Tracking Image dengan Metode feature Lucas-Kanade Sumber: Forsyth & Ponce Chap. 19 Tomashi, Lucas & Kanade: Good Feature to Track Standford Vision & Modeling

Agenda Ulasan metode Lucas-Kanade + Implementasi dengan Matlab Analisa metode Lucas-Kanade Support Maps / Layers: - Robust Norm - Layered Motion - Background Subtraction - Color Layers - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2

Lucas-Kanade: Minimisasi fungsi: Image 1D Intensitas - x + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Linierisasi: Spatial Gradient Temporal Gradient

Lucas-Kanade: Minimisasi fungsi: Image 2D ROI ROI (u,v) + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Spatial Gradient Temporal Gradient

C D C D Lucas-Kanade: Minimisasi fungsi: Image 2D Minimisasi fungsi E(u,v): => C D + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion C -1 D

Impelementasi Lucas-Kanade Step 0: Inisialisasi (dengan manual) Step 1: hitung: C dan D dan cari penyelesaian (u,v): - Hitung image derivatives Fx,Fy,Ft Step 2: re-warp image G: - Sub-pixel image interpolation Step 3: Loop: - Ukur error / terminate + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Impelementasi Lucas-Kanade Step 1: hitung: C dan D dan cari penyelesaian (u,v): - Hitung image derivatives Fx,Fy,Ft Fx, Fy: Filter dengan Gaussian Derivative Kernel: + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Impelementasi Lucas-Kanade Step 1: hitung: C dan D dan cari penyelesaian (u,v): - Hitung image derivatives Fx,Fy,Ft B) Ft: Finite Difference of Blurred F and G: + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Impelementasi Lucas-Kanade Step 1: hitung: C dan D dan cari penyelesaian (u,v): - Hitung image derivatives Fx,Fy,Ft - Hitung dengan Gaussian kernel (menggunakan coarse-to-fine strategy dengan pengurangan sigma) + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Operasi Warping gunakan fungsi interp2 dari Matlab Impelementasi Lucas-Kanade Step 2: re-warp image G: - Sub-pixel image interpolation + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Operasi Warping gunakan fungsi interp2 dari Matlab

Impelementasi Lucas-Kanade Step 3: Loop: - Ukur error / terminate perhatikan: + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Analisa Grafis metode Lucas-Kanade - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2

C D C D Lucas-Kanade: problem singulariti Minimisasi fungsi E(u,v): => C D + lets first talk about the function itself that should be Minimisasi fungsid + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion C -1 D

Lucas-Kanade: problem singulariti + lets first talk about the function itself that should be Minimisasi fungsid + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Fx=0, Fy=0

Lucas-Kanade: problem singulariti + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Fx=0, Fy=0 Fy=0

Lucas-Kanade: problem singulariti + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Fx=0, Fy=0 Fy=0

Lucas-Kanade: Aperture Problem + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Fx=0, Fy=0

Lucas-Kanade: Aperture Problem + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Bergen et al.

Aperture Problem: Bisa diatasi ??? Hindari sebisa mungkin ! Gunakan nilai Eigenvalues untuk inisialisasi “Good Features” (lihat paper “Good Features to track” Shi-Tomasi) - Lokasi Good Feature berada pada: min(eig1,eig2) > a + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Aperture Problem: Bisa diatasi ??? “Good Features” (Shi-Tomasi) + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Aperture Problem: Bisa diatasi ??? coba di Hack ! regularisasi C: + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Aperture Problem: Bisa diatasi ??? Simoncelli et al 1991: + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Aperture Problem: Bisa diatasi ??? Tambah Aperture (window feature) ! Coarse-to-fine Pyramids (Bergen et al, Simoncelli) + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Aperture Problem: Bisa diatasi ??? Tambah Aperture (window feature) ! akibat: integrasi ROI lebih besar -> motion model jadi lebih komplex + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion

Pengembangan Lucas-Kanade secara Affine Affine Motion Model: 2D Translation 2D Rotation Scale in X / Y Shear + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Matlab demo ->

Pengembangan Lucas-Kanade secara Affine Affine Motion Model -> digunakan pada Lucas-Kanade: + lets first talk about the function itself that should be minimized + illustrate it only on a scanline (and generalize it later) + assume it moves by 2 pixel to the right… + we could search + but we need to deal with 6 DOF or higher DOF cases + linearize (old Horn + Schunk) + just matrix inversion Matlab demo ->

Support Maps / Layers: - Robust Norm - Layered Motion - Background Subtraction - Color Based Tracking - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2

Support Maps / Layers L2 Norm vs Robust Norm Bahaya dari fitting secara least square: Akibat adanya Outliers (gangguan pixel luar) menyebabkan square error menjadi sangat besar - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2

Support Maps / Layers L2 Norm vs Robust Norm Bahaya dari fitting secara least square: L2 - title - report on work done together with JM at UCB and together with MC MS at Interval D 2 2

Support Maps / Layers L2 Norm vs Robust Norm Bahaya dari fitting secara least square: L2 robust - title - report on work done together with JM at UCB and together with MC MS at Interval D D 2 2

Support Maps / Layers Robust Norm -- baik untuk menangani outliers nonlinear optimization robust - title - report on work done together with JM at UCB and together with MC MS at Interval D 2 2

Support Maps / Layers Black-Jepson-95 - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2

Support Maps / Layers Layered Motion (Jepson/Black, Weiss/Adelson, …) - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2

Support Maps / Layers Kasus spesial dari Layered Motion: - Background substraction - Outlier rejection (== robust norm) - Kasus sederhana: Tiap Layer punya warna seragam - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2

Support Maps / Layers Color Layers: P(skin | F(x,y)) - title - report on work done together with JM at UCB and together with MC MS at Interval 2 2