Upload presentasi
Presentasi sedang didownload. Silahkan tunggu
Diterbitkan olehEva Fathurrahman Telah diubah "9 tahun yang lalu
1
Image Registration & Tracking dengan Metode Lucas & Kanade
Sumber: Forsyth & Ponce Chap. 19, 20 Tomashi & Kanade: Good Feature to Track
2
Feature Lucas-Kanade(LK)
Extraksi feature dengan metode LK ini adalah sangat populer dalam aplikasi computer vision. Feature diekstraksi dengan mengambil informasi gradient image. Selanjutnya feature ini bisa dimanfaatkan untuk Image registration, yg. Selanjutnya diugnakan utk. tracking, recognition, dan lain-lain Pemilihan feature image yang tepat adalah sangat menentukan keberhasilan proses recognition, tracking, etc.
3
Sejarah Perkembangan LK
Lucas & Kanade (IUW 1981) BAHH ST S BJ HB BL G SI CET SC Bergen, Anandan, Hanna, Hingorani (ECCV 1992) Shi & Tomasi (CVPR 1994) Szeliski & Coughlan (CVPR 1994) Szeliski (WACV 1994) Black & Jepson (ECCV 1996) Hager & Belhumeur (CVPR 1996) Bainbridge-Smith & Lane (IVC 1997) Gleicher (CVPR 1997) Sclaroff & Isidoro (ICCV 1998) Cootes, Edwards, & Taylor (ECCV 1998) LK
4
Image Registration
5
Penerapan metode LK
6
Penerapan pada aplikasi:
Stereo LK BAHH ST S BJ HB BL G SI CET SC
7
Penerapan pada aplikasi:
Stereo Dense optic flow LK BAHH ST SC S BJ HB BL G SI CET
8
Penerapan pada aplikasi:
Stereo Dense optic flow Image mosaics LK BAHH ST SC S BJ HB BL G SI CET
9
Penerapan pada aplikasi:
Stereo Dense optic flow Image mosaics Tracking LK BAHH ST SC S BJ HB BL G SI CET
10
Penerapan pada aplikasi:
Stereo Dense optic flow Image mosaics Tracking Recognition ? LK BAHH ST SC S BJ HB BL G SI CET
11
Derivasi Rumusan Lucas & Kanade #1
12
rumusan L&K 1 I0(x)
13
rumusan L&K 1 I0(x+h) h I0(x)
14
rumusan L&K 1 h I0(x) I(x)
15
rumusan L&K 1 h I0(x) I(x)
16
rumusan L&K 1 I0(x) I(x) R
17
rumusan L&K 1 I0(x) I(x)
18
rumusan L&K 1 h0 I0(x) I(x)
19
rumusan L&K 1 I0(x+h0) I(x)
20
rumusan L&K 1 I0(x+h1) I(x)
21
rumusan L&K 1 I0(x+hk) I(x)
22
rumusan L&K 1 I0(x+hf) I(x)
23
Derivasi Rumusan Lucas & Kanade #2
24
E(h) S [ I(x) - I0(x) - hI0’(x) ]2
rumusan L&K 2 Sum-of-squared-difference (SSD) error E(h) = S [ I(x) - I0(x+h) ]2 x e R E(h) S [ I(x) - I0(x) - hI0’(x) ]2 x e R
25
S I0’(x)2 S 2[I0’(x)(I(x) - I0(x) ) - hI0’(x)2] = 0
rumusan L&K 2 S 2[I0’(x)(I(x) - I0(x) ) - hI0’(x)2] x e R = 0 S I0’(x)(I(x) - I0(x)) x e R h S I0’(x)2
26
S S w(x) S I0’(x)2 w(x)[I(x) - I0(x)] I0’(x) h S I0’(x)[I(x) - I0(x)]
Perbandingan h w(x)[I(x) - I0(x)] S w(x) x S I0’(x) S I0’(x)[I(x) - I0(x)] h S I0’(x)2 x
27
S S w(x) S I0’(x)2 w(x)[I(x) - I0(x)] I0’(x) h S I0’(x)[I(x) - I0(x)]
Perbandingan w(x)[I(x) - I0(x)] S I0’(x) x h S w(x) x S I0’(x)[I(x) - I0(x)] x h S I0’(x)2 x
28
Generalisasi metode Lucas-Kanade
29
Rumus Original S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R
30
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Rumus Original
Dimension of image S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R 1-dimensional LK BAHH ST S BJ HB BL G SI CET SC
31
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Generalisasi 1a
Dimension of image S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R 2D: LK BAHH ST S BJ HB BL G SI CET SC
32
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Generalisasi 1b
Dimension of image S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R Homogeneous 2D: LK BAHH ST SC S BJ HB BL G SI CET
33
Apakah iterasi bisa konvergen?
Permasalahan A Apakah iterasi bisa konvergen? LK BAHH ST S BJ HB BL G SI CET SC
34
Permasalahan A Local minima:
35
Permasalahan A Local minima:
36
S I0’(x)2 -S I0’(x)(I(x) - I0(x)) h Permasalahan B Zero gradient:
x e R h is undefined if S I0’(x)2 is zero x e R S I0’(x)2 x e R LK BAHH ST SC S BJ HB BL G SI CET
37
Permasalahan B Zero gradient: ?
38
S 2 -S (x)(I(x) - I0(x)) hy Permasalahan B’
Aperture problem (mis. Image datar): -S (x)(I(x) - I0(x)) x e R hy S x e R LK BAHH ST SC S BJ HB BL G SI CET
39
Permasalahan B’ No gradient along one direction: ?
40
Jawaban problem A & B Possible solutions: Manual intervention LK BAHH
ST SC S BJ HB BL G SI CET
41
Jawaban problem A & B Possible solutions: Manual intervention
Zero motion default LK BAHH ST SC S BJ HB BL G SI CET
42
Jawaban problem A & B Possible solutions: Manual intervention
Zero motion default Coefficient “dampening” LK BAHH ST SC S BJ HB BL G SI CET
43
Jawaban problem A & B Possible solutions: Manual intervention
Zero motion default Coefficient “dampening” Reliance on good features LK BAHH ST SC S BJ HB BL G SI CET
44
Jawaban problem A & B Possible solutions: Manual intervention
Zero motion default Coefficient “dampening” Reliance on good features Temporal filtering LK BAHH ST SC S BJ HB BL G SI CET
45
Jawaban problem A & B Possible solutions: Manual intervention
Zero motion default Coefficient “dampening” Reliance on good features Temporal filtering Spatial interpolation / hierarchical estimation LK BAHH ST SC S BJ HB BL G SI CET
46
Jawaban problem A & B Possible solutions: Manual intervention
Zero motion default Coefficient “dampening” Reliance on good features Temporal filtering Spatial interpolation / hierarchical estimation Higher-order terms LK BAHH ST SC S BJ HB BL G SI CET
47
Kembali lagi: Rumus Original
[ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R
48
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Rumus Original
Transformations/warping of image S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R Translations: LK BAHH ST SC S BJ HB BL G SI CET
49
Bagaimana bila ada gerakan(motion) tipe lain?
Permasalahan C Bagaimana bila ada gerakan(motion) tipe lain?
50
S [ ] E ( A, h ) = I ( Ax + h ) - I0 ( x ) 2 Generalisasi 2a
Transformations/warping of image S [ ] E ( A, h ) = I ( Ax + h ) - I0 ( x ) 2 x e R Affine: LK BAHH ST SC S BJ HB BL G SI CET
51
Generalisasi 2a Affine:
52
S [ ] E ( A ) = I ( A x ) - I0 ( x ) 2 Generalisasi 2b
Transformations/warping of image S [ ] E ( A ) = I ( A x ) - I0 ( x ) 2 x e R Planar perspective: LK BAHH ST SC S BJ HB BL G SI CET
53
Generalisasi 2b Affine + Planar perspective:
54
Other parametrized transformations
Generalisasi 2c Transformations/warping of image S [ ] E ( h ) = I ( f(x, h) ) - I0 ( x ) 2 x e R Other parametrized transformations LK BAHH ST SC S BJ HB BL G SI CET
55
Other parametrized transformations
Generalisasi 2c Other parametrized transformations
56
S I0’(x)2 -S I0’(x)(I(x) - I0(x)) h ~ -(JTJ)-1 J (I(f(x,h)) - I0(x)) h
Permasalahan B” -S I0’(x)(I(x) - I0(x)) x e R h S I0’(x)2 Generalized aperture problem: ~ -(JTJ)-1 J (I(f(x,h)) - I0(x)) h LK BAHH ST SC S BJ HB BL G SI CET
57
Permasalahan B” Generalized aperture problem: ?
58
Rumus Original S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R
59
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Rumus Original Image type
Grayscale images LK BAHH ST SC S BJ HB BL G SI CET
60
S || || E ( h ) = I ( x + h ) - I0 ( x ) 2 Generalisasi 3 Image type
Color images LK BAHH ST SC S BJ HB BL G SI CET
61
Rumus Original S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R
62
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Rumus Original
Anggapan pixel punya konstan brightness (Constancy assumption) S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R Brightness constancy LK BAHH ST SC S BJ HB BL G SI CET
63
Bagaimana bila iluminasi cahaya bervariasi?
Permasalahan C Bagaimana bila iluminasi cahaya bervariasi?
64
Linear brightness constancy
Generalisasi 4a Constancy assumption S [ ] E ( h,a,b ) = I ( x + h ) - aI0 ( x )+b 2 x e R Linear brightness constancy LK BAHH ST SC S BJ HB BL G SI CET
65
Generalisasi 4a
66
Illumination subspace constancy
Generalisasi 4b Constancy assumption S [ ] E ( h,l ) = I ( x + h ) - lTB ( x ) 2 x e R Illumination subspace constancy LK BAHH ST SC S BJ HB BL G SI CET
67
Bagaimana bila texture berubah?
Permasalahan C’ Bagaimana bila texture berubah?
68
Texture subspace constancy
Generalisasi 4c Constancy assumption S [ ] E ( h,l ) = I ( x + h ) - lTB ( x ) 2 x e R Texture subspace constancy LK BAHH ST SC S BJ HB BL G SI CET
69
Permasalahan D Jelas proses konvergensi menjadi lambat bila jumlah #parameters bertambah !!!
70
Jawaban problem D Percepat konvergensi dengan:
Coarse-to-fine, filtering, interpolation, etc. LK BAHH ST SC S BJ HB BL G SI CET
71
Jawaban problem D Percepat konvergensi dengan:
Coarse-to-fine, filtering, interpolation, etc. Selective parametrization LK BAHH ST SC S BJ HB BL G SI CET
72
Jawaban problem D Percepat konvergensi dengan:
Coarse-to-fine, filtering, interpolation, etc. Selective parametrization Offline precomputation LK BAHH ST SC S BJ HB BL G SI CET
73
Jawaban problem D Percepat konvergensi dengan:
Coarse-to-fine, filtering, interpolation, etc. Selective parametrization Offline precomputation Difference decomposition LK BAHH ST SC S BJ HB BL G SI CET
74
Jawaban problem D Difference decomposition
75
Jawaban problem D Difference decomposition
76
Jawaban problem D Percepat konvergensi dengan:
Coarse-to-fine, filtering, interpolation, etc. Selective parametrization Offline precomputation Difference decomposition Improvements in gradient descent LK BAHH ST SC S BJ HB BL G SI CET
77
Jawaban problem D Percepat konvergensi dengan:
Coarse-to-fine, filtering, interpolation, etc. Selective parametrization Offline precomputation Difference decomposition Improvements in gradient descent Multiple estimates of spatial derivatives LK BAHH ST SC S BJ HB BL G SI CET
78
Jawaban problem D Multiple estimates / state-space sampling
79
Generalisasi metode Lucas-Kanade
Modifikasi yg. Dibuat selama ini adalah: S [ ] I ( x + h ) - I0 ( x ) 2 x e R
80
S [ ] I0 E ( h ) = I ( x + h ) - ( x ) 2 Rumus Original Error norm
Squared difference: LK BAHH ST SC S BJ HB BL G SI CET
81
Permasalahan dengan ourliers? >> Gunakan robust norm
Permasalahan E Permasalahan dengan ourliers? >> Gunakan robust norm
82
S ( ) E ( h ) = r I ( x + h ) - I0 ( x ) Generalisasi 5a Error norm
Robust error norm: LK BAHH ST SC S BJ HB BL G SI CET
83
Rumus Original S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R
84
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Rumus Original
Image region / pixel weighting S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R Rectangular: LK BAHH ST SC S BJ HB BL G SI CET
85
Bagaimana bila background terjadi clutter (bergoyang)?
Permasalahan E’ Bagaimana bila background terjadi clutter (bergoyang)?
86
S [ ] I0 E ( h ) = I ( x + h ) - ( x ) 2 Generalisasi 6a
Image region / pixel weighting S [ I0 ] E ( h ) = I ( x + h ) - ( x ) 2 x e R Irregular: LK BAHH ST SC S BJ HB BL G SI CET
87
Bagaimana bila objek terhalang (foreground occlusion)?
Permasalahan E” Bagaimana bila objek terhalang (foreground occlusion)?
88
S [ ] I0 E ( h ) = I ( x + h ) - ( x ) w(x) 2 Generalisasi 6b
Image region / pixel weighting S [ I0 ] E ( h ) = I ( x + h ) - ( x ) w(x) 2 x e R Weighted sum: LK BAHH ST SC S BJ HB BL G SI CET
89
Generalisasi metode Lucas-Kanade
Modifikasi: S [ ] I ( x + h ) - I0 ( x ) 2 x e R
90
S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 Generalisasi 6c
Image region / pixel weighting S [ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R Sampled: LK BAHH ST SC S BJ HB BL G SI CET
91
Generalisasi metode Lucas-Kanade: Ringkasan
[ ] E ( h ) = I ( x + h ) - I0 ( x ) 2 x e R S ( ) E ( h ) = r I ( f(x, h) ) - lB ( x ) w(x) x e R
92
Ringkasan Generalisasi L&K ? Dimension of image
Image transformations / motion models Pixel type Constancy assumption Error norm Image mask L&K ? Y n
93
Ringkasan Common problems: L&K ? Local minima Aperture effect
Illumination changes Convergence issues Outliers and occlusions L&K ? Y maybe n
94
Ringkasan Penanganan aperture effect: L&K ? Manual intervention
Zero motion default Coefficient “dampening” Elimination of poor textures Temporal filtering Spatial interpolation / hierarchical Higher-order terms L&K ? n Y
Presentasi serupa
© 2024 SlidePlayer.info Inc.
All rights reserved.