Presentasi sedang didownload. Silahkan tunggu

Presentasi sedang didownload. Silahkan tunggu

CHAPTER 5 NON-LINIER OPTIMIZATION. 5.1. Introduction Problem estimisasi secara umum MaX (Min) f (X) s.t. X  y s.t. X  y F disebut objektif function.

Presentasi serupa


Presentasi berjudul: "CHAPTER 5 NON-LINIER OPTIMIZATION. 5.1. Introduction Problem estimisasi secara umum MaX (Min) f (X) s.t. X  y s.t. X  y F disebut objektif function."— Transcript presentasi:

1 CHAPTER 5 NON-LINIER OPTIMIZATION

2 5.1. Introduction Problem estimisasi secara umum MaX (Min) f (X) s.t. X  y s.t. X  y F disebut objektif function. X disebut himpunan feasible. Himpunan feasible biasanya digambarkan dengan persamaan atau pertidaksamaan yang disebut kendala. Contoh linier programming problem : MaX c t X s.t. AX = b X ≥ 0 X ≥ 0 X = {X  R n | AX = b, X ≥0} X = {X  R n | AX = b, X ≥0} Jika tidak ada kendala maka X = R n, dan problem ini disebut optimisasi tanpa kendala

3 5.2. Global dan Local Optima f = D  R dimana D  R n. Fungsi f dikatakan memiliki global (absolute) maximum pada X*  D jhj f(X) ≤ f(X * )  X  D. Titik X* disebut global maximizer (or maximizer) dari f dan f(X*) disebut global maximum dari f dan sebaliknya. Global minimum f(X) ≥ f (X*)  X  D. Fungsi f dikatakan memiliki local (relative) maximum pada X*  D jhj terdapat neighborhood N (X*) seperti : f(X) ≤ f(X*)  X  N (X*)  D. Titik X* disebut local maximizer dari f dan f(X*) disebut local maximum dari f. Dan sebaliknya adalah: Fungsi local minimum X*  D jhj f(X) ≥ f(X*)  X  N(X*)  D Definisi

4 X 1, X 3, X 5 adalah local maximum, X 5 global max X 2, X 4, X 6 adalah local minimum, X 2 global min Contoh Local dan Global Optimum X1X1X1X1 X2X2X2X2 X3X3X3X3 X4X4X4X4 X5X5X5X5 X6X6X6X6 D 0 f(X)

5 (1)Global max digunakan secara relatif to domain D dari f yaitu global maximizer berarti global maximizer to domain (2)Istilah optimum digunakan baik maximum maupun minimum (3)Suatu global maximizer (global minimizer) adalah local maximizer (local minimizer) (4)Local maximum dapat lebih kecil dibanding local minimun (5) Suatu fungsi tidak memiliki baik maximizer ataupun minimizer dalam domainnya. f(X) = X, tidak memiliki max/min dalam interval (0,1) Catatan : Interval ini tidak tertutup meskipun interval tertutup tetapi tidak memiliki batas (bounded) Remark Theorema Suatu fungsi continuous real-valued didefinisikan atas suatu compact subset S dari R n memiliki maximizer & minimizer dalam S.

6 5.3 Necessary (or First-Order) Condition untuk Local Optimum untuk Local Optimum Andaikan f : D  R, dimana D ≤ R n, dan andaikan turunan partial dari f  X(S) pada suatu interior point X* dari D X* adalah local optimizer dari f, maka f’(X) = 0. Proof : Theorema (1) Kasus satu variabel dari theorema (2)Syarat bahwa turunan exist pada local optimizer adalah necessary dalam teorema contoh : f(X) = X 2/3 f’(X) = 2/3X -1/ Remark f(X) f’(X*)=0 0X*X f(X)f’(X*)=0 0X*X

7 Fungsi f memiliki local minimum pada X*=0 but f’(0)≠0. Contoh di atas tidak differentiable pada X*=0 (ini berarti bukan sufficient). (3)Syarat suatu X* adalah interior point dari domainnya adalah necessary. Contoh andaikan gambar 5.5 domain D adalah interval tertutup [0,1]. Memenuhi local minimum pada X*=0 tetapi f’(X*)  0. Note: X* tidak pada interior point dari D. X* f(X) X Gambar X* f’(X*)  0 Gambar 5.5 f(X) = X 2/3 X D

8 (4)Kebalikan dari teorema tidak benar Contoh : f(X) = X 3 f’(X) = 3X 2 f’(X) = 3X 2 f’(0) = 0 tetapi f tidak memiliki local optimum pada X*=0. Contoh tiga dimensi f(X 1,X 2 ) = X 1 2 – X 2 2 Gambar 5.7 gradien f adalah f’(X 1,X 2 ) = Pada X*=[0,0] t, f(X*)=0 tetapi untuk titik dekat X* pada sb X 1 –axis, f(X 1,X 2 )>0 dan titik dekat X* pada sb X 2 f(X 1,X 2 ) 0 dan titik dekat X* pada sb X 2 f(X 1,X 2 )<0. 2X 1 -2X 2 X f(X) f’(X)=X 3 Gambar 5.6

9 Consekuensi X*=0 mendekati local maximizer dan local minimizer sehingga titik tersebut disebut saddle point. Gambar 5.7. Saddle point X1X1 X2X2 X3X3 Titik X* dimana f’(X*)=0 disebut stationary point f. stationary point dan bukan titik optimum disebut saddle point dari f. Dalam pandang theorema titik optimizer f pada domain D ditemukan. (a) Stationary point (figure 5.7.1(a)) (b) The boundary point (figure 5.7.1(b)) (c) f’(X) tidak exist (figure 5.7.1(c)) Definisi

10 a. f’(X*) = 0 b. f’(X*)  0 c. f’(X*)=n tidak diperoleh Gambar Kemudian sufficient condition untuk suatu stationary point menjadi local optimizer ketika f setidaknya dapat di diferensialkan 2 kali secara continuous. 5.4 Sufficient (Second-Order) Condition for Local Optima : The One-Variable Case Local Optima : The One-Variable Case Theorema Andaikan D ≤ R. f : D  R memiliki continuous second-order derivative atas open interval I yang berisi X*. Andaikan f’(X*)=0. (i) Jika f”(X*) > 0, X* adalah local minimizer dari f (ii) Jika f”(X*) < 0, X* adalah local maximizer dari f X* X* X* D D D

11 5.4.2 Contoh Let f : R  R f(X) = 2X 3 – 3X 2 – 12X + 1 f(X) = 2X 3 – 3X 2 – 12X + 1 f’(X) = 6X 2 – 6X – 12 f’(X) = 6X 2 – 6X – 12 Setting f’(X) = 0 diperoleh X* = 2, X** = -1 f”(X) = 12 X – 6 f”(2) = 12(2) – 6 = 18 f”(-1) = 12(-1) – 6 = -18 Ini berarti bahwa X* = 2 adalah local maximizer dari f X** = -1 adalah local minimizer dari f X** = -1 adalah local minimizer dari f Remark Ketika f”(X*)=0, tidak ada kesimpulan yang dapat dibuat. Kita gunakan theorema berikutnya untuk menggeneralisasikan theorema Proof : Proof :

12 Let D C R dan f : D  R memiliki continuous n th -order derivative pada N(X*) C D. Suppose f’(X*) = f”(X*) = … = f (n-1) (X*) = 0 dan f (n)  0 (i) Jika n bilangan genap dan f (n) (X * ) > 0, maka X* adalah local minimum dari f minimum dari f (ii) Jika n bilangan genap dan f (n) (X*) < 0 maka X* adalah local maXimum dari f maXimum dari f (iii) Jika n bilangan ganjil, maka X* adalah saddle point dari f Proof : Proof : Theorema f(X) = (X – 5) 4 f(X) = (X – 5) 4 Necessary condition : f’(X) = 4(X-5) 3 = 0 Stationary Point : X* = 5 Sufficient condition : f”(X) = 12(X-5) 2, f”(5) = 0 f (3) (X) = 24(X–5), f 3 (5) = 0 f (3) (X) = 24(X–5), f 3 (5) = 0 f 4 (X) = 24, F 4 (5) = 24 f 4 (X) = 24, F 4 (5) = C ontoh Kasus ini n=4 dan f 4 (5) > 0, sehingga X*=5 adalah local minimizer pada f.

13 5.5 Sufficient (or Second-Order) Condition for Local Optima : The n-Variable Case Local Optima : The n-Variable Case Theorema Let D C R n + f : D  R mempunyai continous second order partial derivative pada N(X*) C D dan f’(X*) = 0. (i) Jika f”(X*) adalah positif definit, X* adalah local (i) Jika f”(X*) adalah positif definit, X* adalah local minimizer dari f minimizer dari f (ii) Jika f”(X*) adalah negatif definit, X* adalah local (ii) Jika f”(X*) adalah negatif definit, X* adalah local maximizer dari f maximizer dari f (iii) Jika f”(X*) indefinite, X* adalah saddle-point dari f (iii) Jika f”(X*) indefinite, X* adalah saddle-point dari f Proof : Remark Ketika Hessian Matriks f”(X*) adalah positif atau negatif semi definit, tidak ada kesimpulan yang dapat ditarik.

14 5.5.3 Contoh f(X) = -X 1 2 – 3X 2 2 – 2X X 1 – 12X 2 + 8X 3 – 5 -2X X 2 – 12 -4X f’(X) = Gradien dari f : Setting f’(X) = 0, maka titik stationer X* = [2 -2 2] t X* = [2 -2 2] t Hessian matriks dari f pada X* adalah f”(X* ) = [f’’ ij (X*)] = Memiliki leading principle minor adalah |[-2]| = -2 < 0 |[-2]| = -2 < = 12 > 0

15 = -48 < Dari theorema bahwa f”(X*) adalah negatif definite. Alternatif, catatan bahwa f”(X*) adalah matriks diagonal, maka eigenvalue adalah elemen diagonal (theorema ). Sejak eigenvalue negatif maka f”(X*) adalah negatif definit (by Theorema ). Sehingga X* adalah local maximizer dari f (by theorema 5.5.1) Contoh f(X) = 4X X 1 X 2 – 3 / 2 X ½X Gradient dari f adalah f’(X) = 12X X 2 – 3X 1 12X X 2 – 3X 1 X 1 + X 2 Setting f’(X) = 0 diperoleh titik stationer : X* = 00, X** = 1/3 1/3-1/3

16 Hessian matriks dari f adalah f”(X) = 24X Pertimbangankan titik stationer X* maka Hessian matrix dari f pada X* adalah f”(X*) = 24(0) = Matriks Real symmetric dimana eigenvalue adalah  1 = 1 + 5,  2 = 1 – 5 Apabila  1 > 0 dan  2 0 dan  2 < 0 sesuai teorema bahwa f”(X*) adalah indefinite. Konsekuensinya, X* adalah saddle-point dari f by theorema Hessian matrix pada X** adalah f”(X**) = 24(1/3) =

17 Matriks Real symmetric dimana Leading Principal Minors (LPM) adalah |[5]| = 5 > 0, = 4 > 0 |[5]| = 5 > 0, = 4 > Theorema f”(X**) adalah positif definite. Alternatif, calculate eigenvalue dari f”(X**);  1 = 3 + 5,  2 = 3 – 5  1 = 3 + 5,  2 = 3 – 5  1 >0 dan  2 >0, by theorem f”(X**) adalah positif definite. sehingga X** sebagai local minimizer dari f by theorem Contoh Let f : R 2  R didefinisikan dengan f(X) = X X 2 4 f(X) = X X 2 4 Gradient dari f adalah f’(X) = f’(X) = Setting f’(X) = 0, diperoleh stationary point X* = [ 0 0] t 2X 1 4X 2 3

18 Hessian matriks dari f adalah f”(X) = f”(X) = Pada X*, Hessian matriks dari f adalah f”(X*) = f”(X*) = Eigenvalue adalah  1 =2 dan  2 =0 sesuai theorema that f”(X*) adalah positive semidefinite tetapi tidak positif definit sehingga tidak dapat diputuskan X Contoh Let f : R 2  R didefinisikan dengan Let f : R 2  R didefinisikan dengan f(X) = X 1 2 – X 2 4 f(X) = X 1 2 – X 2 4 Gradient pada f adalah Gradient pada f adalah f’(X) = 2X 1 2X 1 - 4X 2 2

19 Setting f’(X) = 0, diperoleh titik stasioner X* = [ 0 0] t X* = [ 0 0] t Hessian matriks dari f adalah f”(X) = f”(X) = Pada X*, Hessian matriks dari f adalah f”(X*) = f”(X*) = Theorem tidak dapat digunakan tetapi f(X*)=0 dan untuk titik X mendekati X* pada X 1 –axis, f(X)=(X 1 ) 2 > 0 sedang untuk titik X mendekati X* pada X 2 –axis, (f(X)=- (X 2 ) 4 0 sedang untuk titik X mendekati X* pada X 2 –axis, (f(X)=- (X 2 ) 4 < 0. Sehingga X* adalah saddle-point dari f (lihat Danao, ) X

20 5.5.7 Contoh Profit Maximization q = output, R(q), C(q),  (q). R dan C continuous second-order derivatives.  (q) = R(q) – C(q)  ’(q*) = R’(q*) – C’(q*) = 0 R’(q*) = C’(q*)  optimal profit R’(q*) < C’(q*)  q    R ’ (q * ) > C ’ ( q*)  q    Contoh The Method least Squares. Garis regresi : Y =  0 +  1 X Untuk setiap i  i = Y i – (  0 +  1 X i )  i = Y i – (  0 +  1 X i ) Untuk estimasi  0 dan  1  (  i ) 2 =  (Y i –  0 –  1 X i ) 2  (  i ) 2 =  (Y i –  0 –  1 X i ) 2 n i=1 n i=1

21 adalah minimum.  disederhanakan sebagai . f(  0,  1 ) =  (Y i –  0 –  1 X i ) 2 f’(  0,  1 ) = 0 untuk minimum  2(Y i –  0 –  1 X i )( -1) = 0  2(Y i –  0 –  1 X i )(-X i ) = 0 Maka n  0 + (  X i )  1 =  Y 1 n  0 + (  X i )  1 =  Y 1 (  X i )  0 + (  X 1 2 )  1 =  Y i X i Normal equation dalam bentuk matrix nXinXiXiXi2XiXi2nXinXiXiXi2XiXi2 B0B0B1B1B0B0B1B1 yiyiyiXiyiXiyiyiyiXiyiXi = Asumsi coefficient matriks adalah non-singular, menggunakan Cramer’s Rule, titik stasioner :  1 * = n  X i Y i – (  X i ) (  Y i ) n  (X i ) 2 – (  X i ) 2  0 * =  Yi – (  X i )  1 * 1n 1n n i=1

22 to verify  * is a global minimizer dari f, dengan melihat Hessian matrix dari f : f”(  0,  1 ) =  2  2X i  2X i  2X i 2 2n2  X i 2  X i 2  X i 2 = LPM adalah : |[2n]| = 2n>0 ; 2n2  X i 2  X i 2  (X i ) 2 = 4n  (X i ) 2 – 4(  X i ) 2 = 4[n  (X i ) 2 – (  X i ) 2 ] = 4[(n-1)  (X i ) 2 –  2 X i X j ] >0 Dari persamaan (5-2) Hessian matriks f”(  0,  1 ) adalah positive definite and so the stationary point  * minimizer f. 5.6 Optima of Concave and Convex Functions Let X,Y  R n, titik Z =  X + (1–  )Y Dimana 0 ≤  ≤ 1 disebut convex combination dari X dan Y Definisi

23 Geometrically, the set of all convex combinations of X and Y is the line segment joining X and Y Remark A subset C of R n adalah convex jhj line segment berhubungan dari titik dalam C berada dalam himpunan C. In symbols, C adalah convex jhj X,Y  C, 0 ≤  ≤ 1 (  X + (1-  )Y)  C X,Y  C, 0 ≤  ≤ 1 (  X + (1-  )Y)  C Definisi 1)Himpunan kosong (O) dan himpunan hanya satu elemen adalah convex set 2) R n adalah convex set Remarks y=1/X Convex Set Strictly Convex Not Strictly Convex

24 Not ConveX Set 3)The following sets are convex : (a) The closed half-spaces : H +( P,  ) = {X  R n | P t X   } H - (P,  ) = {X  R n | P t X ≤  } (b) The hyperplane : H(P,  ) = {X  R n | P t X =  } (c) The Non-negative orthant : R n + = {X  R n | X  0} (d) The positive orthant : R n ++ = {X  R n |X>0} Interseksi dari convex set adalah convex. Proof : Theorema A subset C of R n adalah strictly convex jhj X,Y  C, X  Y, 0 <  < 1 [  X + (1-  )Y]  int (C) Definisi

25 An open convex set is strictly convex. A closed disk is strictly convex while a closed triangle is not. Intuitively, a closed strictly convex set does not have a flat portion on its boundary Remarks A function f : C  R defined on a convex subset C of R n is said to be concave on C if and only if X,Y  C, 0 ≤  ≤ 1, f(  X+(1-  )Y) ≥  f(X)+(1-  )f(Y) X,Y  C, 0 ≤  ≤ 1, f(  X+(1-  )Y) ≥  f(X)+(1-  )f(Y) Definisi f : C  R defined on a convex subset C of R n is said to be strictly concave on C if and only if X,Y  C, X ≠ Y, 0  f(X)+(1-  )f(Y) X,Y  C, X ≠ Y, 0  f(X)+(1-  )f(Y) Definisi f : C  R defined on a convex subset C of R n is said to be strictly convex on C if and only if X,Y  C, X ≠ y, 0 <  < 1, f(  X+(1-  )Y) <  f(X)+(1-  )f(Y) X,Y  C, X ≠ y, 0 <  < 1, f(  X+(1-  )Y) <  f(X)+(1-  )f(Y) Definisi

26 A concave (convex) function is continuous in the interior of its domain Concave function Convex function Theorema f(X 1 ) f(X 2 ) X1X1   (X 1 )+(1-  )X 2 X2X2 f(X 1 ) f(X 2 ) X1X1   X 1 +(1-  )X 2 X2X2   f(X 1 )+(1-  )f(X 2 )  f(  X 1 +(1-  )X 2  f(  (X 1 )+(1-  )X 2 )   f(X 1 )+(1-  )f(X 2 )

27 Concave function Concave function Convex function Convex function Neither concave nor convex functions Figure 5.11 Y=ln(X) Y X Strictly concave function Y=1/X Y X Strictly convex function

28 f : C  R dimana C adalah convex subset of R n (i)if f is concave on C then UC(  ) = {X  C|f(X) ≥  } (i)if f is concave on C then UC f (  ) = {X  C|f(X) ≥  } is convex for every   R (ii)if f is convex on C then LC(  ) = {X  C|f(X) ≤  } (ii)if f is convex on C then LC f (  ) = {X  C|f(X) ≤  } is convex for every   R is convex for every   R Theorema f : D  R adalah function defined on a subset D of R n. - UC(  ) = {X  D| f(X) ≥ ,   R} - UC f (  ) = {X  D| f(X) ≥ ,   R} disebut upper contour set of f disebut upper contour set of f - LC(  ) = {X  D| f(X) ≤ ,   R} - LC f (  ) = {X  D| f(X) ≤ ,   R} disebut lower contour set of f disebut lower contour set of f - C(  ) = {X  D| f(X) = ,   R} - C f (  ) = {X  D| f(X) = ,   R} disebut contour (or level) set of f disebut contour (or level) set of f Definisi Remark

29 Theorem f : C  R adalah continuously differentiable on convex subset of R n (i) f adalah concave on C if and only if X,Y  C, f(Y) - f(X) ≤ [f’(X)] t (Y-X) X,Y  C, f(Y) - f(X) ≤ [f’(X)] t (Y-X) (ii) f adalah convex on C if only if X,Y  C, f(Y) - f(X) ≥ [f’(X)] t (Y-X) X,Y  C, f(Y) - f(X) ≥ [f’(X)] t (Y-X) Theorem f : I  R adalah continuously differentiable on open interval I (i)f adalah concave on I if only if X,Y  I, f(Y) - f(X) ≤ f’(X) (Y-X) (ii) f adalah convex on I if only if X,Y  I, f(Y) - f(X) ≥ f’(X) (Y-X)

30 Theorem f : C  R adalah continuously differentiable on open convex subset C of R n subset C of R n (i)f adalah strictly concave on C if only if X,Y  C, X ≠ Y f(Y) - f(X) < [f’(X)] t (Y-X) X,Y  C, X ≠ Y f(Y) - f(X) < [f’(X)] t (Y-X) (ii) f adalah strictly convex on C if only if X,Y  C, X ≠ Y f(Y) - f(X) > [f’(X)] t (Y-X) X,Y  C, X ≠ Y f(Y) - f(X) > [f’(X)] t (Y-X) Theorem f : C  R adalah twice continuously differentiable on open concex subset C of R n then (i)f is concave on C jhj Hessian matrix f”(X) is negatif semidefinite on C (ii)f is convex on C jhj Hessian matriX f”(X) is positive semidefinite on C

31 f : I  R be twice continuously differentiable on an open interval I X{Ci | if the hessian matriks f”(X) adalah negatif (i) f is concave on I jhj f”(X) ≤ 0 on I (ii) f is conveX on I jhj f”(X) ≥ 0 on I Theorem f have continuous second-order partial derivative on an open convex subset C of R n (i)If Hessian matriks f”(X) is negative definite for every X  C, then f is strictly concave on C (ii) If Hessian matriks f”(X) is positive definite for every X  C, then f is strictly convex on C Proof : Polak Proof : Polak Theorem

32 f : R  R defined f (X) = X 4. f is continuous second-order derivative on R. f is strictly convex on R. But f”(0) = 0 which is not positive definite Remark f : I  R adalah twice continuously differentiable on an open interval I on an open interval I (i) If f”(X) < 0 for every X  I then f is strictly concave on I (ii) If f”(X) > 0 for every X  I then f is strictly convex on I Sum and compositions of concave and convex function Theorema

33 f : C  R dan g : C  R; dimana C is a convex subset on R n (i) If f dan g adalah strictly concave on C then (a) f + g is strictly concave on C (a) f + g is strictly concave on C (b)  f is strictly concave for each  > 0 (b)  f is strictly concave for each  > 0 (c)  f is strictly convex for each  < 0 (c)  f is strictly convex for each  < 0 (ii) If f dan g adalah strictly convex on C then (a) f + g is strictly convex on C (a) f + g is strictly convex on C (b)  f is strictly convex on C for each  > 0 (b)  f is strictly convex on C for each  > 0 (c)  f is strictly concave on C for each  < 0 (c)  f is strictly concave on C for each  < Theorem Theorem f : C  R dan g : C  R where C is a convex subset of R n (i) If f dan g adalah concave on C then (a) f + g is concave on C (a) f + g is concave on C (b)  f is concave on C for each  > 0 (b)  f is concave on C for each  > 0 (c)  f is convex on C for each  < 0 (c)  f is convex on C for each  < 0 (ii) If f dan g adalah convex on C then (a) f + g is convex on C (a) f + g is convex on C (b)  f is convex on C for each  > 0 (b)  f is convex on C for each  > 0 (c)  f is concave on C for each  < 0 (c)  f is concave on C for each  < 0

34 Let : f : C  R be defined on a convex subset C of R n such that f(C) is convex. Let g : f(C)  R be defined on f(C) (i) If f is strictly concave on C and g is strictly concave and increasing of f (C), then the composition of f and g is strictly concave on C. of f (C), then the composition of f and g is strictly concave on C. (ii) If f is strictly convex on f(C), then the composition of f and g is strictly convex on C. strictly convex on C Theorem Let : f : C  R be defined on a convex subset C of R n such that f(C) is convex. Let g : f(C)  R be defined on f(C) (i) If f is concave on C and g is concave and increasing of f (C), then the composition of f and g is concave on C. then the composition of f and g is concave on C. (ii) If f is convex on C and g is convex and increasing on f (C), then the composition of f and g is convex on C. then the composition of f and g is convex on C Theorem

35 (1) f(X) = X X 1 X 3 – X 2 + X X 2 X 3 + 3X 3 2 Hessian matrix of f is Hessian matrix of f is Example f”(X) = LPM are LPM are [2] > 0 [2] > = 4 > = 20 > 0 f”(X) is positive definite for each X  R 3. f”(X) is positive definite for each X  R 3. Therefore, f is strictly conveX on R 3. Therefore, f is strictly conveX on R 3.

36 (2) Consider the production function f : R ++ 2  R f(L,K) = L  – K , 0< ,  <1,  +  <1 f(L,K) = L  – K , 0< ,  <1,  +  <1 The gradient of f is The gradient of f is f’(L,K) =  L  -1 K   L  K  -1 The Hessian matrix of f is f”(L,K) =  (  -1)L  -2 K   L  -1 K  -1  L  -1 K  -1  (  -1)L  K  -2 and its Leading Principal Minors are |[  (  -1)L  -2 K  ]| =  (  -1)L  -2 K  < 0  (  -1)L  -2 K   L  -1 K  -1  (  -1)L  -2 K   L  -1 K  -1  L  -1 K  -1  (  -1)L  K  -2  L  -1 K  -1  (  -1)L  K  -2 = [  (  -1)(  -1) –  2  2 ]L 2  -2 K 2  -2 =  [1-(  +  )]L 2  -2 K 2  -2 > 0 If follows that this function is strictly concave on the positive quadrant.

37 (3) Let f : R ++  R be defined by f(X) = Ln(X) f”(X) = < 0 For all X  R ++. It follows from theorem For all X  R ++. It follows from theorem that f is strictly concave on R ++ that f is strictly concave on R ++ X 2 (4) Let f : R +  R defined by f(X) = X  f’(X) =  X  -1 f”(X) = (  -1)  X  -2 Hence, f is strictly concave if  < 1 Hence, f is strictly concave if  < 1 f is strictly convex if  > 1 f is strictly convex if  > 1 f is concave and convex if  = 1 f is concave and convex if  = 1 Hence, f is concave if  ≤ 1 Hence, f is concave if  ≤ 1 f is convex if  ≥ 1 f is convex if  ≥ 1

38 (5) Let f : R ++ n  R defined by f(X) = C t X =  C i X i then the function g : R ++ n  R definey by g(X) = ln (C t X) = ln (  C i X i ) is concave on R ++ n, since f is concave and ln is a concave and incrasing function. concave and incrasing function. n i=1 n i=1 (6) Let f : R ++ 2  R defined by f(X) =  1 ln (X 1 ) +  2 ln (X 2 ),  1,  2 > 0 f(X) =  1 ln (X 1 ) +  2 ln (X 2 ),  1,  2 > 0 claim that f is strictly concave on R ++ 2 claim that f is strictly concave on R ++ 2

39 (i)Every local maximizer of a concave function is a global maximizer (ii)Every local minimizer of a convex function is a global minimizer Proof : Theorem (i)A local maximizer of strictly concave function is unique (ii)A local minimizer of strictly convex function is unique Theorem Example : Consider example (1) f(X) = X X 1 X 3 – X 2 + X X 2 X 3 + 3X 3 2 f(X) = X X 1 X 3 – X 2 + X X 2 X 3 + 3X 3 2 It was shown that f is strictly convex on R 3. The stationary point are obtained by setting f’(X) = 0, i.e. 2X 1 + X 3 = 0 2X 2 + X 3 = 1 2X 2 + X 3 = 1 2X 1 + X 2 + 6X 3 = 0 2X 1 + X 2 + 6X 3 = 0 Stationary point : X* = [ 1 / 20, 11 / 20, 2 / 20 ] t

40 Hessian matriks f’’(X*) is positive definite then theorem 5.5.1, X* is a local minimizer of f. But f is strictly convex: hence, X* is the unique global minimizer of f by theorem and theorem Example The Linier Programming Problem min Co + C t X S.t. AX ≤ b, A is m x n, X  R n f is strictly conveX on R 3. The stationary point f’(X) = 0 The objective function of the Linier Programming (LP) is both concave and convex. The feasible region X = {X  R n |AX ≤ b, X ≥ 0} is convex since it is the intersection of convex set X = H  R + n Dimana H = {X  R n |AX ≤ b} and R + n = {X  R n |X ≥ 0} Consequently, any optimal solution of the LP is a global optimal solution. Obviously, This is also true of the maximization problem.

41 5.7.1 Definition Function f : C  R defined on convex subset C of R n is said to be quasiconcave on C if and only if 5.7 The Optima of Quasiconcave and Quasiconvex Function Quasiconvex Function Definition Function f : C  R defined on convex subset C of R n is said to be quasiconvex on C if and only if Function f : C  R defined on convex subset C of R n is said to be strictly quasiconcave on C if and only if Definition Definition Function f : C  R defined on convex subset C of R n is said to be strictly quasiconvex on C if and only if Corollary (i)Every concave function is quasiconcave (ii)Every convex function is quasiconvex

42 A linear function is both quasiconcave and quasiconvex Corollary Example Theorem Theorem Theorem (i) A strictly concave function is strictly quasiconcave (ii) A strictly convex function is strictly quasiconvex. Let : f : C  R be continuous on a strictly convex subset C of R n. If is strictly quasiconcave on C, then the upper contour set UC f (  ) is strictly convex for every   R. (i)A differentiable function f : C  R defined on open convex set C C R n is quasiconcave on C if and only if X,Y  C, f(X) ≥ f(Y) [f’(Y)] t (X-Y) ≥ 0 (ii) A differentiable function f : C  R defined on open convex set C C R n is quasiconvex on C if and only if X,Y  C, f(X) ≤ f(Y) [f’(Y)] t (X-Y) ≤ 0 The Normal Distribution Function

43 5.8 Constrained Optimization The general form of the constrained optimization problem may be expressed as follows : The general form of the constrained optimization problem may be expressed as follows : Max (Min) f (X) S.t. G k (X) ≥ 0, k = 1, 2, 3, …, m Where X  R n. The function f is objective function and the in-equalities called the constraints. A vector X satisfies the constraints called a feasible solution and the set of feasible solutions is called feasible set or feasible region. A feasible solution that maximizes (minimizes) the value of the objective function on the feasible region is called maximizer (minimizer). The term optimizer or optimal solution refers to either maximizer or minimizer. The distinction between a constrained optimization problem and an un-constrained problem can be seen fram the geometry of two-variable problem with a single equality constraint. The distinction between a constrained optimization problem and an un-constrained problem can be seen fram the geometry of two-variable problem with a single equality constraint. Suppose that the problem is : Max f(X 1,X 2 ) S.t. P 1 X 1 + P 2 X 2 = Y

44 5.9 Optimization With One Equality Constraint : The Two-Variable Case The Two-Variable Case Solution by Direct Substitution Consider the problem Max (Min) f (X 1, X 2 ) S.t. g (X 1, X 2 ) = 0 Dimana f dan g adalah diferensiabel. If, from the constraint g(X 1, X 2 ) = 0, it is possible to express one variable, say X 2 in terms of the other variable then write X 2 = h(X 1 ) and substitute this in f (X 1, X 2 ). The problem reduces to the un-constrained problem of optimizing f [ X 1, h(X 1 ) ] Example Min (X 1 – 1) 2 + (X 2 ) 2 S.t. 2 X 1 + X 2 ) = 4 Letf (X 1, X 2 ) = (X 1 – 1) 2 + (X 2 ) 2 From the constraint, we get X 2 = h(X 1 ) = 4 - 2(X 1 ) X 1 * = 9/5 X 2 * = 4 – 2(9/5) = 2/5

45 5.9.2 Remark The solution by direct substitution is particularly useful if the constraint is linear since it is easy to express one variable in terms of the other variables. When the constraint is non-linear, it may be difficult or impossible to obtain such an explicit function. This limits the usefulness of this method. On the other hand, the graphical method is limited to two-variable problems. These limitations are not possessed by a third method of solution called the Lagrange Multiplier Method Definition Max (Min) f (X) S.t. g (X 1 ) = b, X  R 2 The function L defined by L( ,X) = f (X) +  [ g(X) – b ] L( ,X) = f (X) +  [ g(X) – b ] Is called the Lagrangean of the problem and the scalar  is called the Lagrange multiplier

46 5.9.4 Remark (1) The Lagrangean is also written as L( ,X) = f (X) –  [ g(X) – b ] L( ,X) = f (X) –  [ g(X) – b ] (2) The Gradient of L is L  ’ ( ,X) g (X) – b L 1 ’ ( ,X) f 1 ’ (X)+  g 1 ’(X) L 2 ’ ( ,X) f 2 ’ (X)+  g 2 ’(X) L’ ( ,X) = = and the Hessian matrix of L is L  ’’ ( ,X) L 1  ’’ ( ,X) L 2  ’’ ( ,X) L  1 ’’ ( ,X) L 11 ’’ ( ,X) L 21 ’’ ( ,X) L  2 ’’ ( ,X) L 12 ’’ ( ,X) L 22 ’’ ( ,X) L’’ ( ,X) = = 0 g 1 ’(X) g 1 ’(X) g 2 ’(X) g 2 ’(X) g 1 ’(X) g 1 ’(X) L 11 ’’( ,X) L 21 ’’( ,X) g 2 ’(X) g 2 ’(X) L 12 ’’ ( ,X) L 22 ’’ ( ,X)

47 5.9.5 Theorem (Necessary or First-Order Condition) Given the problem Max (Min) f (X) S.t. g (X) = b, X  R 2 Where f and g have continuous partial derivatives. Let X* be an optimizer of f on the feasible set and suppose that g’(X*)  0, j=1,2. Then there exists a scalar  * such that L’(  *, X*) = 0 Proof : Theorem (Sufficient or Second-Order Condition) Given the problem Max (Min) f (X) s.t. g(X) = b, X  R 2 Where f and g have continuous second-order partial derivatives. Let X* and  * satisfy L’(  *,X*) = 0 and suppose that g j ’ (X*)  0, j = 1, 2

48 (i) If |L”(  *,X*)| < 0, Then X* is a local minimizer (ii) If |L”(  *,X * )| > 0, Then X * is a local maximizer Proof : For notational convenience, we will suppress the arguments of each function; e.g., g 1 (X) will be written simply is g 1 ’. From the constraint, the total differential is g 1 ’dX 1 + g 1 ’ dX 2 = 0, from which we get g1’g1’g2’g2’g1’g1’g2’g2’ Let Y = f(X 1,X 2 ) ThendY = f 1 ’dX 1 + f 2 ’dX 2 Hence, = f 1 ’ + f 2 ’ = f 1 ’ + f 2 ’ d X 2 dX 1 = – d Y dX 1 d X 2 dX 1 Subtituting …. We know that if < 0 at X*, then X* is a local maximizer. Hence, if the determinant |L’’(  *,X*)| > 0, then X* is a local maximizer. d 2 Y d 2 Y dX 1 2

49 5.9.7 Example Min (X 1 – 1) 2 + (X 2 ) 2 s.t. 2X 1 + X 2 = 4 Lagrangean: L( ,X) = (X 1 – 1) 2 + (X 2 ) 2 +  (2X 1 + X 2 – 4) F.O.C. L  ’ ( ,X) = 2X 1 + X 2 – 4 = 0 L 1 ’( ,X)= 2(X 1 – 1) + 2  = 0 L 1 ’( ,X)= 2(X 1 – 1) + 2  = 0 L 2 ’( ,X) = 2X 2 +  = 0 L 2 ’( ,X) = 2X 2 +  = 0 X 1 * = 9/5, X 2 * = 2/5,  * = – 4/ S.O.C. |L”(u*,X*)| = = -10 < By theorem X* is local minimizer on feasible set. Hessian matrix of the objective function is f”(X) = f”(X) = Which is positive definite for every X  R 2  f is convex. The objective function is strictly convex. X* is the unique global minimizer

50 5.9.8 Example Maxf(X 1, X 2 ) = X 1 X 2 s.t. P 1 X 1 + P 2 X 2 = Y,P 1,P 2,Y > 0 Lagrangean: L(u,X) = X 1 X 2 +  (P 1 X 1 + P 2 X 2 – Y) F.O.C. L  ’( ,X) = P 1 X 1 + P 2 X 2 – Y = 0 L 1 ’( ,X) = X 2 +  P 1 = 0 L 1 ’( ,X) = X 2 +  P 1 = 0 L 2 ’( ,X) = X 1 +  P 2 = 0 L 2 ’( ,X) = X 1 +  P 2 = 0 X 1 * = Y/2P 1, X 2 * = Y/2P 2,  * = – Y/2P 1 P 2 X 1 * = Y/2P 1, X 2 * = Y/2P 2,  * = – Y/2P 1 P 2 0 P 1 P 2 0 P 1 P 2 S.O.C. |L”(u*,X*)| = P = 2P 1 P 2 > 0 P P By theorem X* is a local maximizer on the feasible set. X* the unique global maximizer on the feasible set. They can not opposite signs since their objective function value would be negative which can not be optimal since VOF (X*) = X 1 *X 2 * = > 0 X* is the unique global maximizer on the feasible set. y 2 4P 1 P 2

51 5.10 Optimization With One Equality Constraint : The n-Variable Case The n-Variable Case Given the problem Max(min) f(X) s.t. g(X) = b, X  R n Lagrangean : L( ,X) = f(X) +  [ (g(X) – b ] Definition 1) The gradient of L is Remark L’( ,X) = L  ’( ,X) L 1 ’( ,X)... L n ’( ,X) = g(X) – b g(X) – b f 1 ’(X) +  g’ 1 (X)... f n ’(X) +  g n ’(X) f n ’(X) +  g n ’(X)

52 2) Hessian matriz of L is L’’( ,X) = 0 g 1 ’(X) g 2 ’(X) … g n ’(X) 0 g 1 ’(X) g 2 ’(X) … g n ’(X) g 1 ’(X) L 11 ”( ,X) L 12 ”( ,X) … L 1n ”( ,X) g 2 ’(X) L 21 ”( ,X) L 22 ”( ,X) … L 2n ”( ,X).... g n ’ (X) L n1 ” ( ,X) L n2 ” ( ,X) … L nn ” ( ,X) Leading principal submatrices of order k of the Hessian matrix of L will be denoted by L k ”( ,X) L 1 ’’ ( ,X) = 0 L 2 ” ( ,X) = 0 g 1 ’(X) 0 g 1 ’(X) g 1 ’(X) L 11 ”( ,X) Notation

53 0 g 1 ’(X) g 2 ’(X) 0 g 1 ’(X) g 2 ’(X) g 1 ’(X) L 11 ”( ,X) L 12 ”( ,X) g 2 ’(X) L 21 ” ( ,X) L 22 ”( ,X)...etc L 3 ”( ,X) = Theorem (Necessary of First-Order Condition). given the problem Max(min) f(X) s.t. g(X) = b, X  R n s.t. g(X) = b, X  R n Dimana f dan g have continuous FOC partial derivatives. Let X* be an optimizer over the feasible set such that g’j(X*)  0, j= 1,2, …,n. Then there exists a scalar  * such that L’ (  *,X*) = 0

54 (Sufficient or Second-Order Condition). Given the problem Max (min) f(X) s.t. g(X) = b, X  R n s.t. g(X) = b, X  R n Where f and g have continuous second-order partial derivatives. Suppose that X* and  * satisfy FOC, i.e. L’(  *,X*) = 0. (i)If |L 3 ”(  *,X*)| < 0, |L 4 ”(  *,X*)| < 0, …, |L n ”(  *,X*)| < 0, then X* is a local minimizer of f on the feasible set Theorem (ii) If |L 3 ”(  *,X*)| > 0, |L 4 ”(  *,X*)| 0, |L 4 ”(  *,X*)| < 0, |L” 5 (  *,X*)| > 0, …, (-1) n |L n ”(  *,X*)| > 0, then X* is a local |L” 5 (  *,X*)| > 0, …, (-1) n |L n ”(  *,X*)| > 0, then X* is a local maximizer of f on the feasible set. maximizer of f on the feasible set. Proof : Gue and Thomas (1968)

55 Max (min) X X 1 X 2 + 2X X 3 2 Max (min) X X 1 X 2 + 2X X 3 2 s.t. X 1 – 3X 2 – 4X 3 = 16 s.t. X 1 – 3X 2 – 4X 3 = 16 Lagrangean : L( ,X) = X X 1 X 2 + 2X X  (X 1 – 3X 2 – 4X 3 – 16) First Order Condition : Example L’ ( ,X) = X 1 – 3X 2 – 4X 3 – 16 2X 1 + X 2 +  X 1 + 4X  X 1 + 4X  2X 3 – 4  0000 = This system of equations yields the solution X 1 * = 4, X 2 * = – 4, X 3 * = – 8,  * = – 4

56 Hessian Matrix of L : L”( ,X) = S.O.C. : |L 3 ”(  *,X*)| = = -28 < 0 |L 4 ”(  *,X*)| = = -168 < 0 Hence, X* is a local minimizer on the feasible set. The Hessian Matrix of f is : f”(X) =

57 whose eigenvalues are :  1 = 2,  2 = 3 + 2,  3 = 3 – 2 Which are all positive. Hence, f”(X) is positive definite for all X  R 3. That f is strictly convex on R 3. This implies X* is the unique global minimizer of f on the feasible set Optimization With Several Equality Constraints : The n-Variable Case Constraints : The n-Variable Case Definition Given The Problem Max (min) f(X) s.t. g k (X) = b k k = 1,2 …,m; m < n, X  R n s.t. g k (X) = b k k = 1,2 …,m; m < n, X  R n The function L defined by L( ,X) = f(X) +   k (g k (X) – b k ) is called the Lagrangean of the problem. The variables  1,  2, …,  n are called the lagrange multipliers. n k=1

58 Remark The gradient of L, with the arguments of the functions suppressed, is : L’ = g 1 – b 1 g 2 – b 2... g n – b m ffX1X1ffX1X1 +  k +  k gkgkX1X1gkgkX1X1 ffXnXnffXnXn gkgkXnXngkgkXnXn...

59 Note that the matrix L’’ has dimension (m+n) x (m+n). The Leading Principal Sub Matrix (LPSM) of order k will be denoted by L k ”. L’ = L 11 ” … L 1n ”.... L n1 ” … L nn ” 0 … … 0 The Hessian Matrix of L is : gmgmXnXngmgmXnXn gmgmX1X1gmgmX1X1 g1g1X1X1g1g1X1X1 g1g1XnXng1g1XnXn gmgmX1X1gmgmX1X1 gmgmXnXngmgmXnXn g1g1XiXig1g1XiXi g1g1XnXng1g1XnXn … … … …

60 Theorem (Necessary or F.O.C.). Given the problem Max (min) f(X) Max (min) f(X) s.t. g k (X) = b k, k = 1,2, …, m s.t. g k (X) = b k, k = 1,2, …, m m < n, X  R n m < n, X  R n Where f and g k (k = 1,2, …,m) have continuous first-Order Partial Derivatives. Let X* be an optimizer of f over the feasible set and suppose that the Jacobian determinant  g 1 (X*)  X 1  g 1 (X*)  X n  g m (X*)  X 1  g m (X*)  X n … …  0 0 0 0 Then there exist scalars  k * (k=1,2,…,m) such that L’(  *, X*) = 0 Proof : Panik (1976).

61 Theorem (Sufficient or S.O.C.). Given the problem MaX (min) f (X) MaX (min) f (X) s.t. g k (X) = b k, k = 1,2, …,m, m < n s.t. g k (X) = b k, k = 1,2, …,m, m < n X  R n X  R n Where f and g k (k = 1,2,…,m) have continuous Second-Order Partial Derivatives. Suppose X* and  satisfy the F.O.C., i.e. L’ (  *, X*) = 0 (i) If |L 2m+1 ”(  *,X*)|, |L 2m+2 ”(  *,X*)|,…, |L m+n ”(  *,X*)| have the same sign as (-1) m, then X* is a local minimizer of f have the same sign as (-1) m, then X* is a local minimizer of f on the feasible set. on the feasible set. (ii) If |L 2m+1 ”(  *,X*)|, |L 2m+2 ”(  *,X*)|,…, |L m+n ”(  *,X*)| Alternate in sign with the sign of |L 2m+1 (  *,X*)| being that of Alternate in sign with the sign of |L 2m+1 (  *,X*)| being that of (-1) m+1, then X* is a local maximizer of f on the feasible set. (-1) m+1, then X* is a local maximizer of f on the feasible set. Proof :


Download ppt "CHAPTER 5 NON-LINIER OPTIMIZATION. 5.1. Introduction Problem estimisasi secara umum MaX (Min) f (X) s.t. X  y s.t. X  y F disebut objektif function."

Presentasi serupa


Iklan oleh Google