Mengenal Deep Learning

Presentasi berjudul: "Mengenal Deep Learning"— Transcript presentasi:

1 Mengenal Deep Learning
Author: Muhammad Ghifary

2 Ikhtisar Machine Learning Supervised Learning
Linear Regression & Gradient Descent Logistic / Softmax Regression Pengantar Deep Learning Feed-Forward Neural Networks Useful ingredients pada Deep Learning: Data augmentation Dropout regularization Convolutional & pooling layer (Convolutional Networks) Perkembangan Terkini

3 1. Machine Learning Belajar dari data / pengalaman Aplikasi:
computer vision, language modelling, weather forecast, self driving car, computer graphic & games, recommender system, biomedical analysis

4 1. Machine Learning Supervised Learning Classification Kategori
Regression Unsupervised Learning Clustering, component analysis Semi-supervised Learning Reinforcement Learning Algoritma: Linear/Ridge Regression, Logistic Regression, k-Nearest Neighbour (kNN), Decision Tree, Random Forest, Gradient Boosting (AdaBoost), Probabilistic Graphical Models, Support Vector Machines (SVM), kernel methods, evolutionary computation (Genetic Programming, Genetic Algorithm, PSO, etc), Deep Learning Area terkait: artificial intelligence, pattern recognition, statistics, mathematical optimization, data mining, data science

5 2. Supervised Learning Belajar dari data yang berpasangan (x: input, y: output/label) Cari fungsi yang mengaproksimasi Bagaimana mengukur “kedekatan” antara dan ? y: penyakit jantung x: kadar kolesterol (mg/dl) 1 2 3 4

6 2. Supervised Learning Fungsi loss / cost:
Diberikan n buah labeled sampel selesaikan permasalahan optimisasi berikut: Prediksi / Test:

7 Penting untuk diingat ! 3 unsur utama dari algoritma supervised learning: Model fungsi / hypothesis Fungsi loss Optimisasi

8 3. Linear Regression & Gradient Descent

9 3. Linear Regression Model Problem regresi: 1 dimensi:
d dimensi (multivariat):

10 3. Linear Regression “Belajar” menemukan nilai dan yang sesuai dengan data Generalization : Dapat memprediksi “data yang tak terlihat” Overfitting Hanya akurat memprediksi “data latih”, tapi tidak untuk “data yang tak terlihat”

11 3. Linear Regression: learning
Fungsi loss Squared loss: Problem optimisasi Diberikan n buah “data latih”

12 3. Linear Regression: Gradient Descent
Mencari optimal : gradient descent Gradient / turunan pertama dari loss function Tulis Goal:

13 3. Linear Regression: Gradient Descent
1. Initialize with random values 2. while not until convergence do end

14 3. Linear Regression: Gradient Descent
Ingat kembali, linear regression memiliki Hitung dan

15 3. Linear Regression: Gradient Descent
Batched (offline) Gradient Descent 1. Initialize and with random values 2. while not until convergence do end

16 3. Linear Regression: Gradient Descent
Stochastic (online) Gradient Descent 1. Initialize and with random values 2. while not until convergence do for each endfor endwhile

17 3. Linear Regression: Gradient Descent
Mini-batched Gradient Descent 1. Initialize and with random values 2. while not until convergence do for each endfor endwhile

18 3. Linear Regression: Remark
Linear regression (LR) merupakan algoritma machine learning yang paling sederhana LR memiliki solusi analitis / closed-form, yang memberikan solusi optimal hanya dengan 1 langkah komputasi: tidak perlu gradient descent ! Latihan: hitunglah solusi analitis dari LR

19 4. Logistic / Softmax Regression

20 4. Logistic Regression Problem klasifikasi
e.g. handwritten digit recognition Binary: Multi-class: (0 everywhere, 1 at a specific position)

21 4. Logistic Regression: Binary Classification
Model untuk binary classification [COX1958] Fungsi sigmoid

22 4. Logistic Regression: Binary Classification
Fungsi loss Binary cross-entropy / log-loss Tidak komutatif ! Problem optimisasi Diselesaikan dengan algoritma yang sama dengan Linear Regression: Gradient Descent !

23 4. Logistic Regression: Multiclass Classification
Model untuk multi-class classification Fungsi softmax

24 4. Logistic Regression: Multiclass Classification
Fungsi loss Categorical cross-entropy Problem optimisasi

25 4. Logistic Regression Demo: handwritten digit recognition
Python >= 2.7.x Theano: Keras: IPython Notebook (optional, for a nice GUI) Dataset: USPS Source code:

26

27 5. Pengantar Deep Learning

28 5. Deep Learning: Motivasi
Data representations / features / abstractions sangat penting di machine learning ! “Representasi data bertingkat” telah menjadi solusi standard di berbagai aplikasi (e.g., computer vision) Butuh human expert untuk manual engineering Tujuan deep learning: mengganti manual engineering dengan learning (Handcrafted) Feature Extractor Mid-Level Features Supervised Learning / Classifier

29 5. Deep Learning: Definisi
Deep learning (in the modern time): Himpunan algoritma yang fokus pada pembelajaran representasi data (nonlinear) bertingkat. Contoh: feed-forward neural nets, convolutional nets, recurrent nets, deep belief nets, deep Boltzmann machines, hierarchical probabilistic models, deep PCA, deep CCA, ladder networks, dll. Sangat sukses untuk (large-scale) problem klasifikasi dan sequence modelling: Computer vision: object recognition / classification, object detection, scene labelling, semantic segmentation (see Speech and audio recognition recognition, natural language processing

30 5. Deep Learning: Mengapa “deep”?
Representasi “deep” mampu mengaproksimasi sebuah fungsi lebih efisien dibandingkan “shallow” [Bengio 2009] Terinspirasi dari biological visual system Source:

31 5. Deep Learning: Sejarah
Walaupun istilah “deep” baru populer sekitar tahun 2006, deep learning memiliki sejarah yang panjang [Schmidhuber 2015] [Goodfellow et al. 2015] membagi perkembangan deep learning menjadi 3 gelombang: 1940s – 1960s (cybernetics) Fokus pada memahami biological learning [McCulloch & Pitts 1943, Hebb 1949, Rosenblatt 1958] (connectionist) Fokus pada engineering problem Backpropagation [Rumelhart et al. 1986], Convolutional nets [LeCun et al. 1989] Setelah 1995 kalah populer dengan algoritma machine learning lainnya, e.g., Support Vector Machines, Gradient Boosting, Graphical Models 2006 – sekarang (deep learning) Breakthrough di berbagai real-world application, computing resource advancement (GPU), big data Unsupervised pretraining [Hinton et al. 2006, Bengio et al. 2007, Le et al. 2012] Return of ConvNets [Ciresan et al. 2011, Krizhevsky et al. 2012, Girshick et al. 2013]

32 6. Feed-forward Neural Nets

33 6. Feed-forward Neural Nets
Generalisasi dari Logistic Regression intermediate/hidden representation bertingkat-tingkat di antara input dan output

34 6. Feed-forward Neural Nets
Opsi fungsi aktivasi untuk hidden neuron: Linear Sigmoid / logistic Hyperbolic tangent Rectified Linear Unit Input Hidden

35 6. Feed-forward Neural Nets
Parameter (weights: dan ) i j

36 6. Feed-forward Neural Nets
Algoritma training: backpropagation 2 fase: 1) forward, 2) backward Gradient descent + chain rule ! Source:

37 7. Useful Ingredients pada Deep Learning

38 7. Useful Ingredients Unsupervised Pretraining Autoencoder
Restricted Boltzmann Machines

39 7. Useful Ingredients Data Augmentation
Membuat data sintesis on-the-fly Untuk data berupa images / video: Rotation, scaling, flipping, cropping, noise, etc

40 7. Useful Ingredients Dropout Regularization [Hinton et al. 2012]
Menghindari overfitting Non-aktifkan p% (biasanya p=50) neuron pada layer tertentu secara acak, untuk setiap epoch pada fase training Aktifkan kembali pada fase test/prediksi, namun nilai aktivasinya dikalikan dengan p%

41 7. Useful Ingredients Convolution and Pooling Layer
Khusus untuk data spatial (image / video) Convolution layer mengekspos spatial correlation dan locality Pooling layer meningkatkan level of invariance

42 8. Perkembangan Terkini Handwritten Text Generation [Graves 2014]

43 8. Perkembangan Terkini Story-like Visual Explanation
Aligning books & movies [Zhu et al. 2015]

44 8. Perkembangan Terkini Paint Art [Gatys et al. 2015]

45 8. Perkembangan Terkini Deep Reinforcement Learning
[Mnih et al. 2013] Playing Atari with Deep Reinforcement Learning [Heess et al. 2014] Learning Continuous Control Policies by Stochastic Value Gradients [Mnih et al. 2015] Human-level control through deep reinforcement learning [Balduzzi & Ghifary 2015] Compatible Value Gradients for Reinforcement Learning of Continuous Deep Policies [Silver et al. 2016] Mastering the game of Go with deep neural networks and tree search

46 Kesimpulan Machine learning memungkinkan program komputer belajar dari data / experience Algoritma supervised learning belajar dari data yang berlabel Linear regression & logistic regression merupakan model supervised learning, yang dapat dioptimisasi dengan gradient descent Deep learning merupakan himpunan algoritma yang fokus pada pembelajaran representasi data berhirarkis Istilah baru, tapi memiliki sejarah yang cukup panjang Breakthrough diberbagai aplikasi Cocok untuk large-scale problem Feed-forward neural nets merupakan salah satu algoritma deep learning yang optimisasi dengan back-propagation (gradient descent + chain rule)

47 TERIMA KASIH

48 Kontak Diskusi Bagi yang ingin sharing dan diskusi lebih lanjut, silakan hubungi: Bagi yang ingin bergabung dengan mailing list kami silakan hubungi admin kami di: Dengan Subject ”[Permohonan Subscribe Milis]“

49 Seeing is believing. Listening is understanding. Sharing is inspiring.
© 2015 Seeing is believing. Listening is understanding. Sharing is inspiring.