Clustering.

Presentasi berjudul: "Clustering."— Transcript presentasi:

1 Clustering

2 What is Cluster Analysis?
Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping a set of data objects into clusters Clustering is unsupervised classification: no predefined classes Typical applications As a stand-alone tool to get insight into data distribution As a preprocessing step for other algorithms

3 Contoh Aplikasi Clustering
Marketing: Membantu pemasaran dalam menemukan berbagai segmen pelanggan dan kemudian menggunakan pengetahuai/informasi ini untuk mengembangkan program targeted marketing Land use: Mengidentifikasi area yang serupa dalam penggunakan lahan Insurance: City-planning

4 Bagaimana metode clustering yang baik
Metode clustering yang baik akan menghasilkan kualitas pengelompokan data dengan high intra-class similarity low inter-class similarity Kualitas dari hasil clustering bergantung pada metode pengukuran similaritasnya.

5 Data Structures Data matrix Dissimilarity matrix (two modes)
(one mode)

6 Measure the Quality of Clustering
Ukuran Dissimilarity/Similarity : Similarity dinyatakan dengan fungsi jarak :d(i, j) Terdapat “quality” function yang memastikan bahwa suatu pengelompokan itu baik. Fungsi jarak bermacam macam interval-scaled, boolean, categorical, ordinal dan ratio variables.

7 Major Clustering Approaches
Partitioning algorithms: Construct various partitions and then evaluate them by some criterion Hierarchy algorithms: Create a hierarchical decomposition of the set of data (or objects) using some criterion Density-based: based on connectivity and density functions Grid-based: based on a multiple-level granularity structure Model-based: A model is hypothesized for each of the clusters and the idea is to find the best fit of that model to each other

8 Clustering Approaches
Hierarchical Partitional

9

10 Distance Between Clusters
Single Link: smallest distance between points Complete Link: largest distance between points Group Average Link: average distance between points Centroid: distance between centroids Median method:

11 Single Link: smallest distance between points

12 Matrik disimilaritas

13 Complete Link: largest distance between points

14 Weight Group Average Link: average distance between points

15 Median

16 Centroid: distance between centroids

17 Partitioning Algorithms: Basic Concept
Partitioning method: Mempartisi data D dengan n objek kedalam k kelompok Menentukan k, mencari partisi dengan k clusters yang mengoptimal kriteria partisi Heuristic methods: k-means and k-medoids algorithms k-means (MacQueen’67): Setiap kelompok dinyatakan dengan pusat kelompok (cluster) k-medoids or PAM (Partition around medoids) (Kaufman & Rousseeuw’87): Setiap kelompok data dinyatakan dengan satu objek data dalam kelompok

18 Metode K-Means Clustering
Tentukan k, the k-means algorithm is implemented in four steps: Partition objects into k nonempty subsets Compute seed points as the centroids of the clusters of the current partition (the centroid is the center, i.e., mean point, of the cluster) Assign each object to the cluster with the nearest seed point Go back to Step 2, stop when no more new assignment

19 The K-Means Clustering Method
Example 1 2 3 4 5 6 7 8 9 10 10 9 8 7 6 5 Update the cluster means 4 Assign each objects to most similar center 3 2 1 1 2 3 4 5 6 7 8 9 10 reassign reassign K=2 Arbitrarily choose K object as initial cluster center Update the cluster means

20 Kelebihan dan Kekurangan
Kelebihan: relatif efficient: O(tkn), dimana n adalah # objects, k adalah # clusters, dan t adalah # iterations. Biasanya k, t << n. Dibandingkan : PAM: O(k(n-k)2 ), CLARA: O(ks2 + k(n-k)) Kelemahan Dapat dilkukan ketika bisa mendefinisikan mean, kemudian bagaimana dengan data categorical ? Perlu menentukan k, banyaknya cluster Tidak mampu mengatasi noise dan data outlier

21 Variations of the K-Means Method
Ragam k-means ditentukan oleh Cara memilih initial k means (rata-rata) Cara Menghitung Dissimilarity Strategi untuk menghitung rata-rata dari cluster Untuk categorical data: k-modes (Huang’98) Mengganti rata-rata dari kluster dengan modes/modus Menggunakan ukuran dissimilarity yang lain terkait dengan data categorical Menggunakan frequency-based method untuk memperbaharui modes/modus dari clusters Gabungan data categorical dan data numerik : k-prototype method

22 What is the problem of k-Means Method?
The k-means algorithm is sensitive to outliers ! Since an object with an extremely large value may substantially distort the distribution of the data. K-Medoids: Instead of taking the mean value of the object in a cluster as a reference point, medoids can be used, which is the most centrally located object in a cluster. 1 2 3 4 5 6 7 8 9 10

23 Typical k-medoids algorithm (PAM)
Total Cost = 20 10 9 8 7 Arbitrary choose k object as initial medoids Assign each remaining object to nearest medoids 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 K=2 Randomly select a nonmedoid object,Oramdom Total Cost = 26 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 Do loop Until no change Compute total cost of swapping Swapping O and Oramdom If quality is improved.

24 PAM (Partitioning Around Medoids) (1987)
PAM (Kaufman and Rousseeuw, 1987), built in Splus Use real object to represent the cluster Select k representative objects arbitrarily For each pair of non-selected object h and selected object i, calculate the total swapping cost TCih For each pair of i and h, If TCih < 0, i is replaced by h Then assign each non-selected object to the most similar representative object repeat steps 2-3 until there is no change

25 PAM Clustering: Total swapping cost TCih=jCjih

26 K-Means Example Given: {2,4,10,12,3,20,30,11,25}, k=2
Randomly assign means: m1=3,m2=4 Solve for the rest …. Similarly try for k-medoids

27 Distance Between Clusters
Single Link: smallest distance between points Complete Link: largest distance between points Group Average Link: average distance between points Centroid: distance between centroids Median method:

28 Evaluasi Cluster The root-mean-square standard deviation (RMSSTD) index is used to determine the number of clusters existing in a data set.