Presentasi sedang didownload. Silahkan tunggu

Presentasi sedang didownload. Silahkan tunggu

Seminar Data Mining Business Trouble and Industrial Applications Business Trouble and Industrial Applications Lab Data Mining, Teknik Industri Universitas.

Presentasi serupa


Presentasi berjudul: "Seminar Data Mining Business Trouble and Industrial Applications Business Trouble and Industrial Applications Lab Data Mining, Teknik Industri Universitas."— Transcript presentasi:

1

2 Seminar Data Mining Business Trouble and Industrial Applications Business Trouble and Industrial Applications Lab Data Mining, Teknik Industri Universitas Islam Indonesia 10 Mei, 2008 Budi Santosa

3 11/18/2015Budi Santosa Isi  Pendahuluan  Data  Association rules  Klasifikasi  Clustering  Aplikasi data mining  Commercial tools  Kesimpulan

4 11/18/2015Budi Santosa Pendahuluan  Apa data mining?  Mengapa kita perlu untuk ‘mine’ data?  Jenis data seperti apa yang bisa kita ‘mine’?

5 11/18/2015Budi Santosa Pengertian data mining  Data mining adalah gabungan metode-metode analisis data secara statistik dan algoritma-algoritma untuk memproses data berukuran besar. Data mining merupakan proses menemukan informasi atau pola yang penting dalam basis data berukuran besar.  Bagian dari proses Knowledge Discovery in Data (KDD).  Explorasi dan analisis large quantities of data  Dengan tools secara automatic or semi-automatic  Menemukan meaningful patterns dan rules. Patterns ini memungkinkan suatu company untuk better understand its customers improve its marketing, sales, and customer support operations

6 11/18/2015Budi Santosa

7 11/18/2015Budi Santosa Mengapa data mining? Pertumbuhan yang explosive dalam data collection Penyimpanan data dalam data warehouses Ketersediaan akses data yang semakin meningkat dari Web dan intranet  Kita perlu menemukan cara yang lebih efektif untuk menggunakan data ini dalam proses decision support dari sekedar menggunakan traditional querry languages

8 11/18/2015Budi Santosa Jenis data apa?  Data warehouses  Transactional databases  Advanced database systems Spacial and Temporal Time-series Multimedia, text WWW … Structure - 3D Anatomy Function – 1D Signal Metadata – Annotation

9 11/18/2015Budi Santosa Working with data  Kebanyakan algoritma data mining cocok hanya untuk data numerik  Semua data seharusnya direpresentasikan sebagai bilangan/data numerik sehingga algoritma bisa diterapkan  Data sales, crime rates, text, atau images, kita harus menemukan cara yang tepat untuk mentransform data menjadi bilangan/number.

10 11/18/2015Budi Santosa Knowledge Discovery dan Data Mining  Non-trivial extraction of implicit, unknown, and potentially useful information from databases. oProses Knowledge discovery terdiri dari fase:

11 11/18/2015Budi Santosa Tugas (task) dari Data Mining  Prediksi: Bagaimana perilaku atribut tertentu dalam data dimasa datang? (predictive)  Time series  Pattern Sequence  Independent-dependent relation  Klasifikasi: mengelompokkan data ke dalam kategori berdasarkan sampel yang ada (label diskrit)  Feature selection  Clustering:mengklasterkan obyek tanpa ada sampel sebagai contoh (descriptive)  Association: object association

12 11/18/2015Budi Santosa Association Rules  Tujuan Memberikan aturan yang berkaitan dengan kehadiran set item dengan set item yang lain Contoh:

13 11/18/2015Budi Santosa Association Rules  Market-basket model Mencari kombinasi beberapa produk Letakkan SHOES dekat dengan SOCK sehingga jika seorang customer membeli satu dia akan membeli yang lain  Transaksi: seseorang membeli beberapa items dalam itemset di supermarket

14 11/18/2015Budi Santosa Klasifikasi married salaryAcct balance age Yes <20k Poor risk >=20k <50k Fair risk >=50 Good risk no <5k Poor risk >=25 <25 >5k Fair risk Good risk

15 11/18/2015Budi Santosa RIDMarriedSalaryAcct balanceAgeLoanworthy 1No>=50<5k>=25Yes 2 >=50>=5k>=25Yes 3 20k..50k<5k<25No 4 <20k>=5k<25No 5 <20k<5k>=25No 6Yes20k..50k>=5k>=25Yes Class attribute Expected information Salary I(3,3)=1 Information gain Gain(A) = I-E(A) E(Married)=0.92 Gain(Married)=0.08 E(Salary)=0.33 Gain(Salary)=0.67 E(A.balance)=0.82 Gain(A.balance)=0.18 E(Age)=0.81 Gain(Age)=0.19 age Class is “no” {4,5} >=50k 20k..50k <20k Class is “no” {3}Class is “yes” {6} Class is “yes” {1,2} Entropy <25>=25

16 11/18/2015Budi Santosa Klasifikasi categorical continuous class Training Set Model Learn Classifier Test Set

17 11/18/2015Budi Santosa Text Classification class Training Set Model Learn Classifier text Test Set

18 11/18/2015Budi Santosa Klastering Klastering adalah proses mengelompokkan obyek-obyek yang mirip ke dalam satu klaster. Obyek bisa berasal dari data base customer, produk, gen, mahasiswa, dsb.

19 11/18/2015Budi Santosa Klastering  Berapa Konsep Salah satu hal yang sangat penting adalah penggunaan ukuran kemiripan (similarity) Jika datanya numerik, fungsi kemiripan ( similarity function) berdasarkan jarak sering digunakan Euclidean metric (Euclidean distance), Minkowsky metric, Manhattan metric. Korelasi, cosinus, kovariance Hiraki, Kmeans, Fuzzy, SOM, Support Vector Clustering

20 11/18/2015Budi Santosa Klaster

21 11/18/2015Budi Santosa Aplikasi data mining  Cuaca  Bisnis  Mikrobiologi  Market analysis  Manufacturing and production  Fraud detection dan detection of unusual patterns (outliers) Telecommunication Financial transactions

22 11/18/2015Budi Santosa Aplikasi data mining  Text mining (news group, , documents) and Web mining  DNA and bio-data analysis Diseases outcome Effectiveness of treatments Identify new drugs

23 180 km Elevation Chandler 54 km Cuaca

24 54 km North Azimuth angle Chandler WSR-88D records digital database containing 3 variables: velocity (V), reflectivity (Z), and spectrum width (W).

25 11/18/2015  The current Mesocyclone Detection Algorithm (MDA) was created at the National Severe Storms Laboratory (NSSL), Oklahoma, to work with native variables derived from the WSR- 88D  In order to detect circulations associated with vortices that spin up into tornadoes, the velocity data are exploited  The data are measured for circulation depth, height above the ground, strength of the circulation, shear (change in wind speed or direction with distance), etc.  By relaxing previous threshold values, the MDA is capable of detecting weaker circulations that may eventually spin up into mesocyclones (thereby enhancing the probability of detection)

26 11/18/ base (m) [ ] 2. depth (m) [ ] 3. strength rank [0-25] 4. low-level diameter (m) [ ] 5. maximum diameter (m) [ ] 6. height of maximum diameter (m) [ ] 7. low-level rotational velocity (m/s) [0-65] 8. maximum rotational velocity (m/s) [0-65] 9. height of maximum rotational velocity (m) [ ] 10. low-level shear (m/s/km) [0-175] 11. maximum shear (m/s/km) [0-175] 12. height of maximum shear (m) [ ] 13. low-level gate-to-gate velocity difference (m/s) [0-130] 14. maximum gate-to-gate velocity difference (m/s) [0-130] 15. height of maximum gate-to-gate velocity difference (m) [ ] 16. core base (m) [ ] 17. core depth (m) [0-9000] 18. age (min) [0-200] 19. strength index (MSI) wghtd by avg density of integrated layer [ ] 20. strength index (MSIr) "rank" [0-25] 21. relative depth (%) [0-100] 22. low-level convergence (m/s) [0-70] 23. mid-level convergence (m/s) [0-70]

27 11/18/2015Budi Santosa Medis  Bisa kah saya menggunakan contact lenses?  Possible output: none, soft, hard.  Decision berdasar pada:  - age  - spectacle prescription  - astigmatism  - tear production rate

28 11/18/2015Budi Santosa contoh umurresepastigmatism tear p.r. lenses mudamiopetidakkurang Tdk perlu mudamiopetidaknormalsoft mudahypermetropeyakurang pre- presbyopic miopetidakkurang Tdk perlu presbyopicmiopetidaknormalhard

29 11/18/2015Budi Santosa 28 Prosedur pengklasifikasian  A set of “if-then” rules  A decision tree  A Neural Network  SVM, LSVM, LS-SVM  LDA  KNN  Minimax Prob Machine  Analytic Center Machine  Relevance Vector Machine

30 11/18/2015Budi Santosa Prosedur if -then  If umur = muda and astigmatic = tidak dan tear production rate = normal then rekomendasi = soft  If age = pre-presbyopic and astigmatic = no and tear production rate = normal then rekomendasi = soft  If age = presbyopic and spectacle prescription = myope and  astigmatic = no then rekomendasi = none  If spectacle prescription = hypermetrope and astigmatic = no and tear production rate = normal then rekomendasi = soft  If spectacle prescription = myope and astigmatic = yes and tear production rate = normal then rekomendasi = hard  If age = young and astigmatic = yes and tear production rate = Normal then rekomendasi = hard  If age = pre-presbyopic and spectacle prescription = hypermetrope and astigmatic = yes then rekomendasi = none  If age = presbyopic and spectacle prescription = hypermetrope and astigmatic = yes then rekomendasi = none

31 11/18/2015Budi Santosa Decision tree

32 2004/09/0931  Regression is similar to classification  First, construct a model  Second, use model to predict unknown value  Methods  Linear and multiple regression  Non-linear regression, Neural network, SVR  Regression is different from classification  Classification refers to predict categorical class label  Regression models continuous-valued functions

33 11/18/2015Budi Santosa  Contoh: pemakai Credit card bisa diklasterkan menurut  Berapa sering menggunakan kartu: frequent/seldom usage domestic/foreign transactions high/low amounts of money transactions of specific type …  Untuk setiap klaster, sistem fraud detection bisa dikembangkan. Atau sejumlah produk yang lain yang bisa ditawarkan

34  Attribute 1: (qualitative) Status of existing checking account A11 :... = 200 DM /salary assignments for at least 1 year A14 : no checking account Attribute 2: (numerical) Duration in month Attribute 3: (qualitative) Credit history A30 : no credits taken/all credits paid back duly A31 : all credits at this bank paid back duly A32 : existing credits paid back duly till now A33 : delay in paying off in the past A34 : critical account/other credits existing (not at this bank) 11/18/2015Budi Santosa

35  Attribute 4: (qualitative) Purpose A40 : car (new) A41 : car (used) A42 : furniture/equipment A43 : radio/television A44 : domestic appliances A45 : repairs A46 : education A47 : (vacation - does not exist?) A48 : retraining A49 : business A410 : others 11/18/2015Budi Santosa

36 Checking account durasiCredit hist purposeamount…Good or bad 11/18/2015Budi Santosa Attribute 15: (qualitative) Housing A151 : rent A152 : own A153 : for free Attribute 16: (numerical) Number of existing credits at this bank Attribute 17: (qualitative) Job A171 : unemployed/ unskilled - non-resident A172 : unskilled - resident A173 : skilled employee / official A174 : management/ self-employed/ highly qualified employee/ officer

37 11/18/2015Budi Santosa 36 Cross Selling Cross selling salah satu aplikasi data mining penting yang lain Apa yang merupakan best additional or best next offer (BNO) untuk setiap customer? Misal, sebuah bank ingin bisa menjual automobile insurance ketika seorang customer mendapatkan car loan Bank tersebut mungkin memutuskan untuk mendapatkan a full-service insurance agency

38 11/18/2015Budi Santosa 37 Paying Claims A major manufacturer of diesel engines must also service engines under warranty Warranty claims come in from all around the world Data mining is used to determine rules for routing claims some are automatically approved others require further research Result: The manufacturer saves millions of dollars Data mining also enables insurance companies and the Fed. Government to save millions of dollars by not paying fraudulent medical insurance claims

39 11/18/2015Budi Santosa 38 Finding Prospects A cellular phone company wanted to introduce a new service They wanted to know which customers were the most likely prospects Data mining identified “sphere of influence” as a key indicator of likely prospects Sphere of influence is the number of different telephone numbers that someone calls

40 11/18/2015Budi Santosa Clustering is an undirected data mining technique that finds groups of similar items Based on previous purchase patterns, customers are placed into groups Customers in each group areassumed to have an affinity for the same types of products New product recommendations can be generated automatically based on new purchases made by the group This is sometimes called collaborative filtering Antisipasi Customer Needs 39

41 11/18/2015Budi Santosa Microbiology

42 11/18/2015Budi Santosa 41 Microarray Problem Data Mining Microarray Experiment Image Analysis Biology Application Domain Experiment Design and Hypothesis Data Analysis Knowledge discovery in databases (KDD ) Data Warehouse validasi

43  Enterprise Resources Planning (ERP) systems generate large volumes of data.  Examples of data sources in manufacturing include:  Schedules.  Production capacity, efficiency, failures, etc.  Manufacturing parameters.  Process quality.  Process plans. 11/18/2015Budi Santosa

44 11/18/2015Budi Santosa

45 11/18/2015Budi Santosa

46 11/18/2015Budi Santosa The learning stage focuses on discovering knowledge from manufacturing processes: Step 1: Similar parts and processes are grouped into clusters. Step 2: Relevant processes are associated with each cluster. The exploitation stage takes advantage of the clusters to improve the efficiency of generation of process plans for new parts: Step 3: A new part to be manufactured is matched with a suitable cluster. Step 4: The new part is assigned the relevant process plan. The specialization stage adapts the relevant process for the new part: Step 5: The relevant process is adapted to the new part. Step 6: The new process plan data is incorporated into the database.

47 11/18/2015Budi Santosa

48 11/18/2015Budi Santosa Data Mining to select supplier Input feature set of a performance measure for suppliers FeatureContentFeatureContent FlQuality of material (0, 1, 2, 3)F10Warranty (0/1) F2Track record (0, 1, 2, 3)F11Warehousing (0, 1, 2) F3Technical ability (0, 1, 2)F12Reliability (%) F4 Tools and equipment (0, 1, 2, 3)F13Efficiency (%) F5 Safety practices (0, 1, 2,3)F14Dependability (0, 1, 2) F6 Deliveries/shipments (0, 1, 2, 3)F15Frequency of rejects (time/year) F7Conformance to standards (0, 1, 2)F16 Failure rate (%) F8Applicability of product (0, 1, 2)F17Offered price (0, 1, 2, 3) F9Product development (0, 1)F18Responsiveness to bidding (0, 1, 2)

49  Perencanaan dimulai dari forecasting demand  Dari demand forecasting didapatkan petunjuk:  Apa saja bahan yang dibutuhkan? Berapa kebutuhan per jenis bahan?  Alokasi tenaga kerja Apa saja variabel yang diperlukan? harga, nilai promosi, promosi pesaing, usia customer, permintaan masa lalu Hybrid time series forecasting dan causal relation 11/18/2015Budi Santosa

50 49  Given a set of sequences, find the complete set of frequent subsequences  Applications of sequential pattern  Customer shopping sequences:  First buy computer, then CD-ROM, and then digital camera, within 3 months.  Weblog click streams  Telephone calling patterns SIDsequence Given support threshold min_sup =2, is a sequential pattern

51 11/18/2015Budi Santosa 50 Contoh lain  Direct mailing: siapa yang harus ditawari produk tertentu?  Remote sensing: menentukan water pollution dari spectral images  Forecast beban: prediksi permintaan untuk electric power  Intelligent ATM’s : how much cash will be there tomorrow?  City-planning: Identifying groups of houses according to their house type, value, and geographical location

52 51 Beberapa tahun lalu, UPS mempunyai masalah dengan pekerjanya/pemogokan FedEx mendapati volumenya meningkat Setelah pemogokan, volume FedEx jatuh FedEx mengidentifikasi kustomer yang dulu pindah dan pindah lagi ke jasa lain Kustomer ini menggunakan UPS lagi FedEx memberikan special offers pada Kustomer ini agar mau menggunakan FedEx

53 11/18/2015Budi Santosa Jawab: and Can you find co-location patterns from the following sample dataset?

54 11/18/2015Budi Santosa

55 54 Improves profit by limiting campaign to most likely responders Reduces costs by excluding individuals least likely to respond Using RFM : recency, frequency, monetary

56 55 Predicts response rates to help staff call centers, with inventory control, etc. Identifies most important channel for each customer Discovers patterns in customer data

57 56 A model takes a number of inputs, which often come from databases, and it produces one or more outputs Sometimes, the purpose is to build the best model The best model yields the most accurate output Such a model may be viewed as a black box Sometimes, the purpose is to better understand what is happening This model is more like a gray box

58 57 When the model predicts No, it is right 100/150 = 67% of the time There are 1000 records in the model set When the model predicts Yes, it is right 800/850 = 94% of the time Yes No Yes No Predicted Actual

59 58 The model is correct 800 times in predicting Yes The model is correct 100 times in predicting No The model is wrong 100 times in total The overall prediction accuracy is 900/1000 = 90%

60  MSE  SSE  MAPE  MAD  R 2 11/18/2015Budi Santosa

61 60 Data mining is a tool to achieve goals The goal is better service to customers Only people know what to predict Only people can make sense of rules Only people can make sense of visualizations Only people know what is reasonable, legal, tasteful Human decision makers are critical to the data mining process

62 61 Analyze available data (from the past) Discover patterns, facts, and associations Apply this knowledge to future actions

63 62 Does past data contain the important business drivers? e.g., demographic data Is the business environment from the past relevant to the future? in the e-commerce era, what we know about the past may not be relevant to tomorrow users of the web have changed since late 1990s Are the data mining models created from past data relevant to the future? have critical assumptions changed?

64 63 Form a learning relationship with your customers Notice their needs On-line Transaction Processing Systems Remember their preferences Decision Support Data Warehouse Learn how to serve them better Data Mining Act to make customers more profitable

65 64 Several years ago, Land’s End could not recognize regular Christmas shoppers some people generally don’t shop from catalogs but spend hundreds of dollars every Christmas if you only store 6 months of history, you will miss them Victoria’s Secret builds customer loyalty with a no-hassle returns policy some “loyal customers” return several expensive outfits each month they are really “loyal renters”

66 65 Channels are the way a company interfaces with its customers Examples Direct mail Banner ads Telemarketing Customer service centers Messages on receipts Key data about customers come from channels

67 66 Channels are the source of data Channels are the interface to customers Channels enable a company to get a particular message to a particular customer Channel management is a challenge in organizations CRM is about serving customers through all channels

68 67 The FBI handles numerous, complex cases such as the Unabomber case Leads come in from all over the country The FBI and other law enforcement agencies sift through thousands of reports from field agents looking for some connection Data mining plays a key role in FBI forensics

69 11/18/2015Budi Santosa Contoh penelitian/paper  An application of data mining for marketing in telecommunication  Application of data mining to customer profile analysis in the power electricity  Conditional Market Segmentation by Neural Networks  cluster analysis in Industrial market  marketing segmentation using support vector  Using data mining for manufacturing process selection  Data mining application in credit card business  …..

70 69 More often, a customer is an account Retail banking checking account, mortgage, auto loan, … Telecommunications long distance, local, ISP, mobile, … Insurance auto policy, homeowners, life insurance, … Utilities The account-level view of a customer also misses the boat since each customer can have multiple accounts

71 70 Childhood birth, school, graduation, … Young Adulthood choose career, move away from parents, … Family Life marriage, buy house, children, divorce, … Retirement sell home, travel, hobbies, … Much marketing effort is directed at each stage of life

72 71 It is difficult to identify the appropriate events graduation, retirement may be easy marriage, parenthood are not so easy many events are “one-time” Companies miss or lose track of valuable information a man moves a woman gets married, changes her last name, and merges her accounts with spouse It is hard to track your customers so closely, but, to the extent that you can, many marketing opportunities arise

73 72 Customers begin as prospects Prospects indicate interest fill out credit card applications apply for insurance visit your website They become new customers After repeated purchases or usage, they become established customers Eventually, they become former customers either voluntarily or involuntarily

74 73 Business Processes Organize Around the Customer Lifecycle AcquisitionActivationRelationship Management Winback Former Customer Prospect Established Customer New Customer Low Value High Potential High Value Voluntary Churn Forced Churn

75 74 Prospects receive marketing messages When they respond, they become new customers They make initial purchases They become established customers and are targeted by cross-sell and up-sell campaigns Some customers are forced to leave (cancel) Some leave (cancel) voluntarily Others simply stop using the product (e.g., credit card) Winback/collection campaigns

76 75 The purpose of data warehousing is to keep this data around for decision-support purposes Charles Schwab wants to handle all of their customers’ investment dollars Schwab observed that customers started with small investments

77 76 By reviewing the history of many customers, Schwab discovered that customers who transferred large amounts into their Schwab accounts did so soon after joining After a few months, the marketing cost could not be justified Schwab’s marketing strategy changed as a result

78 77 Prospect acquisition Prospect product propensity Best next offer Forced churn Voluntary churn Bottom line: We use data mining to predict certain events during the customer lifecycle

79 78 Prediction uses data from the past to make predictions about future events (“likelihoods” and “probabilities”) Profiling characterizes past events and assumes that the future is similar to the past (“similarities”) Description and visualization find patterns in past data and assume that the future is similar to the past

80 79 We use the noun churn as a synonym for attrition We use the verb churn as a synonym for leave Why study attrition? it is a well-defined problem it has a clear business value we know our customers and which ones are valuable we can rely on internal data the problem is well-suited to predictive modeling

81 80 Focus on keeping high-value customers Focus on keeping high-potential customers Allow low-potential customers to leave, especially if they are costing money Don’t intervene in every case Topic should be called “managing customer attrition”

82  Weka, (Waikato Environment for Knowledge Analysis) is a Java-based data mining tool developed by Waikato University.  RapidMiner, 11/18/2015Budi Santosa

83 11/18/2015Budi Santosa Commercial tools  Oracle Data Miner miner.html  Data To Knowledge  SAS  Clementine  Intelligent Miner

84 11/18/2015Budi Santosa Kesimpulan  Data mining is a “decision support” process in which we search for patterns of information in data.  This technique can be used on many types of data.

85 11/18/2015Budi Santosa References  Budi Santosa, Data Mining Teknik pemanfaatan data untuk keperluan bisnis  A. Kusiak, International Journal of Production Research,Vol. 44,Data mining: manufacturing and service applications,  Bruno Agard, Data mining for selection of Manufacturing processes, Data mining and knowledge discovery handbook  Michael Berry and Gordon Linoff, Customer Relationship Management Through Data Mining, SAS Institute, 2000  Michael Berry and Gordon Linoff, Mastering Data Mining, John Wiley & Sons, 2000  Trafalis, T.B., M. Richman, and B. Santosa,"Prediction of Rainfall from WSR-88D Radar Using Support Vector Regression", ASME Press, (2002). Book Published of Collection: C.H. Dagli, A.L. Buczak, J. Ghosh, M.J. Embrechts, O. Ersoy, and S.W. Kercel, Intelligent Engineering Systems Through Artificial Neural Networks, Vol. 12 (pp ).  Theodore B. Trafalis, Budi Santosa, and Michael B. Richman, “Learning Networks for Tornado Detection”, International Journal of General Systems, 2005  Sumber dari internet


Download ppt "Seminar Data Mining Business Trouble and Industrial Applications Business Trouble and Industrial Applications Lab Data Mining, Teknik Industri Universitas."

Presentasi serupa


Iklan oleh Google