1 BPS Statistics Indonesia New York, February 2011
Background • Population census in Indonesia is held every ten year. • Indonesia has the fourth largest population and the largest archipelago. • History of data processing for population census o OMR Technology, mainframe o 1980 – data entry, mainframe o 1990 – data entry, mainframe, distributed o 2000 – OCR technology, PC clusters o 2010 – ICR and mobile technology, PC clusters. 2
Data Processing Centers • Located in 33 Provincial Statistics Offices. 3 VPN
4 Flow of Documents in the Fields ENUMERATOR KORTIM BPS DISTRIC SP2010 L, KBC, RT, ART Doc Pool SP2010 L, KBC, RT, ART BPS KECDESA Drop Off Receiving & Handling Queuing Unpack & Checking RepackExpedition Entry SP2010-L PROVINCE Drop Off Receiving & Handling Queuing Unpack Repack Expedition/ Next Queuing Entry Coding BPS
5 Flow of Document in DPC RECEPTION SERVICE QUEUE ROOM DOC PREPREPACKING SCANNING DOCUMENT STORAGE CORRECTIONCOMPLETIONVALIDATION STAGING
6 DPC Personnel Box sorting In Queuing Room RECEPTION AREA - 1) Download Box - 2) Put Boxes in the trolley - 3) Input received Data - 4) Arrange box in Queuing Room Database Box & Block Sensus Database CODING PICKUP OFFICER - 5) Take box from queuing room - 6) Registration of pick up box - 7) Deliver box es to Coding Editing Supervisor CODING EDITING SUPERVISOR - 8) Boxes distribution ke petugas Coding Editing ) Check & Authorization on any pages discrepancies - 13) Update data box that finished coding editing CODING EDITING OFFICER - 9) Box opening - 10) Unbind documents - 11) Pages count - 12) Coding Editing ) Reporting of discrepancies pages Sorting Boxes in Scanning Queue
7 Flow of Processing Documents DROP-OFF SERVICE STAGING DOC PREP REPACKING FUMIGATION DOCUMENT STORAGE Drop Off Receiving & Handling Registering Sorting Unpack & Checking Cutting SCANNING DOCUMENT PREPARATION CORRECTION & COMPLETIONVALIDATION
8 Flow of Work in DPC : Scanning & Warehouse Box Scanning Queue SCANNING PICKUP OFFICER - 1) Pickup box from Scanning Queue - 2) Pickup box registration - 3) Deliver box to Scanning Supervisor Database Box & Block Sensus Database SCANNING OFFICER -5) Register # box -6) Scan docoments REPACKING OFFICER -7) Repacking box -8)Register finished repack STORAGE OFFICER - 9) Trolley from Repacking to Doc Storage STORAGE OFFICER -12) Place box refer to Put-Away STORAGE ADMIN -10) Register box -11) Cetak Put-Away Penyimpanan DATA CAPTURE SERVER
9 Flow of Data in DPC BPSServer INFORMATION TECHNOLOGY CAPTURE SYSTEM SUPPORT APPS Staging Clean Data Data Tabulasi Correction & Completion Validasi Data Validasi Data Staging Image + data RECEPTION SERVICE DOCUMENT STORAGE Status box Lokasi box Scanning Image + data RELEASE
10 Batching System • Document batch o 1 SP 2010 KBC o Consist of = n SP 2010 RT o Each RT consist of = n SP 2010 ART
11 Capture Process • Fixed Form Approach • High speed Auto classification & separation • Accurate High Speed ICR engine • Accurate High Speed OMR engine • Consistency check capability • Inter-page business rule validation • Multipage business rules validation • Low false positive & Tuning
12
13 Solution Components • Guillotine • PCs • Server • Scanner • Software Data Capture • Training & troubleshooting • Template Development • Distribution, installation & implementation in each DPC (33 locations)
14 Fujitsu Scanner Fi-6800 • Scanner Speed : 130 ppm 300 dpi • Duty Cycle : pages/ day • Resolution : 600 dpi • Feeder Capacity : 500 pages • Paper Size : up to A3 • Imprinter capability : Pre and Post
15 Guillotine, workstation, scanner
16 Data Capture Server, Validation Server
17 Server Console, Server Racks
18 Scanner Allocation #DPC - BPS OfficesNo. of Docs Scanner allocation 1NAD Sumatera Utara Sumatera Barat Riau Jambi Sumatera Selatan Bengkulu Lampung Kep Bangka Belitung Kepulauan Riau DKI Jakarta Jawa Barat Jawa tengah DI Yogyakarta Jawa Timur Banten
19 Scanner Allocation #DPC - BPS OfficesNo. of Docs Scanner Allocation 17Bali Nusa Tenggara Barat Nusa Tenggara Timur Kalimantan Barat Kalimantan Tengah Kalimantan Selatan Kalimantan Timur Sulawesi Utara Sulawesi Tengah Sulawesi Selatan Sulawesi Tenggara Gorontalo Sulawesi Barat Maluku Maluku Utara Papua Barat Papua
20 Server, PC Allocation #DPC - BPS OfficesServerPC 1NAD230 2Sumatera Utara 284 3Sumatera Barat 233 4Riau 241 5Jambi 222 6Sumatera Selatan 252 7Bengkulu 214 8Lampung 254 9Kep Bangka Belitung Kepulauan Riau DKI Jakarta Jawa Barat Jawa tengah DI Yogyakarta Jawa Timur Banten 477
21 Server, PC Allocation #DPC - BPS OfficesServerPC 17Bali Nusa Tenggara Barat Nusa Tenggara Timur Kalimantan Barat Kalimantan Tengah Kalimantan Selatan Kalimantan Timur Sulawesi Utara Sulawesi Tengah Sulawesi Selatan Sulawesi Tenggara Gorontalo Sulawesi Barat Maluku Maluku Utara Papua Barat Papua
22Networking #DPC - BPS OfficesSwitch 48 nodeCable (m) 1NAD Sumatera Utara Sumatera Barat Riau Jambi Sumatera Selatan Bengkulu Lampung Kep Bangka Belitung Kepulauan Riau DKI Jakarta Jawa Barat 8 3,780 13Jawa tengah 6 2,650 14DI Yogyakarta Jawa Timur 7 3,140 16Banten 2 870
23Networking #DPC - BPS OfficesSwitch 48 nodeCable (m) 17Bali Nusa Tenggara Barat Nusa Tenggara Timur Kalimantan Barat Kalimantan Tengah Kalimantan Selatan Kalimantan Timur Sulawesi Utara Sulawesi Tengah Sulawesi Selatan Sulawesi Tenggara Gorontalo Sulawesi Barat Maluku Maluku Utara Papua Barat Papua
24
25 ScanScanRecognitionRecognitionCorrectionCorrectionCompletionCompletionReleaseRelease Kofax Implementation Overview
26 Software Data Capture Implementation Doc Template Management o Template Registration o Template Setting • Registration Point • Field Definition • Field Formatting • Multi-Engine Voting • Dictionary • Data Look-Up • Business Rules • Integrity among pages
Correction Validation ReleaseRecognitionScanningQuality Check Provincial Statistics Office Monitoring Compiled Data RBL Data Processing Context 27 Data Entry Quality Check Municipality Statistics Office Head Quarter RBL Listing Statistical Coordinator Validate, Summarize Send SMS KBC C1 Validate, Summarize Send SMSCensus Field Work
Capture Process Flow •Classification •Recognition PC Document Review PC & Scanner Server Data Capture PC Correction PC Completion Server Database PC QUALITY CONTROL
29 Document Preparation • Objective: – To cut the side of forms booklet using paper guillotine – Preparing docs for scanning process
30 Kofax - Module • Scanning : – Scan batch – Page counting of document batch in scanning process • QC: – System ensure that the pages of the doc batch match with the registered sum of pages entry before scanning. • Classification: o System will classify based on template • Document Review: o Unrecognized doc will appear in this module o Operator may re-arrange, delete and re-scan the doc
31 Kofax - Module • Recognition : – Data extraction from processed form – unrecognized Data for Correction & Completion • Correction : – Character correction which un-recognized by system on below a set of confidence level. Correction made field by field. • Completion : – To complete all correction on one set of document in a document batch refer to validation and business rules that have set in the system • Release : – Exporting image to predefine folder and data to predefine database
32 Kofax - Correction • Sample Screen:
33 Kofax – Completion • Sample Screen ENTRY PANEL IN TABULAR FORMAT TO CATEGORISED FIELD
34 Kofax – Completion LOCATION ID CHECKING, DATA LOOKUP TO DATABASE
35 Kofax – Completion VERIFICATION CHILD AGE W/ BIOLOGICAL MOTHER VERIFICATION CHILD NATIONALITY VS BIOLOGICAL FATHER &/ MOTHER Business Rules
36 Kofax - Release • Objective: – Deliver image to folder in the File Server – Deliver data to database Staging BPS • Scope: – Write data to Database Staging
37 Network Architecture of Data Center
38 Network Integration
Population of Indonesia based on the Census, May 2010 (preliminary figures, Released Aug 2010) Male(000)Female(000) Male + Female (000) 119, , ,556 39
Thank You 40