Effective Planning for Managing Environmental Data Susan Gottfried NOAA’s Office of Ocean Exploration and Research Data Management Coordinator

Data Management Planning Data Documentation Data Discovery and Access Data Management Workflow

Presentation Outline • Why is it important? (Mengapa hal itu penting? ) • What is NOAA doing to encourage and enforce data management? (Apa yang NOAA lakukan untuk mendorong dan menegakkan manajemen data?) • OER Data Management • International Standards for Metadata • Innovative Methods and Tools Documentation 3

Why is it important to plan? Documentation 4 • Data Management Plans are an informal agreement between the funding organization and the funded scientists. (Rencana Manajemen Data kesepakatan informal antara organisasi pendanaan dan ilmuwan yang didanai.) • Planning for managing data is improving data submission compliance. (Perencanaan pengelolaan data meningkatkan kemungkinan bahwa para ilmuwan akan menyerahkan data mereka untuk dikelola.) • Planning ensures that the necessary infrastructure is present to facilitate data submission. (Memiliki infrastruktur yang dibutuhkan tersedia untuk memastikan bahwa data dapat disampaikan juga membantu dengan kepatuhan. Perencanaan membantu dengan menyiapkan infrastruktur ini.)

What is NOAA doing to encourage and enforce data management? Documentation 5 • November 2010: NOAA Administrative Order (212-15) for Management of Environmental Data and Information • October 2011: Directives to enforce the NAO : – Data Management Planning – Data Sharing – Data Documentation

NOAA Administrative Order Documentation 6 • Environmental data will be visible, accessible and independently understandable to users, except where limited by law, regulation, policy (such as those applicable to personally identifiable information or protected critical infrastructure information or proprietary trade information) or by security requirements. • Data lingkungan akan terlihat, mudah diakses dan dipahami oleh pengguna, kecuali dibatasi oleh hukum, kebijakan regulasi, (seperti yang berlaku untuk pribadi diidentifikasi informasi atau dilindungi infrastruktur informasi penting atau perdagangan proprietary Informasi) atau dengan persyaratan keamanan.

NOAA Administrative Order Documentation 7 Management of NOAA environmental data will be based upon an end-to-end data management lifecycle that includes: • Determining what environmental data are required to be preserved for the long term and how preservation will be accomplished • Developing and maintaining metadata throughout the environmental data lifecycle that comply with standards • Obtaining user requirements and feedback • Developing and following data management plans that are coordinated with the appropriate NOAA archive for all observing and data management systems • Conducting scientific data stewardship to address data content, access, and user understanding • Providing for delivery to the archive and secure storage • Providing for data access and dissemination • Enabling integration and/or interoperability with other information and products Pengelolaan data NOAA lingkungan akan didasarkan pada sebuah awal untuk mengakhiri Data manajemen siklus hidup yang meliputi: • Menentukan apa data lingkungan yang diperlukan untuk dipertahankan untuk jangka panjang dan bagaimana pelestarian akan dicapai • Mengembangkan dan memelihara metadata seluruh siklus hidup data lingkungan yang sesuai dengan standar • Mendapatkan kebutuhan pengguna dan umpan balik • Mengembangkan dan mengikuti rencana manajemen data yang dikoordinasikan dengan sesuai NOAA arsip untuk semua mengamati dan data manajemen sistem • Melakukan pelayanan data ilmiah untuk mengatasi konten data, akses, dan user pemahaman • Menyediakan untuk pengiriman ke arsip dan penyimpanan yang aman • Memberikan akses data dan diseminasi • Mengaktifkan integrasi dan / atau interoperabilitas dengan informasi lainnya dan produk

Data Management Planning Directive Documentation 8 • The goal of the Data Management plan is to ensure that data are properly collected, documented, made accessible, and preserved for future use in a NOAA Data Center or other long‐term. • Tujuan dari rencana manajemen data adalah untuk memastikan bahwa data yang benar dikumpulkan, didokumentasikan, dibuat dapat diakses, dan diawetkan untuk digunakan di masa depan dalam arsip data center atau jangka panjang lainnya.

Data Sharing Directive Documentation 9 • All NOAA Grantees must share data produced under NOAA grants and cooperative agreements in a timely fashion, except where limited by law, regulation, policy or security requirements. Grantees must address this requirement formally by preparing a Data Sharing Plan as part of their grant project narrative. • Semua Penerima harus berbagi data diproduksi di bawah NOAA hibah dan perjanjian kerja sama secara tepat waktu, kecuali dibatasi oleh hukum, kebijakan regulasi, atau persyaratan keamanan. Penerima harus mengatasi kebutuhan ini secara resmi dengan menyiapkan Rencana Berbagi Data sebagai bagian dari narasi proyek mereka hibah.

Data Documentation Directive Documentation 10 • All NOAA data collections and products derived from these data shall be documented. (Semua NOAA Data koleksi dan produk yang berasal dari data ini harus didokumentasikan.) • Services that provide NOAA data and products shall be documented. (Layanan yang menyediakan akses ke data dan produk harus didokumentasikan.) • Data collections funded by NOAA, and products derived from these collections that are funded by NOAA shall be documented. (Data koleksi didanai oleh NOAA, dan produk-produk yang berasal dari koleksi tersebut harus didokumentasikan.) This Procedural Directive establishes a metadata content standard (International Standards Organization [ISO] Parts 1 and 2) and a recommended representation standard (ISO 19139) for documenting NOAA’s environmental data and information. (Ini Petunjuk prosedural menetapkan standar isi metadata (International Standar Organization [ISO] Bagian 1 dan 2) dan standar representasi direkomendasikan (ISO 19139) untuk mendokumentasikan data lingkungan NOAA dan informasi.)

Expeditions Funded through Announcement of Opportunity (AO) Process Cruise Planning Process & Generation of Data Management Plans and ISO Collection-Level Metadata Record Data Generated During Mission (sensors, video, multibeam, samples, reports, etc ) Post-Mission Data Processing, Metadata Generation, and Archive Prep Data to Archives, Metadata and Data Links Posted to Discovery Tools Exploration Vessel Okeanos Explorer OER Data Management Process Flow

Cruises ditargetkan terhadap penelitian ilmiah dan didanai oleh NOAA Cruise proses perencanaan & Generasi dari rencana manajemen data dan ISO cruise-level rekor metadata Data yang dihasilkan selama misi (sensor, video, multibeam, sampel, laporan, dll) Setelah pelayaran pengolahan data, generasi metadata, dan persiapan untuk arsip Data diserahkan ke bagian arsip, metadata dan data link yang diposting ke alat penemuan Eksplorasi Proses Pengelolaan Data Aliran

Data Management Planning • Annual detailed Data Management Plan for the Vessel – Data management expectations – Standard operating procedures and vocabularies – General data pipelines – Shipboard system specifics • Cruise-specific Data Management Plans – Cruise data collection objectives – Temporal and spatial boundaries – New data pipelines – Data management exceptions – Mission-specific keywords • OER DMP used as an example for EDMC DMP Directive Documentation 13

ISO Metadata Standard • Collaborative work on ISO Metadata Templates • Highly-detailed and Hierarchical – Structure allows for records to have parents and children – Records can be rolled up into metadata inventories at different levels • Convenient and customizable – Use of component metadata – Use of standard vocabularies – Minimal limitations • 2012 Use Case Study Ongoing Documentation 14

ISO Components • “Unresolved” ISO metadata invokes inline references to published components (re-usable ISO segments). NGDC’s Docucomp is the current repository for NOAA components. Documentation 15  Organizations with contact information  Individuals with contact information and calls to organization components  Instruments with technical support and contact info  Platforms (ships, ROVs, Camera Platforms, etc) and their onboard instruments (calls to instrument components)  Standard Vocabularies

Standard Vocabulary Components • OER Program • EX Program • Marine Archaeology Program • Limits of Oceans and Seas • UNOLS Ports • UNOLS Device Types • etc.

ISO Case Study – INDEX-SATAL 2010 Dataset (data object) Series (data type) Collection (cruise) Collection (sub- program) Collection (program) Ocean Exploration Program (Okeanos Explorer) EX1004 Oceanographic Parameters Spreadsheet (Daily Sensor Recordings) Multibeam KMZ (Bathymetric GeoTiff) Video Data MOV (Highlight Video Clip)

Templates • Metadata templates can be made available through publishing in a Web Accessible Folder (WAF) – Collection level (multiple data types collected – hierarchical) – Series level (specific data type collections) – Dataset level (data file or data product) • Templates pre-populated with the UUID of its parent records are possible for convenience and encouragement.