This event has ended. Create your own event on Sched.
Data to Action: Increasing the Use and Value of Earth Science Data and InformationFor 20 years, ESIP meetings have brought together the most innovative thinkers and leaders around Earth observation data, thus forming a community dedicated to making Earth observations more discoverable, accessible and useful to researchers, practitioners, policymakers, and the public.

The ESIP Summer Meeting has already taken place, but check out the ESIP Summer Meeting Highlights Webinar: https://youtu.be/vbA8CuQz9Rk.
Back To Schedule
Thursday, July 18 • 1:30pm - 3:00pm
HDF Town Hall

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Data in HDF file formats continues to play an important role for Earth Scientists in the U.S. and around the world. The HDF Group will update ESIP members on the state of HDF software and HDF5 Roadmap, and will share our experience on working with HDF5 in the Cloud. We will discuss our technical approaches, and lessons learned from different projects including a NASA ACCESS project that transformed NASA HDF data into GeoTIFF in AWS. We will also update ESIP members on our involvement in standardization efforts and demonstrate how HDF tools support ESDIS data from product initial design to production, and to compliance with the standards. We will encourage ESIP members participating in the session to share their experiences with the HDF software and to contribute to the HDF5 Roadmap.

Google Colaboratory for HDF-EOS - Joe Lee Abstract: Google provides a free Jupyter notebook environment called Colaboratory (also known as Colab).  It is simple, easy, and awesome Python environment for data scientists. We present how NASA Earthdata in HDF can be used with Google Colab using the existing comprehensive example on HDF-EOS Tools and Information Center website (http://hdfeos.org/zoo). We also present how OPeNDAP can be used with Colab to achieve 100%-cloud data analysis.

Keywords: Python, Google Colab, Jupyter notebook, HDF-EOS, OPeNDAP, Cloud computing.
Slides: https://doi.org/10.6084/m9.figshare.8976464

Leveraging the Cloud for HDF Software Testing - Larry Knox

Abstract: In this talk we will discuss how we leverage the Cloud for HDF software daily regression testing including testing of the HDF5 parallel library on the Cloud cluster using Orange FS.
Keywords: HDF5, Cloud, CI testing.
Parallel Computing with HDF Server - John Readey

Abstarct: To deal with really big data you need to be able to harness the power of multiple machines, but many users are put off by the complexity involved in setting up a cluster and then figuring out to effectively utilize it.   However, by using HDF Server (HSDS) with Kubernetes, it’s much easier than you would think.  In this talk we’ll walk through some examples of using xarray, h5netcdf, and h5py with HSDS to illustrate how you can scale up your compute to match your data size.
Keywords: HDF5, h5netcdf, h5py

 HDF5 Roadmap 2019-2020 - Elena Pourmal

Abstract: In this talk we will give an overview of the new features of the upcoming HDF5 release 1.12.0, and outline the HDF5 roadmap for the next year. We will demonstrate new open source file drivers to access HDF5 files via Amazon Simple Storage Service (Amazon S3) and on Hadoop Distributed File system (HDFS). We will use this presentation to get feedback on the HDF5 roadmap from the ESDIS users and application developers.
Keywords: HDF5, Amazon S3, HDFS, Cloud, Object Store.

Session recording here.


Aleksandar Jelenek

The HDF Group


John Readey

The HDF Group

Larry Knox

The HDF Group

Elena Pourmal

Engineering Director, HDF Group
avatar for Hyokyung Joe Lee

Hyokyung Joe Lee

Software Engineer, The HDF Group
Data Modeling: HDF Product DesignerData Format: HDF(-EOS) / netCDF / Parquet / ONNX / ArcGIS CRF / GDALData Service: OPeNDAP (Hyrax / THREDDS / Pydap) / ArcGIS EnterpriseData @Scale: Cloud / AWS S3 & Lambda & ECS / Docker & Kubernetes / Conda & DaskData Analytics: Big data / Apache... Read More →

Thursday July 18, 2019 1:30pm - 3:00pm PDT
Room 318
  Room 318, Breakout