Loading…
This event has ended. Create your own event on Sched.
Data to Action: Increasing the Use and Value of Earth Science Data and InformationFor 20 years, ESIP meetings have brought together the most innovative thinkers and leaders around Earth observation data, thus forming a community dedicated to making Earth observations more discoverable, accessible and useful to researchers, practitioners, policymakers, and the public.

The ESIP Summer Meeting has already taken place, but check out the ESIP Summer Meeting Highlights Webinar: https://youtu.be/vbA8CuQz9Rk.
Back To Schedule
Tuesday, July 16 • 2:45pm - 4:15pm
Cloud Data Optimization: Emerging Best Practices I

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
When data is shared in the cloud, anyone can analyze it without having to download it or store it themselves, which lowers the cost of new product development, reduces the time to scientific discovery, and can accelerate innovation. However, staging large-scale datasets for analysis in the cloud requires consideration of how data should be prepared and organized to allow fast, efficient, and programmatic access from distributed computing systems. This workshop provides a forum for members of the community to share lessons learned as they explore ways to use the cloud to expand data access. It seeks to encourage dialog between users interested in leveraging data in the AWS Cloud for research and application development for Earth Sciences.

View Session Recording

Session Description:
When data is shared in the cloud, anyone can analyze it without having to download it or store it themselves, which lowers the cost of new product development, reduces the time to scientific discovery, and can accelerate innovation. However, staging large-scale datasets for analysis in the cloud requires consideration of how data should be prepared and organized to allow fast, efficient, and programmatic access from distributed computing systems. This workshop provides a forum for members of the community to share lessons learned as they explore ways to use the cloud to expand data access. It seeks to encourage dialog between users interested in leveraging data in the AWS Cloud for research and application development for Earth Sciences.

Workshop Format: 
Workshop includes 1.5 hours of presentations (Cloud Data Optimization: Emerging Best Practices I) followed by 1.5 hours of discussion on emerging best practices and identifying needs to move this space forward.

Presentations (10 minutes each)
Full Abstracts can be found in the attached file.
  1. Title: STAC, sat-utils, and Open Data - Prioritizing Data Use (10 min)
    Presenter: Dan Pilone (Element 84)
  2. Title: Radiant ML Hub, A cloud based commons for geospatial training datasets (10 min)
    Presenter: Hamed Alemohammad (Radiant Earth Foundation)
    Slides: https://doi.org/10.6084/m9.figshare.9696446
  3. Title: One data format pattern to rule them all (10 min)
    Presenter: Grega Milcinski (Sinergise)
    Slides: https://doi.org/10.6084/m9.figshare.9121991
  4. Title: Improved Cloud Raster Format for multidimensional raster storage and analysis (10 min)
    Presenters: Hong Xu (Esri) & Sudhir Raj Shrestha (Esri)
    Slides: https://doi.org/10.6084/m9.figshare.9762866
  5. Title: Optimization of CESM LENS on AWS S3 (10 min)
    Presenter: Jeff de La Beaujardiere (NCAR)
    Slides: https://doi.org/10.6084/m9.figshare.9633314
  6. Title: The Zarr format
    Presenter: Rich Signell (USGS)
    Slides: https://doi.org/10.6084/m9.figshare.9701684
  7. Title: NOAA’s Big Data Project - A Data Broker’s Perspective
    Presenter: Otis Brown (NC State University/NCICS)
    Slides: https://doi.org/10.6084/m9.figshare.9693776
  8. Title: HDF Data Service for the Cloud
    Presenter: John Readey (The HDF Group)


Session Take-Aways
  1. Moving to cloud infrastructure offers a chance to reevaluate best practices, though some of these may not be purely cloud-related (e.g., data formats) but the discussions are coming along for the ride!
  2. It is unclear who will own the cloud-optimized datasets and it will likely be different from dataset to dataset. Until (if/when) cloud-optimized formats become the norm, they may often be provided by other groups (or created on the fly).
  3. There is a lot of focus on datasets in these conversations, but we need to also focus on tooling/services and education.



Speakers
avatar for Jeff de La Beaujardiere

Jeff de La Beaujardiere

Director, Information Systems Division, NCAR
I am the Director of the NCAR/CISL Information Systems Division. My focus is on the entire spectrum of geospatial data usability: ensuring that Earth observations and model outputs are open, discoverable, accessible, documented, interoperable, citable, curated for long-term preservation... Read More →
avatar for Dan Pilone

Dan Pilone

Chief Technologist, Element 84
Dan Pilone is CEO/CTO of Element 84 and oversees the architecture, design, and development of Element 84's projects including supporting NASA, the USGS, Stanford University School of Medicine, and commercial clients. He has supported NASA's Earth Observing System for nearly 13 years... Read More →
avatar for Sudhir Shrestha

Sudhir Shrestha

Technical Director Web and Dissemination, NOAA NWS Office of Water Prediction
avatar for Hamed Alemohammad

Hamed Alemohammad

Executive Director and Chief Data Scientist, Radiant Earth Foundation
JR

John Readey

The HDF Group
avatar for Sudhir Raj Shrestha

Sudhir Raj Shrestha

Solution Engineer Researcher, Esri
Solution Engineer and Scientific Data enthusiast with keen interest in making data easily Discoverable and Interoperable. Passionate about geospatially driven Hydrological Modeling and Heuristic Soil Modeling and develop, implement new and innovative geospatial methods, techniques... Read More →
avatar for Grega Milcinski

Grega Milcinski

CEO and Co-founder, Sinergise
Sentinel Hub and general availability of EO data in the clouds
avatar for Ana Pinheiro Privette

Ana Pinheiro Privette

Amazon Sustainability Data Initiative (ASDI) Lead, Amazon
Dr. Ana Pinheiro Privette is a senior program manager with Amazon's Sustainability group and she leads the Amazon Sustainability Data Initiative (ASDI), a Tech-for-Good program that seeks to leverage Amazon’s scale, technology, and infrastructure to help create global innovation... Read More →



Tuesday July 16, 2019 2:45pm - 4:15pm PDT
Ballrm A
  Ballrm A, Workshop