Loading…
This event has ended. Create your own event on Sched.
Data to Action: Increasing the Use and Value of Earth Science Data and InformationFor 20 years, ESIP meetings have brought together the most innovative thinkers and leaders around Earth observation data, thus forming a community dedicated to making Earth observations more discoverable, accessible and useful to researchers, practitioners, policymakers, and the public.

The ESIP Summer Meeting has already taken place, but check out the ESIP Summer Meeting Highlights Webinar: https://youtu.be/vbA8CuQz9Rk.

Tuesday, July 16 • 2:45pm - 4:15pm
Cloud Data Optimization: Emerging Best Practices I

Sign up or log in to save this to your schedule and see who's attending!

Feedback form is now closed.
When data is shared in the cloud, anyone can analyze it without having to download it or store it themselves, which lowers the cost of new product development, reduces the time to scientific discovery, and can accelerate innovation. However, staging large-scale datasets for analysis in the cloud requires consideration of how data should be prepared and organized to allow fast, efficient, and programmatic access from distributed computing systems. This workshop provides a forum for members of the community to share lessons learned as they explore ways to use the cloud to expand data access. It seeks to encourage dialog between users interested in leveraging data in the AWS Cloud for research and application development for Earth Sciences.

View Session Recording

Session Description:
When data is shared in the cloud, anyone can analyze it without having to download it or store it themselves, which lowers the cost of new product development, reduces the time to scientific discovery, and can accelerate innovation. However, staging large-scale datasets for analysis in the cloud requires consideration of how data should be prepared and organized to allow fast, efficient, and programmatic access from distributed computing systems. This workshop provides a forum for members of the community to share lessons learned as they explore ways to use the cloud to expand data access. It seeks to encourage dialog between users interested in leveraging data in the AWS Cloud for research and application development for Earth Sciences.

Workshop Format: 
Workshop includes 1.5 hours of presentations (Cloud Data Optimization: Emerging Best Practices I) followed by 1.5 hours of discussion on emerging best practices and identifying needs to move this space forward.

Presentations (10 minutes each)
Full Abstracts can be found in the attached file.
  1. Title: STAC, sat-utils, and Open Data - Prioritizing Data Use (10 min)
    Presenter: Dan Pilone (Element 84)
  2. Title: Radiant ML Hub, A cloud based commons for geospatial training datasets (10 min)
    Presenter: Hamed Alemohammad (Radiant Earth Foundation)
    Slides: https://doi.org/10.6084/m9.figshare.9696446
  3. Title: One data format pattern to rule them all (10 min)
    Presenter: Grega Milcinski (Sinergise)
    Slides: https://doi.org/10.6084/m9.figshare.9121991
  4. Title: Improved Cloud Raster Format for multidimensional raster storage and analysis (10 min)
    Presenters: Hong Xu (Esri) & Sudhir Raj Shrestha (Esri)
    Slides: https://doi.org/10.6084/m9.figshare.9762866
  5. Title: Optimization of CESM LENS on AWS S3 (10 min)
    Presenter: Jeff de La Beaujardiere (NCAR)
    Slides: https://doi.org/10.6084/m9.figshare.9633314
  6. Title: The Zarr format
    Presenter: Rich Signell (USGS)
    Slides: https://doi.org/10.6084/m9.figshare.9701684
  7. Title: NOAA’s Big Data Project - A Data Broker’s Perspective
    Presenter: Otis Brown (NC State University/NCICS)
    Slides: https://doi.org/10.6084/m9.figshare.9693776
  8. Title: HDF Data Service for the Cloud
    Presenter: John Readey (The HDF Group)


Session Take-Aways
  1. Moving to cloud infrastructure offers a chance to reevaluate best practices, though some of these may not be purely cloud-related (e.g., data formats) but the discussions are coming along for the ride!
  2. It is unclear who will own the cloud-optimized datasets and it will likely be different from dataset to dataset. Until (if/when) cloud-optimized formats become the norm, they may often be provided by other groups (or created on the fly).
  3. There is a lot of focus on datasets in these conversations, but we need to also focus on tooling/services and education.



Speakers
avatar for Jeff de La Beaujardiere

Jeff de La Beaujardiere

Director, Information Systems Division, NCAR/CISL
Big data, cloud computing, object storage, data management.
avatar for Dan Pilone

Dan Pilone

Chief Technologist, Element 84, Inc.
avatar for Rich Signell

Rich Signell

Oceanographer, USGS
Ocean Modeling, Python, NetCDF, THREDDS, ERDDAP, UGRID, SGRID, CF-Conventions, Jupyter, JupyterHub, CSW, TerriaJS
avatar for Hamed Alemohammad

Hamed Alemohammad

Chief Data Scientist, Radiant Earth Foundation
JR

John Readey

The HDF Group
avatar for Sudhir Shrestha

Sudhir Shrestha

Solution Engineer Researcher, Esri
Solution Engineer and Scientific Data enthusiast with keen interest in making data easily Discoverable and Interoperable. Passionate about geospatially driven Hydrological Modeling and Heuristic Soil Modeling and develop, implement new and innovative geospatial methods, techniques... Read More →
avatar for Grega Milcinski

Grega Milcinski

CEO and Co-founder, Sinergise
Sentinel Hub and general availability of EO data in the clouds



Tuesday July 16, 2019 2:45pm - 4:15pm
Ballrm A

Attendees (63)




Twitter Feed