| A bit about me... |
|---|
| 💼 Engineer @ Earthmover |
| 🌎 Active member of the Pangeo Community |
| 🌕 Core developer of Xarray |
| 🧪 PhD in fusion plasma physics from Uni. York & CCFE |
I also work on several other open-source projects for science.
Recent blog posts¶
Unlocking scientific data archives for human, agentic and AI model usage requires a truly scalable data storage architecture. We show from first principles what architecture is necessary.
GRIB data is notoriously hard to work with, so we made it easy to create virtual Icechunk stores from GRIB archives such as the NOAA National Blend of Models, allowing cloud-optimized access to GRIB data archives without copying the data.
Delivering enterprise-grade access control for virtual chunks - surprisingly subtle!
Access billions of chunks of satellite imagery as a single Zarr store, without copying any data!
The article I wish I could have read back when I first heard of Zarr in 2018. Explains how object storage and conventional filesystems are different, and the key properties that make Zarr work so well in cloud object storage.