zarr trial run #28
Description
Hello Team,
As mentioned elsewhere, I am working on a R-native Zarr package, aptly named zarr. The package is in the very early stages of development but it already supports most of the core v.3 specification (notable misses are some data types and the sharding codec). I'd like to invite you to test run the package available here and let me know what you think and point out the (many?) errors that you encounter (please use issues on the zarr site). You are the first people out there who are invited to look at this. A few things of note:
- This is for Zarr v.3. Existing Zarr v.2 stores can not be accessed (adding this is high on my development list).
- The only stores that are currently supported are the local filesystem store and a memory store.
- The local file system store is brittle in the sense that it will fail with out-of-space issues and access rights problems. Also, there is no check on concurrent access or other pernicious access issues.
- No support yet for attributes or other non-mandatory properties.
- Documentation is sparse but have a look at the
test_that/test-zarr.Rfile for additional features. - @thisistaimur Taimur, would be great if you could throw some Urdu at it, or some other non-Latin languages such as Arabic or Punjabi, when naming groups and/or arrays.
I am developing on a MacBook so hoping that you can test run on Windows and/or Linux.
Any views on how this might work with ClimHub very welcome. On my development plan (after a first release is out on CRAN) are:
- Support for Zarr v.2
- Adding http and S3 stores
- Making I/O asynchronous
- Make the package extendable with support for new extension points
- Make the package multi-lingual
Hoping to release to CRAN by the end of this month.