No project description provided
Project description
Zarrita
Zarrita is an experimental implementation of Zarr v3 including sharding. This is only a technical proof of concept meant for generating sample datasets. Not recommended for production use.
Setup
import zarrita
import numpy as np
store = zarrita.LocalStore('testdata') # or zarrita.RemoteStore('s3://bucket/test')
Create an array
a = await zarrita.Array.create_async(
store,
'array',
shape=(6, 10),
dtype='int32',
chunk_shape=(2, 5),
codecs=[zarrita.codecs.blosc_codec()],
attributes={'question': 'life', 'answer': 42}
)
await a.async_[:, :].set(np.ones((6, 10), dtype='int32'))
Open an array
a = await zarrita.Array.open_async(store, 'array')
assert np.array_equal(await a.async_[:, :].get(), np.ones((6, 10), dtype='int32'))
Create an array with sharding
a = await zarrita.Array.create_async(
store,
'sharding',
shape=(16, 16),
dtype='int32',
chunk_shape=(16, 16),
chunk_key_encoding=('v2', '.'),
codecs=[
zarrita.codecs.sharding_codec(
chunk_shape=(8, 8),
codecs=[zarrita.codecs.blosc_codec()]
),
],
)
data = np.arange(0, 16 * 16, dtype='int32').reshape((16, 16))
await a.async_[:, :].set(data)
assert np.array_equal(await a.async_[:, :].get(), data)
Create a group
g = await zarrita.Group.create_async(store, 'group')
g2 = await g.create_group_async('group2')
a = await g2.create_array_async(
'array',
shape=(16, 16),
dtype='int32',
chunk_shape=(16, 16),
)
await a.async_[:, :].set(np.arange(0, 16 * 16, dtype='int32').reshape((16, 16)))
Open a group
g = await zarrita.Group.open_async(store, 'group')
g2 = g['group2']
a = g['group2/array']
assert np.array_equal(await a.asnyc_[:, :].get(), np.arange(0, 16 * 16, dtype='int32').reshape((16, 16)))
Credits
This is a largely-rewritten fork of zarrita
by @alimanfoo. It implements the Zarr v3 draft specification created by @alimanfoo, @jstriebel, @jbms et al.
Licensed under MIT
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
zarrita-0.1.0a5.tar.gz
(21.4 kB
view hashes)
Built Distribution
zarrita-0.1.0a5-py3-none-any.whl
(27.0 kB
view hashes)
Close
Hashes for zarrita-0.1.0a5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a34062f60b9a072a15bd342223b5212dc73ce98525fe29ae8d17a49d7cd60bb7 |
|
MD5 | 531cb5f2cd1b8ec8893e395c21573843 |
|
BLAKE2b-256 | f0232e5fcf06234084232558be0772539fdfbe440899bd22e9986fe186d76a76 |