Retrieve information content and compress accordingly.
Project description
xbitinfo: Retrieve information content and compress accordingly
This is an xarray
-wrapper around BitInformation.jl to retrieve and apply bitrounding from within python.
The package intends to present an easy pipeline to compress (climate) datasets based on the real information content.
How the science works
Paper
Klöwer, M., Razinger, M., Dominguez, J. J., Düben, P. D., & Palmer, T. N. (2021). Compressing atmospheric data into its real information content. Nature Computational Science, 1(11), 713–724. doi: 10/gnm4jj
Video
Julia Repository
How to install
Preferred installation
conda install -c conda-forge xbitinfo
Alternative installation
pip install xbitinfo
# ensure to install julia manually
How to use
import xarray as xr
import xbitinfo as xb
ds = xr.tutorial.load_dataset(inpath)
bitinfo = xb.get_bitinformation(ds, dim="lon") # calling bitinformation.jl.bitinformation
keepbits = xb.get_keepbits(bitinfo, inflevel=0.99) # get number of mantissa bits to keep for 99% real information
ds_bitrounded = xb.xr_bitround(ds, keepbits) # bitrounding keeping only keepbits mantissa bits
ds_bitrounded.to_compressed_netcdf(outpath) # save to netcdf with compression
Credits
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
xbitinfo-0.0.3.tar.gz
(48.4 kB
view hashes)
Built Distribution
Close
Hashes for xbitinfo-0.0.3-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3acd10d05495043f05b07280feb28ffd5a6832b644b3d01486aaba87dee021dd |
|
MD5 | 3b6405ef3191fd14cc7ca16422c29cfe |
|
BLAKE2b-256 | 82121ae1ce880a290b91c55c3570c42fe560b5c22bc2c227377d8dc764cca640 |