Automatically store/load data in a tidy, efficient way.
Project description
Tidypath
For people that have to compute and store a large variety of data and/or figures.
Keep your files tidy!
Don't spend time creating directories, deciding filenames, saving, loading, etc. Decorators savefig & savedata will do it for you.
savedatacomputes output and stores it in the first function call. Future calls reads it from memory. Default LZMA compressionsavefigsaves output figure.
Although recommended, it is not needed to start a new project using tidypath. You can continue using your previous code and apply tidypath on new code.
savedata
Example function slow_computation in module package.subpackages.module
@savedata("x+z")
def slow_computation(x, y, *args, z=1, **kwargs):
...
return result
- Apply to function (result of any type).
- Choose the variables to record in the filenames.
- Optionally, choose file extension and other specifications. Supported:
lzma(default),bz2,npz,csv,JSON. - Result will be saved at
data/subpackages/module/slow_computation/x-'x'_z-'z'_.lzma('x' = value of x passed toslow_computationduring call) - If you want to recompute and overwrite, you can pass
overwrite=Truetoslow_computation. The decorator adds the arguments:save,overwrite,keysandfuncname_in_filename.
savefig
@savefig("kwargs")
def plot_results(*args, **kwargs):
...
return figure
- Same steps as
savedata. Only difference is the output type. - Supports
matplotlibandplotlyand all figure extensions (png,eps, ...) includinghtml(plotly). - Decorator adds the same arguments as
savedataplusreturn_fig(bool).
Adaptable to code modifications
Caching data depends on the specific variables set to store, since they define the filenames. Suppose we want to add a new variable method indicating a new method for computing the results, but already computed results are still useful. We can
-
Modify the variables to record in the
savedatadecorator:@savedata("x+z") => @savedata("x+z+method") -
Assign
method='original'to all existing pre-computed files:add_arg(slow_computation, method='original') -
Now access is granted for the already computed data, and data corresponding to new methods will be stored in separate files.
Use the functions add_arg, modify_arg, delete_arg to ensure cached data is loaded after modifying function arguments.
Example
Docs
Install
pip install tidypath
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
