Rclone wrapper to manage multiple datasets in a project
Project description
rkale
Install
Install lantern in your project using poetry:
poetry add rkale
Use pip if you want a global installation:
pip install rkale
Configuration
Global
~/.config/rkale/rkale.conf
:
[data]
root = "path to data folder where datasets are stored"
[aliases]
wasabi = "optional alias for remote in rclone.conf"
If alases are empty the remote name from the project config is used in the rclone lookup.
Project
Configure project datasets in the pyproject.toml file:
<project path>/pyproject.toml
:
[[tool.rkale.dataset]]
name = "dataset_1"
remote = "remote_1"
[[tool.rkale.dataset]]
name = "dataset_2"
remote = "remote_2"
The remote specified for the dataset must match a remote in the rclone.conf
or an alias in the global rkale configuration.
Usage
Python interface
from rkale.config import dataset_paths
DATASET_PATH = dataset_paths()["dataset_1"]
Syncing datasets
Syncs the local datasets to be identical to the remote
rkale psync
Syncs the remote datasets to be identical to the local
rkale psync --upstream
Same as rclone sync but checks differences first and asks for confirmation
rkale sync <source> <destination>
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.