Stratocumulus is the backend component of Cumulus.
Project description
Stratocumulus is the backend component of Cumulus, which aims to providing a unified backend interface for different cloud environments.
Installation
Stratocumulus does not include any Cloud SDK. You need to install the corresponding SDK separately, depending on which Cloud platform you use:
Google Cloud: Install Google Cloud SDK.
Amazon AWS: Install AWS CLI.
Local machine: Works for Linux and macOS. Please make sure rsync is installed, as some commands will use it.
After that, install Stratocumulus in your Python environment by command:
pip install stratocumulus
Usage
Stratocumulus has 4 commands:
cp: Copy file(s)/folder(s).
sync: Synchronize two folders, usually used for copying one folder.
rm: Remove file(s)/folder(s).
exist: Check if a (remote) file/folder exists.
These commands have options specific to backend. For now, Stratocumulus supports the following backends:
aws: Amazon AWS.
gcp: Google Cloud.
local: Local machine. In specific, macOS or Linux.
cp
This command copies file(s) or folder(s). See examples below:
# AWS upload strato cp --backend aws file1 folder2 s3://my-bucket/target_folder/ # AWS download strato cp --backend aws s3://my-bucket/source_folder/*.zip /target_folder/ # GCP upload strato cp --backend gcp -m -r --ionice file1 folder2 gs://my-bucket/target_folder/ # GCP download mkdir /target_folder strato cp --backend gcp -m gs://my-bucket/source_folder/*.zip /target_folder/ # Local Machine strato cp --backend local -r file1 folder2 /target_folder/
Notice that:
For AWS backend, you must explicitly attach a trailing slash for source folder.
For Google Cloud download, you’ll have to explicitly create the target folder, and then run strato cp command.
Wildcards are acceptable.
-r option is not needed for AWS, and copying folders is always recursive.
-m and --ionice options only work for Google Cloud.
For details on the options, try command strato cp -h.
sync
This command synchronizes two folders. Notice that this synchronization will delete content in the target folder not existing in the source folder.
See examples below:
# AWS strato sync --backend aws source_folder s3://my-bucket/target_folder # GCP strato sync --backend gcp -m --ionice source_folder gs://my-bucket/target_folder # Local Machine strato sync --backend local source_folder target_folder
Notice that:
-m and --ionice options only work for Google Cloud.
For details on the options, try command strato sync -h.
rm
This command deletes file(s) or folder(s). See examples below:
# AWS strato rm --backend aws s3://my-bucket/file1 s3://my-bucket/folder2/ # GCP strato rm --backend gcp -m gs://my-bucket/file1 gs://my-bucket/folder2 gs://my-bucket/folder3/*.zip # Local Machine strato rm --backend local file1 folder2
Notice that:
-m option only works for Google Cloud.
For AWS backend, wildcards are not yet accepted.
For details on the options, try command strato rm -h.
exists
This command checks if a (remote) file or folder exists. If it exists, the command terminates without any output. Otherwise, the command will break with Python subprocess.CalledProcessError exception.
See examples below:
# AWS strato exists --backend aws s3://my-bucket/file1 # GCP strato exists --backend gcp gs://my-bucket/folder2/ # Local Machine strato exists --backend local folder2/
Notice that this command works for both file and folder, regardless of backend.
For details on the options, try command strato exists -h.
help
Type strato -h or strato --help to check available Stratocumulus commands.
Check version
Type strato -v or strato --version to check the version of Stratocumulus currently installed on your machine.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for stratocumulus-0.1.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3dab109e9cda1bb4293f1b0498c37ffe645ddbb5a9fc13f276fd06c71e91f6f4 |
|
MD5 | 949eb42d939b1093b9853f941595897f |
|
BLAKE2b-256 | ac7d8607ded9a6d5cdce7beb405c26a349e636ee0837d055bce46fbff69d0f68 |