Sync s3 from one source to another place
Project description
s3-sync
Sync AWS S3 storage:
- Sync file from s3 -> s3
- Sync file from s3 -> local storage
- Sync file from local storage -> s3
Installation
pip install s3sync
S3 sync s3 to s3
Read from parameters
s3-sync --sync-s3-bucket true \
# using 5 concurrent thread default is 2
--thread 5 \
--source-bucket-name source-bucket \
--source-region-name ap-southeast-1 \
--source-access-key-id source-access-key \
--source-secret-access source-secret \
--target-bucket-name target-bucket \
--target-region-name ap-southeast-3 \
--target-access-key-id target-access-key \
--target-secret-access target-secret
Read from json file
- Create example json file as bellow example, you can name this file whatever
{
"source": {
"bucket_name": "source-bucket",
"region_name": "ap-southeast-1",
"access_key_id": "source-access-key",
"secret_access_key": "source-secret-key"
},
"target": {
"bucket_name": "target-bucket",
"region_name": "ap-southeast-3",
"access_key_id": "target-access-key",
"secret_access_key": "target-secret-key"
}
}
- Then execute command
s3-sync --sync-s3-bucket true -json /home/ubuntu/config.json --thread 5
S3 sync s3 to local (Give absolute path for this)
Read from parameters
s3-sync --sync-s3-local true \
# using 5 concurrent thread default is 2
--thread 5 \
--source-bucket-name source-bucket \
--source-region-name ap-southeast-1 \
--source-access-key-id source-access-key \
--source-secret-access source-secret-key \
--target-local-path /home/ubuntu/sync-s3/my-folder
Read from json file
- Create example json file as bellow example, you can name this file whatever
{
"source": {
"bucket_name": "source-bucket",
"region_name": "ap-southeast-1",
"access_key_id": "source-access-key",
"secret_access_key": "source-secret-key"
},
"target": {
"path": "/home/ubuntu/sync-s3/my-folder"
}
}
- Then execute command
s3-sync --sync-s3-local true -json /home/ubuntu/config.json --thread 5
S3 sync local to s3
Read from parameters
s3-sync --sync-local-s3 true \
# using 5 concurrent thread default is 2
--thread 5 \
--source-local-path /home/ubuntu/sync-s3/my-folder \
--target-bucket-name target-bucket \
--target-region-name ap-southeast-3 \
--target-access-key-id target-access-key \
--target-secret-access target-secret-key
Read from json file
- Create example json file as bellow example, you can name this file whatever
{
"source": {
"path": "/home/ubuntu/sync-s3/my-folder"
},
"target": {
"bucket_name": "target-bucket-name",
"region_name": "ap-southeast-3",
"access_key_id": "target-access-key",
"secret_access_key": "target-secret-key"
}
}
- Then execute command
s3-sync --sync-local-s3 true -json /home/ubuntu/config.json --thread 5
Help & Bugs
If you are still confused or found a bug, please open the issue. All bug reports are appreciated, some features have not been tested yet due to lack of free time.
License
s3-sync released under MIT. See LICENSE
for more details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
s3sync-0.1.1.tar.gz
(6.4 kB
view hashes)
Built Distribution
s3sync-0.1.1-py3-none-any.whl
(10.4 kB
view hashes)