High-level Python abstraction layer for Amazon S3
Project description
High-level Python abstraction layer for Amazon S3:
An `open <https://docs.python.org/2/library/functions.html#open>`__-like context handler which allows using S3 keys and local files interchangeably.
When reading S3, contents are first written to a temporary local file.
When writing S3, contents are written to a temporary local file, and uploaded on close.
An s3 CLI for listing, copying, syncing, and other common activities.
Features
Works without an S3 connection (with local files).
Supports multiprocess parallelism for copying lots of files.
Supports Python 2.7 and uses boto2.
Supports OS X, Linux, and Windows.
Tested and production-hardened.
Examples
with s3.open('s3://example/info.txt', 'w') as f:
f.write('hello')
with s3.open('file:///home/me/info.txt', 'w') as f:
f.write('hello')
with s3.open('s3://example/info.txt', 'r') as f:
contents = f.readlines()
with s3.open('file:///home/me/info.txt', 'r') as f:
contents = f.readlines()s3 cp foo.txt s3://example/bar.txt
s3 cp s3://example/bar.txt s3://another-example/bazinga.txt
s3 rm s3://example/bar.txtDevelopment
pip install -r requirements_dev.txt
rake test
rake lintTODO
Migrate credentials to ~/.aws/credentials or env, and deprecate AWS credential support in ~/.bodylabs.
Move baiji.util.parallel into a separate library.
Upgrade to boto3.
Contribute
Issue Tracker: github.com/bodylabs/baiji/issues
Source Code: github.com/bodylabs/baiji
Pull requests welcome!
Support
If you are having issues, please let us know.
License
The project is licensed under the Apache license, version 2.0.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
