Concat files in s3
Project description
S3 Concat is used to concatenate many small files in an s3 bucket into fewer larger files.
Install
Comming soon: :sub:`pip install s3-concat`
Usage
This installs a cli called s3-concat. Find out how to use it by
running s3-concat -h.
It can also be imported and used in any other python scripts by foing
import s3_concat.
More docs to follow on that.
Limitations
This uses the multipart upload of s3 and its limits are
https://docs.aws.amazon.com/AmazonS3/latest/dev/qfacts.html
TODO: give details on how it deals with files smaller then 5MB because
it is handled differently.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
s3-concat-0.0.3.tar.gz
(4.1 kB
view hashes)
