A Client for managing batch interactions with AWS services
Project description
Boto3 Batch Utils
This library offers some functionality to assist in writing records to AWS services in batches, where your data is not naturally batched. This helps to achieve significant efficiencies when interacting with those AWS services as batch writes are often much more efficient than individual writes.
Installation
The package can be installed using pip:
pip install boto3-batch-utils
You may install a specific version of the package:
pip install boto3-batch-utils==3.0.0
Boto3 and Configuration
Boto3 Batch Utils is an abstraction around AWS' Boto3 library. boto3 is a dependency and will be installed
automatically, if it is not already present.
You will need to configure your AWS credentials and roles in exactly the same way as you would if using boto3
directly.
For more information on boto3 configuration, refer to the AWS documentation
here.
Concepts
The library is very simple to use. To use it, you must initialise a client, send it the payloads you want to transmit and finally tell the client to clear down.
To use the package you do not need care how to batch up the payloads and send them into their target service. The
package will take care of this for you. This allows you to utilise the significant efficiencies of boto3's batch
send/put/write methods, without the headaches of error handling and batch sizes.
Each of the supported services has it's own dispatcher client. Each has the same 2 methods with which to interact. So interacting with each of the various service clients is similar and follows the same 3 steps:
- Initialise: Instantiate the batch dispatcher, passing in the required configuration.
- submit_payload: pass in a payload (e.g. a single message, metric etc).
- flush_payloads: send all payloads in the backlog.
If you are using
boto3-batch-utilsin AWS Lambda, you should call.flush_payloads()at the end of every invocation.
Documentation
Full documentation is available here: boto3-batch-utils Docs
Contributing
For more information about contributing to this project, please refer to the Developer documentation.
Acknowledgements
[ ~ Dependencies scanned by PyUp.io ~ ]
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for boto3_batch_utils-5.1.0-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | edebf29124067a5a1f96531f41e6e89e5b3f064f03aa112728c533fd324a6200 |
|
| MD5 | 8a1597da771afc97008915eb30ed7904 |
|
| BLAKE2b-256 | efd8018860beceed782ed70735f386209ced802752c061b6d823d8e728a65c6b |
