amqtt_db

DB persistence for amqtt.

Objective

amqtt_db persists payloads received by the amqtt broker into performant relational databases. SQLAlchemy as well as timescaleBD are the target RMDB-Systems.

amqtt_db will do four steps to persist the amqtt data:

1. decoding the payload (e.G. from binary, JSON or which ever encoding)
2. deserializing the payload to typed Python entities
3. structure the session, topic, property, value information into a relational model of your choice
4. generate the necessary tables, columns to store the data

All of this steps can be configured via the amqtt yaml config. And you can even replace any these steps for each topic by your code in terms of Python plugins. amqtt is designed to be enhanced and extended.

Performance

Flexibility comes with a penalty on performance. The more layers of classes and filters we implement the higher the performance penalty.

So we optimize the data flow by an optimistic approach.

amqtt_db expects that the decoding, deserializing, transformations, target DB, target tables, table colums etc. are all well in place if it deals with a single incoming packet. If the handling of that package fails, exceptions will be raised, and the error handling rushes in to deal with the problem.

Since the change rate on the decoding, deserializing, database model is quite low this optimistic approach will be quite performant.

Documentation

Please have a look at the documentation.