Dead simple task queue using redis
Project description
Dead simple task queue using redis.
# tasks.py
import dsq
manager = dsq.create_manager()
@manager.task(queue='normal')
def add(a, b):
print a + b
if __name__ == '__main__':
add(1, 2)$ python tasks.py
$ dsq worker -bt tasks normalSee full DSQ documentation.
Features
Low latency.
Expiring tasks (TTL).
Delayed tasks (ETA).
Retries (forever or particular amount).
Dead letters.
Queue priorities.
Worker lifetime.
Task execution timeout.
Task forwarder from one redis instance to another.
HTTP interface.
Inspect tools.
Supports 2.7, 3.4, 3.5 and PyPy.
100% test coverage.
The goal is a simple design. There is no worker manager, one can use supervisord/circus/whatever to spawn N workers. Simple storage model. Queue is a list and scheduled tasks are a sorted set. There are no task keys. Tasks are items of list and sorted set. There is no any registry to manage workers, basic requirements (die after some lifetime and do not hang) can be handled by workers themselves. Worker do not store result by default.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
