Skip to main content

Simple Redis work queue with added features (priorities, pop multiple items at once)

Project description

Pypi Build Status codecov MIT licensed

redis-priority-queue is a simple work queue similar to Redis lists with the following added features:

  • An item can be added with a priority (between -9007199254740992 and 9007199254740992)

  • Queues are automatically de-duplicated (duplicate items are voided when pushing them)

  • Multiple items can be popped from the queue at the same time

  • A queue monitoring tool to easily see how many items are in each queue

redis-priority-queue is based on Redis sorted sets and all sorted sets commands can be used alongside this project.

Clients

Basic usage

Bash example

-- Generic
redis-cli --eval src/redis-priority-queue.lua null null , [push|pop|peek|count] my_list (arg1, arg2...)

-- Push an item
redis-cli --eval src/redis-priority-queue.lua null null , push my_super_list my_item

-- Pop an item
redis-cli --eval src/redis-priority-queue.lua null null , pop my_super_list

push: Push an item in a queue

Usage

push my_list item [(int) priority (default: 100)]

Examples

-- Push an item with the default priority (100)
push my_super_list my_item

-- Push an item with a priority of 200
push my_super_list my_item 200

Return

Output is similar to ZADD

pop: Pop an item from the queue

Usage

pop my_list [(string) asc/desc (default: 'desc')] [(int) numer_of_items (default: 1')]

Examples

-- Pop 1 item ordered by descending priority
pop my_super_list

-- Pop 1 item ordered by ascending priority
pop my_super_list asc

-- Pop 5 items ordered by descending priority
pop my_super_list desc 5

Return

Output is similar to ZRANGEBYSCORE

peek: View a set of items from the list

Aliases: list, view

Usage

Same as pop but items are not removed from the list.

count: Count items in a queue

Alias: size

Usage

count my_list [(int) priority_min] [(int) priority_max]

Examples

-- Count all items from the list
count my_super_list

-- Count all items with a priority between 1 and 1110
count my_super_list 1 1110

Return

Output is similar to ZCOUNT

Queue monitoring

The queues can be easily monitored with the Python script src/queue_monitor.py

To use the queue monitor, you need to ensure python is installed and use the following command:

# Installation
pip3 install rpq

# Usage
rpq_monitor

Usage example

# Basic usage
rpq_monitor -H [host] -p [port] (-a [auth] -n [dbnum])
+-------------------+-------+-----------+----------+
| Queue name        | Total | Up to 100 | From 101 |
+-------------------+-------+-----------+----------+
| book_orders       |    44 |        12 |       32 |
| book_recycle      |   223 |       123 |      100 |
| book_returns      |    13 |        13 |        0 |
| late_fees_pending |   112 |        56 |       56 |
| new_books         | 1,144 |     1,120 |       24 |
+-------------------+-------+-----------+----------+

# Specify your own groups
rpq_monitor -H [host] -p [port] (-a [auth] -n [dbnum]) -s "[[0, 1000], [1001, 2000], [2001, 3000]]"
+-------------------+-------+------------+----------------+----------------+
| Queue name        | Total | 0 to 1,000 | 1,001 to 2,000 | 2,001 to 3,000 |
+-------------------+-------+------------+----------------+----------------+
| book_orders       |    44 |         24 |              9 |             11 |
| book_recycle      |   223 |        127 |             40 |             56 |
| book_returns      |    13 |         13 |              0 |              0 |
| late_fees_pending |   112 |         58 |             13 |             41 |
| new_books         | 1,144 |      1,142 |              2 |              0 |
+-------------------+-------+------------+----------------+----------------+

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rpq-2.2.tar.gz (8.0 kB view hashes)

Uploaded Source

Built Distribution

rpq-2.2-py2.py3-none-any.whl (8.9 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page