techies 0.1.4

Queue Implementations (backed by Redis)

1. techies.Queue, based on Redis List. Interfaces are almost standard queue compatible.

2. techies.UniQueue, based on Redis Sorted Set. Inherits techies.Queue but ignores repetitive items, keeps items unique. Score of the sorted set member is epoch timestamp from time.time().

3. techies.CountQueue, based on Redis Sorted Set. Inherits techies.UniQueue but score is used as a count of item appearance, that the item has the highest count gets placed in front to be get first.

logging.Handler Implementation

1. techies.QueueHandler, inherits standard logging.Handler that emit to any standard Queue compatible implementations, including all the Queue implementations in this library.

Misc Tools

1. techies.StateCounter, based on Redis Hash. To see an example of its usage, see `tidehunter <https://github.com/woozyking/tidehunter#example-2-without-limit>`__.

Queue

from techies import Queue

q = Queue(key='demo_q', host='localhost', port=6379, db=0)

# put, or enqueue
q.put('lol')
q.put('dota')
q.put('skyrim')
q.put('dota')

# Check size of the queue, two ways
print(q.qsize())  # 4
print(len(q))  # 4

# get, or dequeue
print(q.get())  # 'lol'
print(q.get())  # 'dota'
print(q.get())  # 'skyrim'
print(q.get())  # 'dota'
print(q.get())  # ''

# clear the queue
q.clear()

UniQueue

from techies import UniQueue

q = UniQueue(key='demo_q', host='localhost', port=6379, db=0)

# put, or enqueue
q.put('lol')
q.put('dota')
q.put('skyrim')
q.put('dota')  # this one is ignored

# Check size of the unique queue, two ways
print(q.qsize())  # 3
print(len(q))  # 3

# get, or dequeue
print(q.get())  # 'lol'
print(q.get())  # 'dota'
print(q.get())  # 'skyrim'
print(q.get())  # ''  # only 3 unique items

# clear the queue
q.clear()

CountQueue

from techies import CountQueue

q = CountQueue(key='demo_q', host='localhost', port=6379, db=0)

# put, or enqueue
q.put('lol')
q.put('dota')
q.put('skyrim')
q.put('dota')  # increment the count of the existing 'dota'

# Check size of the unique queue, two ways
print(q.qsize())  # 3
print(len(q))  # 3

# get, or dequeue
print(q.get())  # ('dota', 2)  # the one with the most count is returned first
print(q.get())  # ('lol', 1)
print(q.get())  # ('skyrim', 1)
print(q.get())  # ()  # only 3 unique items still

# clear the queue
q.clear()

QueueHandler

import logging
from techies import (
    Queue, UniQueue, CountQueue, QueueHandler, REF_LOG_FORMAT
)
from techies.compat import xrange

if __name__ == '__main__':
    key = 'test_q'
    q = Queue(key=key, host='localhost', port=6379, db=0)
    uq = UniQueue(key=key, host='localhost', port=6379, db=1)
    cq = CountQueue(key=key, host='localhost', port=6379, db=2)

    logger = logging.getLogger(__name__)

    for i in [q, uq, cq]:
        handler = QueueHandler(i)
        handler.setFormatter(logging.Formatter(REF_LOG_FORMAT))
        logger.addHandler(handler)

    # Enqueue multiple times of the same error
    for i in xrange(3):
        try:
            1 / 0
        except ZeroDivisionError as e:
            logger.error(e)

    # simple queue, should print error log 3 times
    while len(q):
        print(q.get())

    # unique queue, should just have 1 item in this case
    print(len(uq) == 1)
    print(uq.get())

    # count queue, should just have 1 item as unique queue
    print(len(cq) == 1)
    print(cq.get()[1])  # 3, the count of the same item

    for i in [q, uq, cq]:
        i.clear()

0.1.4 (2014-01-22)

• Added StateCounter, a state counter based on Redis Hash. To see an example of its usage, see `tidehunter <https://github.com/woozyking/tidehunter#example-2-without-limit>`__.

• Included hiredis in requirements.txt for added performance gain.

0.1.3 (2014-01-20)

• Added QueueHandler, inherits standard logging.Handler that emit to any standard Queue compatible implementations, including all the Queue implementations in this library

• Exposed direct accessibility of all intended APIs, for example, from techies import Queue; or if you will, from techies import *

• Behavior of CountQueue.get has changed from just returning the item value to a tuple of item value and its number of appearances in the queue, eg: ('dota', 2); an empty tuple is returned when the CountQueue is empty

0.1.2 (2014-01-17)

• Added CountQueue, inherits UniQueue but the score is used as a count of item appearance, that the item has the highest count gets placed in front to be get first

• Behavior of all Queue and its subclasses’ put and put_nowait methods changed from return certain value to not return anything (more Python standard Queue like)

0.1.1 (2014-01-16)

• Python 3.2 and 3.3 supported now

• Behavior of both Queue and UniQueue’s get method changed from returning None to empty “native string” (unicode for Python 2.x, str for Python 3.x) when attempting to dequeue from an empty queue

0.1.0 (2014-01-16)

• Initial release with Redis backed Queue and UniQueue