Skip to main content

A gevent based python client for NSQ.

Project description

https://img.shields.io/pypi/v/gnsq.svg?style=flat https://img.shields.io/travis/wtolson/gnsq/master.svg?style=flat https://img.shields.io/pypi/dm/gnsq.svg?style=flat

A gevent based python client for NSQ distributed messaging platform.

Features include:

  • Free software: BSD license

  • Documentation: https://gnsq.readthedocs.org

  • Battle tested on billions and billions of messages </sagan>

  • Based on gevent for fast concurrent networking

  • Fast and flexible signals with Blinker

  • Automatic nsqlookupd discovery and back-off

  • Support for TLS, DEFLATE, and Snappy

  • Full HTTP clients for both nsqd and nsqlookupd

Installation

At the command line:

$ easy_install gnsq

Or even better, if you have virtualenvwrapper installed:

$ mkvirtualenv gnsq
$ pip install gnsq

Currently there is support for Python 2.6+, Python 3.3+ and PyPy.

Usage

First make sure nsq is installed and running. Next create a nsqd connection and publish some messages to your topic:

import gnsq
conn = gnsq.Nsqd(address='localhost', http_port=4151)

conn.publish('topic', 'hello gevent!')
conn.publish('topic', 'hello nsq!')

Then create a Reader to consume messages from your topic:

reader = gnsq.Reader('topic', 'channel', 'localhost:4150')

@reader.on_message.connect
def handler(reader, message):
    print 'got message:', message.body

reader.start()

Dependencies

Optional snappy support depends on the python-snappy package which in turn depends on libsnappy:

# Debian
$ sudo apt-get install libsnappy-dev

# Or OS X
$ brew install snappy

# And then install python-snappy
$ pip install python-snappy

Contributing

Feedback, issues, and contributions are always gratefully welcomed. See the contributing guide for details on how to help and setup a development environment.

History

0.3.2 (2016-04-10)

  • Add support for Python 3 and PyPy.

  • #7 - Fix undeclared variable in compression socket.

0.3.1 (2015-11-06)

  • Fix negative in flight causing not throtteling after backoff.

0.3.0 (2015-06-14)

  • Fix extra backoff success/failures during backoff period.

  • Fix case where handle_backoff is never called.

  • Add backoff parameter to message.requeue().

  • Allow overriding backoff on NSQRequeueMessage error.

  • Handle connection failures while starting/completing backoff.

0.2.3 (2015-02-16)

  • Remove disconnected nsqd messages from the worker queue.

  • #4 - Fix crash in Reader.random_ready_conn (thanks @ianpreston).

0.2.2 (2015-01-12)

  • Allow finishing and requeuing in sync handlers.

0.2.1 (2015-01-12)

  • Topics and channels are now valid to 64 characters.

  • Ephemeral topics are now valid.

  • Adjustable backoff behaviour.

0.2.0 (2014-08-03)

  • Warn on connection failure.

  • Add extra requires for snappy.

  • Add support for nsq auth protocal.

0.1.4 (2014-07-24)

  • Preemptively update ready count.

  • Dependency and contributing documentation.

  • Support for nsq back to 0.2.24.

0.1.3 (2014-07-08)

  • Block as expected on start, even if already started.

  • Raise runtime error if starting the reader without a message handler.

  • Add on_close signal to the reader.

  • Allow upgrading to tls+snappy or tls+deflate.

0.1.2 (2014-07-08)

  • Flush delfate buffer for each message.

0.1.1 (2014-07-07)

  • Fix packaging stream submodule.

  • Send queued messages before closing socket.

  • Continue to read from socket on EAGAIN

0.1.0 (2014-07-07)

  • First release on PyPI.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gnsq-0.3.2.tar.gz (37.7 kB view hashes)

Uploaded Source

Built Distribution

gnsq-0.3.2-py2.py3-none-any.whl (25.9 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page