skip to navigation
skip to content

Not Logged In

redis-shard 0.3.2

Redis Sharding API

Redis Shard
###########

.. image:: https://img.shields.io/travis/zhihu/redis-shard.svg?style=flat
:target: https://travis-ci.org/zhihu/redis-shard
:alt: Build Status

.. image:: https://pypip.in/version/redis-shard/badge.svg?style=flat
:target: https://pypi.python.org/pypi/redis-shard
:alt: Latest Version

.. image:: https://pypip.in/py_versions/redis-shard/badge.svg?style=flat
:target: https://pypi.python.org/pypi/redis-shard
:alt: Supported Python versions

.. image:: https://pypip.in/license/redis-shard/badge.svg?style=flat
:target: https://pypi.python.org/pypi/redis-shard
:alt: License

A Redis sharding implementation.

Redis is great. It's fast, lightweight and easy to use. But when we want to store
a mass of data into one single instance, we may encounter some problems such as performance
degradation and slow recovery and we need to scale it.

Usage
=====

First, Create an RedisShardAPI instance with multiple nodes, node ``name`` **must be unique**::

from redis_shard.shard import RedisShardAPI

servers = [
{'name': 'server1', 'host': '127.0.0.1', 'port': 10000, 'db': 0},
{'name': 'server2', 'host': '127.0.0.1', 'port': 11000, 'db': 0},
{'name': 'server3', 'host': '127.0.0.1', 'port': 12000, 'db': 0},
]

client = RedisShardAPI(servers, hash_method='md5')

Then, you can access the Redis cluster as you use `redis-py <https: github.com="" andymccurdy="" redis-py="">`_::

client.set('test', 1)
client.get('test') # 1
client.zadd('testset', 'first', 1)
client.zadd('testset', 'second', 2)
client.zrange('testset', 0, -1) # [first, second]


Hash Tags
---------

If you want to store specific keys on one node for some reason (such as you prefer single instance pipeline, or
you want to use multi-keys command such as ``sinter``), you should use Hash Tags::

client.set('foo', 2)
client.set('a{foo}', 5)
client.set('b{foo}', 5)
client.set('{foo}d', 5)
client.set('d{foo}e', 5)

client.get_server_name('foo') == client.get_server_name('a{foo}') == client.get_server_name('{foo}d') \
== client.get_server_name('d{foo}e') # True

The string in a braces of a key is the Hash Tag of the key. The hash of a Hash Tag will be treated the hash of the key.
So, keys ``foo``, ``bar{foo}`` and ``b{foo}ar`` will be sotred in the same node.

Tag method
~~~~~~~~~~~

Just add ``tag_`` prefix, you can use any of the normal redis method on the same hash tag::

client.tag_mget("{user:1}question1", "{user:1}question2")
client.tag_delete("{user:1}question1", "{user:1}question2")


Config Details
--------------
There are three parameters ``servers``, ``hash_method`` and ``sentinel`` in the :class:`redis_shard.shard.RedisShardAPI`.

``servers`` is a list. Each element in it should be a dict or a URL schema.

- dict::

[
{'name': 'server1', 'host': '127.0.0.1', 'port': 10000, 'db': 0},
{'name': 'server2', 'host': '127.0.0.1', 'port': 11000, 'db': 0},
{'name': 'server3', 'host': '127.0.0.1', 'port': 12000, 'db': 0},
]

- URL schema::

[
'redis://127.0.0.1:10000/0?name=node1',
'redis://127.0.0.1:11000/0?name=node2',
'redis://127.0.0.1:12000/0?name=node3'
]

If the following parameter ``sentinel`` is enabled, only **name** is needed for the ``servers`` config.

``hash_method`` is a string which indicate the method of generating the hash key of the consistent hash ring.
The default value is **crc32**. It also supports **md5** and **sha1**.


``sentinel`` is the config for `Redis Sentinel <http: redis.io="" topics="" sentinel="">`_. With the sentinel support, redis-shard
will do read/write splitting. Config is like this::

{"hosts": [('localhost', 26379)], "socket_timeout": 0.1}



Limitations
===========

* Redis Shard dose not support all Redis commands.
* As mentioned above, Redis Shard does not support multi-keys command crossing different nodes,
you have to use Hash Tag to work with those commands.
* Redis Shard does not have any replication mechanism.


How it Works
============

Redis Shard is basically inspired by `this article <http: oldblog.antirez.com="" post="" redis-presharding.html="">`_.


Change History
==============

0.3.2 (2015-06-02)
--------------------
- add strict redis support

0.3.1 (2015-02-02)
------------------
- transaction support

0.3.0 (2014-11-20)
------------------
- shard key generate function now support **md5** and **sha1** .
- add Redis Sentinel support

0.2.4 (2014-06-26)
------------------
- remove gevent dependency

0.2.3 (2014-06-02)
------------------
- better pipeline support

0.2.1 (2013-08-07)
------------------
- add evel method

0.2.0 (2013-06-02)
------------------
- add python3 support

0.1.11 (2013-05-06)
-------------------
- add mset support

0.1.10 (2013-04-13)
-------------------
- add mget support ,thks to @Yuekui

0.1.9 (2013-03-04)
------------------
- add an reshard example
- tidy the pipeline code
- add more shard methods

0.1.8 (2013-01-21)
------------------
- add `append` and `getrange` method, thks @simon-liu

0.1.7 (2012-11-19)
------------------
- use new redis url config instead of dict

0.1.5 (2012-07-16)
------------------
- Add many new methods, support socket_timeout and password

0.1.4 (2011-07-20)
------------------
- modify hash key algor, support suffix match, thks to dkong
- support more redis methods, include `keys`.

0.1.3 (2011-06-20)
------------------
- support 2.4.X version of redis-py

0.1.2 (2011-06-01)
------------------
- add MANIFEST.in file

0.1.1 (2011-05-29)
------------------
- create hashring use server's name config.

0.1 (2011-05-28)
----------------
- first version  
File Type Py Version Uploaded on Size
redis-shard-0.3.2.tar.gz (md5) Source 2015-06-02 12KB
  • Downloads (All Versions):
  • 142 downloads in the last day
  • 661 downloads in the last week
  • 2346 downloads in the last month