Skip to main content

Dump and load redis databases

Project description

https://api.travis-ci.org/p/redis-dump-load.png

Dumps Redis data sets into a format suitable for long-term storage (currently JSON) and loads data from such dump files back into Redis.

redis-dump was there first, but it is written in Ruby.

The output format is intended to be compatible with redis-dump.

Usage

redis-dump-load may be used as a module and also as a command line tool.

Module Usage

redis-dump-load exports a pickle-like interface, namely load, loads, dump and dumps functions. For example:

import redisdl

json_text = redisdl.dumps()

with open('path/to/dump.json', 'w') as f:
    # streams data
    redisdl.dump(f)

json_text = '...'
redisdl.loads(json_text)

with open('path/to/dump.json') as f:
    # currently does not stream data
    redisdl.load(f)

Note that while dump will stream data, load currently will not (load will read the entire file contents into a string, parse it, then walk the resulting data structure and load it into redis).

Dump and load methods accept options as keyword arguments:

json_text = redisdl.dumps(encoding='iso-8859-1', pretty=True)

The arguments should always be passed in as keyword, i.e, do not rely on the order in which the parameters are currently listed. Options take string values unless otherwise noted. The options are as follows:

  • host: host name or IP address for redis server

  • port: port number for redis server

  • unix_socket_path: connect to redis via a Unix socket instead of TCP/IP; specify the path to the socket

  • password: specify password to connect to redis

  • db (integer): redis database to connect to

  • encoding: encoding to use for encoding or decoding the data, see Unicode section below

  • pretty (boolean, dump only): produce a pretty-printed JSON which is easier to read; currently this makes dump load entire data set into memory rather than stream it

  • empty (boolean, load only): empty the redis data set before loading the data

Command Line Usage

redisdl.py can be used as a command line tool as follows:

# dump database 0
./redisdl.py > dump.json
./redisdl.py -o dump.json

# load into database 0
./redisdl.py -l < dump.json
./redisdl.py -l dump.json

For convenience, redisdl.py can be hard or soft linked as follows:

ln redisdl.py redis-dump
ln redisdl.py redis-load

Now it can be used thusly:

# dump database 0
./redis-dump > dump.json
./redis-dump -o dump.json

# load into database 0
./redis-load < dump.json
./redis-load dump.json

Symlinks work as well. “load” in the executable name triggers the loading mode, “dump” triggers the dumping mode, otherwise the default is to dump and -l option switches into the loading mode.

All options supported my the module API are accepted when redisdl is invoked as a command line tool. The command line options are:

  • -h/--help: help text

  • -H HOST/--host HOST: specify redis host

  • -p PORT/--port PORT: specify redis port

  • -s SOCKET_PATH/--socket SOCKET_PATH: connect to Unix socket at the specified path

  • -w PASSWORD/--password PASSWORd: password to use when connecting to redis

  • -d DATABASE/--db DATABASE: redis database to connect to (integer)

  • -E ENCODING/-encoding ENCODING: specify encoding to use

  • -o PATH/--output PATH: write dump to PATH rather than standard output

  • -y/--pretty (dumping only): pretty-print JSON

  • -e/--empty (loading only): empty redis data set before loading

Dependencies

Unicode

Redis operates on bytes and has no concept of Unicode or encodings. JSON operates on Unicode strings and cannot serialize binary data. Therefore, redis-dump-load has to encode Unicode strings into byte strings when loading data into Redis and decode byte strings into Unicode strings when dumping data from Redis. By default redis-dump-load uses utf-8 for encoding and decoding. This behavior matches py-redis, whose default encoding is utf-8. A different encoding can be specified.

Currently redis-py is broken on Python 3 with any encoding which is not a superset of ascii (https://github.com/andymccurdy/redis-py/issues/430). Data sets in such encodings can still be dumped and loaded on Python 2.

Concurrent Modifications

redis-dump-load does not lock the entire data set it is dumping, because Redis does not provide a way to do so. As a result, modifications to the data set made while a dump is in progress affect the contents of the dump.

License

Released under the 2 clause BSD license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

redis-dump-load-0.3.0.tar.gz (6.2 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page