Skip to main content

A faster python pickle replacement

Project description

This module can be used as transparent replacement for pickle.

Difference to original python pickle:

  • no memo attribute.

  • no clear_memo().

  • no persistent_id interface.

  • byte strings are always assumed to be utf-8 encoded.

  • Pickler has an additional with_refs parameter. Setting with_refs to false, the pickler will ignore object references. This can result in an extra speed boost.

Installation

larch-pickle needs the boost library for compilation. If boost is not in the standard include path install it with:

python build_ext -I /path/to/boost install

Speed compared to some other pickler packages

dump Dictionaries (10 loops)

Package

Seconds

Size

marshal

0.46141

15975952

larch-pickle

0.664333

9714411

ujson

1.94214

13141354

msgpack

2.04704

9843459

json

2.14117

14365311

cPickle

4.09599

12781866

load Dictionaries (10 loops)

Package

Seconds

msgpack

1.38378

marshal

1.49512

larch-pickle

1.51489

ujson

2.42431

cPickle

2.42524

json

6.04131

dump Objects (10 loops)

Package

Seconds

Size

larch-pickle

1.07443

10094415

ujson

2.55502

13331354

cPickle

4.70805

12961909

load Objects (10 loops)

Package

Seconds

larch-pickle

1.6615

ujson

2.45786

cPickle

2.58697

dump Strings (10 loops)

Package

Seconds

Size

marshal

0.56747

32826787

msgpack

0.975226

29464951

larch-pickle

1.16368

12103729

json

1.19005

31966498

ujson

1.51648

30987525

cPickle

4.12952

19871780

load Strings (10 loops)

Package

Seconds

larch-pickle

0.353255

marshal

0.435936

msgpack

0.469068

cPickle

1.72308

ujson

1.82656

json

3.01798

dump Lists (10 loops)

Package

Seconds

Size

marshal

0.861886

42703907

larch-pickle

2.33574

16054577

json

3.11054

35917346

ujson

4.4284

34938373

msgpack

5.55321

31440375

cPickle

15.5531

33700258

load Lists (10 loops)

Package

Seconds

larch-pickle

2.07305

marshal

2.14713

msgpack

2.46054

ujson

4.14067

json

5.20221

cPickle

8.1974

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

larch-pickle-1.1.3.tar.gz (3.9 MB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page