Skip to main content

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.

Project description

A pickleable wrapper for sharing NumPy ndarrays between processes using POSIX shared memory.

SharedNDArrays are designed to be sent over multiprocessing.Pipe and Queue without serializing or transmitting the underlying ndarray or buffer. While the associated file descriptor is closed when the SharedNDArray is garbage collected, the underlying buffer is not released when the process ends: you must manually call the unlink() method from the last process to use it.

Usage

from __future__ import print_function

import multiprocessing as mp

import numpy as np
from shared_ndarray import SharedNDArray

try:
    shm = SharedNDArray((4, 4))
    shm.array[0, 0] = 1
    p = mp.Process(target=lambda shm: print(shm.array), args=(shm,))
    p.start()
    p.join()
finally:
    shm.unlink()

This should print:

[[ 1.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]

There are also convenience methods to create a new SharedNDArray from an existing NumPy array:

arr = np.array([0, 0])
shm1 = SharedNDArray.copy(arr)
shm2 = SharedNDArray.zeros_like(arr)
shm1.unlink()
shm2.unlink()

Dependencies

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shared-ndarray-1.1.1.tar.gz (3.3 kB view hashes)

Uploaded Source

Built Distributions

shared_ndarray-1.1.1-py3-none-any.whl (6.0 kB view hashes)

Uploaded Python 3

shared_ndarray-1.1.1-py2-none-any.whl (6.0 kB view hashes)

Uploaded Python 2

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page