Skip to main content

Python package for splitting arrays into sub-arrays (i.e. rectangular-tiling and rectangular-domain-decomposition), similar to ``numpy.array_split``.

Project description

array_split python package Build Status Documentation Status Coveralls Status MIT License array_split python package

The array_split python package is a modest enhancement to the numpy.array_split function for sub-dividing multi-dimensional arrays into sub-arrays (slices). The main motivation comes from parallel processing where one desires to split (decompose) a large array (or multiple arrays) into smaller sub-arrays which can be processed concurrently by other processes (multiprocessing or mpi4py) or other memory-limited hardware (e.g. GPGPU using pyopencl, pycuda, etc).

Quick Start Example

>>> from array_split import array_split, shape_split
>>> import numpy as np
>>>
>>> ary = np.arange(0, 4*9)
>>>
>>> array_split(ary, 4) # 1D split into 4 sections (like numpy.array_split)
[array([0, 1, 2, 3, 4, 5, 6, 7, 8]),
 array([ 9, 10, 11, 12, 13, 14, 15, 16, 17]),
 array([18, 19, 20, 21, 22, 23, 24, 25, 26]),
 array([27, 28, 29, 30, 31, 32, 33, 34, 35])]
>>>
>>> shape_split(ary.shape, 4) # 1D split into 4 sections, slice objects instead of numpy.ndarray views
array([(slice(0, 9, None),), (slice(9, 18, None),), (slice(18, 27, None),), (slice(27, 36, None),)],
      dtype=[('0', 'O')])
>>>
>>> ary = ary.reshape(4, 9) # Make ary 2D
>>> split = shape_split(ary.shape, axis=(2, 3)) # 2D split into 2*3=6 sections
>>> split.shape
(2, 3)
>>> split
array([[(slice(0, 2, None), slice(0, 3, None)),
        (slice(0, 2, None), slice(3, 6, None)),
        (slice(0, 2, None), slice(6, 9, None))],
       [(slice(2, 4, None), slice(0, 3, None)),
        (slice(2, 4, None), slice(3, 6, None)),
        (slice(2, 4, None), slice(6, 9, None))]],
      dtype=[('0', 'O'), ('1', 'O')])
>>> sub_arys = [ary[tup] for tup in split.flatten()] # Split ary into sub-array views using the slice tuples.
>>> sub_arys
[array([[ 0,  1,  2], [ 9, 10, 11]]),
 array([[ 3,  4,  5], [12, 13, 14]]),
 array([[ 6,  7,  8], [15, 16, 17]]),
 array([[18, 19, 20], [27, 28, 29]]),
 array([[21, 22, 23], [30, 31, 32]]),
 array([[24, 25, 26], [33, 34, 35]])]

Latest sphinx documentation examples at http://array-split.readthedocs.io/en/latest/examples/.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

array_split-0.1.2.tar.gz (23.9 kB view hashes)

Uploaded Source

Built Distributions

array_split-0.1.2-py3.5.egg (54.6 kB view hashes)

Uploaded Source

array_split-0.1.2-py3.4.egg (54.7 kB view hashes)

Uploaded Source

array_split-0.1.2-py3.3.egg (55.4 kB view hashes)

Uploaded Source

array_split-0.1.2-py2.7.egg (53.6 kB view hashes)

Uploaded Source

array_split-0.1.2-py2.6.egg (53.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page