skip to navigation
skip to content

Not Logged In

sharedmem 0.1

Easy routines for coding on sharedmem machines

Latest Version: 0.3

Dispatch your trivially parallizable jobs with sharedmem.

Now also supports Python 3.

  • sharedmem.empty creates numpy arrays to child processes.
  • sharedmem.MapReduce dispatches work to child processes.
  • sharedmem.MapReduce.ordered and sharedmem.MapReduce.critical provides the equivlant concept of OpenMP ordered and OpenMP critical sections.

Functions and variables are inherited from a fork and copy-on-write. Pickability is not a concern.

Easier to use than multiprocessing.Pool, at the cost of not supporting Windows.

For documentation, please refer to http://rainwoodman.github.io/sharedmem .

>>>
>>> input = numpy.arange(1024 * 1024 * 128, dtype='f8')
>>> output = sharedmem.empty(1024 * 1024 * 128, dtype='f8')
>>> with MapReduce() as pool:
>>>    chunksize = 1024 * 1024
>>>    def work(i):
>>>        s = slice (i, i + chunksize)
>>>        output[s] = input[s]
>>>        return i, sum(input[s])
>>>    def reduce(i, r):
>>>        print('chunk', i, 'done')
>>>        return r
>>>    r = pool.map(work, range(0, len(input), chunksize), reduce=reduce)
>>> print numpy.sum(r)
>>>
 
File Type Py Version Uploaded on Size
sharedmem-0.3.tar.gz (md5) Source 2015-07-31 12KB
  • Downloads (All Versions):
  • 44 downloads in the last day
  • 195 downloads in the last week
  • 195 downloads in the last month