gevent-cooperative child processes and inter-process communication.
Project description
What can gipc do for you?
Naive usage of multiprocessing in the context of a gevent-powered application may raise various problems and most likely breaks the application in some way. That is where gipc comes into play: it is developed with the motivation to solve these issues transparently and make using gevent in combination with multiprocessing a no-brainer again.
With gipc (pronunciation “gipsy”) multiprocessing.Process-based child processes can safely be created anywhere within your gevent-powered application. Furthermore, gipc provides gevent-cooperative inter-process communication and useful helper constructs.
gipc is lightweight and very simple to integrate. In the following code snippet, a Python object is sent from a greenlet in the main process to a child process:
import gevent
import gipc
obj = 0
def child(reader):
assert reader.get() == obj
if __name__ == "__main__":
with gipc.pipe() as (reader, writer):
writelet = gevent.spawn(lambda w: w.put(obj), writer)
readchild = gipc.start_process(child, args=(reader,))
writelet.join()
readchild.join()
Can’t I do this with just gevent+multiprocessing?
It requires care: child process creation via multiprocessing in the context of gevent yields an undesired event loop state in the child. Greenlets spawned before forking are duplicated in the child. Furthermore, blocking method calls such as join() on a multiprocessing.Process or the send()/recv() methods on a multiprocessing.Connection are not gevent-cooperative. gipc overcomes these challenges for you transparently and in a straight-forward fashion. It allows for simple integration of child processes in your application – on POSIX-compliant systems as well as on Windows.
Documentation
The documentation with technical notes, API details, installation instructions, requirements, and code examples can be found at http://gehrcke.de/gipc.
Availability
Releases are available at PyPI. The development version can be received from the mercurial repository at bitbucket.
Contact
Your feedback is highly appreciated. You can contact me at jgehrcke@googlemail.com or use the Bitbucket issue tracker.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
