Run multiple subprocesses asynchronous/in parallel with streamed output/non-blocking reading
Project description
shelljob
A simple way to manage several parallel subprocesses. This provides for asynchronous processes and non-blocking reading of their output.
Parallel Subprocesses
Using the Job system is the quickest approach to just run processes and log their output (by default in files named ‘/tmp/job_ID.log’)
from shelljob import job
jm = job.FileMonitor()
jm.run([
[ 'ls', '-alR', '/usr/local' ],
[ 'my_prog' ],
[ 'build', 'output', 'input' ],
])
Note the command items are lists passed directly to subprocess.Popen.
The lower level Group class provides a simple container for more manual job management.
from shelljob import proc
g = proc.Group()
p1 = g.run( [ 'ls', '-al', '/usr/local' ] )
p2 = g.run( [ 'program', 'arg1', 'arg2' ] )
while g.is_pending():
lines = g.readlines()
for proc, line in lines:
sys.stdout.write( "{}:{}".format( proc.pid, line ) )
Simple Subprocess calls
A simplified call function allows timeouts on subprocesses and easy acces to their output.
from shelljob import proc # capture the output output = proc.call( 'ls /tmp' ) # this raises a proc.Timeout exception proc.call( 'sleep 10', timeout = 0.1 )
Issues
You can use my Launchpad project to submit issues.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
