Distribute commands to many nodes
Project description
cooperate is a shell command that execute commands in a cooperative manner, by distributing them to many nodes.
It requires Python >= 3.3 and asyncio.
For example:
cooperate --local -- echo FOO
Will execute the job “echo FOO” locally.
This one:
cooperate --ssh me@my.node -- echo FOO
Is barelly equivalent to:
ssh me@my.node echo FOO
You can declare as many nodes as you want. For example:
cooperate --local --ssh me@my.node --ssh me@my.second.node -- echo FOO
Is equivalent to:
echo FOO ssh me@my.node echo FOO ssh me@my.second.node echo FOO
You can also declare many jobs at once. For example:
cooperate --local --command "echo FOO" --command "echo BAR"
Is equivalent to:
echo FOO echo BAR
Installation
pip install cooperate
Nodes
Commands can be distribued thru these kind of nodes:
–local execute locally
–ssh execute thru ssh
–docker execute in a local docker container
–lxc execute in a local lxc container
These options can be repeated as often as needed.
Modes
By default, it spawns every commands to every nodes.
The -m, –mode allow to configure the desired mode.
The all mode executes all commands in all nodes:
cooperate --local --ssh me@my.node --ssh me@my.second.node \ --command="echo FOO" --command="echo BAR"
Is equivalent to:
echo FOO echo BAR ssh me@my.second.node echo FOO ssh me@my.second.node echo BAR
The distribute mode share the commands out among all the nodes:
cooperate --local --ssh me@my.node --ssh me@my.second.node \ --command="echo FOO" --command="echo BAR" --mode=distribute
Is equivalent to:
echo FOO ssh me@my.second.node echo BAR
Concurrency
By default, it executes all jobs simultaneously.
The -b, –batch option allows to execute on only a specify number of jobs at a time. Both percentages and finite numbers are supported:
cooperate --local --concurrence 1 \ --command="echo FOO" --command="echo BAR" --command="echo BAZ" cooperate --local --concurrence 33% \ --command="echo FOO" --command="echo BAR" --command="echo BAZ"
The concurrency system maintains a window of running jobs. When a job returns then it starts a remnant job and so on.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cooperate-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2350196d588470cfb4991870ef3cd47fd95630e4ce053d9200ed4e5d7e50ce2 |
|
MD5 | 6bdf5732b8861203e86e7bc6064fc757 |
|
BLAKE2b-256 | 140864479fde167119728397272a8fd46204e869742725ccbbdf10618373e62e |