caravan 0.2.2

About this Project

Caravan is being used at Ludia for projects like marketing campaign system and on-demand distributed processing systems (thanks to hundreds of Lambda functions).

Feedbacks, ideas and contributions are highly welcomed. (Just open a Github issue).

• Code on Github

• PyPi

• Tests

• Doc: TODO

Focus of this project

Similar projects exists (like Simpleflow which seems mature). Here is where Caravan differs from existing projects:

• Support of AWS Lambda tasks

• Boto3

• KISS

• Bring your own workflow framework (standard implementations as contribs)

• No coupling between Decider code and Activity code

• Paster compatible config file

Features

• Decider worker

• Activity task worker TODO

• Commands to start/signal/terminate an arbitrary workflow execution

• Command to list open workflow execution

• Command to register a domain / list domains

Configuration

Caravan uses Boto3 to connect to AWS. See Boto 3 configuration guide for the complete documentation.

[default]
# The access key for your AWS account
aws_access_key_id=<YOUR ACCESS KEY ID>

# The secret key for your AWS account
aws_secret_access_key=<YOUR SECRET KEY>

[default]
# The default region when making requests
region=<REGION NAME>

[profile dev-profile]
# The default region when using the dev-profile account
region=<REGION NAME>

Demo

Setup a SWF domain to run this example:

$ caravan-domain-register -n CaravanDemo --retention-days 1

Write a workflow type (see full demo)

from caravan import Workflow


class Demo(Workflow):

    """Noop workflow using the bare caravan API."""

    name = 'Demo'
    version = '0.1'
    default_execution_start_to_close_timeout = '600'
    default_task_start_to_close_timeout = '10'

    def run(self):
        self.task.print_events()
        self.task.add_decision('CompleteWorkflowExecution')

Run the decider with the Demo workflow:

$ caravan-decider -d CaravanDemo -m caravan.examples.demo -t default --verbose

Start an execution of the Demo workflow:

$ caravan-start -d CaravanDemo -n Demo -v 0.1 -i 1

(The Demo workflow will wait for 5 minutes)

List the executions:

$ caravan-list -d CaravanDemo
$ caravan-list -d CaravanDemo --oldest 2015-01-01

Send a signal to an execution:

$ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Hello World!'
$ caravan-signal -d CaravanDemo -i 1 -s PRINT --input 'Lorem ipsum'
$ caravan-signal -d CaravanDemo -i 1 -s STOP

Terminate an execution:

$ caravan-terminate -d CaravanDemo -i 1

Similar projects

Python:

• Simpleflow: https://github.com/botify-labs/simpleflow

• Flowy: https://github.com/severb/flowy

• Garcon: https://github.com/xethorn/garcon

Ruby:

• AWS Flow: https://github.com/aws/aws-flow-ruby

References

• AWS Developer Guide for SWF

• AWS API reference for SWF

Development

Possibly use virtualenvwrapper to manage your virtualenvs.

$ mkvirtualenv caravan

or

$ virtualenv caravan

Clone and install development dependencies:

(caravan)$ git clone git@github.com:pior/caravan.git
(caravan)$ cd caravan
(caravan)$ pip install -e .[dev]

Run tests:

(caravan)$ nosetests

Release

The release process use zest.releaser:

$ fullrelease

License

MIT licensed. See the bundled LICENSE file for more details