Skip to main content
PyCon US is happening May 14th-22nd in Pittsburgh, PA USA.  Learn more

Data processing in distributed environments

Project description

PyPI release Conda version Build status BSD license Supported Python versions Code style: black Language grade: Python Total alerts

Workflows enables light-weight services to process tasks in a message-oriented environment.

It is comprised of a communications layer (workflows.transport) that provides a common interface to queues and topics over different transport providers, a service abstraction layer (workflows.frontend) which encapsulates a service in a separate process, tools for status reporting, logging and controlling services via a transport mechanism, and a service class skeleton alongside example services (workflows.services.common_service et al.). To achieve more complex workflows services can be interconnected using recipes whereby the output of one service can be directed onwards to other services.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page