Skip to main content

Bonobo

Project description

🐵 bonobo
=========

Data-processing. By monkeys. For humans.

Bonobo is a data-processing library for python 3.5+ that emphasis writing
simple, atomic, plain old python functions and chaining them using a basic
acyclic graph. The nodes will need a bit of plumbery to be runnable in
different means (iteratively, in threads, in processes, on different machines
...) but that should be as transparent as possible.

The only thing asked to the developer is to either write "pure" functions to
process data (create a new dict, don't change in place, etc.), and everything
should be fine from this point.

It's a young rewrite of an old python2.7 tool that ran millions of
transformations per day for years on production, so as though it may not yet
be complete or fully stable (please, allow us to reach 1.0), the underlying
concepts work.

* Documentation: http://docs.bonobo-project.org/
* Release announcements: http://eepurl.com/csHFKL
* Old project (for reference, don't use anymore, instead, help us recode the missing parts in bonobo): http://etl.rdc.li/


.. image:: https://travis-ci.org/python-bonobo/bonobo.svg?branch=0.2
:target: https://travis-ci.org/python-bonobo/bonobo
:alt: Continuous Integration

.. image:: https://landscape.io/github/python-bonobo/bonobo/0.2/landscape.svg?style=flat
:target: https://landscape.io/github/python-bonobo/bonobo/0.2
:alt: Code Health

.. image:: https://img.shields.io/coveralls/python-bonobo/bonobo.svg
:target: https://coveralls.io/github/python-bonobo/bonobo?branch=0.2
:alt: Coverage

.. image:: https://readthedocs.org/projects/bonobo/badge/?version=0.2
:target: http://docs.bonobo-project.org/
:alt: Documentation

.. image:: https://img.shields.io/github/downloads/python-bonobo/bonobo/total.svg
:target: https://github.com/python-bonobo/bonobo/releases
:alt: Downloads

.. image:: https://img.shields.io/pypi/dm/bonobo.svg
:target: https://pypi.python.org/pypi/bonobo
:alt: Python Package on PyPI

----

Made with ♥ by `Romain Dorgueil <https://twitter.com/rdorgueil>`_ and `contributors <https://github.com/python-bonobo/bonobo/graphs/contributors>`_.

----

Roadmap (in progress)
:::::::::::::::::::::

Bonobo is young. This roadmap is alive, and will evolve. Its only purpose is to
write down incoming things somewhere.

Version 0.2
-----------

* Changelog
* Migration guide
* Update documentation
* Threaded does not terminate anymore (fixed ?)
* More tests

Bugs:

- KeyboardInterrupt does not work anymore. (fixed ?)
- ThreadPool does not stop anymore. (fiexd ?)

Configuration
.............

* Support for position arguments (options), required options are good candidates.

Context processors
..................

* Be careful with order, especially with python 3.5. (done)
* @contextual decorator is not clean enough. Once the behavior is right, find a
way to use regular inheritance, without meta.
* ValueHolder API not clean. Find a better way.

Random thoughts and things to do
................................

* Class-tree for Graph and Nodes

* Class-tree for execution contexts:

* GraphExecutionContext
* NodeExecutionContext
* PluginExecutionContext

* Class-tree for ExecutionStrategies

* NaiveStrategy
* PoolExecutionStrategy
* ThreadPoolExecutionStrategy
* ProcesPoolExecutionStrategy
* ThreadExecutionStrategy
* ProcessExecutionStrategy

* Class-tree for bags

* Bag
* ErrorBag
* InheritingBag

* Co-routines: for unordered, or even ordered but long io.

* "context processors": replace initialize/finalize by a generator that yields only once


* "execute" function:

.. code-block:: python

def execute(graph: Graph, *, strategy: ExecutionStrategy, plugins: List[Plugin]) -> Execution:
pass

* Handling console. Can we use a queue, and replace stdout / stderr ?

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bonobo-0.2.0.macosx-10.12-x86_64.tar.gz (121.3 kB view hashes)

Uploaded Source

Built Distributions

bonobo-0.2.0-py3.6.egg (162.6 kB view hashes)

Uploaded Source

bonobo-0.2.0-py2.py3-none-any.whl (111.6 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page