poopy 0.2

Poopy - An unusable map-reduce engine over AMPQ

Install

From PYPI

1. pip install poopy (please use virtualenv)

2. Download this file https://bitbucket.org/leliel12/poopy/raw/tip/example/iris.arff?at=default

3. Download this file https://bitbucket.org/leliel12/poopy/raw/tip/example/randomforest.py and put in the same directory of iris.arff

From REPO

1. Install rabitMQ

2. Clone this repo

3. pip install -e .

4. The archives iris.arff and randomforest.py are inside in examples/

Running

1. Open two consoles (consoleA, consoleB)

2. In consoleB run poopy deploy ampq://localhost

3. In consoleA execute poopy upload ampq://localhost path/to/iris.arff poopFS://iris.arff now your file are uploaded to the “distributed file sistem”

4. In consoleA run poopy run ampq://localhost path/to/randomforest.py Script out

5. Your output model are serialized in out/localtime

Reading the model

In python console

from poopy import serializers

with open("out/file") as fp:
    model = serializers.load(fp)

TODO:

• Implement correctly timeouts.

• Implement an error exchange.

• Real distributed file system.

• More than one map or reduce.

• More output formats than b64-pkl.

• Not use the central node as main memory.

• Reorder some modules into packages

In your directory if all work ok you will find a file with the text “hello world” inside.

PS: probably nothing works