Skip to main content

PlanOut is a framework for online field experimentation.

Project description

# PlanOut

PlanOut is a Python-based toolkit and language for online field experimentation. PlanOut was created to make it easy to run more sophisticated experiments and to quickly iterate on these experiments, while satisfying the constraints of deployed Internet services with many users.

Developers integrate PlanOut by defining experiments that detail how _units_ (e.g., users, cookie IDs) should get mapped onto conditions. For example, to create a 2x2 experiment randomizing both the color and the text on a button, you create a class like this:

```python
class MyExperiment(SimpleExperiment):
def assign(self, params, userid):
params.button_color = UniformChoice(choices=['#ff0000', '#00ff00'], unit=userid)
params.button_text = UniformChoice(choices=['I voted', 'I am a voter'], unit=userid)
```

Then, in the application code, you query the Experiment object to find out what values the current user should be mapped onto:
```python
my_exp = MyExperiment(userid=101)
color = my_exp.get('button_color')
text = my_exp.get('button_text')
```

PlanOut takes care of randomizing each ``userid`` into the right bucket. It does so by hashing the input, so each ``userid`` will always map onto the same values for that experiment.

This release currently includes:
* An extensible Python class for defining experiments. This class comes with objects that make it easy to implement randomized assignments, and automatically logs key data.
* An extensible class for managing multiple mutually exclusive experiment namespaces.
* The reference implementation of the PlanOut interpreter, which executes serialized code generated by the PlanOut domain specific language.

### Who is PlanOut for?
This software release is a Python implementation of PlanOut designed for researchers, students, and small businesses wanting to run experiments. It is designed to be extensible, so that it may be adapted for use with large production environments. The implementation here mirrors many of the key components of Facebook's PHP implementation of PlanOut which is used to conduct experiments with hundreds of millions of users.

### Full Example

To create a basic PlanOut experiment, you subclass ``SimpleExperiment`` object, and implement an assignment method. You can use PlanOut's random assignment operators by setting ``e.varname``, where ``params`` is the first argument passed to the ``assign()`` method, and ``varname`` is the name of the variable you are setting.
```python

from planout.experiment import SimpleExperiment
from planout.ops.random import *

class FirstExperiment(SimpleExperiment):
def assign(self, params, userid):
params.button_color = UniformChoice(choices=['#ff0000', '#00ff00'], unit=userid)
params.button_text = WeightedChoice(
choices=['Join now!', 'Sign up.'],
weights=[0.3, 0.7], unit=userid)

my_exp = FirstExperiment(userid=12)
# parameters may be accessed via the . operator
print my_exp.get('button_text'), my_exp.get('button_color')

# experiment objects include all input data
for i in xrange(6):
print FirstExperiment(userid=i)
```

which outputs:
```
Join now! #ff0000
{'inputs': {'userid': 0}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#ff0000', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 1}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#ff0000', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 2}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#00ff00', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 3}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#ff0000', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 4}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#00ff00', 'button_text': 'Join now!'}}
{'inputs': {'userid': 5}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#00ff00', 'button_text': 'Sign up.'}}
```

The ``SimpleExperiment`` class will automatically concatenate the name of the experiment, ``FirstExperiment``, the variable name, and the input data (``userid``) and hash that string to perform the random assignment. Parameter assignments and inputs are automatically logged into a file called ``firstexperiment.log'``.

### Installation
Check out the github and type:

```
sudo python setup.py install
```

### Learn more
Learn more about PlanOut by [reading the PlanOut paper](http://www-personal.umich.edu/~ebakshy/planout.pdf). You can cite PlanOut as "Designing and Deploying Online Field Experiments". Eytan Bakshy, Dean Eckles, Michael S. Bernstein. Proceedings of the 23rd ACM conference on the World Wide Web. April 7–11, 2014, Seoul, Korea, or by copying and pasting the bibtex below:
``` bibtex
@inproceedings{bakshy2014www,
Author = {Bakshy, E. and Eckles, D. and Bernstein, M.S.},
Booktitle = {Proceedings of the 23rd ACM conference on the World Wide Web},
Organization = {ACM},
Title = {Designing and Deploying Online Field Experiments},
Year = {2014}}
```

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

PlanOut-0.1.0.tar.gz (14.7 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page