Skip to main content

Django app for managing external side effects.

Project description

https://travis-ci.org/hugorodgerbrown/django-side-effects.svg?branch=master https://badge.fury.io/py/django-side-effects.svg https://codecov.io/gh/hugorodgerbrown/django-side-effectsside-effects/branch/master/graph/badge.svg

Django Side Effects

Django app for managing external side effects.

Background

This project was created to try and bring some order to the use of external side-effects within the YunoJuno platform. External side-effects are (as defined by us) those actions that affect external systems, and that are not part of the core application integrity. They fall into two main categories within our application - notifications and updates, and are best illustrated by example:

Notifications

  • HipChat messages

  • SMS (via Twilio)

  • Push notifications (via Google Cloud Messaging)

  • Email

Updates

  • Base CRM (sales)

  • Mailchimp CRM (marketing)

  • Elasticsearch (full-text index)

There are some shared aspects of all of these side-effects:

  1. They can all be processed asynchronously (queued)

  2. They can all be ‘replayed’ (and are idempotent)

  3. They can be executed in any order

  4. They are not time ‘critical’

  5. They do not affect the state of the Django application

As we have continued to build out YunoJuno our use of these side-effects has become ever more complex, and has in some areas left us with functions that are 80% side-effects:

def foo():
    # do the thing the function is supposed to do
    update_object(obj)
    # spend the rest of the function working out which side-effects to fire
    if settings.notify_account_handler:
        send_notification(obj.account_handler)
    if obj.has_changed_foo():
        udpate_crm(obj)

This results in a codebase is:

  • Hard to read

  • Hard to test

  • Hard to document^

^ Barely a week goes by without someone asking “what happens when X does Y - I thought they got email Z?”

Solution

This project aims to address all three of the issues above by:

  • Removing all side-effects code from core functions

  • Simplifying mocking / disabling of side-effects in tests

  • Simplifying testing of side-effects only

  • Automating documentation of side-effects

It does this with a combination of function decorators that can be used to build up a global registry of side-effects.

The first decorator, has_side_effects, is used to mark a function as one that has side effects:

# mark this function as one that has side-effects. The label
# can be anything, and is used as a dict key for looking up
# associated side-effects functions
@side_effects.decorators.has_side_effects('update_profile')
def foo(*args, **kwargs):
    pass

The second decorator, is_side_effect_of, is used to bind those functions that implement the side effects to the origin function:

# bind this function to the event 'update_profile'
@side_effects.decorators.is_side_effect('update_profile')
def send_updates(*args, **kwargs):
    """Update CRM system."""
    pass

# bind this function also to 'update_profile'
@side_effects.decorators.is_side_effect('update_profile')
def send_notifications(*args, **kwargs):
    """Notify account managers."""
    pass

In the above example, the updates and notifications have been separated out from the origin function, which is now easier to understand as it is only responsible for its own functionality. In this example we have two side-effects bound to the same origin, however this is an implementation detail - you could have a single function implementing all the side-effects, or split them out further into the individual external systems.

Internally, the app maintains a registry of side-effects functions bound to origin functions using the text labels. The docstrings for all the bound functions can be grouped using these labels, and then be printed out using the management command display_side_effects:

$ ./manage.py display_side_effects

This command prints out the first line from the docstrings of all functions
registered using the @is_side_effect decorator, grouped by label.

update_profile:

    - Update CRM system.
    - Notify account managers.

close_account:

    - Send confirmation email to user.
    - Notify customer service.

Why not use signals?

The above solution probably looks extremely familiar - and it is very closely related to the built-in Django signals implementation. In fact, under the hood we rely on signals to hand off between functions.

Installation

The project is available through PyPI as django-side-effects:

$ pip install django-side-effects

And the main package itself is just side_effects:

>>> from side_effects import decorators

Usage

TBC

Settings

The following settings can be specified as environment settings or within the Django settings.

TBC

Tests

The project has pretty good test coverage (>90%) and the tests themselves run through tox.

$ pip install tox
$ tox

If you want to run the tests manually, make sure you install the requirements, and Django.

$ pip install -r requirements.txt
$ pip install django==1.8  # your version goes here
$ python manage.py test onfido.tests

If you are hacking on the project, please keep coverage up.

Contributing

Standard GH rules apply: clone the repo to your own account, create a branch, make sure you update the tests, and submit a pull request.

Status

This project is very early in its development. We are using it at YunoJuno, but ‘caveat emptor’. It does what we need it to do right now, and we will extend it as we evolve. If you need or want additional features, get involved :-).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django-side-effects-0.1.tar.gz (7.6 kB view hashes)

Uploaded Source

Built Distribution

django_side_effects-0.1-py2-none-any.whl (12.3 kB view hashes)

Uploaded Python 2

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page