Skip to main content

Small utility library for coroutine-based asynchronous generic programming

Project description

paco Build Status PyPI Coverage Status Documentation Status Stability Quality Versions

Small and idiomatic utility library for coroutine-driven asynchronous generic programming in Python +3.4.

Built on top of asyncio, paco provides missing capabilities from Python stdlib in order to write asynchronous cooperative multitasking in a nice-ish way. Also, paco aims to port some of functools and itertools standard functions to the asynchronous world.

paco can be your utility belt to deal with asynchronous, I/O-bound, non-blocking concurrent code in a cleaner and idiomatic way.

Features

  • Simple and idiomatic API, extending Python stdlib with async coroutines gotchas.

  • Built-in configurable control-flow concurrency support (throttle).

  • No fancy abstractions: it just works with the plain asynchronous coroutines.

  • Useful iterables, decorators, functors and convenient helpers.

  • Coroutine-based functional helpers: compose, throttle, partial, timeout, times, until, race

  • Asynchronous coroutines port of Python built-in functions: filter, map, dropwhile, filterfalse, reduce

  • Supports asynchronous iterables and generators (PEP0525)

  • Concurrent iterables and higher-order functions.

  • Better asyncio.gather() and asyncio.wait() with optional concurrency control and ordered results.

  • Works with both async/await and yield from coroutines syntax.

  • Reliable coroutine timeout limit handler via context manager.

  • Designed for intensive I/O bound concurrent non-blocking tasks.

  • Good interoperability with asyncio and Python stdlib functions.

  • Composable pipelines of functors via | operator overloading.

  • Small and dependency free.

  • Compatible with Python +3.4.

Installation

Using pip package manager:

pip install --upgrade paco

Or install the latest sources from Github:

pip install -e git+git://github.com/h2non/paco.git#egg=paco

API

Examples

Asynchronously and concurrently execute multiple HTTP requests.

import paco
import aiohttp

async def fetch(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as res:
            return res

async def fetch_urls():
    urls = [
        'https://www.google.com',
        'https://www.yahoo.com',
        'https://www.bing.com',
        'https://www.baidu.com',
        'https://duckduckgo.com',
    ]

    # Map concurrent executor with concurrent limit of 3
    responses = await paco.map(fetch, urls, limit=3)

    for res in responses:
        print('Status:', res.status)

# Run in event loop
paco.run(fetch_urls())

Concurrent pipeline-style composition of transform functors over an iterable object.

import paco

async def filterer(x):
    return x < 8

async def mapper(x):
    return x * 2

async def drop(x):
    return x < 10

async def reducer(acc, x):
    return acc + x

async def task(numbers):
    return await (numbers
                   | paco.filter(filterer)
                   | paco.map(mapper)
                   | paco.dropwhile(drop)
                   | paco.reduce(reducer, initializer=0))

# Run in event loop
number = paco.run(task((1, 2, 3, 4, 5, 6, 7, 8, 9, 10)))
print('Number:', number) # => 36

License

MIT - Tomas Aparicio

History

v0.2.3 / 2018-10-23

  • Merge pull request #42 from dylanjw/fix_syntax_error

  • Use getattr to avoid async keyword

0.2.2 / 2018-10-09

  • Merge pull request #40 from thatmattbone/master

  • add loop param to paco.interval()

  • fix(setup): use space based indentation

  • fix(travis): use cpython 3.7-dev release

  • refactor(errors): use “paco” prefix in exception messages

  • chore(History): add version changes

v0.2.1 / 2018-03-21

  • fix(#37): allow overload function signatures with variadic arguments

  • refactor(timeout_test): remove print statement

  • fix(docs): bad link to API reference

  • refactor(docs): remove codesponsor

v0.2.0 / 2017-10-21

  • refactor(api): API breaking change that modifies behavior by raising any legit exception generated by a coroutine.

  • feat(examples): add examples file

  • feat(v2): v2 pre-release, propagate raise exception if return_exceptions is False

  • refactor(tests): add map error exception assertion test

  • Merge branch ‘master’ of https://github.com/h2non/paco

  • refactor(tests): add map error exception assertion test

  • feat(docs): add sponsor banner

  • feat(docs): add sponsor banner

  • feat(LICENSE): update copyright year

  • Update setup.py

v0.1.11 / 2017-01-28

  • feat(api): add paco.interval function.

v0.1.10 / 2017-01-11

  • fix(each.py,map.py): fixed return_exceptions kwarg

  • fix(setup.py): add author email

  • fix(Makefile): remove package file

v0.1.9 / 2017-01-06

  • feat(api): add identity function

  • feat(#31): add thunk function

  • feat(package): add wheel package distribution

  • refactor(wraps): simplify implementation

  • fix(History): remove indentation

v0.1.8 / 2016-12-29

  • feat(requirements): force upgrade setuptools

  • feat(#29): support async iterable and generators

  • fix(docs): link to examples

  • chore(travis): use Python 3.6 stable release

0.1.7 / 2016-12-18

  • feat(#26): add curry function.

0.1.6 / 2016-12-11

  • feat(pipe): isolate pipe operator overload code

  • refactor: decorator and util functions

  • feat(#11): timeout limit context manager.

  • refactor(core): several minor refactors

  • fix(docs): comment out latex sphinx settings

  • fix(docs): use current package version

  • Documentation examples improvements (#27)

  • feat(history): update

  • feat: add pool length magic method

0.1.5 (2016-12-04)

  • fix(#25): allow empty iterables in iterators functions, such as map, filter, reduce.

0.1.4 (2016-11-28)

  • fix(#24): explicitly pass loop instance to asyncio.wait.

0.1.3 (2016-10-27)

  • feat(#17): add flat_map function.

  • feat(#18): add pipeline-style operator overloading composition.

0.1.2 (2016-10-25)

  • fix(setup.py): fix pip installation.

  • refactor(api): minor refactors in several functions and tests.

0.1.1 (2016-10-24)

  • refactor(name): use new project name.

0.1.0 (2016-10-23)

  • First version (beta)

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

paco-0.2.3.tar.gz (26.3 kB view hashes)

Uploaded Source

Built Distribution

paco-0.2.3-py2.py3-none-any.whl (46.9 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page