Skip to main content

gevent based Streaming API consumer

Project description

Abstract base classes for a long running, streaming API consumer and a specific implementation that puts the data into a redis queue and processes it in batches.

Requires gevent. The example implementation depends on redis.

Coded initially to send batches of statuses from the Twitter Streaming API to an appengine webhook. May (or may not) be usable for other purposes.

Presumes the data coming down the pipe is Chunked Transfer Encoded.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page