Skip to main content

A full async docsplit previewer server based on Tornado and Redis

Project description

Introduction

We are working to find the best solution for our previewer media server.

The case is simple, we have PDF and we want to preview them (small, medium, large, header).

We first started with Insight which allows people to ask for sync and async previews generations from an url with caching.

The system is able to register engines so that we can manipulate documents on the flow.

The idea of Insight-Reloaded is to remove the sync mode and delegate it to Nginx with disk or S3 storage.

If the document preview is not ready, we want a 404 error.

We will get the document preview url with a callback.

The API

Simple example

curl -X GET "http://localhost:8888/?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg"
{"insight_reloaded": "Job added to queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/status
{"insight_reloaded": "There is 14 job in the queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/
{"version": "1.5", "insight_reloaded": "Bonjour", "name": "insight-reloaded"}

Multi-queues example

You must to configure REDIS_QUEUE_KEYS in your settings.py.

REDIS_QUEUE_KEYS = ['urgent', 'normal']
DEFAULT_REDIS_QUEUE_KEY = 'normal'

Then you can use:

curl -X GET "http://localhost:8888/urgent?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg"
{"insight_reloaded": "Job added to queue 'normal'.", "number_in_queue": 14}

curl -X GET http://localhost:8888/urgent/status
{"insight_reloaded": "There is 14 job in the 'urgent' queue.", "number_in_queue": 14}

curl -X GET http://localhost:8888/
{"version": "1.5", "insight_reloaded": "Bonjour", "name": "insight-reloaded"}

Set hash

By default, insight compute preview hash using url (sha1(url)). You can also send it within the request if you want, be sure that each preview has an uniq hash:

curl -X GET "http://localhost:8888/?url=http://my_file_url.com/file.pdf&callback=http://requestb.in/12vsewg?hash=123456789"
{"insight_reloaded": "Job added to queue.", "number_in_queue": 14}

Service architecture

https://raw.github.com/novapost/insight-reloaded/master/docs/_static/InsightReloaded.png

Server provisioning

You can find insight-reloaded chef cookbooks here : https://github.com/novapost/insight-installer

This will helps you install all requirements to run your insight-reloaded server.

Run the tests

make test

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

insight_reloaded-1.6.4.tar.gz (31.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page