skip to navigation
skip to content

Not Logged In

drf_to_s3 0.7.1

Django REST Framework Interface for direct upload to S3

Latest Version: 0.7.7

drf-to-s3
=========

`Django REST Framework <http://django-rest-framework.org/>`__ interface
for direct upload to S3. Designed for browser-based clients using `Fine
Uploader <http://fineuploader.com/>`__ and API clients.

Features
--------

Handles browser-based uploads
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Signs `policy
   documents <http://docs.aws.amazon.com/AmazonS3/latest/dev/HTTPPOSTForms.html#HTTPPOSTConstructPolicy>`__
   for the `POST
   API <http://docs.aws.amazon.com/AmazonS3/latest/dev/HTTPPOSTForms.html>`__.
2. Provides an empty response to use as a success action redirect with
   old browsers (IE 9 and Android 2.3.x) which do not support the File
   API, instead using a dynamically generated iframe.
3. Provides a framework for your upload-complete callback, which should
   copy the file to storage and take whatever other action is needed,
   such as creating model objects.

See this `Fine Uploader blog
post <http://blog.fineuploader.com/2013/08/16/fine-uploader-s3-upload-directly-to-amazon-s3-from-your-browser/>`__
for a long explanation of these responsibilities.

Handles API-client uploads
~~~~~~~~~~~~~~~~~~~~~~~~~~

The library provides a streamlined interface suitable for programmatic
uploads by non-browser-based API clients:

1. Provides signed URIs for the `REST PUT Object
   API <http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html>`__.
2. Provides an upload-complete callback.

Designed for security
---------------------

This library's design goal is to be secure by default. To that end, it
makes two recommendations:

1. Send your uploads to an "uploads" bucket, and make them private. This
   simplifies the namespacing used to segregate uploads by user, and
   discourages read-write ACLs.
2. Create an account which has the minimum permission required on your
   uploads bucket, and use that account to sign users' policy documents.
3. Move the files into a storage bucket during the completion callback,
   with either private or read-only ACLs. The library provides a view
   and serializer you can subclass to accomplish this easily.
4. Use Fine Uploader's
   ```objectProperties.key`` <http://docs.fineuploader.com/api/options-s3.html#objectProperties.key>`__
   property to incorporate the username as a prefix in the key. If you
   don't already have access to the username in the client, you can use
   the library's middleware to set a cookie with the prefix.
5. Be sure to specify an ``https`` endpoint url when you configure Fine
   Uploader.
6. Set a one-day expiration policy which automatically deletes stale,
   incomplete uploads. This step is mainly to save you money.

If you're willing to take what hopefully is sensible-sounding advice, go
on to the next section. If you want to know *why*, see the discussion in
SECURITY.md.

If you don't want to use it as designed, you can use the utility classes
and naive serializers to create your own components.

Status
------

This project is functional pre-alpha. Most significantly it needs better
documentation.

|Build Status|

Installation
------------

Requires `Django REST Framework <http://django-rest-framework.org/>`__,
a great toolkit for building Web APIs in Django.

::

        pip install drf_to_s3

This will install the remaining dependencies:
`boto <https://github.com/boto/boto>`__, and
`querystring\_parser <https://github.com/bernii/querystring-parser>`__
which handles nested keys within ``uploadSuccess.params``.

How to use
----------

1. Include ``drf_to_s3.urls`` in your site (or if you prefer, redefine
   them).
2. If you want to use nested dictionaries in your success callback, you
   must disable Django REST Framework's options for overriding the HTTP
   method and content. You probably aren't using these options, and they
   `interfere with the view's use of a custom form
   parser <https://github.com/tomchristie/django-rest-framework/issues/1346>`__.

   ::

       REST_FRAMEWORK = {
           'FORM_METHOD_OVERRIDE': None,
           'FORM_CONTENT_OVERRIDE': None,
       }

3. Create an temporary bucket for uploads.
4. Set the CORS policy on that bucket.
5. Create a user which only has PutObject access to that bucket.
6. Add Fine Uploader to your front end.
7. Configure Fine Uploader:

   -  Keys
   -  ``request.key``
   -  access key

Limitations
-----------

Users must be logged in to upload. Anonymous uploads currently aren't
supported.

Deletes during upload are not supported, but would be easy to add.

Contributing
------------

-  Issue Tracker: https://github.com/bodylabs/drf-to-s3/issues
-  Source Code: https://github.com/bodylabs/drf-to-s3

Unit tests:
~~~~~~~~~~~

::

    rake create_venv
    . .venv/bin/activate
    rake install
    rake test

Unit tests against S3:
~~~~~~~~~~~~~~~~~~~~~~

1. If it's not already installed on your system, install ``foreman``.
   You can get it from RubyGems with ``gem install foreman`` or install
   `Heroku Toolbelt <https://toolbelt.heroku.com/>`__.
2. Create an S3 bucket to use for testing.
3. Create a ``.env`` file at the project root with the following three
   lines:

   -  AWS\_TEST\_BUCKET=...
   -  AWS\_ACCESS\_KEY\_ID=...
   -  AWS\_SECRET\_ACCESS\_KEY=...

4. Run the tests:

   ::

       source venv/bin/activate
       foreman run drf_to_s3/runtests/runtests.py

Integration tests:
~~~~~~~~~~~~~~~~~~

1. Install ``foreman``, create an S3 bucket, and set up your ``.env``
   file as described above.

2. Install Node, NPM, the build dependences for Fine Uploader, and
   Chromium Driver:

   ::

       rake install_integration

3. If you're not using Mac OS / Homebrew, you need to install Chromium
   Driver some other way.

4. Choose a version of Fine Uploader to test:

   ::

       rake install_fine

5. Build it into ``drf_to_s3/integration/static``:

   ::

       rake install_fine[4.2.2]

6. Run the tests

   ::

       rake integration

Running integration tests on Sauce Labs:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Create a `Sauce Labs <https://saucelabs.com/>`__ account.
2. In .env, set ``SAUCE_USERNAME`` and ``SAUCE_ACCESS_KEY``.
3. Install `Sauce
   Connect <http://saucelabs.com/downloads/Sauce-Connect-latest.zip>`__.
4. Start Sauce Connect:

   foreman run sh -c 'java -jar
   ~/code/Sauce-Connect-latest/Sauce-Connect.jar
   :math:`SAUCE_USERNAME `\ SAUCE\_ACCESS\_KEY'

5. Run the tests:

   WITH\_SAUCE=1 rake integration

Building the package for PyPi:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This readme is written in Markdown, so there are dependencies for
converting it to reStructuredText. You only need this if you want to
generate the PyPi package with long\_description intact. Without it,
you'll just get a warning.

::

    rake install_dist

If you're not using MacOS / Homebrew, you'll need to install Pandoc some
other way.

License
-------

This project is licensed under the MIT license.

.. |Build Status| image:: https://travis-ci.org/bodylabs/drf-to-s3.png?branch=master
   :target: https://travis-ci.org/bodylabs/drf-to-s3
 
File Type Py Version Uploaded on Size
drf_to_s3-0.7.1.tar.gz (md5) Source 2014-02-10 15KB
  • Downloads (All Versions):
  • 9 downloads in the last day
  • 205 downloads in the last week
  • 3159 downloads in the last month