Skip to main content

A WSGI middleware to get some stats on large files upload,and provide a progress bar for your users

Project description

The Sphinx version of this documentation can be found here.

A demo page is also available.

If you found a bug submit an issue <http://www.bitbucket.org/gawel/gpfileupload/issues/?status=new>.

Description

gp.fileupload is a set of wsgi middleware to deal with large file upload.

  • gp.fileupload.FileUpload is a wsgi middleware to get the stat of large files uploaded to the server. Optional javascript code is provided to animate a progress bar.

  • gp.fileupload.Storage is a wsgi middleware to store files on file system and avoid big transaction in your application.

Upload middleware

The principle is to count uploaded chunks in the wsgi loop, transmit them to the application, and provide a link with up-to-date json information. The optional javascript code is able to use an existing form and replace it with a progress bar during upload. It can also generate its own upload form, then upload several files sequentially with multiple progress bars. gp.fileupload can be used without modifying your application, just by adding it in the wsgi stack.

It has currently been tested with Pylons and Zope 3.

Middleware

Wrap your wsgi application with the middleware:

>>> from gp.fileupload import FileUpload

>>> def my_application(environ, start_response):
...     start_response('200 OK', [('Content-Type', 'txt/html')])
...     return ['<html><body>My app</body></html>']

>>> app = FileUpload(my_application, tempdir=TEMP_DIR,
...                  max_size=None)

>>> def application(environ, start_response):
...     return app(environ, start_response)

The FileUpload middleware has the following options:

  • tempdir: A path to a temporary folder

  • max_size: Max allowed size. If the file size is larger than max_size a RuntimeError is raised.

  • include_files: A list of static files that the middleware should include in your html body (see below). You might want to include the files by yourself in the relevant pages, or they will be included in all of your pages.

  • require_session: a boolean flag that disable the middleware for requests that don’t provide the SESSION_name in the query.

Application code

Write an html form like this:

<form enctype="multipart/form-data"
      method="POST"
      action=".?gp.fileupload.id=1">
  <input type="file" name="file" />
  <input type="submit" />
</form>

Where 1 is the session id. The session id must be a digit.

When the form is submitted, you can use some ajax stuff to get the stats of the upload with the url:

http://yourhost/gp.fileupload.stat/1

This will return some JSON data like:

{'state': 1, 'percent': 69}

state can have the following values:

  • 0: nothing done yet.

  • 1: upload is active

  • -1: file is larger than max_size.

You can use this to display the upload progress.

Storage middleware

The storage middleware provide a way to avoid long transaction in your application.

POST content is written to a temporary directory. If the POST contains some files then the files are moved to the storage directory. The files content is replaced by the path of the real file relative to the storage root.

This way, your application receive always a few Ko of data.

Usage

Wrap your wsgi application with the middleware:

>>> from gp.fileupload import Storage
>>> from gp.fileupload import purge_files
>>> import cgi

>>> def my_application(environ, start_response):
...     """simple app to read the file path from the request and remove it
...     from the storage directory
...     """
...     if environ['REQUEST_METHOD'] == 'POST':
...         fields = cgi.FieldStorage(fp=environ['wsgi.input'],
...                                   environ=environ,
...                                   keep_blank_values=1)
...         relative_path = fields['file'].read()
...         # remove file from storage
...         purge_files(environ, relative_path)
...     start_response('200 OK', [('Content-Type', 'txt/html')])
...     return ['<html><body>My app</body></html>']

>>> app = Storage(my_application,
...               upload_to='/tmp/share/files',
...               tempdir='/tmp/upload_tmp',
...               )

>>> def application(environ, start_response):
...     return app(environ, start_response)

The Storage middleware has the following options:

  • upload_to: the root directory of the storage tree.

  • tempdir: A path to a temporary folder

  • exclude_paths: a list of regular expressions. All matched PATH_INFO will be ignored. If this parameters is not None, then the middleware will also catch non-HTML content send by your application. WARNING: This options is experimental and not well tested.

  • require_session: a boolean flag that disable the middleware for requests that don’t provide the SESSION_name in the query.

Paste factories

The package provides a filter factory usable in PasteDeploy configuration files.

The factory provides the middleware itself:

[pipeline:main]
pipeline = fileupload egg:myapp

[filter:fileupload]
use = egg:gp.fileupload
# temporary directory to write streams to
tempdir = %(here)s/data/fileupload

# file to inject in the html code
include_files = fileupload.css jquery.*

# if you already have jquery in your application, use this line
#include_files = fileupload.css jquery.fileupload.*

# max upload size is 50Mo
max_size = 50

# use this options to also wrap your application with a Storage middleware
#upload_to = %(here)s/storage
#exclude_paths = /@@

Then you can access the javascript stuff at /gp.fileupload.static/.

The include_files parameters will inject these tags in your application:

<link type="text/css" rel="Stylesheet" media="screen"
      href="/gp.fileupload.static/fileupload.css"/>
<script type="text/javascript"
        src="/gp.fileupload.static/jquery.js"/>
<script type="text/javascript"
        src="/gp.fileupload.static/jquery.fileupload.js"/>
<script type="text/javascript"
        src="/gp.fileupload.static/jquery.fileupload.auto.js"/>

And feel free to use ajax stuff. Notice that these tags are included at the end of the html body.

Available files are:

  • jquery.js: jquery 1.2.6

  • jquery.fileupload.js: the jQuery().fileUpload plugin.

  • jquery.fileupload.auto.js: auto bind form tags with a multipart/form-data enctype.

  • fileupload.css: a few css to display the progress bar.

Ajax stuff

Description

A jQuery plugin is provided as a helper.

To use it, you only need to add a script tag to your html head section:

<script type="text/javascript">
    jQuery(document).ready(function() { jQuery('#sample').fileUpload(); });
</script>

The fileUpload plugin has the following options:

  • replace_existing_form: replace the existing form with a generated one.

    Defaults to false.

  • submit_label: The label of the general submit button.

  • field_name: A string to use as file field name for each form. Default to file.

  • hidden_submit_name: A string to use in the hidden field name located below

    each file field. The hidden field is supposed to replace the missing button, and is required by form frameworks (such as z3c.form) which expect a button name (bound to an action). Default to submit.

  • submit_empty_forms: if true, submit forms with empty file fields. Default: true.

  • use_iframes: If set to false the form will be submitted as a normal form. If true the form target become an iframe and the page is not reloaded.

  • stat_delay: delay between each stat request. Default: 1500.

  • success: A javascript function evaluated when all files are uploaded. The default one does nothing.

Examples

If you want multiple file forms:

<div id="forms"></div>

<script type="text/javascript">
  jQuery(document).ready(function() {
    jQuery('#forms').fileUpload({action:'/upload', field_name:'file_field'})
  });
</script>

This will show a form with addable file field and upload them to /upload. The forms are submitted in iframes as target so the page does not change after uploading.

If you already have forms. This is what is done in the jquery.fileupload.auto.js:

<script type="text/javascript">
  jQuery(document).ready(function() {
    jQuery('form[enctype^='multipart/form-data]')
        .fileUpload({use_iframes: false});
  });
</script>

This will show a progress bar when the form is uploaded, then redirect to the application page when the upload is completed. So the usage is totally transparent for you.

Contributors

Gael Pasgrimaud <gael@gawel.org>

Christophe Combelles <ccomb@free.fr>

News

1.2 (2012-09-03)

  • Use json instead of simplejson if you have Python2.6.

  • Fix middleware to work on Windows.

  • Add a flag require_session that enable the middleware only if there is SESSION_NAME in the query string. By default the behavior is off, meaning it behave like before.

1.1

  • Unrecorded changes.

1.0

  • Unrecorded changes.

0.9 (2010-03-20)

  • support multiple file with same name and special chars (by pfalcon)

  • allow JSON callback on stats (by pfalcon)

0.8 (2009-02-20)

  • fix installation problem

0.7 (2009-02-08)

  • fix #1. bug in js when the environ have a SCRIPT_NAME

  • storage improvement. (by trollfot)

0.6 (2008-11-24)

  • improve middleware internals using WebOb

0.5 (2008-08-28)

  • support readline(int). Thanks to Tony Caduto for bug report.

  • Added a hidden field to provide an action name required by form frameworks such as z3c.form

  • updated the doc

  • raised the stat update delay to 1500ms

  • added an option to replace the existing form with a generated one.

  • changed the progress filename from a span to a div

  • improved usability by adding a form on input:file change event

  • handle only POST method

  • improve Storage middleware

0.4 (2008-08-12)

  • fix egg install script

  • add a storage module

0.3 (2008-07-27)

  • fix Content-Length header sent after code injection.

  • add e.stopPropagation() to the Add more file link

0.2 (2008-07-25)

  • add docs/ folder to auto generate documentation with sphinx

  • use jquery packed version

  • IE javascript fixes

0.1 (2008-07-24)

  • first version

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gp.fileupload-1.2.tar.gz (37.4 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page