Skip to main content

Data pipeline framework for machine learning

Project description

https://img.shields.io/coveralls/mila-udem/fuel.svg https://travis-ci.org/mila-udem/fuel.svg?branch=master https://readthedocs.org/projects/fuel/badge/?version=latest https://img.shields.io/scrutinizer/g/mila-udem/fuel.svg https://requires.io/github/mila-udem/fuel/requirements.svg?branch=master https://img.shields.io/badge/license-MIT-blue.svg

Fuel

Fuel provides your machine learning models with the data they need to learn.

  • Interfaces to common datasets such as MNIST, CIFAR-10 (image datasets), Google’s One Billion Words (text), and many more

  • The ability to iterate over your data in a variety of ways, such as in minibatches with shuffled/sequential examples

  • A pipeline of preprocessors that allow you to edit your data on-the-fly, for example by adding noise, extracting n-grams from sentences, extracting patches from images, etc.

  • Ensure that the entire pipeline is serializable with pickle; this is a requirement for being able to checkpoint and resume long-running experiments. For this, we rely heavily on the picklable_itertools library.

Fuel is developed primarily for use by Blocks, a Theano toolkit that helps you train neural networks.

If you have questions, don’t hesitate to write to the mailing list.

Citing Fuel

If you use Blocks or Fuel in your work, we’d really appreciate it if you could cite the following paper:

Bart van Merriënboer, Dzmitry Bahdanau, Vincent Dumoulin, Dmitriy Serdyuk, David Warde-Farley, Jan Chorowski, and Yoshua Bengio, “Blocks and Fuel: Frameworks for deep learning,” arXiv preprint arXiv:1506.00619 [cs.LG], 2015.

Documentation

Please see the documentation for more information.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fuel-0.1.1.tar.gz (161.7 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page