Deep Learning Optimizer Benchmark Suite
Project description
# DeepOBS - A Deep Learning Optimizer Benchmark Suite
![DeepOBS](docs/deepobs_banner.png “DeepOBS”)
[![PyPI version](https://badge.fury.io/py/deepobs.svg)](https://badge.fury.io/py/deepobs) [![Documentation Status](https://readthedocs.org/projects/deepobs/badge/?version=latest)](https://deepobs.readthedocs.io/en/latest/?badge=latest) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
> This branch contains the beta of version 1.2.0: <br> It contains the latest changes planed for the release of DeepOBS 1.2.0, including support for PyTorch. Not all features are implemented and most notably we currently don’t provide baselines for this version.
DeepOBS is a benchmarking suite that drastically simplifies, automates and improves the evaluation of deep learning optimizers.
It can evaluate the performance of new optimizers on a variety of real-world test problems and automatically compare them with realistic baselines.
DeepOBS automates several steps when benchmarking deep learning optimizers:
Downloading and preparing data sets.
Setting up test problems consisting of contemporary data sets and realistic deep learning architectures.
Running the optimizers on multiple test problems and logging relevant metrics.
Reporting and visualization the results of the optimizer benchmark.
![DeepOBS Output](docs/deepobs.jpg “DeepOBS_output”)
The code for the current implementation working with TensorFlow can be found on [Github](https://github.com/fsschneider/DeepOBS). A PyTorch version is currently developed (see News section below).
The documentation of the beta version is available on readthedocs: https://deepobs.readthedocs.io/en/v1.2.0-beta0/
The paper describing DeepOBS has been accepted for ICLR 2019 and can be found here: https://openreview.net/forum?id=rJg6ssC5Y7
If you find any bugs in DeepOBS, or find it hard to use, please let us know. We are always interested in feedback and ways to improve DeepOBS.
## News
We are currently working on a new and improved version of DeepOBS, version 1.2.0. It will support PyTorch in addition to TensorFlow, has an easier interface, and many bugs ironed out. You can find the latest version of it in [this branch](https://github.com/fsschneider/DeepOBS/tree/v1.2.0-beta0).
A pre-release, version 1.2.0-beta0, will be available shortly and a full release is expected in a few weeks.
Many thanks to [Aaron Bahde](https://github.com/abahde) for spearheading the developement of DeepOBS 1.2.0.
## Installation
pip install deepobs
We tested the package with Python 3.6, TensorFlow version 1.12, Torch version 1.1.0 and Torchvision version 0.3.0. Other versions of Python and TensorFlow (>= 1.4.0) might work, and we plan to expand compatibility in the future.
If you want to create a local and modifiable version of DeepOBS, you can do this directly from this repo via
pip install -e git+https://github.com/fsschneider/DeepOBS.git#egg=DeepOBS
for the latest stable version, or
pip install -e git+https://github.com/fsschneider/DeepOBS.git@v1.2.0-beta0#egg=DeepOBS
to get the preview of DeepOBS 1.2.0.
Further tutorials and a suggested protocol for benchmarking deep learning optimizers can be found on https://deepobs.readthedocs.io/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for deepobs-1.2.0b0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cd2cb32959b156b3871746aca9822b7896b9b4d887281766487b9e04da125642 |
|
MD5 | 14f0f705ce10332154ef98268198ca31 |
|
BLAKE2b-256 | 136271911beb82ab07576725478af0a71fdc43e632fefe7d5551dc42ec8c9645 |