Skip to main content

Linux tool enabling reproducible experiments (unpacker)

Project description

ReproZip is a tool aimed at scientists using Linux distributions, that simplifies the process of creating reproducible experiments from programs.

It uses the ptrace facilities of Linux to trace the processes and files that are part of the experiment and build a comprehensive provenance graph for the user to review.

Then, it can pack these files in a package to allow for easy reproducibility elsewhere, either by unpacking and running on a compatible machine or by creating a virtual machine through Vagrant.

Quickstart

Packing

On the machine where the experiment is setup, install reprozip:

$ pip install reprozip

Then run your experiment under reprozip’s monitoring (simply prefix reprozip trace):

$ reprozip trace ./myexperiment -my --options inputs/somefile.csv other_file_here.bin

This leaves you with a .reprozip directory, in which you’ll find the config.yml configuration file. In it, you can edit the command-line and environment variables, and choose which files are to be packed.

If you are using one of the supported Linux distributions (currently Debian and Ubuntu), these files are organized by package, plus an other_files: section at the end. You can add or remove files, or choose not to include a package by changing packfiles: true to packfiles: false; this allows you to make a smaller packs, since reprounzip can get these files from the package manager easily (however versions might differ).

Once done editing .reprozip/config.yml, or if you left everything as-is, simply run the packer: (note that you should review the configuration to be sure not to include sensitive information)

$ reprozip pack my_experiment.rpz

Unpacking

$ pip install reprounzip

# Installing the packages on the host machine
$ reprounzip installpkgs my_experiment.rpz

# Unpacking in a directory
$ reprounzip directory my_experiment.rpz mydirectory
$ mydirectory/script.sh
# Building a chroot environment
$ reprounzip chroot my_experiment.rpz mychroot
$ sudo mychroot/script.sh

# Building a virtual machine (requires Vagrant)
$ pip install reprounzip-vagrant
$ reprounzip vagrant --use-chroot my_experiment.rpz mytemplate
$ cd mytemplate; vagrant up
$ vagrant ssh
vagrant@vm$ sh /vagrant/script.sh

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

reprounzip-0.2.tar.gz (12.0 kB view hashes)

Uploaded Source

Built Distribution

reprounzip-0.2-py2.py3-none-any.whl (16.9 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page