skip to navigation
skip to content

slapos.toolbox 0.60

SlapOS toolbox.


SlapOS toolbox contains :

HTTP Cache Checker

Crawl a website and check the HTTP configuration.


python2 -o result.txt cachechecker.conf

Example of configuration

url =

working_directory = /tmp/cachechecker

# debug_level = info
debug_level = debug
file_log_path = /tmp/cachechecker.log

traffic_server_cache = cHs

Last-Modified = True
Expires = True
Vary = Cookie,Authorization,Accept-Encoding
Cache-Control = max-age=600, stale-while-revalidate=360000, public

[no_header url=.*]
Set-Cookie = None

[no_header content-type=(image/.*|application/|application/x-font-woff|font/truetype)]
Vary = None

[header content-type=image/.*]
Cache-Control = max-age=3600, stale-if-error=360000, stale-while-revalidate=360000, public

[header content-type=(text/css|text/plain|application/x-javascript|.*/javascript|application/|application/x-font-woff|font/truetype)]
Vary = Accept-Encoding
Cache-Control = max-age=3600, stale-if-error=360000, stale-while-revalidate=360000, public

# Captcha
[header url=(.*/contactus$)]
Cache-Control = max-age=0, no-cache

# # IE PDF Bug
[header url=.*format=pdf]
Cache-Control = Private

[header url=.*format=]
Vary = Cookie,Authorization,Accept-Encoding
Cache-Control = max-age=600, stale-while-revalidate=360000, public

prohibited_folder_name_list = web_page_module
prohibited_file_name_list = WebSection_viewAsWeb




onetimeupload is an HTTP application which only accepts one file to be uploaded.


The network cache server is a NoSQL storage with a REST API.

How does it works

                 /               \
                 |               |
          ------>| NETWORKCACHED |
          |  ----|               |<-----
          |  |   \_______________/     |
GET /key  |  |                         |
          |  | File                    | PUT / <- data
        __|__v______             ______|_____
       |            |           |            |
       |   Client   |           |   Client   |
       |____________|           |____________|

Basically, the networkcached archives the files through HTTP PUT method. When a client want to download the file it just need to provide the key value and the server will send a response with the file data.

 PUT / :
   parameter: file uploaded
   Used to upload/modify an entry

 GET /<key>
   Return raw content
   Raise HTTP error (404) if key does not exist


$ python2.6 install

Now it is time to create the ‘networkcached.conf’ file in /etc/networkcached.conf directory, using your preferred text editor (gedit, kate, vim.). Follow text shall be put in this file:

host =
port = 5001
cache_base_folder = /var/cache/networkcached/

Run the server:

# networkcached networkcached.conf

Setup Develoment Environment

$ mkdir -p ~/networkcached/downloads
$ cd ~/networkcached

Now it is time to create ‘buildout.cfg’ file in ~/networkcached directory, using your preferred text editor (gedit, kate, vim.). Follow text shall be put in this file:

extensions = mr.developer
auto-checkout = slapos.tool.networkcached
download-cache = /nexedi/buildout-networkcached/downloads
eggs-directory = /nexedi/buildout-networkcached/eggs

parts =

slapos.tool.networkcached = svn

recipe = zc.recipe.egg
eggs =

Now you bootstrap the buildout:

$ python -S -c 'import urllib;print urllib.urlopen(\
     ).read()' | python -S -

Run the buildout:

$ bin/buildout -v

Now it is time to create the ‘networkcached.conf’ file in ~/networkcached directory, using your preferred text editor (gedit, kate, vim.). Follow text shall be put in this file:

host =
port = 5001
cache_base_folder = ~/networkcached/networkcached-database

Now you can start your networkcached server: $ bin/networkcached networkcached.conf or $ bin/networkcached -d networkcached-database -a -p 5002


0.60 (2016-10-05)

  • monitor: write pid to file while processing
  • monitor: run all promises in a single crontask with timeout of 12 seconds per promises.

0.59 (2016-09-16)

  • runner: add tests
  • Add scripts to generate RSS feeds and to create promises on RSS feeds
  • notifier: new arguments to generate verobose logging

0.58 (2016-09-07)

  • monitor: multiple fixes and improvements
  • resilient: fix tests
  • agent: fix console script

0.57 (2016-08-09)

  • monitor: Included monitor code with unit tests.

0.56 (2016-07-25)

  • slaprunner: remove create user at first launch, clone repository can be done by slaprunner at startup.
  • Remove slapos.builder and slapos.monitor as it only contains unused and dead code.

0.55 (2016-03-01)

  • Switch to mysqlclient, that is a maintained fork of MySQL-python.
  • Add an option in is-process-older-than-dependency-set to kill the process if older than dependency set.

0.54 (2016-02-25)

  • Added is-process-older-than-dependency-set script.

0.53 (2015-11-26)

  • networkbench: Minor fixes

0.52 (2015-09-02)

  • testagent: Fully reimplementation
  • networkbench: Accept external configurations and split file logs

0.51 (2015-08-17)

  • networkbench: Include ping6 and TTFB support

0.50 (2015-07-20)

  • slaprunner: Allow to configure the window title through ‘instance-name’ parameter.

0.49 (2015-07-17)

  • slaprunner: filter out useless file extensions in treeview

0.48 (2015-04-09)

  • slaprunner: Support command exiting with non 0 status in minishell

0.47.3 (2015-02-17)

  • resiliency tests: hide output of slapos node instance.
  • equeue: move takeover detection in thread lock.
  • add erp5 resiliency test suite.
  • harcode slaprunner request to use dev branch for erp5.
  • slaprunner resiliency test suite: compatibility with python 2.7.9.
  • resiliency test suite: do takeover from web interface.

0.47.2 (2015-02-14)

  • equeue: prevent to run import script if takeover-triggered file exists.
  • Advertise development of new version.

0.47.1 (2015-02-05)

  • equeue: unlock lockfile at boot.

0.47 (2015-02-05)

  • runner: fix syntax when stopping all processes.
  • equeue: realtime logging.

0.46.1 (2015-01-30)

  • runner: enhance software/instance explanation to user.
  • runner: do not remove whole instance root directory, but only instance home directories.
  • slaprunner resilient test: update API about slapgridResult.

0.46.0 (2015-01-16)

  • slaprunner: don’t shutdown supervisord.
  • slaprunner: Simplify process management and make sure ‘process killer’ handler works.
  • webrunner: add getConnectionParameter API.
  • equeue: create a lockfile when running importer command.
  • Equeue: do not register command in database if it was not successful.

0.45.3 (2015-01-12)

  • slaprunner: Quick and dirty support for ace editor autocompletion

0.45.2 (2014-12-02)

  • resiliencytest framework: adapt abstract test suite to work with a single clone.
  • resiliency test suites: send log to erp5 master.
  • KVM resiliency test suite: use test framework

0.45.1 (2014-11-24)

  • Fix logrotate and introduce a delay for networkbench.

0.45.0 (2014-11-24)

  • Resiliency tests: allow to use from inside of a classical erp5testnode.

0.44.0 (2014-11-22)

  • Add new networkbench command.

0.43.0 (2014-11-13)

  • Add new slapos-kill command, deprecating killpidfromfile
  • Make pycurl egg an optional dependency

0.42.0 (2014-10-28)

  • The egg provides check-web-page-http-cache-hit script to check if a web page is well cached by Squid or TrafficServer.

0.41.0 (2014-10-20)

  • The egg provides is-local-tcp-port-opened script to check if an ip address + port are currently opened on the local machine by reading in /proc/net/tcp and /proc/net/tcp6.

0.40.4 (2014-10-14)

  • The equeue server now supports additional parameters to the callbacks, separated by 0 in the JSON protocol. The pubsubnotifier accepts a –transaction-id parameter. If missing, the current timestamp is used. The pubsubnotifier then provides the transaction-id value in the POST call to /notify.

    Together, these three changes allow us to put in the crontab:

    0 0 * * * /srv/slapgrid/slappartXX/bin/exporter –transaction-id date +%s

    and the same timestamp value is reported as transaction id in the equeue logs for BOTH the pull and push operations.

0.40.3 (2014-10-13)

  • Slaprunner: stop proxy when removing instances content. This fixes the “partition11 table does not exist” error.

0.40.2 (2014-09-25)

  • Slaprunner: Various bug fix
  • Slaprunner: Update supervisorctl interaction to work with latest slapos.core revision

0.40.1 (2014-09-02)

  • Slaprunner: Fix a bad behaviour within slapos

0.40 (2014-09-01)

  • Slaprunner: Supervisord runs slapos commands
  • Slaprunner: fixes for git repositories management

0.39.5 (Unreleased)

  • Slaprunner: Add slave support test.

0.39.4 (2014-08-11)

  • Slaprunner: Add a minishell feature, using far less bandwith than shellinabox
  • Slaprunner: Slapgrid state is correctly rendered while compiling
  • Slaprunner: consider new parameter’s values without needing to restart [719da31a]
  • Slaprunner: slapgrid’s last result is always accessible through a stored file [a0bc26e9]
  • Slaprunner: Fix dangerous bug concerning custom parameters [50ca3386]
  • Slaprunner: various fixes and improvements

0.39.3 (2014-07-11)

  • Slaprunner: Add folder to favorites [Alain T.]
  • Slaprunner: Fix https git clone not possible if no login [Alain T.]
  • Slaprunner: Improve editor full window [Alain T.]
  • Pubsub: More explicit RSS displayed in reversed chronoligical order [Cedric LN]
  • Pubsub-notifier: add logger [Cedric LN]
  • Slaprunner: Various fixes

0.39.2 (2014-05-16)

  • Slaprunner: Add tab support [Alain T.]
  • Slaprunner: Add upload file [Alain T.]
  • Slaprunner: Improve editor full window [Alain T.]
  • Slaprunner: Various fixes

0.39.1 (2014-04-30)

  • Slaprunner: Various fixes

0.39 (2014-02-20)

  • Slaprunner: new web interface design
  • Slaprunner: one function handle both “run software” and “run instance” [9c660c0]
  • Slaprunner: building and deploying can be customized [0db1f6b, b33bd1f]
  • Slaprunner: adds a multi-user feature [efad6d]
  • Slaprunner: add fullscreen mode for text edition [83d1dc]
  • Slaprunner: direct access to monitoring of running instance, if it exists [f8e7bf3]

0.38.1 (2013-12-06)

  • Slaprunner: do not delete proxy.db on each run software [71777fc0]

0.38 (2013-12-03)

  • Slaprunner: adds an integrated shell [ca6a670a]
  • Slaprunner: uses basic authentification [05913751]
  • Slaprunner: adds automated deployment of a Software Release [c8ab1273]
  • Slaprunner: flask development server replaced by Gunicorn, a WSGI server [48d60d0f]
  • Slaprunner: new test scenario for auto-deployment [c6007954]
  • Runner resiliencytestsuite: adds basic auth support [3c03f12b]
  • Runner resiliencytestsuite: tests can be done on only one Slapos node [07198d87]

0.37.4 (2013-10-15)

  • Improve QEMU QMP wrapper by adding drive-backup method and other helpers. [0afb7d6, 95d0c8b]

0.37.3 (2013-10-10)

  • pubsub: don’t swallow output of subprocess to allow debug. [c503484]

0.37.2 (2013-10-10)

  • Add QEMU QMP wrapper. [9e819a8]
  • KVM resiliency test: update docstring about how to setup disk image. [dbe347f]
  • KVM resiliency test: change key for each clone. [7ef1db3]

0.37.1 (2013-10-03)

  • pubsub notifier: handle timeout and other connection errors. [ac4c75c]
  • equeue: cast str(timestamp) to please gdbm. [8b067d6]

0.37 (2013-09-30)

  • equeue: log output of subprocess. [1694937]
  • slaprunner: don’t send 200 when login is bad. [4a8e10bf]
  • Improve reliability of resiliency tests.

0.36 (2013-09-05)

  • Add resiliency test system.
  • Add kvm resiliency test suite.
  • Add slaprunner resiliency test suite.
  • slaprunner: don’t tell user false assumptions [71dc4a08e].
  • slaprunner: allow profile/instance to be run by GET [4b454814a].
  • slaprunner: remove boring username/password length check [fcce0336c].

0.35.1 (2013-08-05)

  • slaprunner: add support for reverse proxy [6c379644].

0.35.0 (2013-07-08)

  • Resiliency: equeue is rewriten extending SocketServer.ThreadingUnixStreamServer (Thanks to Julien M.). So far outputs of commands invoked by equeue are redirected to /dev/null to avoid locking the whole process. [Cedric Le Ninivin]
  • Add htpasswd script. This script is meant to produce htpasswd file without installing apache. [Cedric Le Ninivin]
  • slaprunner: Grammar improvements.

0.34.0 (2013-03-28)

  • runner: Log textarea is resizable. [Cedric de Saint Martin]
  • runner: Fix bug where slaproxy hangs: Redirect slapproxy output to stdout. [Cedric de Saint Martin]
  • lampconfigure: Add support for running SQL scripts in lampconfigure. [Cedric de Saint Martin]
  • runner: software_type now survives to slaprunner reboot. [Cedric de Saint Martin]

0.33.1 (2013-02-12)

  • runner: Add default values for git repository while cloning. [Cedric de Saint Martin]
  • runner: Don’t remove git repo if error occurred during clone. [Cedric de Saint Martin]
  • runner: Fix bug where software_type of main instance is impossible to set reliably. [Cedric de Saint Martin]

0.33 (2013-01-24)

  • agent: run tests in order. [Marco Mariani]
  • slaprunner: Add process killer killing all processes when instance is stopped. [Cedric Le Ninivin]
  • slaprunner: slapgrid-cp will be run at startup. [Cedric Le Ninivin]
  • slaprunner: Improved design. [Alain Takoudjou]
  • slaprunner: Don’t set arbitrary limitations on the complexity of password. [Cedric de Saint Martin]

0.32 (2012-12-13)

  • pubsub: support multiple notifications and callbacks. [Marco Mariani]
  • pubsub: print/return errors from subprocess or notifications. [Marco Mariani]
  • agent: ignore SR status, retrieve instance status with .getStatus(). [Marco Mariani]
  • slapmonitor: added slapmonitor-xml function + entry point. [Marco Mariani]

0.31.1 (2012-10-02)

  • slaprunner: Add new file manager. [Alain Takoudjou]
  • slaprunner: Ask confirm before delete important folder. [Alain Takoudjou]

0.31 (2012-10-02)

  • slaprunner: Add correct support for software_type. [Cedric de Saint Martin]

0.30 (2012-08-29)

  • Add promise support on slapcontainer script. [Antoine Catton]

0.29.1 (2012-08-23)

  • Minor fix on slapcontainer. [Antoine Catton]

0.29 (2012-08-22)

  • Add slapcontainer stateless script. [Antoine Catton]

0.28.2 (2012-08-17)

  • Slaprunner: Move password, process pid and config files to etc_dir [Alain Takoudjou]

0.28.1 (2012-08-15)

  • Update ACE to 08.11.2012 and JQuery to 1.8.0. [Cedric de Saint Martin]

0.28 (2012-08-09)

  • slaprunner: add support for authentication management. [Alain Takoudjou]

0.27 (2012-08-07)

  • pubsub: add support for multiple notification servers. [Timothée Lacroix]

0.26.1 (2012-07-09)

  • runner: Add support to slapos.core 0.26.2 [Cedric de Saint Martin]

0.26 (2012-07-06)

  • slaprunner now supports authentication and instance parameters. [Alain Takoudjou]
  • Redisign of Test Agent so that it supports proper instantiation test and report. [Vincent Pelletier]

0.25 (2012-06-19)

  • slapmonitor is fully functional. [Mohamadou Mbengue]

0.24 (2012-04-19)

  • Slap Test Agent : test software release installation and instance allocation [Yingjie Xu]

0.22 (2012-04-05)

  • Install argparse if missing [Łukasz Nowak]

0.21 (2012-04-04)

  • zodbpack : simple tool to pack ZODB [Łukasz Nowak]

0.20 (2012-03-28)

  • slaprunner : fix recursive include of .gif files [Alain Takoudjou]

0.19 (2012-03-28)

  • slaprunner : Add Support for more instances at the same time. [Alain Takoudjou]
  • slaprunner : Add slapgrid process control and management. [Alain Takoudjou]
  • slaprunner : Add Cloud9 IDE. [Alain Takoudjou]
  • slaprunner : Add develop location of cloned git repository. [Alain Takoudjou]

0.18 (2012-02-23)

  • slaprunner : Add git commit and push support. [Alain Takoudjou]
  • slaprunner : Solve UI problems in project management. [Alain Takoudjou]
  • slaprunner : Solve bug where ACE modifies carrier returns. [Alain Takoudjou]
  • slaprunner : Add ajax view of build log. [Alain Takoudjou]
  • slaprunner : Allow to edit any file in repository. [Alain Takoudjou]

0.17 (2012-02-10)

  • slapbuilder: Update slapbuilder to new OpenSuse 12.1. [Cedric Le Ninivin]
  • slapbuilder: Added automatic IPv6 support thanks to ViFiB OpenVPN. [Cedric Le Ninivin]
  • slapbuilder: Added virtual machine support. [Cedric Le Ninivin]
  • slapbuilder: Added “One Disk” feature. [Cedric Le Ninivin]

0.16 (2012-01-20)

  • Fix [Cedric de Saint Martin]

0.15 (2012-01-20)

  • slaprunner: Adding project and git repository management [Alain Takoudjou]

0.14 (2012-01-09)

  • onetimeupload: SSL should be provided by an http frontend [Romain Courteaud]

0.13 (2011-12-30)

  • builder: fix usage of mkstemp [Romain Courteaud]

0.12 (2011-12-29)

  • builder: dynamic offset calculation [Romain Courteaud]

0.11 (2011-12-23)

  • equeue: Add execution queue server. [Antoine Catton]
  • pubsub: Add pubsub server using pubsubhubbub for notifications. [Antoine Catton]

0.10 (2011-11-07)

  • slapbuilder: Support banging. [Łukasz Nowak]
  • lamp : Fix bug when using file_token option [Alain Takoudjou]

0.9 (2011-10-29)

  • Add LAMP tool to manage lamp instances. [Alain Takoudjou]
  • Add onetimedownload : a simple script which download any http resource and save it into a specified file_path. [Nicolas Delaby]

0.8 (2011-10-24)

  • Add new style to runner. [Alain Takoudjou]

0.7 (2011-10-17)

  • generic: Added systool with killpidfromfile. [Łukasz Nowak]
  • slaprunner: Avoid error when trying to remove instance directory containing not writable directories. [Alain Takoudjou]

0.6 (2011-07-13)

  • slapbuilder: Typo during PKI repository creation fixed. [Łukasz Nowak]

0.5 (2011-07-12)

  • slapbuilder: Do not play too much with buildout, SlapOS installation is responsible for correctly featured zc.buildout [Łukasz Nowak]

0.4 (1011-07-11)

  • Update slaprunner to run on newer slapos (requires at least 0.10 to work). [Cedric de Saint Martin]

0.3 (2011-06-27)

  • slapbuilder: use slapos provided networkcached buildout.
  • slapbuilder: fix network issues on busy networks (“Neighbour table overflow.”)

0.2 (2011-06-10)

  • In slapbuilder: drop to 100 partitions by default, move out PKI from USB key to real disk. [Łukasz Nowak]

0.1 (2011-05-27)

  • Created slapos.toolbox from slapos.tool.builder, slapos.tool.runner, slapos.tool.cloudmgr, slapos.tool.ontimeupload and slapos.tool.networkcache. [Łukasz Nowak]
File Type Py Version Uploaded on Size
slapos.toolbox-0.60.tar.gz (md5) Source 2016-10-05 1MB