skip to navigation
skip to content

batou 1.0b24

A utility for automating multi-host, multi-environment software buildsand deployments.

Package Documentation

Latest Version: 1.7.2


batou is a multi-(component|host|environment|…) deployment utility.

Deployments are described in a model using “components” and can be deployed locally and remotely. Models are written as Python code.

See the batou documentation for details on how to use it.


1.0b24 (2014-07-04)

  • Actually fix unzipping.
  • Remove environment option connect_method, and use autodetection instead: If we’re not the configured service_user after the plain ssh connect, run sudo.
  • Refactor attributes and overrides: attributes allow template expansion now and also converting default values. The Attribute class has moved to the batou component module.
  • Yield nonzero returncode on remote exceptions.
  • Add convenience converter for attributes “list” to handle comma separated lists of strings.
  • Force safety of Nagios service description field: prohibit “,” as its not safe for Nagios business processes.
  • Allow specifying custom rsync options for SyncDir: the ‘–inplace’ option is a good idea in most cases, but not all. Fixes #11846.
  • Make supervisor restart/reload more robust towards non-running supervisors with lying PID files. Fixes #12744.
  • Remove harmful decoding step in File verify that caused wrong results.

1.0b23 (2014-06-30)

  • Allow unzip to overwrite existing files if the archive has newer copies.

1.0b22 (2014-06-30)

  • Fix uncaught import-error on mysql module.

1.0b21 (2014-06-30)

  • Fix bug in bundle shipping that made batou depend on the locale (#13465).
  • Add MySQL components.
  • Introduce batou.utils.Attribute() to avoid imperative type conversions in configure().

1.0b20 (2014-05-13)

  • Start using python-requests for HTTP/S downloads to support authentication (and other goodies).
  • Improve error logging.

1.0b19 (2014-04-07)

  • Allow specifying alternate find-links in requirements.txt
  • Remove remnants of the buildout component’s extends option.

1.0b18 (2014-03-31)

  • Introduce environment option connect_method, either ‘ssh+sudo’ (default) or plain ‘ssh’.

  • Improve update code: existing installations would not update cleanly to a new version due to wrong use of pkg_resources API. Existing installations need to be cleaned manually. :/

  • Archive: recognize .tar.xz files and extract them with tar.

  • Fix bug in CMMI verify that would always raise UpdateNeeded due to a path mismatch (#13145).

  • Component.cmd() now performs expand() on the given command by default. Saves typing:

    self.cmd(‘rm -rf {{component.workdir}}’)

    instead of

    self.cmd(self.expand(‘rm -rf {{component.workdir}}’))

  • File component: apply character encoding when reading and writing files. (#13165)

  • Mercurial clone: changed update behaviour yet again: unless configured differently, always achieve target state, even by losing uncommitted changes.

  • Allow unnamed access to the last added/ored component using the ‘self._’ attribute:

    def configure(self):

    self += Download(…) self += Extract(

1.0b17 (2013-12-06)

  • Mercurial clone: refuse to update if there are uncommitted changes or outgoing changesets.
  • Mercurial clone: added an option to avoid updating existing clones, which is often desirable during development.

1.0b16 (2013-11-29)

  • Improve the bootstrap/upgrade fix for #13049: the bootstrap template needed a second improvement.

1.0b15 (2013-11-29)

  • Try to fix #13049 - will only work once the dev version is already at least at b15. Not sure whether it really does the job, though. Hard to test.

  • Make environment and host arguments for running batou local optional. dev and localhost are the defaults now. Fixes #13034

  • Allow specifying secrets with [component:xy] in addition to [xy] to make it easier to discover when used to the environment overrides. Both options are supported now. Fixes #13003

  • Refactor Python virtualenv and package management. Provide explicit knowledge about Python versions 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, and 3.3 to use compatible virtualenv/pip/setuptools versions and commands. Fixes #12941

    NOTE: The API for batou.lib.python.Package has changed in backwards-incompatible ways:

    Package() needs to be added to a VirtualEnv component. Before:

    self += VirtualEnv(...)
    self += Package(...)


    venv = VirtualEnv(...)
    self += venv
    venv += Package(...)

    Package parameter pip_install_options has been renamed to install_options.

  • Use urllib module instead of wget command for cross platform compatibility.

1.0b14 (2013-11-26)

  • Make Buildout component install zc.buildout without dependencies (which is just setuptools), since that can break the version pinning of setuptools.
  • Check whether packages required via a -e expression in the local requirements.txt are already installed.
  • #12946: Make full path to python executable configurable for virtualenv (and buildout) component.
  • Remove cleanup code that removes batou-0.2-style virtualenv directories (bin, lib, include etc.) from remote deployments.
  • Add mailto setting to crontab component.
  • Added some documentation.

1.0b13 (2013-11-19)

  • Introduce a plugin mechanism: if a local requirements.txt file is present, install its contents into the .batou virtualenv.
  • Load components in alphabetical order. This enables importing from other components (when naming the directories appropriately).
  • #12986: Only try to unbundle if there actually are changes to unbundle.
  • Raise an error if a cron job is configured without a timing value.
  • Add platform support for Debian (crontab, logrotate, supervisor).
  • Added some documentation.

1.0b12 (2013-11-04)

  • Added branch argument to mercurial.Clone. Setting a branch automatically updates to the branch head on deploy. This is mostly useful for development environments.
  • Create the ‘secrets’ directory if it doesn’t exist, yet. Also, disallow editing secret files for non-existing environments.
  • Support continuing remote bootstrapping if we failed after creating the initial remote directory but were unable to use Mercurial.
  • #12898: build.Configure component was broken when using the default prefix.

1.0b11 (2013-10-17)

  • #12897: Use non-SSL pypi mirror for downloading virtualenv to fix tests failing randomly on machines that (for some reason) can’t validate PyPI’s certificate.

  • #12911: Ensure that we can configure file owners when they don’t exist during configure phase yet.

  • #12912: Fix untested and broken file ownership management.

  • #12847: Clean up unicode handling for File and Content components and templating.

  • #12910: Remote deployments failed when using bundles for transfers if no changes needed bundling.

  • #12766: Allow bootstrapping a batou project in an existing directory to support migration from 0.2.

  • #12283: Recognize files as ‘is_template’ by default. Auto-detect source files in the definition directory if they have the same basename. This is what you want in 99% of all cases. Explicitly stating either the ‘content’ or ‘source’ parameter disables auto-detection.

    Now you can write this:


    and have components/x/foo recognized as the source file and handled as a template.

  • Use ConfigParser instead of configobj which is effectively unmaintained (see and support lists separated by newlines in addition to commas.

1.0b10 (2013-09-27)

  • Package our own virtualenv instead of depending on the system-installed one. This should alleviate troubles due to old virtualenv versions that package distribute, which causes conflicts with recent setuptools versions (#12874).
  • Update supervisor version to 3.0.

1.0b9 (2013-08-22)

  • Update Package component so it ignores installed packages when installing. This way, we actually install setuptools even when distribute is installed. (Otherwise it’s a no-op since distribute tells pip that setuptools is already satisfied).
  • Fix update process: wrong call to old ‘.batou/bin/batou’ failed and early bootstrapping would downgrade temporarily which is confusing and superfluous. Fixes #12739.

1.0b8 (2013-08-17)

  • Remove superfluous mkdir call during remote bootstrap.
  • Make batou init print that it’s working. Bootstrapping can take a while, so at least signal that something’s going on.

1.0b7 (2013-08-17)

  • Depend on Python2.7 to be available on the PATH during early bootstrap. Otherwise our chances to get a 2.7 virtualenv are pretty small, too.
  • Improve project template: ignore the work/ directory by default.

1.0b6 (2013-08-17)

  • More MANIFEST inclusions: bootstrap-template.

1.0b5 (2013-08-17)

  • Improve MANIFEST so we actually package the init template and other generated files, like version.txt and requirements.txt.

1.0b4 (2013-08-17)

  • Provide a simple project-creation command, both for pip-installed batou’s as well as spawning new projects from existing ones. Fixes #12730
  • Fix #12679: make timeouts configurable.
  • Removed re-imports from batou main module to support light-weight self-installation and bootstrapping. I.e. ‘from batou import Component’ no longer works.
  • Provide a single main command together with a ‘bootstrap’ wrapper that you can check into your project and that is maintained during updates automatically. It also provides fully automatic bootstrapping, installation, upgrading and other maintenance.
  • Fix Python package installation version check.
  • Don’t use bin/buildout bootstrap command anymore. PIP installs a sufficient bin/buildout so buildout can do the rest internally.
  • Install zc.buildout during bootstrapping phase using PIP to avoid problems.
  • Shorten URLs in the Build component to their basename.
  • Add ‘assert_cmd’ API to support simpler assertions for verify when needing to check the result of an external command.
  • Switch to asking pip installing eggs instead of flat installations as namespaces seem to collide otherwise.
  • Remove non-functional deprecated ‘md5sum’ attribute.
  • Components are context managers now. If you provide __enter__ it will be called before verify() and if you provide __exit__ this will be called after update (always - even if update isn’t actually called). This allows you to manage temporary state on the target system more gracefully. See the DMGExtractor for an example.
  • Major refactoring of internal data structures to simplify and improve test coverage. Some breakage to be expected:
    • Components do not have a ‘service’ attribute any longer. If you need access to the service base directory use ‘environment.base_dir’.
  • Another attempt at getting convergency for package installation: actually leverage pkg_resources to check what version of a package is installed. PIP does not necessarily list or freeze setuptools, even if forced to reinstall.
  • Remove superfluous ‘–update’ from pip calls that would temporarily install the incorrect package version.
  • Improve convenience of the ‘secrets-edit’ wrapper script to allow re-encrypting without re-entering the editor.
  • Consistently switch to using setuptools.
  • Fix #12399: incorrect stat attributes for Owner and Group
  • Add exclude parameter to Directory component.
  • Add env parameter to Component.cmd() (and corresponding build_environment parameter to the Build component) to allow adding/overriding environment variables.

1.0b3 (2013-07-09)

  • Enable logging in the remote core to see what’s going on on the remote side.
  • Try to better format exceptions from the remote side.
  • Try harder to get virtualenv back into a working state.
  • Allow remote deployments from root of repository.
  • Make PIP management more robust.

1.0b2 (2013-07-09)

  • Add component to manage PIP within a virtual env.
  • Add component to manage packages with PIP within a virtual env.
  • Restructure buildout component to make it more robust regarding setuptools/distribute preparation. Also remove usage of bootstrap completely as we rely on virtualenv anyway.

1.0b1 (2013-07-09)

  • Apply semantic versioning: initial development is over, so this is 1.0 now.
  • Major revamp of secrets management:
    • switch to GPG (instead of aespipe)
    • turn secrets into a core feature, removing the need for a special component
  • Add ‘–single’ to suppress parallel bootstrapping.
File Type Py Version Uploaded on Size (md5) Source 2014-07-04 109KB