skip to navigation
skip to content

batou 1.0b26

A utility for automating multi-host, multi-environment software builds and deployments.

Package Documentation

Latest Version: 1.6.1


batou is a multi-(component|host|environment|...) deployment utility.

Deployments are described in a model using "components" and can be deployed
locally and remotely. Models are written as Python code.

See the `batou documentation <http:"">`_ for details
on how to use it.


1.0b26 (2014-08-05)

- Enhance git Clone component to support the full suite of functionality the
mercurial one has (incoming/outgoing/local changes etc.).

NOTE: This changes ``batou.lib.git.Clone`` in backward-incompatible ways:
1. The namevar is now only the ``url``. The previously also accepted usage
of passing the ``target`` as the namevar no longer works.
2. ``update_unpinned`` is gone, use the ``branch``/``revision``/``vcs_update``
mechanism instead.

1.0b25 (2014-07-31)

- Improve gpg handling: don't do --no-tty or --batch any longer as we actually
only run the gpg code in (local) terminal environments usually.

1.0b24 (2014-07-04)

- Actually fix unzipping.

- Remove environment option ``connect_method``, and use autodetection instead:
If we're not the configured ``service_user`` after the plain ssh connect,
run sudo.

- Refactor attributes and overrides: attributes allow template expansion now
and also converting default values. The Attribute class has moved to the
batou component module.

- Yield nonzero returncode on remote exceptions.

- Add convenience converter for attributes "list" to handle comma separated
lists of strings.

- Force safety of Nagios service description field: prohibit "," as its not
safe for Nagios business processes.

- Allow specifying custom rsync options for SyncDir: the '--inplace' option
is a good idea in most cases, but not all. Fixes #11846.

- Make supervisor restart/reload more robust towards non-running supervisors
with lying PID files. Fixes #12744.

- Remove harmful decoding step in File verify that caused wrong results.

1.0b23 (2014-06-30)

- Allow unzip to overwrite existing files if the archive has newer copies.

1.0b22 (2014-06-30)

- Fix uncaught import-error on mysql module.

1.0b21 (2014-06-30)

- Fix bug in bundle shipping that made batou depend on the locale (#13465).

- Add MySQL components.

- Introduce batou.utils.Attribute() to avoid imperative type conversions in

1.0b20 (2014-05-13)

- Start using python-requests for HTTP/S downloads to support authentication
(and other goodies).

- Improve error logging.

1.0b19 (2014-04-07)

- Allow specifying alternate find-links in requirements.txt

- Remove remnants of the buildout component's ``extends`` option.

1.0b18 (2014-03-31)

- Introduce environment option ``connect_method``, either 'ssh+sudo' (default)
or plain 'ssh'.

- Improve update code: existing installations would not update cleanly to a new
version due to wrong use of pkg_resources API.
Existing installations need to be cleaned manually. :/

- Archive: recognize ``.tar.xz`` files and extract them with ``tar``.

- Fix bug in CMMI verify that would always raise UpdateNeeded due to a path
mismatch (#13145).

- Component.cmd() now performs expand() on the given command by
default. Saves typing:

self.cmd('rm -rf {{component.workdir}}')

instead of

self.cmd(self.expand('rm -rf {{component.workdir}}'))

- File component: apply character encoding when reading and writing files.

- Mercurial clone: changed update behaviour yet again: unless configured
differently, always achieve target state, even by losing uncommitted changes.

- Allow unnamed access to the last added/ored component using the
'self._' attribute:

def configure(self):
self += Download(...)
self += Extract(

1.0b17 (2013-12-06)

- Mercurial clone: refuse to update if there are uncommitted changes or
outgoing changesets.

- Mercurial clone: added an option to avoid updating existing clones, which is
often desirable during development.

1.0b16 (2013-11-29)

- Improve the bootstrap/upgrade fix for #13049: the bootstrap template needed a
second improvement.

1.0b15 (2013-11-29)

- Try to fix #13049 - will only work once the dev version is already at least
at b15. Not sure whether it really does the job, though. Hard to test.

- Make environment and host arguments for running ``batou local`` optional.
``dev`` and ``localhost`` are the defaults now. Fixes #13034

- Allow specifying secrets with [component:xy] in addition to [xy] to make it
easier to discover when used to the environment overrides. Both options are
supported now. Fixes #13003

- Refactor Python virtualenv and package management. Provide explicit knowledge
about Python versions 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, and 3.3 to use compatible
virtualenv/pip/setuptools versions and commands. Fixes #12941

NOTE: The API for batou.lib.python.Package has changed in
backwards-incompatible ways:

Package() needs to be added to a VirtualEnv component. Before::

self += VirtualEnv(...)
self += Package(...)


venv = VirtualEnv(...)
self += venv
venv += Package(...)

Package parameter ``pip_install_options`` has been renamed to

- Use ``urllib`` module instead of ``wget`` command for cross platform

1.0b14 (2013-11-26)

- Make Buildout component install zc.buildout without dependencies (which is
just setuptools), since that can break the version pinning of setuptools.

- Check whether packages required via a ``-e`` expression in the local
``requirements.txt`` are already installed.

- #12946: Make full path to python executable configurable for virtualenv
(and buildout) component.

- Remove cleanup code that removes batou-0.2-style virtualenv directories (bin,
lib, include etc.) from remote deployments.

- Add mailto setting to crontab component.

- Added some documentation.

1.0b13 (2013-11-19)

- Introduce a plugin mechanism: if a local ``requirements.txt`` file is
present, install its contents into the ``.batou`` virtualenv.

- Load components in alphabetical order. This enables importing from other
components (when naming the directories appropriately).

- #12986: Only try to unbundle if there actually are changes to unbundle.

- Raise an error if a cron job is configured without a timing value.

- Add platform support for Debian (crontab, logrotate, supervisor).

- Added some documentation.

1.0b12 (2013-11-04)

- Added ``branch`` argument to ``mercurial.Clone``. Setting a branch
automatically updates to the branch head on deploy. This is mostly useful for
development environments.

- Create the 'secrets' directory if it doesn't exist, yet. Also, disallow
editing secret files for non-existing environments.

- Support continuing remote bootstrapping if we failed after creating the
initial remote directory but were unable to use Mercurial.

- #12898: build.Configure component was broken when using the default

1.0b11 (2013-10-17)

- #12897: Use non-SSL pypi mirror for downloading virtualenv to fix tests
failing randomly on machines that (for some reason) can't validate PyPI's

- #12911: Ensure that we can configure file owners when they don't
exist during configure phase yet.

- #12912: Fix untested and broken file ownership management.

- #12847: Clean up unicode handling for File and Content components and

- #12910: Remote deployments failed when using bundles for transfers if no
changes needed bundling.

- #12766: Allow bootstrapping a batou project in an existing directory to
support migration from 0.2.

- #12283: Recognize files as 'is_template' by default. Auto-detect source files
in the definition directory if they have the same basename. This is what you
want in 99% of all cases. Explicitly stating either the 'content' or 'source'
parameter disables auto-detection.

Now you can write this:


and have `components/x/foo` recognized as the source file and handled as a

- Use ConfigParser instead of configobj which is effectively unmaintained
(see and support
lists separated by newlines in addition to commas.

1.0b10 (2013-09-27)

- Package our own virtualenv instead of depending on the system-installed one.
This should alleviate troubles due to old virtualenv versions that package
``distribute``, which causes conflicts with recent ``setuptools`` versions

- Update supervisor version to 3.0.

1.0b9 (2013-08-22)

- Update Package component so it ignores installed packages when installing.
This way, we actually install setuptools even when distribute is installed.
(Otherwise it's a no-op since distribute tells pip that setuptools is
already satisfied).

- Fix update process: wrong call to old '.batou/bin/batou' failed and early
bootstrapping would downgrade temporarily which is confusing and superfluous.
Fixes #12739.

1.0b8 (2013-08-17)

- Remove superfluous mkdir call during remote bootstrap.

- Make ``batou init`` print that it's working. Bootstrapping can take a while,
so at least signal that something's going on.

1.0b7 (2013-08-17)

- Depend on Python2.7 to be available on the PATH during early bootstrap.
Otherwise our chances to get a 2.7 virtualenv are pretty small, too.

- Improve project template: ignore the work/ directory by default.

1.0b6 (2013-08-17)

- More MANIFEST inclusions: bootstrap-template.

1.0b5 (2013-08-17)

- Improve MANIFEST so we actually package the init template and other generated
files, like version.txt and requirements.txt.

1.0b4 (2013-08-17)

- Provide a simple project-creation command, both for pip-installed batou's as
well as spawning new projects from existing ones. Fixes #12730

- Fix #12679: make timeouts configurable.

- Removed re-imports from batou main module to support light-weight
self-installation and bootstrapping. I.e. 'from batou import Component' no
longer works.

- Provide a single main command together with a 'bootstrap' wrapper that you
can check into your project and that is maintained during updates
automatically. It also provides fully automatic bootstrapping, installation,
upgrading and other maintenance.

- Fix Python package installation version check.

- Don't use `bin/buildout bootstrap` command anymore. PIP installs a sufficient
`bin/buildout` so buildout can do the rest internally.

- Install zc.buildout during bootstrapping phase using PIP to avoid problems.

- Shorten URLs in the Build component to their basename.

- Add 'assert_cmd' API to support simpler assertions for verify when needing to
check the result of an external command.

- Switch to asking pip installing eggs instead of flat installations as
namespaces seem to collide otherwise.

- Remove non-functional deprecated 'md5sum' attribute.

- Components are context managers now. If you provide __enter__ it will be
called before verify() and if you provide __exit__ this will be called after
update (always - even if update isn't actually called). This allows you to
manage temporary state on the target system more gracefully. See the
DMGExtractor for an example.

- Major refactoring of internal data structures to simplify and improve test
coverage. Some breakage to be expected:

* Components do not have a 'service' attribute any longer. If you need access
to the service base directory use 'environment.base_dir'.

- Another attempt at getting convergency for package installation: actually
leverage pkg_resources to check what version of a package is installed. PIP
does not necessarily list or freeze setuptools, even if forced to reinstall.

- Remove superfluous '--update' from pip calls that would temporarily install
the incorrect package version.

- Improve convenience of the 'secrets-edit' wrapper script to allow
re-encrypting without re-entering the editor.

- Consistently switch to using setuptools.

- Fix #12399: incorrect stat attributes for Owner and Group

- Add ``exclude`` parameter to Directory component.

- Add ``env`` parameter to ``Component.cmd()`` (and corresponding
``build_environment`` parameter to the Build component) to allow
adding/overriding environment variables.

1.0b3 (2013-07-09)

- Enable logging in the remote core to see what's going on on the remote side.

- Try to better format exceptions from the remote side.

- Try harder to get virtualenv back into a working state.

- Allow remote deployments from root of repository.

- Make PIP management more robust.

1.0b2 (2013-07-09)

- Add component to manage PIP within a virtual env.

- Add component to manage packages with PIP within a virtual env.

- Restructure buildout component to make it more robust regarding
setuptools/distribute preparation. Also remove usage of bootstrap completely
as we rely on virtualenv anyway.

1.0b1 (2013-07-09)

- Apply semantic versioning: initial development is over, so this is 1.0 now.

- Major revamp of secrets management:

- switch to GPG (instead of aespipe)

- turn secrets into a core feature, removing the need for a special component

- Add '--single' to suppress parallel bootstrapping.

Developing batou

:Issue tracker:

:Source code:  
File Type Py Version Uploaded on Size (md5) Source 2014-08-05 374KB