Skip to main content

A utility for automating multi-host, multi-environment software builds and deployments.

Project description

batou
=====

batou is a multi-(component|host|environment|...) deployment utility.

Deployments are described in a model using "components" and can be deployed
locally and remotely. Models are written as Python code.

See the `batou documentation <http://batou.readthedocs.org>`_ for details
on how to use it.


=========
Changelog
=========

1.0b26 (2014-08-05)
===================

- Enhance git Clone component to support the full suite of functionality the
mercurial one has (incoming/outgoing/local changes etc.).

NOTE: This changes ``batou.lib.git.Clone`` in backward-incompatible ways:
1. The namevar is now only the ``url``. The previously also accepted usage
of passing the ``target`` as the namevar no longer works.
2. ``update_unpinned`` is gone, use the ``branch``/``revision``/``vcs_update``
mechanism instead.


1.0b25 (2014-07-31)
===================

- Improve gpg handling: don't do --no-tty or --batch any longer as we actually
only run the gpg code in (local) terminal environments usually.


1.0b24 (2014-07-04)
===================

- Actually fix unzipping.

- Remove environment option ``connect_method``, and use autodetection instead:
If we're not the configured ``service_user`` after the plain ssh connect,
run sudo.

- Refactor attributes and overrides: attributes allow template expansion now
and also converting default values. The Attribute class has moved to the
batou component module.

- Yield nonzero returncode on remote exceptions.

- Add convenience converter for attributes "list" to handle comma separated
lists of strings.

- Force safety of Nagios service description field: prohibit "," as its not
safe for Nagios business processes.

- Allow specifying custom rsync options for SyncDir: the '--inplace' option
is a good idea in most cases, but not all. Fixes #11846.

- Make supervisor restart/reload more robust towards non-running supervisors
with lying PID files. Fixes #12744.

- Remove harmful decoding step in File verify that caused wrong results.


1.0b23 (2014-06-30)
===================

- Allow unzip to overwrite existing files if the archive has newer copies.


1.0b22 (2014-06-30)
===================

- Fix uncaught import-error on mysql module.


1.0b21 (2014-06-30)
===================

- Fix bug in bundle shipping that made batou depend on the locale (#13465).

- Add MySQL components.

- Introduce batou.utils.Attribute() to avoid imperative type conversions in
configure().

1.0b20 (2014-05-13)
===================

- Start using python-requests for HTTP/S downloads to support authentication
(and other goodies).

- Improve error logging.


1.0b19 (2014-04-07)
===================

- Allow specifying alternate find-links in requirements.txt

- Remove remnants of the buildout component's ``extends`` option.



1.0b18 (2014-03-31)
===================

- Introduce environment option ``connect_method``, either 'ssh+sudo' (default)
or plain 'ssh'.

- Improve update code: existing installations would not update cleanly to a new
version due to wrong use of pkg_resources API.
Existing installations need to be cleaned manually. :/

- Archive: recognize ``.tar.xz`` files and extract them with ``tar``.

- Fix bug in CMMI verify that would always raise UpdateNeeded due to a path
mismatch (#13145).

- Component.cmd() now performs expand() on the given command by
default. Saves typing:

self.cmd('rm -rf {{component.workdir}}')

instead of

self.cmd(self.expand('rm -rf {{component.workdir}}'))

- File component: apply character encoding when reading and writing files.
(#13165)

- Mercurial clone: changed update behaviour yet again: unless configured
differently, always achieve target state, even by losing uncommitted changes.

- Allow unnamed access to the last added/ored component using the
'self._' attribute:

def configure(self):
self += Download(...)
self += Extract(self._.target)


1.0b17 (2013-12-06)
===================

- Mercurial clone: refuse to update if there are uncommitted changes or
outgoing changesets.

- Mercurial clone: added an option to avoid updating existing clones, which is
often desirable during development.


1.0b16 (2013-11-29)
===================

- Improve the bootstrap/upgrade fix for #13049: the bootstrap template needed a
second improvement.

1.0b15 (2013-11-29)
===================

- Try to fix #13049 - will only work once the dev version is already at least
at b15. Not sure whether it really does the job, though. Hard to test.

- Make environment and host arguments for running ``batou local`` optional.
``dev`` and ``localhost`` are the defaults now. Fixes #13034

- Allow specifying secrets with [component:xy] in addition to [xy] to make it
easier to discover when used to the environment overrides. Both options are
supported now. Fixes #13003

- Refactor Python virtualenv and package management. Provide explicit knowledge
about Python versions 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, and 3.3 to use compatible
virtualenv/pip/setuptools versions and commands. Fixes #12941

NOTE: The API for batou.lib.python.Package has changed in
backwards-incompatible ways:

Package() needs to be added to a VirtualEnv component. Before::

self += VirtualEnv(...)
self += Package(...)

Now::

venv = VirtualEnv(...)
self += venv
venv += Package(...)

Package parameter ``pip_install_options`` has been renamed to
``install_options``.

- Use ``urllib`` module instead of ``wget`` command for cross platform
compatibility.


1.0b14 (2013-11-26)
===================

- Make Buildout component install zc.buildout without dependencies (which is
just setuptools), since that can break the version pinning of setuptools.

- Check whether packages required via a ``-e`` expression in the local
``requirements.txt`` are already installed.

- #12946: Make full path to python executable configurable for virtualenv
(and buildout) component.

- Remove cleanup code that removes batou-0.2-style virtualenv directories (bin,
lib, include etc.) from remote deployments.

- Add mailto setting to crontab component.

- Added some documentation.


1.0b13 (2013-11-19)
===================

- Introduce a plugin mechanism: if a local ``requirements.txt`` file is
present, install its contents into the ``.batou`` virtualenv.

- Load components in alphabetical order. This enables importing from other
components (when naming the directories appropriately).

- #12986: Only try to unbundle if there actually are changes to unbundle.

- Raise an error if a cron job is configured without a timing value.

- Add platform support for Debian (crontab, logrotate, supervisor).

- Added some documentation.


1.0b12 (2013-11-04)
===================

- Added ``branch`` argument to ``mercurial.Clone``. Setting a branch
automatically updates to the branch head on deploy. This is mostly useful for
development environments.

- Create the 'secrets' directory if it doesn't exist, yet. Also, disallow
editing secret files for non-existing environments.

- Support continuing remote bootstrapping if we failed after creating the
initial remote directory but were unable to use Mercurial.

- #12898: build.Configure component was broken when using the default
prefix.


1.0b11 (2013-10-17)
===================

- #12897: Use non-SSL pypi mirror for downloading virtualenv to fix tests
failing randomly on machines that (for some reason) can't validate PyPI's
certificate.

- #12911: Ensure that we can configure file owners when they don't
exist during configure phase yet.

- #12912: Fix untested and broken file ownership management.

- #12847: Clean up unicode handling for File and Content components and
templating.

- #12910: Remote deployments failed when using bundles for transfers if no
changes needed bundling.

- #12766: Allow bootstrapping a batou project in an existing directory to
support migration from 0.2.

- #12283: Recognize files as 'is_template' by default. Auto-detect source files
in the definition directory if they have the same basename. This is what you
want in 99% of all cases. Explicitly stating either the 'content' or 'source'
parameter disables auto-detection.

Now you can write this:

File('foo')

and have `components/x/foo` recognized as the source file and handled as a
template.

- Use ConfigParser instead of configobj which is effectively unmaintained
(see https://twitter.com/voidspace/status/380699699042213889) and support
lists separated by newlines in addition to commas.


1.0b10 (2013-09-27)
===================

- Package our own virtualenv instead of depending on the system-installed one.
This should alleviate troubles due to old virtualenv versions that package
``distribute``, which causes conflicts with recent ``setuptools`` versions
(#12874).

- Update supervisor version to 3.0.


1.0b9 (2013-08-22)
==================

- Update Package component so it ignores installed packages when installing.
This way, we actually install setuptools even when distribute is installed.
(Otherwise it's a no-op since distribute tells pip that setuptools is
already satisfied).

- Fix update process: wrong call to old '.batou/bin/batou' failed and early
bootstrapping would downgrade temporarily which is confusing and superfluous.
Fixes #12739.

1.0b8 (2013-08-17)
==================

- Remove superfluous mkdir call during remote bootstrap.

- Make ``batou init`` print that it's working. Bootstrapping can take a while,
so at least signal that something's going on.


1.0b7 (2013-08-17)
==================

- Depend on Python2.7 to be available on the PATH during early bootstrap.
Otherwise our chances to get a 2.7 virtualenv are pretty small, too.

- Improve project template: ignore the work/ directory by default.


1.0b6 (2013-08-17)
==================

- More MANIFEST inclusions: bootstrap-template.


1.0b5 (2013-08-17)
==================

- Improve MANIFEST so we actually package the init template and other generated
files, like version.txt and requirements.txt.

1.0b4 (2013-08-17)
==================

- Provide a simple project-creation command, both for pip-installed batou's as
well as spawning new projects from existing ones. Fixes #12730

- Fix #12679: make timeouts configurable.

- Removed re-imports from batou main module to support light-weight
self-installation and bootstrapping. I.e. 'from batou import Component' no
longer works.

- Provide a single main command together with a 'bootstrap' wrapper that you
can check into your project and that is maintained during updates
automatically. It also provides fully automatic bootstrapping, installation,
upgrading and other maintenance.

- Fix Python package installation version check.

- Don't use `bin/buildout bootstrap` command anymore. PIP installs a sufficient
`bin/buildout` so buildout can do the rest internally.

- Install zc.buildout during bootstrapping phase using PIP to avoid
bootstrap.py problems.

- Shorten URLs in the Build component to their basename.

- Add 'assert_cmd' API to support simpler assertions for verify when needing to
check the result of an external command.

- Switch to asking pip installing eggs instead of flat installations as
namespaces seem to collide otherwise.

- Remove non-functional deprecated 'md5sum' attribute.

- Components are context managers now. If you provide __enter__ it will be
called before verify() and if you provide __exit__ this will be called after
update (always - even if update isn't actually called). This allows you to
manage temporary state on the target system more gracefully. See the
DMGExtractor for an example.

- Major refactoring of internal data structures to simplify and improve test
coverage. Some breakage to be expected:

* Components do not have a 'service' attribute any longer. If you need access
to the service base directory use 'environment.base_dir'.

- Another attempt at getting convergency for package installation: actually
leverage pkg_resources to check what version of a package is installed. PIP
does not necessarily list or freeze setuptools, even if forced to reinstall.

- Remove superfluous '--update' from pip calls that would temporarily install
the incorrect package version.

- Improve convenience of the 'secrets-edit' wrapper script to allow
re-encrypting without re-entering the editor.

- Consistently switch to using setuptools.

- Fix #12399: incorrect stat attributes for Owner and Group

- Add ``exclude`` parameter to Directory component.

- Add ``env`` parameter to ``Component.cmd()`` (and corresponding
``build_environment`` parameter to the Build component) to allow
adding/overriding environment variables.


1.0b3 (2013-07-09)
==================

- Enable logging in the remote core to see what's going on on the remote side.

- Try to better format exceptions from the remote side.

- Try harder to get virtualenv back into a working state.

- Allow remote deployments from root of repository.

- Make PIP management more robust.

1.0b2 (2013-07-09)
==================

- Add component to manage PIP within a virtual env.

- Add component to manage packages with PIP within a virtual env.

- Restructure buildout component to make it more robust regarding
setuptools/distribute preparation. Also remove usage of bootstrap completely
as we rely on virtualenv anyway.


1.0b1 (2013-07-09)
==================

- Apply semantic versioning: initial development is over, so this is 1.0 now.

- Major revamp of secrets management:

- switch to GPG (instead of aespipe)

- turn secrets into a core feature, removing the need for a special component

- Add '--single' to suppress parallel bootstrapping.




================
Developing batou
================

:Issue tracker:
https://projects.gocept.com/projects/batou/

:Source code:
https://bitbucket.org/gocept/batou/

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

batou-1.0b26.zip (383.6 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page