Skip to main content

Python egg releaser

Project description

mkrelease is a no-frills Python egg releaser. It is designed to take the cumber out of building and distributing Python eggs.

Also see jarn.viewdoc.

Motivation

Python eggs are great, and we strive to release all software in egg form. However, as projects grow larger and are comprised of more and more eggs, release requirements can become a burden.

This is because it takes some work to put a new egg on a distribution server. After preparing a package for release (update version strings, etc.), we typically have to:

  1. Commit modified files.

  2. Tag the release.

  3. Package up an egg.

  4. Distribute the egg via scp or upload it to an index server.

Now imagine doing this a lot, and the need for automation becomes obvious.

Installation

mkrelease works with Python 2.6 - 3.4 and all released versions of setuptools and distribute.

Use easy_install jarn.mkrelease to install the mkrelease script. Then put it on your system PATH by e.g. symlinking it to /usr/local/bin.

Usage

mkrelease [options] [scm-url [rev]|scm-sandbox]

Options

-C, --no-commit

Do not commit modified files from the sandbox.

-T, --no-tag

Do not tag the release in SCM.

-S, --no-upload

Do not upload the release to dist-location.

-n, --dry-run

Dry-run; equivalent to -CTS.

--svn, --hg, --git

Select the SCM type. Only required if the SCM type cannot be guessed from the argument.

-d dist-location, --dist-location=dist-location

An scp or sftp destination specification, an index server configured in ~/.pypirc, or an alias name for either. This option may be specified more than once.

-s, --sign

Sign the release with GnuPG.

-i identity, --identity=identity

The GnuPG identity to sign with. Implies -s.

-p, --push

Push sandbox modifications upstream.

-e, --develop

Allow version number extensions (i.e. don’t ignore respective setup.cfg options). Implies -T.

-b, --binary

Release a binary (bdist) egg.

-q, --quiet

Suppress output of setuptools commands.

-c config-file, --config-file=config-file

Use config-file instead of the default ~/.mkrelease.

-l, --list-locations

List known dist-locations and exit.

-h, --help

Print the help message and exit.

-v, --version

Print the version string and exit.

scm-url

The URL of a remote SCM repository. The optional rev argument specifies a branch or tag to check out.

scm-sandbox

A local SCM sandbox. Defaults to the current working directory.

Examples

Release my.package and upload it to PyPI:

$ mkrelease -d pypi src/my.package

Release my.package and upload it via scp to the jarn.com server:

$ mkrelease -d jarn.com:/var/dist/public src/my.package

Release my.package using the repository URL instead of a local working copy:

$ mkrelease -d pypi https://svn.jarn.com/public/my.package/trunk

Release a development egg of my.package while suppressing setuptools output:

$ mkrelease -qed jarn.com:/var/dist/private src/my.package

Configuration

mkrelease reads available index servers from the distutils configuration file ~/.pypirc. This file must contain your PyPI account information:

[distutils]
index-servers =
    pypi

[pypi]
username = fred
password = secret

mkrelease also reads its own configuration file ~/.mkrelease. Here’s an example:

[mkrelease]
distbase =
distdefault = public

[aliases]
public =
    jarn.com:/var/dist/public
customerA =
    jarn.com:/var/dist/customerA
world =
    pypi
    public

(Note that pypi refers to the index server pypi as configured in ~/.pypirc above.)

Armed with this configuration we can shorten example 2 to:

$ mkrelease -d public src/my.package

And because public is the default location, we can omit -d entirely:

$ mkrelease src/my.package

Working with SCP

The simplest distribution location is a server directory shared through Apache. Releasing an egg just means scp-ing it to the appropriate place on the server:

$ mkrelease -d jarn.com:/var/dist/public src/my.package

We have a distribution point for every project, so customer A does not see customer B’s releases:

$ mkrelease -d jarn.com:/var/dist/customerB src/my.package

Typing the full destination every time is tedious, even setting up an alias for each and every customer is, so we configure distbase instead:

[mkrelease]
distbase = jarn.com:/var/dist
distdefault = public

[aliases]
world =
    pypi
    public

The distbase is prepended when an scp destination does not contain a host part. We can now write:

$ mkrelease -d public src/my.package
$ mkrelease -d customerB src/my.package

Working with SFTP

To upload via sftp instead of scp, specify the destination in URL form:

$ mkrelease -d sftp://jarn.com/var/dist/public src/my.package

For consistency scp URLs are supported as well:

$ mkrelease -d scp://jarn.com/var/dist/public src/my.package

Note: Unlike scp, the sftp client does not prompt for login credentials. This means that for sftp non-interactive login must be configured on the destination server.

Working with Index Servers

Another way of distributing Python eggs is by uploading them to dedicated index servers, notably PyPI. Given the ~/.pypirc file from above we can release to PyPI by typing:

$ mkrelease -d pypi src/my.package

Index servers are not limited to PyPI though. For example, in the Plone world it is common practice to upload packages to plone.org as well as to PyPI.

We extend our ~/.pypirc to add a second index server:

[distutils]
index-servers =
    pypi
    plone

[pypi]
username = fred
password = secret

[plone]
repository = https://plone.org/products
username = fred
password = secret

This allows us to release to plone.org by typing:

$ mkrelease -d plone src/my.package

The -d option can be specified more than once:

$ mkrelease -d pypi -d plone src/my.package

Alternatively, we can group the servers by creating an alias in ~/.mkrelease:

[aliases]
plone-world =
    pypi
    plone

And type:

$ mkrelease -d plone-world src/my.package

Note: Setuptools rebuilds the egg for every index server it uploads it to. This means that MD5 sums and GnuPG signatures will differ between servers. If this is not what you want, upload to only one server and distribute from there by other means.

Releasing a Tag

Release my.package from an existing Subversion tag:

$ mkrelease -T https://svn.jarn.com/public/my.package/tags/1.0

With Mercurial and Git we can use the second argument to specify the tag:

$ mkrelease -T git@github.com:Jarn/my.package 1.0

Using GnuPG

Release my.package and sign the archive with GnuPG:

$ mkrelease -s -i fred@bedrock.com src/my.package

The -i flag is optional, and GnuPG will pick your default key if not given. In addition, defaults for -s and -i can be configured in ~/.pypirc, on a per-server basis:

[distutils]
index-servers =
    pypi
    plone

[pypi]
username = fred
password = secret
sign = yes
identity = fred@bedrock.com

[plone]
repository = https://plone.org/products
username = fred
password = secret
sign = no

Requirements

The following commands must be available on the system PATH (you only need what you plan to use):

  • svn

  • hg

  • git

  • scp

  • sftp

  • gpg

Limitations

Subversion

The release tag can only be created if the repository follows one of these layouts:

  • The standard Subversion layout: my.package/trunk, my.package/branches, and my.package/tags.

  • The singular-form layout variant: my.package/trunk, my.package/branch, and my.package/tag.

Changelog

3.8 - 2013-11-21

  • Support Python 3.x. [stefan]

3.7 - 2012-08-22

  • Fix compilation of Python source files when the -b option is given. [stefan]

  • Run check command as part of sdist and register commands. [stefan]

  • Add SFTP support. [stefan]

  • Allow sftp:// and scp:// URLs as dist-locations. [stefan]

3.6 - 2012-07-11

  • Handle return code changes in Mercurial 2.1. [stefan]

  • Add setuptools-subversion dependency. [stefan]

  • Support Subversion 1.7 with the help of setuptools-subversion. [stefan]

3.5 - 2011-11-25

  • Allow multiple values for the distdefault config file option. [stefan]

  • Defer list-locations until after all arguments have been parsed. [stefan]

  • Make tests run twice as fast by avoiding Subversion checkouts. [stefan]

3.4 - 2011-11-10

  • Warn if -p is given but no upstream location is found. [stefan]

  • Always push to default in Mercurial. [stefan]

  • Avoid reading empty lines from terminating subprocesses. [stefan]

  • Fix bug in handling of distbase. [stefan]

3.3 - 2011-10-31

  • Add setuptools to the PYTHONPATH for subprocesses. [stefan]

  • Unset any PYTHONPATH while executing SCM commands. [stefan]

  • Support Git’s short-form ssh:// URLs. [stefan]

  • Add -c option to specify a config file other than ~/.mkrelease. [stefan]

3.2 - 2011-10-21

  • Fix the environment passed to subprocesses; Mercurial did not appreciate the mangled PYTHONPATH. [stefan]

  • Allow to specify the branch or tag to check out from Git and Mercurial repositories. [stefan]

  • Adapt to new status output in Subversion 1.6. [stefan]

  • Always include distdefault in list-locations. [stefan]

  • Detect Subversion repos from file:// URLs. [stefan]

  • Detect bare Git repos from file:// URLs. [stefan]

3.1 - 2011-07-19

  • Pass the PYTHONPATH to subprocesses so mkrelease works in zc.buildout environments. [stefan]

  • Improve SCM detection in situations where one or more SCMs are nested. [stefan]

  • Add support for relative file: URLs. [stefan]

  • Depend on lazy instead of carrying a local implementation. [stefan]

3.0.10 - 2011-07-07

  • Add -l option to list known dist-locations (i.e. servers and aliases). [stefan]

  • Drop support for server URLs as dist-locations. Server URLs are not unique. [stefan]

  • Update the Mercurial test repository so tagging tests don’t fail under Mercurial 1.8. [stefan]

3.0.9 - 2010-12-31

  • Rename [defaults] configuration file section to [mkrelease]. [stefan]

  • Various internal code cleanups. [stefan]

3.0.8 - 2010-08-13

  • Avoid underscores in dependency names. [stefan]

  • Handle return code changes in Mercurial 1.6. [stefan]

3.0.7 - 2010-07-07

  • Improve documentation and error messages. [stefan]

3.0.5 - 2010-03-23

  • Allow per-server configuration of -s and -i defaults. [stefan]

  • Support the codespeak.net Subversion repository layout. [stefan]

3.0.4 - 2010-03-16

  • Status checks didn’t use the same path restrictions as commits (Mercurial and Git.) [stefan]

3.0.3 - 2010-03-16

  • Change how we check for existing tags in Subversion repositories. [stefan]

  • Make sandbox-status checks more robust in all three SCMs. [stefan]

3.0.2 - 2010-03-12

  • Add support for Git 1.7. [stefan]

3.0.1 - 2010-02-07

  • Stop when -d pypi is given but no configuration can be found. [stefan]

  • Use gnu_getopt to parse the command line. [stefan]

3.0 - 2010-01-15

  • Switch to -n for dry-run to be consistent with other tools. [stefan]

  • Rename –skip-* long options to –no-* for the same reason. [stefan]

  • Fix a bug in Mercurial and Git sandbox detection. [stefan]

  • Prepare for standalone distutils. [stefan]

2.0.4 - 2010-01-10

  • Improve Git support to handle remotes other than origin. [stefan]

  • Fix SCM detection in ssh:// URLs. [stefan]

2.0.3 - 2010-01-03

  • Add -b option for releasing binary eggs. [stefan]

  • Don’t choke on dirty sandboxes when dry-running. [stefan]

2.0.2 - 2009-08-29

  • Filter meta files (.svn*, .hg*, .git*) and never include them in releases. [stefan]

  • Make sure to clean up all temporary files. [stefan]

2.0.1 - 2009-07-24

  • Fixed bug which could cause mkrelease to issue eggs with faulty manifest files (Symptom: data files not installed). [stefan]

  • The -e flag now implies -T. We never want to tag a development release. [stefan]

2.0 - 2009-07-16

  • Allow command line options to appear after the argument. As in: mkrelease src/my.package -q -d pypi. [stefan]

2.0b2 - 2009-07-09

  • Improve user feedback in the SCM-detection part. [stefan]

  • Document the -e flag. [stefan]

  • Drop global configuration file for YAGNI. [stefan]

  • Allow to set default values for -s and -i in ~/.mkrelease. [stefan]

2.0b1 - 2009-07-03

  • By default, ignore all version number extensions (dev-r12345) that may be configured in setup.cfg. Passing the -e flag disables this safeguard. [witsch, stefan]

  • Delete any existing signature file before signing anew. This keeps GnuPG from complaining about existing (but left-over) files. [stefan]

2.0a2 - 2009-06-27

  • Drop configurable Python and use sys.executable. This also means we now require Python 2.6. [stefan]

  • Force setuptools to only use file-finders for the selected SCM type. This is required to support multi-SCM sandboxes (think git-svn). [stefan]

  • Treat Subversion sandboxes just like the others and avoid the temporary checkout step. [stefan]

  • Remove the -u flag for being pointless. [stefan]

2.0a1 - 2009-06-14

  • Added support for Mercurial and Git. [stefan]

  • Added 250+ unit tests. [stefan]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

jarn.mkrelease-3.8.zip (158.0 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page