btrfs-backup 0.3.1

btrfs-backup
============
This project supports incremental backups for *btrfs* using *snapshots*
and *send/receive* between filesystems. Think of it as a basic version
of Time Machine.

Backups can be stored locally and/or remotely (e.g. via SSH). Multi-target
setups are supported as well as dealing with transmission failures
(e.g. due to network outage).

Its main goals are to be **reliable** and **functional** while
maintaining **user-friendliness**. It should be easy to get started in
just a few minutes without detailled knowledge on how btrfs send/receive
works. However, you should have a basic understanding of snapshots and
subvolumes.

btrfs-backup has almost no dependencies and hence is well suited for
many kinds of setups with only minimal maintenance effort.

Originally, it started as a fork of a project with the same name,
written by Chris Lawrence. Since then, most of the code has been
refactored and many new features were added before this repository
has been transferred to me. Many thanks to Chris for his work.
The old code base has been tagged with ``legacy``. If, for any reason,
you want to continue using it and miss the new features, you can check
that out.

:Latest release: v0.3.0
:Downloads: http://pypi.python.org/pypi/btrfs_backup
:Source: https://github.com/efficiosoft/btrfs-backup
:Platforms: Linux >= 3.12, Python >= 3.3
:Keywords: backup, btrfs, snapshot, send, receive, ssh


Features
--------
- Initial creation of full backups
- Incremental backups on subsequent runs
- Different backup storage engines:

- Local storage
- Remote storage via SSH
- Custom storage: Alternatively, the output of ``btrfs send`` may be
piped to a custom shell command.

- Multi-target support with tracking of which snapshots are missing at
each location.
- Retransmission on errors (e.g. due to network outage).
- Simple and configurable retention policy for local and remote
snapshots
- Optionally, create snapshots without transferring them anywhere
and vice versa.
- Creation of backups without root privileges, if some special
conditions are met
- Detailled logging output with configurable log level


Installation
------------
Requirements
~~~~~~~~~~~~
- Python 3.3 or later
- Appropriate btrfs-progs; typically you'll want **at least** 3.12 with
Linux 3.12/3.13
- (optional) OpenSSH's ``ssh`` command - needed for remote backup pulling
and pushing via SSH
- (optional) ``sshfs`` - only needed for pulling via SSH
- (optional) ``pv`` command for displaying progress during backups

Install via PIP
~~~~~~~~~~~~~~~
The easiest way to get up and running with the latest stable version is via PIP. If ``pip3`` is missing
on your system and you run a Debian-based distribution, simply install
it via:

::

$sudoapt-getinstallpython3-pippython3-wheelThen,youcanfetchthelatestversionofbtrfs-backup:::$ sudo pip3 install btrfs_backup

Pre-built packages
~~~~~~~~~~~~~~~~~~
There are pre-built packages available for the following distributions.

- `Arch Linux <https://aur.archlinux.org/packages/python-btrfs-backup>`_ (thanks to XenGi for maintaining this)

Manual installation
~~~~~~~~~~~~~~~~~~~
Alternatively, clone this git repository

::

$gitclonehttps://github.com/efficiosoft/btrfs-backup$ cd btrfs-backup
$\text{You can't use 'macro parameter character \#' in math mode}$ sudo ./setup.py install


Sample usage
------------
Not every feature of btrfs-backup is explained in this README, since
there is a detailled and descriptive help included with the command.

However, there are some sections about the general concepts and different
sample usages to get started as quick as possible.

For reference, a copy of the output of ``btrfs-backup --help`` is
attached below.

As root:

::

$btrfs-backup/home/backupThiswillcreatearead-onlysnapshotof‘‘/home‘‘in‘‘/home/snapshot/YYMMDD-HHMMSS‘‘,andthensenditto‘‘/backup/YYMMDD-HHMMSS‘‘.Onfutureruns,itwilltakeanewread-onlysnapshotandsendthedifferencebetweentheprevioussnapshotandthenewone.\ast \ast Note:Bothsourceanddestinationneedtobeonbtrfsfilesystems.Additionally,thesourcehastobeeithertherootoranyothersubvolume,butnotjustanordinarydirectorybecausesnapshotscanonlybecreatedofsubvolumes.\ast \ast Forthebackuptobesensible,sourceanddestinationshould{n}^{\prime }tbethesamefilesystem.Otherwiseyoucouldjustsnapshotandsavethehassle.Youcanbackupmultiplesubvolumestomultiplesubfoldersorsubvolumesatthedestination.Forexample,youmightwanttobackupboth‘‘/‘‘and‘‘/home‘‘.Themaincaveatisyo{u}^{\prime }llwanttoputthebackupsinseparatefoldersonthedestinationdrivetoavoidconfusion.::$ btrfs-backup / /backup/root
$btrfs-backup/home/backup/homeIfyoureallywanttostorebackupsofdifferentsubvolumesatthesamelocation,youhavetospecifyaprefixusingthe‘‘-p/--snapshot-prefix‘‘option.Withoutthat,btrfs-backupca{n}^{\prime }tdistinguishbetweenyourdifferentbackupchainsandwillmixthemup.Usingtheexamplefromabove,itcouldlooklikethefollowing:::$ btrfs-backup --snapshot-prefix root / /backup
$btrfs-backup--snapshot-prefixhome/home/backupYoucanspecify‘‘-N/--num-snapshots<num>‘‘toonlykeepthelatest‘‘<num>‘‘numberofsnapshotsonthesourcefilesystem.‘‘-n/--num-backups<num>‘‘doesthesamethingforthebackuplocation.RemotebackupsBackinguptoaremoteserverviaSSHisaseasyas:::$ btrfs-backup /home ssh://server/mnt/backups

btrfs-backup doesn't need to be installed on the remote side for this
to work. It is recommended to set up public key authentication to
eliminate the need for entering passwords. A full description of how
to customize the ``ssh`` call can be found in the help text.

Pulling backups from a remote side is now supported as well! Simply use
the ``ssh://`` scheme as source.

You could even do something like:

::

$btrfs-backupssh://sourc{e}_{s}erver/homessh://des{t}_{s}erver/mnt/backupstopullbackupsfrom‘‘sourc{e}_{s}erver‘‘andstorethemat‘‘des{t}_{s}erver‘‘.Thismightbeusedifyouca{n}^{\prime }tinstallbtrfs-backuponeitherremotehostforanyreason.Butkeepinmindthatthisprocedurewillgeneratedoubletraffic(from‘‘sourc{e}_{s}erver‘‘toyouandfromyouto‘‘des{t}_{s}erver‘‘).Okay,justonelastexample,becauseIreallylikethatone:::$ btrfs-backup ssh://source_server/home \
/mnt/backups \
ssh://dest_server/mnt/backups

Can you guess what it does? Well, it does the same as the command before +
an extra sending to your local ``/mnt/backups`` folder. Please note that
btrfs-backup is not smart enough to prevent the same data from being
pulled from ``source_server`` twice. But that wouldn't be easy to
implement with the current design.


Help text
---------
This is the output of ``btrfs-backup --help``. Taking a look at it,
you should get a good insight in what it can and can't do (yet).

::

Cooming at the release.


Configuration files
-------------------
By default, btrfs-backup doesn't read any configuration file. However,
you can create one or more and specify them at the command line:

::

$\text{You can't use 'macro parameter character \#' in math mode}$ btrfs-backup --no-snapshot --locked-dests /home

to retry all failed backup transfers of snapshots of ``/home``. This
could be executed periodically because it just does nothing if there
are no locks.

Snapshots for which locks or parent notes exist are excluded from the
retention policy and won't be purged until the locks are removed either
automatically (because the partially transferred snapshots could be
deleted from the destination) or manually (see below).

As a last resort for removing locks for transfers you don't want to retry
anymore, there is a flag called ``--remove-locks``. Use it with caution
and only if you can assure that there are no corrupt snapshots at the
destinations you apply the flag on.

::

$\text{You can't use 'macro parameter character \#' in math mode}$ mkdir /home/snapshot
$btrfssend/backup/YYMMDD-HHMMSS|btrfsreceive/home/snapshotThenyouneedtotaketheread-onlysnapshotandturnitbackintoarootfilesystem:::$ cp -aR --reflink /home/snapshot/YYMMDD-HHMMSS /home

You might instead have some luck taking the restored snapshot and
turning it into a read-write snapshot, and then re-pivoting your mounted
subvolume to the read-write snapshot.


Alternative workflow
--------------------
An alternative structure is to keep all subvolumes in the root directory

::

/
/active
/active/root
/active/home
/inactive
/snapshot/root/YYMMDD-HHMMSS
/snapshot/home/YYMMDD-HHMMSS

and have corresponding entries in ``/etc/fstab`` to mount the subvolumes
from ``/active/*``. One benefit of this approach is that restoring a
snapshot can be done entirely with btrfs tools:

::

$btrfssend/backup/root/YYMMDD-HHMMSS|btrfsreceive/snapshot/home$ btrfs send /backup/home/YYMMDD-HHMMSS | btrfs receive /snapshot/root
$mv/active/root/inactive$ mv /active/home /inactive
$btrfssubvolumesnapshot/snapshot/root/YYMMDD-HHMMSS/active/root$ btrfs subvolume snapshot /snapshot/home/YYMMDD-HHMMSS /active/home

The snapshots from btrfs-backup may be placed in ``/snapshots/`` by
using the ``--snapshot-folder`` option.


Issues and Contribution
-----------------------
As in every piece of software, there likely are bugs. When you find one,
please open an issue on GitHub. If you do so, please include the output
with debug log level (``-v debug``) and provide steps to reproduce
the problem. Thank you!

If you want to contribute, that's great! You can create issues (even
for feature requests), send pull requests or contact me via email at
r.schindler@efficiosoft.com.


Copyright
---------
.. |copy| unicode:: U+000A9 .. COPYRIGHT SIGN
| Copyright |copy| 2017 Robert Schindler <r.schindler@efficiosoft.com>
| Copyright |copy| 2014 Chris Lawrence <lawrencc@debian.org>