-
bup: It backs things up
=======================
for you, but we don't know why. It is also missing some
probably-critical features.
- - It requires python >= 2.4, a C compiler, and an installed git version >=
+ - It requires python >= 2.5, a C compiler, and an installed git version >=
1.5.3.1.
- - It currently only works on Linux, MacOS X >= 10.4, or Windows (with
- Cygwin). Patches to support other platforms are welcome.
+ - It currently only works on Linux, MacOS X >= 10.4,
+ NetBSD, Solaris, or Windows (with Cygwin). Patches to support
+ other platforms are welcome.
Getting started
----------------
+===============
+
+
+From source
+-----------
- Check out the bup source code using git:
- git clone git://github.com/apenwarr/bup
+ git clone git://github.com/bup/bup
+
+ - Install the needed python libraries (including the development
+ libraries).
+
+ On Debian/Ubuntu this is usually sufficient (run as root):
+
+ apt-get install python2.6-dev python-fuse
+ apt-get install python-pyxattr python-pylibacl
+ apt-get install linux-libc-dev
+
+ Substitute python2.5-dev if you have an older system. Alternately,
+ on newer Debian/Ubuntu versions, you can try this:
+
+ apt-get build-dep bup
+
+ On CentOS (for CentOS 6, at least), this should be sufficient (run
+ as root):
+
+ yum groupinstall "Development Tools"
+ yum install python python-dev
+ yum install fuse-python pyxattr pylibacl
+ yum install perl-Time-HiRes
+
+ In addition to the default CentOS repositories, you may need to add
+ RPMForge (for fuse-python) and EPEL (for pyxattr and pylibacl).
+
+ On Cygwin, install python, make, rsync, and gcc4.
- - install the python 2.5 development libraries. On Debian or Ubuntu, this
- is:
- apt-get install python2.5-dev
-
- Build the python module and symlinks:
-
+
make
- Run the tests:
(The tests should pass. If they don't pass for you, stop here and send
me an email.)
-
+
+ - You can install bup via "make install", and override the default
+ destination with DESTDIR and PREFIX.
+
+ Files are normally installed to "$DESTDIR/$PREFIX" where DESTDIR is
+ empty by default, and PREFIX is set to /usr. So if you wanted to
+ install bup to /opt/bup, you might do something like this:
+
+ make install DESTDIR=/opt/bup PREFIX=''
+
+
+From binary packages
+--------------------
+
+Binary packages of bup are known to be built for the following OSes:
+
+ - Debian:
+ http://packages.debian.org/search?searchon=names&keywords=bup
+ - Ubuntu:
+ http://packages.ubuntu.com/search?searchon=names&keywords=bup
+ - pkgsrc (NetBSD, Dragonfly, and others)
+ http://pkgsrc.se/sysutils/bup
+ http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/sysutils/bup/
+
+
+Using bup
+---------
+
+ - Initialize the default BUP_DIR:
+
+ bup init
+
- Try making a local backup as a tar file:
tar -cvf - /etc | bup split -n local-etc -vv
GIT_DIR=~/.bup git log local-etc
- Make a backup on a remote server (which must already have the 'bup' command
- somewhere in the PATH, and be accessible via ssh; make sure to replace
- SERVERNAME with the actual hostname of your server):
+ somewhere in the server's PATH (see /etc/profile, etc/environment,
+ ~/.profile, or ~/.bashrc), and be accessible via ssh.
+ Make sure to replace SERVERNAME with the actual hostname of your server):
tar -cvf - /etc | bup split -r SERVERNAME: -n local-etc -vv
That's all there is to it!
+Notes on FreeBSD
+----------------
+
+- FreeBSD's default 'make' command doesn't like bup's Makefile. In order to
+ compile the code, run tests and install bup, you need to install GNU Make
+ from the port named 'gmake' and use its executable instead in the commands
+ seen above. (i.e. 'gmake test' runs bup's test suite)
+
+- Python's development headers are automatically installed with the 'python'
+ port so there's no need to install them separately.
+
+- To use the 'bup fuse' command, you need to install the fuse kernel module
+ from the 'fusefs-kmod' port in the 'sysutils' section and the libraries from
+ the port named 'py-fusefs' in the 'devel' section.
+
+- The 'par2' command can be found in the port named 'par2cmdline'.
+
+- In order to compile the documentation, you need pandoc which can be found in
+ the port named 'hs-pandoc' in the 'textproc' section.
+
+
+Notes on NetBSD/pkgsrc
+----------------------
+
+ - See pkgsrc/sysutils/bup, which should be the most recent stable
+ release and includes man pages. It also has a reasonable set of
+ dependencies (git, par2, py-fuse-bindings).
+
+ - The "fuse-python" package referred to is hard to locate, and is a
+ separate tarball for the python language binding distributed by the
+ fuse project on sourceforge. It is available as
+ pkgsrc/filesystems/py-fuse-bindings and on NetBSD 5, "bup fuse"
+ works with it.
+
+ - "bup fuse" presents every directory/file as inode 0. The directory
+ traversal code ("fts") in NetBSD's libc will interpret this as a
+ cycle and error out, so "ls -R" and "find" will not work.
+
+ - It is not clear if extended attribute and POSIX acl support does
+ anything useful.
+
+
+Notes on Cygwin
+---------------
+
+ - There is no support for ACLs. If/when some entrprising person
+ fixes this, adjust t/compare-trees.
+
+ - In t/test.sh, two tests have been disabled. These tests check to
+ see that repeated saves produce identical trees and that an
+ intervening index doesn't change the SHA1. Apparently Cygwin has
+ some unusual behaviors with respect to access times (that probably
+ warrant further investigation). Possibly related:
+ http://cygwin.com/ml/cygwin/2007-06/msg00436.html
+
+
+Notes on OS X
+-------------
+
+ - There is no support for ACLs. If/when some entrprising person
+ fixes this, adjust t/compare-trees.
+
+
How it works
-------------
+============
Basic storage:
Help with any of these problems, or others, is very welcome. Join the
mailing list (see below) if you'd like to help.
- - 'bup save' doesn't know about file metadata.
+ - 'bup save' and 'bup restore' have immature metadata support.
- That means we aren't saving file attributes, mtimes, ownership, hard
- links, MacOS resource forks, etc. Clearly this needs to be improved.
+ On the plus side, they actually do have support now, but it's new,
+ and not remotely as well tested as tar/rsync/whatever's. If you'd
+ like to help test, please do (see t/compare-trees for one
+ comparison method).
+
+ In addition, at the moment, if any strip or graft-style options
+ are specified to 'bup save', then no metadata will be written for
+ the root directory. That's obviously less than ideal.
+
+ - bup is overly optimistic about mmap. Right now bup just assumes
+ that it can mmap as large a block as it likes, and that mmap will
+ never fail. Yeah, right... If nothing else, this has failed on
+ 32-bit architectures (and 31-bit is even worse -- looking at you,
+ s390).
+
+ To fix this, we might just implement a FakeMmap[1] class that uses
+ normal file IO and handles all of the mmap methods[2] that bup
+ actually calls. Then we'd swap in one of those whenever mmap
+ fails.
+
+ This would also require implementing some of the methods needed to
+ support "[]" array access, probably at a minimum __getitem__,
+ __setitem__, and __setslice__ [3].
+
+ [1] http://comments.gmane.org/gmane.comp.sysutils.backup.bup/613
+ [2] http://docs.python.org/2/library/mmap.html
+ [3] http://docs.python.org/2/reference/datamodel.html#emulating-container-types
- - There's no 'bup restore' yet.
-
- 'bup save' saves files in the standard git 'tree of blobs' format, so you
- could then "restore" the files using something like 'git checkout'. But
- that's a git command, not a bup command, so it's hard to explain and
- doesn't support retrieving objects from a remote bup server without first
- fetching and packing an entire (possibly huge) pack, which could be very
- slow. Also, like 'bup save', you would need extra features in order to
- properly restore file metadata. And files that bup has split into
- chunks will need to be recombined. Although there's no restore tool,
- 'bup fuse' does accomplish some of this already.
-
- 'bup index' is slower than it should be.
It's still rather fast: it can iterate through all the filenames on my
We'll have to do it in a totally different way. There are lots of
options. For now: make sure you've got lots of disk space :)
- - bup has never been tested on anything but Linux, MacOS, and Linux+Cygwin.
+ - bup has never been tested on anything but Linux, MacOS, and Windows+Cygwin.
There's nothing that makes it *inherently* non-portable, though, so
that's mostly a matter of someone putting in some effort. (For a
"native" Windows port, the most annoying thing is the absence of ssh in
a default Windows installation.)
+
+ - bup needs better documentation.
+
+ According to a recent article about git in Linux Weekly News
+ (https://lwn.net/Articles/380983/), "it's a bit short on examples and
+ a user guide would be nice." Documentation is the sort of thing that
+ will never be great unless someone from outside contributes it (since
+ the developers can never remember which parts are hard to understand).
+
+ - bup is "relatively speedy" and has "pretty good" compression.
+
+ ...according to the same LWN article. Clearly neither of those is good
+ enough. We should have awe-inspiring speed and crazy-good compression.
+ Must work on that. Writing more parts in C might help with the speed.
- bup has no GUI.
Actually, that's not stupid, but you might consider it a limitation.
There are a bunch of Linux GUI backup programs; someday I expect someone
will adapt one of them to use bup.
+
+
+More Documentation
+------------------
+
+bup has an extensive set of man pages. Try using 'bup help' to get
+started, or use 'bup help SUBCOMMAND' for any bup subcommand (like split,
+join, index, save, etc.) to get details on that command.
+
+For further technical details, please see ./DESIGN.
How you can help
-----------------
+================
bup is a work in progress and there are many ways it can still be improved.
If you'd like to contribute patches, ideas, or bug reports, please join the
bup-list+subscribe@googlegroups.com
+Please see <a href="HACKING">./HACKING</a> for
+additional information, i.e. how to submit patches (hint - no pull
+requests), how we handle branches, etc.
+
+
Have fun,
Avery