]> arthur.barton.de Git - bup.git/log
bup.git
10 years agoAccommodate systems where paths inherit their group from the parent dir.
Rob Browning [Mon, 11 Nov 2013 18:03:25 +0000 (12:03 -0600)]
Accommodate systems where paths inherit their group from the parent dir.

Thanks to Thomas Klausner <tk@giga.or.at> for reporting the problem
and helping test the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years ago_helpers.c: be smarter when converting unknown integer types to python.
Rob Browning [Sun, 10 Nov 2013 17:21:46 +0000 (11:21 -0600)]
_helpers.c: be smarter when converting unknown integer types to python.

For now assume that long long and unsigned long long are always the
the largest integer types, and convert unknown size integers to Python
via PyLong_FromUnsignedLongLong() or PyLong_FromLongLong(), depending
on the value's sign.

Thanks to Thomas Klausner <tk@giga.or.at> for reporting the problem
and helping  track down the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoOnly allow "index --clear" to clear the default index (BUP_DIR/bupindex).
Rob Browning [Fri, 8 Nov 2013 17:35:44 +0000 (11:35 -0600)]
Only allow "index --clear" to clear the default index (BUP_DIR/bupindex).

For now, disallow clearing "external" indexes.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoMakefile: avoid using -Werror with clang for now.
Rob Browning [Sun, 10 Nov 2013 20:11:44 +0000 (14:11 -0600)]
Makefile: avoid using -Werror with clang for now.

Apparently the default compiler on OS X and FreeBSD is now clang, and
it doesn't like our CHECK_VALUE_FITS() definition.  So for now, just
don't specify -Werror when we detect clang.

Thanks to Alexander Barton <alex@barton.de>, Thomas Klausner
<tk@giga.or.at>, Sebastian Schumb <sebastian@sebastians-site.de>, and
Zoran Zaric <zz@zoranzaric.de> for reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Alexander Barton <alex@barton.de>
10 years agotag-cmd.py: add "FIXME: review for safe writes.".
Rob Browning [Sat, 9 Nov 2013 20:17:11 +0000 (14:17 -0600)]
tag-cmd.py: add "FIXME: review for safe writes.".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoAdd 'tag -f' support.
Ben Kelly [Sat, 9 Nov 2013 19:21:04 +0000 (14:21 -0500)]
Add 'tag -f' support.

Add a new flag, -f/--force, to bup tag. By analogy with 'git tag',
'bup tag -f foo commit' will create tag *foo* even if a tag with that
name already exists; the existing tag will be silently overwritten.

By analogy with 'rm -f', add 'bup tag -d -f'. All this does is
suppress the error when you attempt to 'bup tag -d' a tag that doesn't
exist; it is primarily for use in scripts that want to delete a tag
that may or may not exist in the first place without 'bup tag'
reporting a failure.

Signed-off-by: Ben Kelly <btk@google.com>
[rlb@defaultvalue.org: add missing "tag" and wrap text in bup-tag.md;
 put "-d" first in tag-cmd.py optspec line and bup-tag.md; adjust
 commit message.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agobup-index.md: make it clear that *bup* doesn't expand --exclude paths.
Rob Browning [Sat, 9 Nov 2013 18:30:54 +0000 (12:30 -0600)]
bup-index.md: make it clear that *bup* doesn't expand --exclude paths.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoxstat.py: include "s" for socket in mode strings, i.e. "srw-r-----".
Rob Browning [Sat, 9 Nov 2013 18:08:21 +0000 (12:08 -0600)]
xstat.py: include "s" for socket in mode strings, i.e. "srw-r-----".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agotest-meta.sh: add a missing WVPASS in the ownership restoration tests.
Rob Browning [Fri, 8 Nov 2013 17:26:11 +0000 (11:26 -0600)]
test-meta.sh: add a missing WVPASS in the ownership restoration tests.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoindex.py: compute the index metadata offset more carefully.
Rob Browning [Fri, 8 Nov 2013 00:33:43 +0000 (18:33 -0600)]
index.py: compute the index metadata offset more carefully.

Previously when reading bupindex.meta (or similar), bup computed the
metadata file offset for it's internal lookup table by just
subtracting the size of a re-encoding of the metadata from file.tell()
after the read.  This works fine until we change the length of the
metadata representation (which we just did); then bup will get the
wrong length from the re-encoded data.

To fix this, do what we should have done originally and capture the
index metadata offset before reading the metadata via tell(), and use
that instead.

Thanks to Mark J Hewitt <mjh@idnet.com> for being the first (known)
victim and for helping track down the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agocurrent-filesystem(): accommodate NetBSD and SunOS.
Rob Browning [Thu, 7 Nov 2013 18:29:37 +0000 (12:29 -0600)]
current-filesystem(): accommodate NetBSD and SunOS.

Thanks to Thomas Klausner <tk@giga.or.at> for reporting the problem,
and providing the alternate commands.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agobup-restore.md: add more detail to the restore --map-* documentation.
Rob Browning [Wed, 6 Nov 2013 17:50:21 +0000 (11:50 -0600)]
bup-restore.md: add more detail to the restore --map-* documentation.

Thanks to Gabriel Filion <gabster@lelutin.ca> for the suggestions.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
10 years agoAdd --map-user --map-group --map-uid and --map-gid options to restore.
Rob Browning [Tue, 15 Oct 2013 19:20:10 +0000 (14:20 -0500)]
Add --map-user --map-group --map-uid and --map-gid options to restore.

Note that the usual metadata rules still appply, so a user or group
entry will normally take precedence over a uid/gid unless
--numeric-ids is specified.  So if you want to map a uid, for example,
you'll either need --numeric-ids, or you'll need to clear the
user/group like this:

  bup restore ... --map-user rlb= --map-uid 1000=2000

These options should also make it possible to recover from archives
that were broken as a result of our incorrect handling of
signed/unsigned stat values (recently fixed).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
10 years agoTreat stat fields correctly as signed, unsigned, or unspecified (cf. POSIX).
Rob Browning [Sat, 12 Oct 2013 18:59:07 +0000 (13:59 -0500)]
Treat stat fields correctly as signed, unsigned, or unspecified (cf. POSIX).

Patrick reported a problem with restore on Cygwin that was being
caused by bup's handling of unknown uid/gid values; it turns out that
Cygwin returns -1 in that situation:

  http://www.cygwin.com/cygwin-ug-net/ntsec.html
  "Special values of user and group ids"

However, bup didn't see -1; bup saw 4294967295, because it was
incorrectly converting some stat field values as unsigned when POSIX
says that they should be signed (or doesn't specify):

  http://pubs.opengroup.org/onlinepubs/009695299/basedefs/sys/types.h.html

To fix the problem, convert the stat fields correctly -- using the
widest type available (PY_LONG_LONG), and add a new common metadata
record (_rec_tag_common_v2) to store them correctly.  Test at runtime
for potential overflows that we can't rule out at compile time.

Store all of the fields (excepting the timestamp ns values, which we
already force to be non-negative) as signed vints because the overhead
should be negligible, and doing so means that we won't have to break
backward compatibility again if we ever decide we actually want to
store negative values.

Continue to read the legacy format as before, with the realization
that this may leave some existing archives broken.  At the moment, the
only known case is the Cygwin "unknown uid/gid" problem cited above,
and to help with that and similar cases, we'll be adding
--map-{user,group,uid,gid} options to restore.

Thanks to Patrick Rouleau <prouleau72@gmail.com> for the initial
report leading to the deeper problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
10 years agoCorrect thanks for commit ff5e91840c92fbb3af70731ede8696114dfee7aa.
Rob Browning [Wed, 6 Nov 2013 22:29:57 +0000 (16:29 -0600)]
Correct thanks for commit ff5e91840c92fbb3af70731ede8696114dfee7aa.

Thanks to Ben Kelly <btk@google.com> for tracking down the problem,
rather than Alexander Barton <alex@barton.de>.

Original message:

  Author: Rob Browning <rlb@defaultvalue.org>
  Date:   Tue Nov 5 17:17:45 2013 -0600

    Don't include atime when determining hardlink compatibility.

    Don't include the atimes since we're comparing two paths stat()ed at
    different times, and depending on the
    platform/filesystem/mount-options, that may result in two different
    atimes, even when the paths refer to the same file.

    Thanks to Alexander Barton <alex@barton.de> for tracking down the
    problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoDon't include atime when determining hardlink compatibility.
Rob Browning [Tue, 5 Nov 2013 23:17:45 +0000 (17:17 -0600)]
Don't include atime when determining hardlink compatibility.

Don't include the atimes since we're comparing two paths stat()ed at
different times, and depending on the
platform/filesystem/mount-options, that may result in two different
atimes, even when the paths refer to the same file.

Thanks to Alexander Barton <alex@barton.de> for tracking down the
problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
10 years agoUse "type -p" instead of "which" in test.sh.
Rob Browning [Tue, 5 Nov 2013 22:39:31 +0000 (16:39 -0600)]
Use "type -p" instead of "which" in test.sh.

Which isn't portable (if I recall correctly), and we know we're in
Bash.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoAdd current-filesystem and path-filesystems to t/lib.sh; use to guard test.
Rob Browning [Tue, 5 Nov 2013 22:29:53 +0000 (16:29 -0600)]
Add current-filesystem and path-filesystems to t/lib.sh; use to guard test.

Add current-filesystem which reports the current filesystem type, and
path-filesystems which reports all the filesystems back up to the
root, and use the latter to skip the "save --strip-path (no match)"
test whenever all the filesystems involved aren't the same type.

This issue was discovered while testing on
/some/ext4/tree/containing/a/btrfs/tree.  When the test indexes/saves,
it picks up the metadata for the parents above the btrfs, which may
have (for example) linux +e attrs.  Then when it tries to restore the
tree fully within the btrfs, the restore defers an error when it tries
to restore the +e, which isn't valid for btrfs.

Normally we can avoid this problem via --strip, but this particular
test is actually checking the behavior when --strip doesn't apply.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoInclude the chattr value in the ApplyError.
Rob Browning [Tue, 5 Nov 2013 22:25:54 +0000 (16:25 -0600)]
Include the chattr value in the ApplyError.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years ago_apply_linux_xattr_rec(): do nothing if no rec; fix restricted access test.
Rob Browning [Tue, 5 Nov 2013 22:17:04 +0000 (16:17 -0600)]
_apply_linux_xattr_rec(): do nothing if no rec; fix restricted access test.

If the metadata object has no linux_xattr, don't do anything at all in
_apply_linux_xattr_rec(), and don't test the filesystem for xattr
support to determine whether or not to expect an xattr error, because
EACCES trumps the other errors.

This should be more efficient and fix a
test_apply_to_path_restricted_access() failure.

Thanks to Sebastian Schumb <sebastian@sebastians-site.de> for
reporting the problem and helping track down the fix.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agotest.sh: remove "wc -l" quoting until/unless we fix the broader issue.
Rob Browning [Mon, 4 Nov 2013 15:34:17 +0000 (09:34 -0600)]
test.sh: remove "wc -l" quoting until/unless we fix the broader issue.

On some platforms "wc -l" returns whitespace along with the count,
which breaks the one case where we quoted the result, i.e. "$(... | wc
-l").  For now, remove the quoting until/unless we decide to fix all
of the affected invocations.

Thanks to Thomas Klausner <tk@giga.or.at> for reporting the problem on
NetBSD.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agotest-meta.sh: prefer the more portable test '=' to '=='.
Thomas Klausner [Sun, 3 Nov 2013 11:55:27 +0000 (12:55 +0100)]
test-meta.sh: prefer the more portable test '=' to '=='.

Signed-off-by: Thomas Klausner <wiz@NetBSD.org>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoCall date with -d since it's more portable than --date.
Thomas Klausner [Sun, 3 Nov 2013 11:55:25 +0000 (12:55 +0100)]
Call date with -d since it's more portable than --date.

Signed-off-by: Thomas Klausner <wiz@NetBSD.org>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoimport-rsnapshot-cmd.sh: add a few missing failure checks.
Rob Browning [Sun, 3 Nov 2013 17:35:37 +0000 (11:35 -0600)]
import-rsnapshot-cmd.sh: add a few missing failure checks.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoSet pipefail so that "x | y || exit $?" will work properly.
Rob Browning [Sat, 2 Nov 2013 17:55:49 +0000 (12:55 -0500)]
Set pipefail so that "x | y || exit $?" will work properly.

Set pipefail so that constructs like this will work properly:

  x | y | z || exit $?

or

  WVPASS x | WVPASS y | WVPASS z || exit $?

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agobup-index.md: indicate that non-rx --exclude paths are taken literally.
Rob Browning [Sat, 2 Nov 2013 16:26:26 +0000 (11:26 -0500)]
bup-index.md: indicate that non-rx --exclude paths are taken literally.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoindex-cmd.py: pass exclude_rxs as an argument to update_index.
Rob Browning [Sat, 2 Nov 2013 16:15:15 +0000 (11:15 -0500)]
index-cmd.py: pass exclude_rxs as an argument to update_index.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agotest-cat-file.sh: use -l instead of -b; explain differences.
Rob Browning [Thu, 31 Oct 2013 15:23:24 +0000 (10:23 -0500)]
test-cat-file.sh: use -l instead of -b; explain differences.

It turns out that cmp -b doesn't work on at least NetBSD, so use -l
instead, and when there's a problem, diff the "bup meta" output as
well.

Thanks to Thomas Klausner <tk@giga.or.at> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoDon't expect an attr error when the metadata has no attr.
Rob Browning [Fri, 1 Nov 2013 23:32:22 +0000 (18:32 -0500)]
Don't expect an attr error when the metadata has no attr.

When the metadata object has no Linux attr, there won't be an attr
related failure, so don't look for it.

Thanks to Alexander Barton <alex@barton.de> and Mark J Hewitt
<mjh@idnet.com> for reporting and helping track down the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agotmetadata.py: don't look for attr and xattr errors when impossible.
Rob Browning [Tue, 29 Oct 2013 21:21:01 +0000 (16:21 -0500)]
tmetadata.py: don't look for attr and xattr errors when impossible.

Don't look for the errors when the current (file)system doesn't
support the relevant features.

Thanks to Alexander Barton <alex@barton.de> and Mark J Hewitt
<mjh@idnet.com> for reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoMake force-delete more flexible, and more likely to succeed.
Rob Browning [Tue, 29 Oct 2013 01:23:38 +0000 (20:23 -0500)]
Make force-delete more flexible, and more likely to succeed.

Thanks to Alexander Barton <alex@barton.de> for helping track down
the correct chattr invocation.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoDon't use "set -e" in import-rdiff-backup.sh.
Rob Browning [Mon, 28 Oct 2013 18:02:45 +0000 (13:02 -0500)]
Don't use "set -e" in import-rdiff-backup.sh.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoRemove all of the remaining test-related "set -e" usage.
Rob Browning [Sun, 27 Oct 2013 22:42:39 +0000 (17:42 -0500)]
Remove all of the remaining test-related "set -e" usage.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix xattr tests and bugs revealed (fully remove set -e from test-meta.sh).
Rob Browning [Sun, 27 Oct 2013 18:23:20 +0000 (13:23 -0500)]
Fix xattr tests and bugs revealed (fully remove set -e from test-meta.sh).

Fixing the xattr tests to run correctly pointed out that bup stops
trying to apply metadata after the first failure (i.e. after any
metadata record application fails).  Change it so that bup always
tries every metadata record.  Otherwise, bup may skip some forms of
metadata without any notification -- and it shouldn't.  For example,
it shouldn't skip ACLs just because the current filesystem doesn't
support xattrs.

Fix two additional problems that were exposed by the changes described
above.

Check for EACCES when trying to apply an attr record, and properly
convert it to an ApplyError.  This problem was revealed by
test_apply_to_path_restricted_access.

Check for EACCES when trying to read the current filesystem xattrs
during the process of applying the metadata xattrs, and when
encountered, convert it to an ApplyError.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix ACL tests, and the bugs revealed thereby.
Rob Browning [Sun, 27 Oct 2013 18:02:08 +0000 (13:02 -0500)]
Fix ACL tests, and the bugs revealed thereby.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoRemove more of the "set -e" dependency from test-meta.sh.
Rob Browning [Sun, 27 Oct 2013 16:25:53 +0000 (11:25 -0500)]
Remove more of the "set -e" dependency from test-meta.sh.

Remove the "set -e" dependency from the remaining code where doing so
doesn't reveal any bugs.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix the "set -e" problem in the test-meta.sh chattr tests.
Rob Browning [Sat, 26 Oct 2013 20:03:12 +0000 (15:03 -0500)]
Fix the "set -e" problem in the test-meta.sh chattr tests.

Start fixing the "set -e" problem, beginning with the test-meta.sh
chattr tests, and add support to the test filesystem for options
needed by the chattr calls.

To allow us to produce a more digestable sequence of patches for the
broader "set -e" problem, temporarily establish BUP_SKIP_BROKEN_TESTS
which will do what it says when set, and until we finish repairing
everything, it must be set during any testing:

  BUP_SKIP_BROKEN_TESTS=1 make check

The fundamental issue with "set -e" is that it has some unexpected
(but documented) semantics.  For example:

  set -e
  foo() { false; rm -rf /; }

  # Effectively harmless
  foo

  # Erase the filesystem
  foo || true

If nothing else, this behavior appears to have been masking test
failures.  Better to just avoid "set -e" altogether.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoAdd /t/tmp/ to .gitignore.
Phil Hord [Fri, 1 Nov 2013 21:25:01 +0000 (17:25 -0400)]
Add /t/tmp/ to .gitignore.

Signed-off-by: Phil Hord <hordp@cisco.com>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoREADME.md: fix typo; cited article is about 'git'.
Phil Hord [Fri, 1 Nov 2013 21:25:00 +0000 (17:25 -0400)]
README.md: fix typo; cited article is about 'git'.

Signed-off-by: Phil Hord <hordp@cisco.com>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agometa-cmd.py: actually respond to recursion options.
Rob Browning [Thu, 31 Oct 2013 16:57:01 +0000 (11:57 -0500)]
meta-cmd.py: actually respond to recursion options.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoRemove misleading -1 from BUP_WINDOWSIZE definition.
Aidan Hobson Sayers [Sun, 20 Oct 2013 19:32:15 +0000 (20:32 +0100)]
Remove misleading -1 from BUP_WINDOWSIZE definition.

Signed-off-by: Aidan Hobson Sayers <aidanhs@cantab.net>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
10 years agoCorrect DESIGN to refer to 64 byte rolling window.
Aidan Hobson Sayers [Sun, 20 Oct 2013 19:32:14 +0000 (20:32 +0100)]
Correct DESIGN to refer to 64 byte rolling window.

Signed-off-by: Aidan Hobson Sayers <aidanhs@cantab.net>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
10 years agoHACKING: correct output folder in the format-patch example
Aidan Hobson Sayers [Thu, 24 Oct 2013 19:57:31 +0000 (20:57 +0100)]
HACKING: correct output folder in the format-patch example

Signed-off-by: Aidan Hobson Sayers <aidanhs@cantab.net>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix top-level non-dir metadata restoration with /foo/latest/target/ syntax.
Rob Browning [Tue, 22 Oct 2013 17:06:56 +0000 (12:06 -0500)]
Fix top-level non-dir metadata restoration with /foo/latest/target/ syntax.

Previously something like "bup restore -C dest /foo/latest/bar/" would
fail to restore metadata for top-level, non-directory paths inside
bar.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoIgnore atime when testing bup cat-file --meta.
Rob Browning [Wed, 23 Oct 2013 20:43:38 +0000 (15:43 -0500)]
Ignore atime when testing bup cat-file --meta.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoDon't create ACL objects until they're needed.
Rob Browning [Mon, 21 Oct 2013 20:57:40 +0000 (15:57 -0500)]
Don't create ACL objects until they're needed.

Previously bup would store the POSIX1e ACL information in metadata
objects as pylibacl objects.  That meant bup would have to be able to
create those objects immediately when loading the metadata record.

Aside from being unnecessary, that was causing top-level crashes in
cases where the ACL object creation failed.  That's because bup would
throw an exception (deferred via add_error()) without skipping the ACL
metadata record content.  After that, all metadata reads would be out
of sync, resulting in erroneous reads, and an eventual EOFError.

To fix the problem, don't convert the encoded ACL data to pylibacl
objects until necessary (i.e. unless trying to apply the ACLs to the
filesystem via bup restore).  With that change bup should no longer
get out of sync when reading the metadata, and in cases where it would
be impossible to create the pylibacl objects (i.e. the local machine's
user/groups won't allow it), bup won't just drop the metadata.  For
example,

  bup meta --edit --set-user rlb < metadata.old > metadata.new

will no longer silently drop ACL information in metadata.old on error.

Thanks to Anton Eliasson <devel@antoneliasson.se> for reporting the
problem and helping track it down.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agometadata_file(): don't load metadata objects for all the files in a dir.
Rob Browning [Fri, 18 Oct 2013 23:30:01 +0000 (18:30 -0500)]
metadata_file(): don't load metadata objects for all the files in a dir.

When only asked for the .bupm via metdata_file(), don't also parse the
.bupm and create all the metadata objects for the dir.  Aside from
being potentially wasteful, that may also make it impossible to
retrieve a broken .bupm via "cat-file --bupm" (for analysis).

Thanks to Anton Eliasson <devel@antoneliasson.se> for helping find the
problem and test the fix.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoAdd "bup cat-file [--meta|--bupm] /branch/revision/[path]".
Rob Browning [Tue, 15 Oct 2013 02:29:45 +0000 (21:29 -0500)]
Add "bup cat-file [--meta|--bupm] /branch/revision/[path]".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix id-other-than usage message.
Rob Browning [Tue, 22 Oct 2013 15:56:19 +0000 (10:56 -0500)]
Fix id-other-than usage message.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoOnly forward signals when appropriate; on SIGTSTP suspend parent too.
Rob Browning [Fri, 18 Oct 2013 18:46:05 +0000 (13:46 -0500)]
Only forward signals when appropriate; on SIGTSTP suspend parent too.

Finish the main.py work, started in
aa6f2c87e6f1292f1fa22f618532b65a5565d604, which began forwarding
SIGTSTP and SIGCONT to the child process so that it would
suspend/resume with the parent.

Only forward signals to the child when appropriate -- guarded by p and
forward_signals.  After forwarding SIGTSTP (as SIGSTOP) to the child,
reset the SIGTSTP handler to its default behavior and re-send SIGTSTP
to the parent so that it will actually suspend too.

Thanks to Karl Kiniger <karl.kiniger@med.ge.com> for reporting the
problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoUpdate version extraction for _version.py to work with the new tag syntax.
Patrick Rouleau [Mon, 30 Sep 2013 04:10:50 +0000 (00:10 -0400)]
Update version extraction for _version.py to work with the new tag syntax.

The version tag used to be in the form "bup-<version>".

The syntax is now just "<version>" to allow github to produce
reasonable archives (i.e. not bup-bup-VERSION).

Signed-off-by: Patrick Rouleau <prouleau72@gmail.com>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agobloom-cmd.py: close open file object for tfname before attempting a rename.
Rob Browning [Sun, 6 Oct 2013 19:33:35 +0000 (14:33 -0500)]
bloom-cmd.py: close open file object for tfname before attempting a rename.

Ben Kelly reported a an IOException (permission denied) during "bup
bloom" on Cygwin.  The exception was caused by the attempt to
os.rename(tfname, outfilename) in do_bloom().

This appears to be because Cygwin doesn't allow you to rename a file
while it's still open.  At first, we thought this might be because
bloom.py create() opens tfname, passes the resulting file to ShaBloom,
and then no one ever explicitly closes it.  So if the gc hasn't
collected the ShaBloom object before the rename, we're in trouble.

However, the problem may also have been caused by an unrelated process
scanning the file at the same time.

In any case, for the moment, explicitly close the ShaBloom object
before the rename.  That shouldn't hurt, and may help.

Thanks to Ben Kelly <btk@google.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFall back from mknod() to socket()/bind() on EPERM for SOCK_STREAM.
Rob Browning [Tue, 8 Oct 2013 16:18:00 +0000 (11:18 -0500)]
Fall back from mknod() to socket()/bind() on EPERM for SOCK_STREAM.

Apparently OS X is throwing EPERM for mknod(... SOCK_STREAM), and the
current OS X manpage does state that mknod() requires superuser
privileges.

Thanks to Zoran Zaric <zz@zoranzaric.de> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoHandle EOPNOTSUPP as an indicator that Linux attrs are not supported.
Rob Browning [Sat, 5 Oct 2013 18:01:37 +0000 (13:01 -0500)]
Handle EOPNOTSUPP as an indicator that Linux attrs are not supported.

Add EOPNOTSUPP to the set of errno values in _add_linux_attr() that
indicate Linux attrs are not supported by the target filesystem.  The
problem was observed on ZFS-on-Linux, 0.6.2.

Thanks to Will Rouesnel <w.rouesnel@gmail.com> for the report and an
initial patch.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoBe more careful when testing that repeated saves can produce the same tree.
Rob Browning [Sat, 5 Oct 2013 01:58:07 +0000 (20:58 -0500)]
Be more careful when testing that repeated saves can produce the same tree.

Previously, test.sh would run multiple saves with no intervening
modifications, and check to see that the top-level "save -t" hash
didn't change, but with the addition of metadata this may fail on some
platforms where the test process itself (the save runs, etc.) affects
atime values.  That's because save is pulling the metadata for
unindexed parent directories directly from the filesystem.  And note
that atime is only special here because the test itself may
incidentally alter it -- we would see the same problem if any of the
other metdata for one of the unindexed parent directories changed
during the test run (i.e. mtime, uid, gid, ...).

To fix this, move the relevant test code from test.sh to
test-redundant-saves.sh, and use a newly added subdir-hash tool so
that we can compare the hashes of the roots of the indexed subtrees
instead of the top-level "save -t" hashes, ignoring the unindexed
parents entirely.

Thanks to Robert Edmonds <edmonds@debian.org> for running rc3 through
the Debian buildds which lead to the kFreeBSD test failure that
demonstrated the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoMove _helper.c's Python.h back up (now to the top), because Python. 0.25-rc3
Rob Browning [Mon, 23 Sep 2013 00:18:24 +0000 (19:18 -0500)]
Move _helper.c's Python.h back up (now to the top), because Python.

Move the Python.h include in _helper.c to the top because Python says
so:

  http://docs.python.org/2/c-api/intro.html#include-files

Though according to this, it's been fixed upstream:

  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=206805

Which might explain why I didn't see the problem here.

Thanks to Chandan Kumar <chandank.kumar@gmail.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoMove non-system includes after system includes in _helpers.c.
Rob Browning [Fri, 20 Sep 2013 18:33:02 +0000 (13:33 -0500)]
Move non-system includes after system includes in _helpers.c.

Having Python.h before assert.h was apparently causing trouble on
OpenBSD:

  _helpers.c: In function 'bup_xutime_ns':
  _helpers.c:842: warning: implicit declaration of function 'utimensat'
  _helpers.c:842: error: 'AT_FDCWD' undeclared (first use in this function)
  _helpers.c:842: error: (Each undeclared identifier is reported only once
  _helpers.c:842: error: for each function it appears in.)
  _helpers.c:843: error: 'AT_SYMLINK_NOFOLLOW' undeclared (first use in this function)
  error: command 'cc' failed with exit status 1

...and moving the non-sytem includes lower is reasonable anyway.

Thanks to Ryan Hinton <iobass@email.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoconfigure: since the *BSDs have GNU make as gmake, look for that too.
Rob Browning [Fri, 20 Sep 2013 17:14:09 +0000 (12:14 -0500)]
configure: since the *BSDs have GNU make as gmake, look for that too.

Thanks to Ryan Hinton <iobass@email.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoIgnore unimplemented metadata record during read (_rec_tag_nfsv4_acl).
Rob Browning [Wed, 18 Sep 2013 17:20:28 +0000 (12:20 -0500)]
Ignore unimplemented metadata record during read (_rec_tag_nfsv4_acl).

Don't test for _rec_tag_nfsv4_acl and then try to call an
unimplemented method in Metadata.read().  Otherwise, if/when we do add
support, older versions of bup will choke on it.

Thanks to Alexander Barton <alex@barton.de> for pointing this out.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoQuash stat(2) st_rdev unless we're going to need it for mknod().
Rob Browning [Tue, 17 Sep 2013 01:47:28 +0000 (20:47 -0500)]
Quash stat(2) st_rdev unless we're going to need it for mknod().

On some platforms (i.e. kFreeBSD), the st_rdev value isn't completely
stable.  For example, given "date > foo; cp -a foo foo-2", the st_rdev
value may differ between the two files.

Since we only need the st_rdev value for the call to mknod() when
restoring character and block special files, set it to zero for
anything else.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agowrite_idx: make filename char* non-const (fixes python 2.6.6 compatibility).
Rob Browning [Tue, 17 Sep 2013 22:17:55 +0000 (17:17 -0500)]
write_idx: make filename char* non-const (fixes python 2.6.6 compatibility).

Thanks to Alexander Barton <alex@barton.de> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoMention --git-ids in bup split -h output sample commands.
Rob Browning [Tue, 17 Sep 2013 21:33:49 +0000 (16:33 -0500)]
Mention --git-ids in bup split -h output sample commands.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoUpdate bup split -h output (optspec) to distinguish between split "modes".
Gabriel Filion [Sun, 8 Sep 2013 03:26:16 +0000 (23:26 -0400)]
Update bup split -h output (optspec) to distinguish between split "modes".

Split it into three lines to clearly show that some modes don't fare
well with others.

Signed-off-by: Gabriel Filion <gabster@lelutin.ca>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoUpdate bup-split(1); document -d, clarify the split "modes", and reorganize.
Gabriel Filion [Sun, 8 Sep 2013 03:26:15 +0000 (23:26 -0400)]
Update bup-split(1); document -d, clarify the split "modes", and reorganize.

Update the synopsis, document the "-d DATE" option, and make a clearer
distinction between normal options and the options that select the
split "mode".

Signed-off-by: Gabriel Filion <gabster@lelutin.ca>
[rlb@defaultvalue.org: add dashes to seconds-since-epoch; adjust the
 MODES text a bit and move -b down; make a few other edits and adjust
 commit message.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix ./configure handling of OS_* for GNU/kFreeBSD.
Rob Browning [Tue, 17 Sep 2013 02:11:09 +0000 (21:11 -0500)]
Fix ./configure handling of OS_* for GNU/kFreeBSD.

Thanks to Robert Edmonds <edmonds@debian.org> for the initial report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix path ownership restoration problems on Cygwin.
Rob Browning [Sat, 14 Sep 2013 18:47:23 +0000 (13:47 -0500)]
Fix path ownership restoration problems on Cygwin.

It turns out that Cygwin won't allow you to chown() a path to an
unknown uid or gid, even when "root".

For now, make that a deferred error on Cygwin, rework the tests to
avoid it when possible, and disable the tests (on Cygwin) that require
it.

For the record, it appears that tar doesn't normally hit this problem
on Cygwin because it uses "geteuid() == 0" to detect super-user
status, which won't be true in the normal case, even if the user is an
administrator.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoTreat ENOSYS like EOPNOTSUPP in _add_posix1e_acl().
Rob Browning [Mon, 16 Sep 2013 21:47:53 +0000 (16:47 -0500)]
Treat ENOSYS like EOPNOTSUPP in _add_posix1e_acl().

Thanks to Robert Edmonds <edmonds@debian.org> for reporting the
relevant failure on a Debian kFreeBSD buildd.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoFix minor typo in Metadata.read() in metadata.py.
Rob Browning [Mon, 16 Sep 2013 18:07:15 +0000 (13:07 -0500)]
Fix minor typo in Metadata.read() in metadata.py.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoAlways fall back to socket()/bind() when os.mknod(...S_IFSOCK) fails.
Rob Browning [Mon, 16 Sep 2013 17:56:27 +0000 (12:56 -0500)]
Always fall back to socket()/bind() when os.mknod(...S_IFSOCK) fails.

Previously bup would use socket()/bind() instead of os.mknod(... |
stat.S_IFSOCK) on Cygwin, but this issue isn't Cygwin specific.
Remove the platform conditionalization, and fall back to
socket()/bind() any time mknod() fails with EINVAL.

Thanks to Robert Edmonds <edmonds@debian.org> for reporting the
relevant failure on a Debian kFreeBSD buildd.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoDon't try to load POSIX1e ACLs when the library isn't installed.
Rob Browning [Mon, 16 Sep 2013 17:55:12 +0000 (12:55 -0500)]
Don't try to load POSIX1e ACLs when the library isn't installed.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agobup-init.md: update after dropping automatic ~/.bup init.
Zoran Zaric [Tue, 13 Aug 2013 06:01:39 +0000 (08:01 +0200)]
bup-init.md: update after dropping automatic ~/.bup init.

Signed-off-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: adjust commit message; remove escape from
 BUP\_DIR after list discussion; remove additional "bup init" from
 README.md example.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoStop interleaving stream and mmap IO operations when writing the index.
Rob Browning [Fri, 23 Aug 2013 03:57:03 +0000 (22:57 -0500)]
Stop interleaving stream and mmap IO operations when writing the index.

Previously bup would write the index via a simultaneous combination of
stream operations and mmap writes to non-overlapping, but adjacent
regions of the same file.  This was causing index corruption in some
cases.

Instead, make an extra pass over the data in memory to precompute the
size of the final index section, which contains any 31+-bit offsets.
Then mmap and write the entire set of tables directly, avoiding the
need for simultaneous stream operations.

Reported-by: Alan Jenkins <alan.christopher.jenkins@gmail.com>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Greg Troxel <gdt@lexort.com>
Reviewed-by: Greg Troxel <gdt@lexort.com>
10 years agoMention tmp/review/ branches in HACKING.
Zoran Zaric [Wed, 28 Aug 2013 14:56:48 +0000 (16:56 +0200)]
Mention tmp/review/ branches in HACKING.

Signed-off-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: make minor wording changes.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoPreserve filesystem timestamps during "make install".
Christopher Meng [Sun, 25 Aug 2013 14:32:27 +0000 (10:32 -0400)]
Preserve filesystem timestamps during "make install".

A good idea regardless, and recommended by the Fedora and Debian
packaging guidelines.

From debian-policy:

  The rationale is that there is some information conveyed by knowing
  the age of the file, for example, you could recognize that some
  documentation is very old by looking at the modification time, so it
  would be nice if the modification time of the upstream source would
  be preserved.

It should also help prevent unnecessary backup churn after upgrades.

Signed-off-by: Christopher Meng <cickumqt@gmail.com>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
Tested-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoAdd -l and --human-readable options to "bup ls".
Kumar Appaiah [Sun, 25 Aug 2013 19:02:18 +0000 (15:02 -0400)]
Add -l and --human-readable options to "bup ls".

When -l is specified, include the size in bytes for each item.

When --human-readable is also specified, print sizes like 3.5K, 1.8G,
etc., instead of the exact byte count.

Signed-Off-By: Kumar Appaiah <a.kumar@alumni.iitm.ac.in>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: adjust commit message; adjust newlines; squash
 two of Kumar's patches into this one.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoAdd --human-readable option to "bup web".
Kumar Appaiah [Sun, 25 Aug 2013 19:02:16 +0000 (15:02 -0400)]
Add --human-readable option to "bup web".

When --human-readable is specified, print sizes like 3.5K, 1.8G,
etc., instead of the exact byte count.

Signed-Off-By: Kumar Appaiah <a.kumar@alumni.iitm.ac.in>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: adjust commit message; squash two of Kumar's
 patches into this one.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agosave-cmd.py: don't write an irrelevant and incomplete .bupm fragment.
Rob Browning [Sun, 25 Aug 2013 22:33:42 +0000 (17:33 -0500)]
save-cmd.py: don't write an irrelevant and incomplete .bupm fragment.

When finishing up (leaving) a directory and preparing to store its
related tree in the repository, don't write the pending .bupm (which
is still incomplete) to the pack if we know the directory already
exists in the repository (i.e. when we're not going to write it
either).

This problem was discovered when Zoran noticed that two consecutive
saves without an intervening index could produce a repository with an
unreferenced blob (according to git fsck).

Reported-by: Zoran Zaric <zz@zoranzaric.de>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agomain.py: forward SIGTSTP/SIGCONT so "C-z" will actually suspend everything.
Rob Browning [Fri, 23 Aug 2013 17:23:17 +0000 (12:23 -0500)]
main.py: forward SIGTSTP/SIGCONT so "C-z" will actually suspend everything.

Catch and forward SIGTSTP (as SIGSTOP) and SIGCONT to the subprocess
as we already do for SIGTERM and SIGINT so that the subprocess will
also suspend/resume.

This still leaves bup with potentially unexpected behavior since the
(detached) subprocess will never see a SIGSTOP delivered to the parent
(because SIGSTOP can't be intercepted and forwarded).  This is due to
the os.setsid() call that was originally introduced to support current
newliner arrangement (cf. b7a524ccb662c9ed3ebd786da0f45f459929ef45).

Thanks to Kalle for the report.

Reported-by: krichter722@aol.de
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agosplit-cmd.py: change -N to --noop (which actually exists) in error messages.
Gabriel Filion [Fri, 2 Aug 2013 20:31:03 +0000 (16:31 -0400)]
split-cmd.py: change -N to --noop (which actually exists) in error messages.

Signed-off-by: Gabriel Filion <gabster@lelutin.ca>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: adjusted commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoREADME.md: add a "bup init" step to "Using bup" since we killed auto-init.
Rob Browning [Sat, 17 Aug 2013 19:59:45 +0000 (14:59 -0500)]
README.md: add a "bup init" step to "Using bup" since we killed auto-init.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agogit.py: don't automatically initialize ~/.bup if it doesn't exist.
Zoran Zaric [Mon, 12 Aug 2013 14:20:12 +0000 (16:20 +0200)]
git.py: don't automatically initialize ~/.bup if it doesn't exist.

bup had a convenience feature where commands would automagically
initialize a repo in ~/.bup if it didn't exist and no other BUP_DIR
was given.

This had the odd effect that when one forgot to specify BUP_DIR, a bup
repo would be initialized in ~/.bup even though only a browsing
command was used.

This patch drops that behaviour.  Now all repositories must be
explicitly intiialized via "bup init".

Signed-off-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: edit commit message; fix test ("set +e" during
 init run).]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agofuse-cmd.py: match the style of the import failure message in web-cmd.py.
Rob Browning [Sat, 17 Aug 2013 19:14:42 +0000 (14:14 -0500)]
fuse-cmd.py: match the style of the import failure message in web-cmd.py.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoweb-cmd.py: accommodate HTTPServer s/_socket/_sockets/ in newer versions.
Rob Browning [Sat, 17 Aug 2013 19:07:10 +0000 (14:07 -0500)]
web-cmd.py: accommodate HTTPServer s/_socket/_sockets/ in newer versions.

Newer versions of tornado changed HTTPServer._socket to
HTTPServer._sockets, with slightly different semantics.  Handle both.

Thanks to Andy Somerville <andy.somerville@gmail.com> for reporting
the problem and proposing an initial patch.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoweb-cmd.py: add a tornado import guard like the one for fuse in fuse-cmd.py.
Robert S. Edmonds [Sat, 10 Aug 2013 20:01:57 +0000 (16:01 -0400)]
web-cmd.py: add a tornado import guard like the one for fuse in fuse-cmd.py.

In Debian, we replace the embedded copy of tornado with a Recommends on
the python-tornado package. Print a nice error message instead of a
backtrace if the user doesn't have it installed.

Signed-off-by: Robert S. Edmonds <edmonds@debian.org>
[rlb@defaultvalue.org: edited commit and error message]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agotest-meta.sh: use the more portable "touch -t" instead of "touch -d".
Rob Browning [Fri, 16 Aug 2013 01:23:20 +0000 (20:23 -0500)]
test-meta.sh: use the more portable "touch -t" instead of "touch -d".

Thanks to Greg Troxel <gdt@work.lexort.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoDESIGN: fix typo.
Rob Browning [Sun, 11 Aug 2013 20:05:41 +0000 (15:05 -0500)]
DESIGN: fix typo.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years ago_helpers.c: be careful about memory alignment in bloom operations.
Julien Cristau [Thu, 8 Aug 2013 22:30:26 +0000 (18:30 -0400)]
_helpers.c: be careful about memory alignment in bloom operations.

Debian (Bug#641488) discovered via the bloom test that bup crashes on
sparc because it's trying to access unaligned memory.  Fix that.

Signed-off-by: Julien Cristau <jcristau@debian.org>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agoDocumentation/bup-tag.md: fix spelling error "retreive"
Robert S. Edmonds [Fri, 9 Aug 2013 01:12:48 +0000 (21:12 -0400)]
Documentation/bup-tag.md: fix spelling error "retreive"

Signed-off-by: Robert S. Edmonds <edmonds@debian.org>
Reviewed-by Zoran Zaric <zz@zoranzaric.de>

10 years agoSkip test_handling_of_incorrect_existing_linux_xattrs() when under fakeroot.
Robert S. Edmonds [Fri, 9 Aug 2013 00:31:38 +0000 (20:31 -0400)]
Skip test_handling_of_incorrect_existing_linux_xattrs() when under fakeroot.

Don't try to do a loopback mount when running under fakeroot ("mount:
only root can do that").

Signed-off-by: Robert S. Edmonds <edmonds@debian.org>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
10 years agot/compare-trees: check rsync capability support correctly.
Rob Browning [Sun, 4 Aug 2013 18:39:55 +0000 (13:39 -0500)]
t/compare-trees: check rsync capability support correctly.

Handle older versions of rsync (where nothing was printed), and for
newer versions, check for "no FOO" rather than "FOO", since FOO will
always be listed one way or the other.

Handle ACL and xattr support independently.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Zoran Zaric <zz@zoranzaric.de>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
10 years agoconfig/configure: ignore utimensat on GNU/kFreeBSD.
Gonéri Le Bouder [Thu, 8 Aug 2013 22:30:28 +0000 (18:30 -0400)]
config/configure: ignore utimensat on GNU/kFreeBSD.

Even though we'll find it, it won't work.

See also:

  http://bugs.debian.org/651104
  http://lists.debian.org/debian-bsd/2011/03/msg00063.html

Signed-off-by: Gonéri Le Bouder <goneri@rulezlan.org>
[rlb@defaultvalue.org: adjust indentation, comments, and message.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years ago_apply_linux_attr_rec: treat ENOSYS like EOPNOTSUPP (seen via sshfs). bup-0.25-rc2
Rob Browning [Wed, 31 Jul 2013 16:34:29 +0000 (11:34 -0500)]
_apply_linux_attr_rec: treat ENOSYS like EOPNOTSUPP (seen via sshfs).

After reports that set_linux_file_attr() produces ENOSYS on sshfs,
raise raise ApplyError for ENOSYS too.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years ago_create_via_common_rec: treat rmdir() EEXIST like ENOTEMPTY.
Rob Browning [Wed, 31 Jul 2013 16:32:17 +0000 (11:32 -0500)]
_create_via_common_rec: treat rmdir() EEXIST like ENOTEMPTY.

See rmdir(2):

       ENOTEMPTY
              pathname contains entries other than .  and ..  ; or,
              pathname has ..  as its final component.  POSIX.1-2001
              also allows EEXIST for this condition.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoCurrently bup only indexes physical parent paths; adjust tests.
Zoran Zaric [Mon, 29 Jul 2013 17:23:10 +0000 (19:23 +0200)]
Currently bup only indexes physical parent paths; adjust tests.

Fix tests that fail on systems where components of the parent path are
symlinks (like /tmp on OS X).  i.e. if you run:

  bup index /x/y/z

bup might actually store

  /a/z...

if y is a symlink to a.  See index-cmd.py and index.reduce_paths().

Signed-off-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: handle test.sh fix differently; add realpath();
 adjust commit message.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoMakefile: add any environmental CFLAGS to the end of the effective CFLAGS.
Rob Browning [Fri, 5 Jul 2013 17:06:16 +0000 (12:06 -0500)]
Makefile: add any environmental CFLAGS to the end of the effective CFLAGS.

For now, just do the simplest thing.

Thanks to Greg Troxel <gdt@ir.bbn.com> for reporting the issue.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoCompletely disable ACL support on OS X for now.
Rob Browning [Thu, 4 Jul 2013 20:32:07 +0000 (15:32 -0500)]
Completely disable ACL support on OS X for now.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoconfig/configure: add and use bup_find_prog; remove unused MF_PATH_INCLUDEs.
Rob Browning [Thu, 4 Jul 2013 19:12:22 +0000 (14:12 -0500)]
config/configure: add and use bup_find_prog; remove unused MF_PATH_INCLUDEs.

Pull all the common program search code into bup_find_prog() and use
it everywhere.  This also makes configure's output a bit more
consistent in style/content.

Make it obvious that at the moment bup doesn't pay any attention to a
PYTHON or GIT environment variable value. i.e. bup currently ignores a
setting like PYTHON=python2.7.

Bup also doesn't use MF_PATH_INCLUDE settings yet, so remove them.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agogit.py: flush idx_map before close so FILE* operations will see changes.
Rob Browning [Sat, 8 Jun 2013 15:25:33 +0000 (10:25 -0500)]
git.py: flush idx_map before close so FILE* operations will see changes.

Flush idx_map (msync()) before closing it, since it doesn't look like
POSIX guarantees that a matching FILE* (i.e. idx_f) will see the
parallel changes if we don't.

From the original report:

  After `bup save`, `git fsck` would show messages like
  "Packfile index for %s SHA1 mismatch"
  This indicated a bad trailing checksum on the pack index file.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reported-by: Alan Jenkins <alan.christopher.jenkins@gmail.com>
Reviewed-by: Alan Jenkins <alan.christopher.jenkins@gmail.com>
Tested-by: Alan Jenkins <alan.christopher.jenkins@gmail.com>
10 years agoFix completely broken invocations of t/root-status.
Rob Browning [Fri, 7 Jun 2013 05:22:58 +0000 (00:22 -0500)]
Fix completely broken invocations of t/root-status.

No idea what I was thinking.

Thanks to Sascha Silbe <sascha-ml-reply-to-2013-2@silbe.org> for the
report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
10 years agoSkip the large bloom filter test if mmap fails on 32-bit archtectures.
Rob Browning [Sat, 25 May 2013 20:25:11 +0000 (15:25 -0500)]
Skip the large bloom filter test if mmap fails on 32-bit archtectures.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>