bup.git
2 weeks agoUpdate base_version for 0.32 release master
Rob Browning [Sat, 9 Jan 2021 22:11:02 +0000 (16:11 -0600)]
Update base_version for 0.32 release

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 weeks agoUpdate README, HACKING, and release notes for 0.32
Rob Browning [Sat, 9 Jan 2021 22:02:38 +0000 (16:02 -0600)]
Update README, HACKING, and release notes for 0.32

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
3 weeks agoprep-for-macos-build: don't run brew update
Rob Browning [Wed, 6 Jan 2021 01:43:50 +0000 (19:43 -0600)]
prep-for-macos-build: don't run brew update

The update was causing tests to fail, and the previous attempt to
address the issue e244aa8ec2ea4b754d6242ee81dd0d3c6dace543 didn't.
Thanks to Johannes Berg proposing the fix.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
3 weeks agoREADME: add markdown mode indicator
Rob Browning [Wed, 6 Jan 2021 01:18:34 +0000 (19:18 -0600)]
README: add markdown mode indicator

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
3 weeks agoindex: add newline to 'corrupt index' message
Johannes Berg [Thu, 31 Dec 2020 22:07:10 +0000 (23:07 +0100)]
index: add newline to 'corrupt index' message

This otherwise prints the traceback afterwards starting
on the same line, which looks odd.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
3 weeks agoindex: remove pointless git.repo() invocation
Johannes Berg [Thu, 31 Dec 2020 21:29:38 +0000 (22:29 +0100)]
index: remove pointless git.repo() invocation

Since the only way we can get to clear_index() is with
indexfile having previously been assigned

    indexfile = git.repo(b'bupindex')

since opt.clear and opt.indexfile are explicitly mutually
exclusive, there's no point in calling git.repo() again,
it has no effect.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
3 weeks agoget: remove pointless git.repo() call
Johannes Berg [Tue, 29 Dec 2020 22:28:34 +0000 (23:28 +0100)]
get: remove pointless git.repo() call

There's no point in doing "opt.source or git.repo()" since
we just pass the result to LocalRepo(), which already does
exactly the same thing.

Remove the src_dir variable entirely.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
3 weeks agoprune-older: fix output with --pretend
Johannes Berg [Wed, 30 Dec 2020 22:41:57 +0000 (23:41 +0100)]
prune-older: fix output with --pretend

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
3 weeks agobup-web: handle 4 item getsockname() result for IPv6 sockets
Christian Brabandt [Tue, 29 Dec 2020 14:19:48 +0000 (15:19 +0100)]
bup-web: handle 4 item getsockname() result for IPv6 sockets

So bup-web will create a new network socket to listen on for HTTP
connections and it may use either IPv4 or IPv6 (or both) addresses,
depending on the configuration of the system. This is done using the
tornado.netutil.bind_sockets().

Currently, bup-web does not restrict the address family to either
AF_INET/AF_INET6 and according to the documentation of
[tornado](https://www.tornadoweb.org/en/stable/netutil.html?highlight=netutil#tornado.netutil.bind_sockets)
it will use both address families and listen for connections on IPv4 and
IPv6 addresses.

> Family may be set to either socket.AF_INET or socket.AF_INET6 to
> restrict to IPv4 or IPv6 addresses, otherwise both will be used if
> available.

However, when your system has only IPv6 enabled, the print() statement
may fail with:

Traceback (most recent call last):
  File "/opt/bup/lib/bup/cmd/bup-web", line 310, in <module>
    print('Serving HTTP on %s:%d...' % sockets[0].getsockname())
TypeError: not all arguments converted during string formatting

For systems that have IPv4 as well as IPv6 enabled, it may or may not
fail, depending on whether the first returned socket by getsockname() is
for AF_INET6 or AF_INET4

The reason is, getsockname() for a AF_INET6 connections returns a tuple with
4 items: `(host, port, flowinfo, scope_id)`, while for AF_INET4
addresses, it contains only `(host, port)`.

Since we are only interested in the first 2 items, make sure to return
only the first 2 items, which are `host` and `port` for IPv4 (AF_INET)
and IPv6 (AF_INET6).

Signed-off-by: Christian Brabandt <cb@256bit.org>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: adjust commit summary]

3 weeks agorestore: fix error message for inaccessible items
Johannes Berg [Tue, 29 Dec 2020 21:26:40 +0000 (22:26 +0100)]
restore: fix error message for inaccessible items

This actually shows up during tests, but it's hidden as it's
an error where we _expect_ bup to fail (and don't check why).
Fix the message in this case.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
4 weeks agoReplace atoi with explicit checks; rm unused atof
Rob Browning [Tue, 22 Dec 2020 17:17:45 +0000 (11:17 -0600)]
Replace atoi with explicit checks; rm unused atof

Replace atoi with explicit checks, making the intention a bit clearer
at each call site, and avoiding trouble we've had in the past with the
default to 0 for any unrecognized input.

This does make us stricter about the values (and the environment),
i.e. BUP_TTY_WIDTH=random-noise will now cause a crash.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
4 weeks agossh: don't pass through bad BUP_TTY_WIDTH
Johannes Berg [Mon, 21 Dec 2020 20:45:55 +0000 (21:45 +0100)]
ssh: don't pass through bad BUP_TTY_WIDTH

If there's no BUP_TTY_WIDTH variable already, then environ.get()
will result in None, which cannot be converted to an integer so
we get 0. Then we pass 0 to the other side in the BUP_TTY_WIDTH
environment variable.

Unfortunately, 0 is a really bad default, since textwrap cannot
work with a width of 0, causing tracebacks such as the one Mark
J Hewitt reported:
https://groups.google.com/g/bup-list/c/rfIw3STN2EY/m/C5Y3UrMIAgAJ

Fix this by not passing it through if there's no good value.

Reported-by: Mark J Hewitt <mjh@idnet.com>
Reported-by: Alexander Barton <alex@barton.de>
Fixes: cd3647c81a25 ("bup: pass TTY width through to sub-command and remote")
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
4 weeks agoPrefer python 3, and mention intent to drop python 2 support
Rob Browning [Sun, 20 Dec 2020 01:36:09 +0000 (19:36 -0600)]
Prefer python 3, and mention intent to drop python 2 support

For now, set the minimum supported 3 version at 3.7 since I just
tested that on a debian/buster system, and update the installation
instructions after seeing what was required to run the tests from
scratch there.  Continue to search for 3.6 and 2.6 for now, since no
one's complained, and we can always lower the requirement later if it
seems appropriate.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 weeks agoprep-for-macos-build: only install brew when not found
Rob Browning [Sun, 20 Dec 2020 19:15:38 +0000 (13:15 -0600)]
prep-for-macos-build: only install brew when not found

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 weeks agoprep-for-macos-build: update brew install command
Rob Browning [Sun, 20 Dec 2020 19:00:25 +0000 (13:00 -0600)]
prep-for-macos-build: update brew install command

cirrus-ci builds started failing like this:

  Error:
    homebrew-core is a shallow clone.
  To `brew update`, first run:
    git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow

and I noticed that the install url at https://brew.sh/ differed so try
using that one.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 weeks agoREADME: update build requirements for Centos
danpawlik [Mon, 21 Sep 2020 14:33:22 +0000 (16:33 +0200)]
README: update build requirements for Centos

Some package names have changed, so building bup was problematic.

Signed-off-by: Daniel Pawlik <dpawlik@redhat.com>
[rlb@defaultvalue.org: adjust commit message]

5 weeks agodev: macos: install sha1sum
Johannes Berg [Tue, 18 Aug 2020 19:32:01 +0000 (21:32 +0200)]
dev: macos: install sha1sum

Install the sha1sum utility for the tests.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
5 weeks agotest-web: redirect stdio to make test work standalone
Johannes Berg [Tue, 4 Aug 2020 14:49:36 +0000 (16:49 +0200)]
test-web: redirect stdio to make test work standalone

Python (at least 3) behaves differently if somehow
stdio is connected to the terminal, and external
SIGTERM then doesn't cause the same behaviour as
actually pressing Ctrl-C. Redirect the output to
a file to make the test run standalone.

Print the file if any errors occur so that if there's
anything interesting in there, we can see it.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 weeks agorestore: when -vv is specified, don't crash when printing paths
Jean-Paul Marmorat [Fri, 6 Nov 2020 15:07:39 +0000 (16:07 +0100)]
restore: when -vv is specified, don't crash when printing paths

Without the fix restore would crash like this:

  TypeError: can't concat str to bytes

Signed-off-by: Jean-Paul Marmorat <jean-paul.marmorat@mines-paristech.fr>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: add angle brackets around email address in
 signed-off-by; adjust commit message]
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 weeks agoMention homebrew installation
Gustavo Goretkin [Tue, 15 Dec 2020 23:43:06 +0000 (18:43 -0500)]
Mention homebrew installation

Signed-off-by: Gustavo Nunes Goretkin <gustavo.goretkin@gmail.com>
5 weeks agoREADME: reflect report that WSL works fine
Rob Browning [Wed, 16 Dec 2020 02:18:29 +0000 (20:18 -0600)]
README: reflect report that WSL works fine

Thanks to William Stein for the update.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
7 weeks agoMigrate tests to pytest
Rob Browning [Sat, 5 Dec 2020 20:59:00 +0000 (14:59 -0600)]
Migrate tests to pytest

2 months agocirrus freebsd: install pytest via pip for python 2
Rob Browning [Fri, 27 Nov 2020 18:51:51 +0000 (12:51 -0600)]
cirrus freebsd: install pytest via pip for python 2

I didn't see a port, and pkg warns that python 2 may be removed
entirely at the end of the year, so just use pip for now.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agofreebsd: fix pytest setup for python 3
Rob Browning [Thu, 26 Nov 2020 19:11:57 +0000 (13:11 -0600)]
freebsd: fix pytest setup for python 3

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoUse pytest item results to preserve tmpdirs on failure
Rob Browning [Tue, 17 Nov 2020 04:41:09 +0000 (22:41 -0600)]
Use pytest item results to preserve tmpdirs on failure

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoAdd bup map including test results to pytest items
Rob Browning [Tue, 17 Nov 2020 01:52:42 +0000 (19:52 -0600)]
Add bup map including test results to pytest items

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoRemove extensions from test command names
Rob Browning [Sun, 15 Nov 2020 19:53:42 +0000 (13:53 -0600)]
Remove extensions from test command names

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoPull some tests forward to improve parallel runtimes
Rob Browning [Fri, 13 Nov 2020 18:25:01 +0000 (12:25 -0600)]
Pull some tests forward to improve parallel runtimes

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoInstall xdist in prep-for-*-build
Johannes Berg [Fri, 29 May 2020 13:53:20 +0000 (15:53 +0200)]
Install xdist in prep-for-*-build

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
[rlb@defaultvalue.org: adjust commit message after moving some code]

2 months agoAutodetect xdist and translate make -j to xdist -n
Rob Browning [Fri, 13 Nov 2020 20:27:52 +0000 (14:27 -0600)]
Autodetect xdist and translate make -j to xdist -n

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoMigrate the remaining (standalone) test/ext tests to pytest
Johannes Berg [Thu, 28 May 2020 21:14:53 +0000 (23:14 +0200)]
Migrate the remaining (standalone) test/ext tests to pytest

Enable the test/ext/conftest.py collector to allow pytest to handle
all of the test/ext standalone tests too.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: show the pytest invocations; switch to use
 test/ext/conftest.py collector; use the release mark in check and
 distcheck; augment commit message; augment the information in
 HACKING]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agoMove ext/test.sh to ext/test-misc to match test-* pattern
Rob Browning [Sun, 15 Nov 2020 18:54:50 +0000 (12:54 -0600)]
Move ext/test.sh to ext/test-misc to match test-* pattern

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoAdd pytest collector for all test/cmd/test-* files
Rob Browning [Sun, 8 Nov 2020 18:13:55 +0000 (12:13 -0600)]
Add pytest collector for all test/cmd/test-* files

Add a collector for all the test/cmd tests that runs them as
individual subprocesses.  Add support for a pytest "release" mark and
apply it to the test-release-archive test.  Disable the collector
until the upcoming changes that rely on it are in place.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agotests: convert python-based tests to pytest
Johannes Berg [Thu, 28 May 2020 20:50:43 +0000 (22:50 +0200)]
tests: convert python-based tests to pytest

This converts the existing test scripts that are written
in python already to pytest based tests.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
[rlb@defaultvalue.org: change wvtest imports to wvpytest; remove
 wvtest.py here; drop buptest.test_tempdir here]

2 months agotest-ftp: set TZ=UTC to produce consistent names
Rob Browning [Sun, 15 Nov 2020 18:09:08 +0000 (12:09 -0600)]
test-ftp: set TZ=UTC to produce consistent names

This dependency was exposed by the upcoming switch to pytest which may
run this test alongside others in the same python process.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agotest-prune-older: accommodate unified_diff break in python 3.5
Rob Browning [Sun, 15 Nov 2020 17:44:27 +0000 (11:44 -0600)]
test-prune-older: accommodate unified_diff break in python 3.5

Apparently python 3.5 changed unified_diff to crash when given bytes
arguments, and added a new diff_bytes function.  Use the latter when
the version is >= 3.5.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agotests: partially convert to pytest
Johannes Berg [Thu, 28 May 2020 20:05:45 +0000 (22:05 +0200)]
tests: partially convert to pytest

Convert all the python-based library unit tests to pytest,
and add a pytest.ini to only look in the test/ directory.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
[rlb@defaultvalue.org: run pytest outside wvtest check wrapper so the
 output is visible; restore some of buptest.py and wvtest.py so the
 other tests can still run; rename pytest shim to wvpytest to
 distinguish from ./wvtest.py.; add ./pytest to centralize common
 options; switch .wvtest import to wvtest now that lib/test is a proper
 module dir; narrow pytest.ini test dirs to test/int and test/ext; move
 sort-z to test/bin; fix some dependencies in dev/prep-for-*]

2 months agoEnsure pytests preserve the cd $(pwd -P) behavior
Rob Browning [Sun, 15 Nov 2020 18:21:28 +0000 (12:21 -0600)]
Ensure pytests preserve the cd $(pwd -P) behavior

Previously handled in the Makefile.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoAdd pytest context manager to save/restore test environment
Rob Browning [Sun, 1 Nov 2020 20:03:41 +0000 (14:03 -0600)]
Add pytest context manager to save/restore test environment

Without this, test_git.py could fail because test_ftp had left its
GIT_DIR lying around, and other tests could fail because it changed
and didn't restore the working directory.  Make sure to call tzset()
after any TZ changes.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agoMove t*.py tests to test_*.py for pytest autodetection
Rob Browning [Sun, 1 Nov 2020 16:39:17 +0000 (10:39 -0600)]
Move t*.py tests to test_*.py for pytest autodetection

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agowvtest.py: respect BUP_TEST_SEED
Rob Browning [Sat, 14 Nov 2020 17:28:50 +0000 (11:28 -0600)]
wvtest.py: respect BUP_TEST_SEED

Without this, the test counts can vary given (for example) the
tests in validate_vfs_{streaming,seeking}* in tvfs.py.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agotests: move all tests to test/
Johannes Berg [Wed, 27 May 2020 17:58:38 +0000 (19:58 +0200)]
tests: move all tests to test/

Instead of having some testing code interleaved with bup's
real code in lib/bup/test/ and lib/bup/t/ move everything
related to testing into test/, which is also renamed from
just t/.

This aligns with pytest for future changes, and, almost more
importantly, cleans up the lib/ directory to only have the
operational code.

The latter is necessary to properly 'libify' bup in order to
be able to use tox properly to test with different versions
of python in venv later.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: also move utilities like compare-trees to dev/;
 make test/lib a test-specific module tree and have wvtest.py include
 it; use test/ext/ and test/int/ (external and internal) as the test
 dir names.]

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agocirrus: allow failures on macos with python 2
Rob Browning [Wed, 25 Nov 2020 20:37:01 +0000 (14:37 -0600)]
cirrus: allow failures on macos with python 2

It looks like macos has changed something that makes it very difficult
to detect and use the *utime* related functions.  Apparently Emacs
just decided to live without it, and Python had to go to substantial
lengths to sort it out.  So until/unless we have a reasonable fix,
just mark that combination as expected to fail.

Python 3 should work fine since we rely on it to sort out the problem
for us.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agosystem-info: include uname -a output
Rob Browning [Wed, 25 Nov 2020 20:35:12 +0000 (14:35 -0600)]
system-info: include uname -a output

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agocirrus: make macos image match reality (catalina)
Rob Browning [Wed, 25 Nov 2020 20:33:57 +0000 (14:33 -0600)]
cirrus: make macos image match reality (catalina)

Apparently it's actually choosing catalina anyway (see the labels
during the runs), and that appears to be all that's supported now at
https://cirrus-ci.org/guide/macOS/

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
2 months agoDepend on python 3 for utime when using 3
Rob Browning [Tue, 24 Nov 2020 00:21:00 +0000 (18:21 -0600)]
Depend on python 3 for utime when using 3

Since python 3.3+ now claims to implement what we need (follow
symlinks, ns resolution), rely on it for utime when possible in
preference to our own C code in _helpers.

This should fix new failures on macos due to changes to the
availability of utimensat, etc.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agotest.sh: set LC_ALL=C for git fsck to avoid localized output
Karl-Philipp Richter [Tue, 1 Sep 2020 19:09:03 +0000 (21:09 +0200)]
test.sh: set LC_ALL=C for git fsck to avoid localized output

Observed on Ubuntu 20.04 with German as the primary language. The test
clearly parses output in English and since adapting the parsing to the
language of the test environment is not what the test tests,
specifying the language is the way to go.

The original problem can be reproduced via

  LC_ALL=de_DE.utf8  ./wvtest run t/test.sh

if your system has de_DE.utf8 avalilable.

Signed-off-by: Karl-Philipp Richter <krichter@posteo.de>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: adjust commit message; set LC_ALL=C just for
 fsck rather than setting LANGUAGE globally]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agotest-meta: use grep -a on xstat/meta
Johannes Berg [Thu, 19 Nov 2020 20:56:31 +0000 (21:56 +0100)]
test-meta: use grep -a on xstat/meta

xattr data may contain arbitrary bytes, including NUL.
If we use grep on such data without -a, then it just
prints "Binary file (standard input) matches" and we
haven't tested anything.

Use grep -a for the appropriate invocations to avoid
this issue, we only care about ascii lines here.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
2 months ago_helpers: remove argv mangling on python 3.9
Johannes Berg [Wed, 18 Nov 2020 20:29:31 +0000 (21:29 +0100)]
_helpers: remove argv mangling on python 3.9

Since python 3.9 exports the argv as wchar_t now, we
can no longer attempt to mangle it. Remove this code
on python 3.9 so it builds again.

This fixes Debian's FTBFS https://bugs.debian.org/972763, and this
part of our Cirrus test builds on MacOS.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
2 months agowrite_idx: make the 64-bit offset count uint32_t not int
Rob Browning [Sun, 22 Nov 2020 18:27:27 +0000 (12:27 -0600)]
write_idx: make the 64-bit offset count uint32_t not int

It's a 31-bit offset into the 64-bit object offsets table.

https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agowrite_idx: fix handling of signed/unsigned values
Rob Browning [Fri, 20 Nov 2020 07:29:02 +0000 (01:29 -0600)]
write_idx: fix handling of signed/unsigned values

Coerce the incoming Py_ssize_t count directly to the required uint32_t
while checking for overflow.  The underlying problem was identified
when we hit two sign-compare warnings on (32-bit) platforms where
ssize_t is 32 bits.

Explicitly add -Wsign-comapare to CFLAGS, just to be sure it's
included.

Reported-by: Dominique Leuenberger
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agoINTEGER_TO_PY(): include necessary compiler pgramas
Rob Browning [Wed, 18 Nov 2020 02:00:18 +0000 (20:00 -0600)]
INTEGER_TO_PY(): include necessary compiler pgramas

Move the pragmas it needs into INTEGER_TO_PY() so we don't need to
scatter them around.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agoconfigure: Search for python3.x and python3
Greg Troxel [Tue, 29 Sep 2020 00:07:11 +0000 (00:07 +0000)]
configure: Search for python3.x and python3

Previously, only specific python2.x and python2 were saerched for.

Signed-off-by: Greg Troxel <gdt@lexort.com>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agotest-import-duplicity: adjust to FreeBSD's rsync output
Johannes Berg [Wed, 18 Nov 2020 10:56:53 +0000 (11:56 +0100)]
test-import-duplicity: adjust to FreeBSD's rsync output

rsync on FreeBSD now outputs 12 status chars, adjust to
that and simplify the check a little bit.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
2 months agorm: handle writer.abort exceptions when exception pending
Rob Browning [Sat, 12 Sep 2020 22:21:43 +0000 (17:21 -0500)]
rm: handle writer.abort exceptions when exception pending

And drop the else clause since it's unnecessary here.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agogc: always clean up the packwriter; abort on errors
Rob Browning [Sat, 12 Sep 2020 22:15:46 +0000 (17:15 -0500)]
gc: always clean up the packwriter; abort on errors

Make sure to always close or abort the packwriter.  Abort on errors
under the assumption that any packfile in progress should only contain
blobs duplicated from the packfiles that were being garbage collected
to produce it, and those packfiles should never be removed until we've
finished writing and closing the one they're being swept into.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
2 months agocompat: add "with pending_raise(ex)" to simplify nested exceptions
Rob Browning [Sat, 12 Sep 2020 22:14:21 +0000 (17:14 -0500)]
compat: add "with pending_raise(ex)" to simplify nested exceptions

Thanks to Johannes Berg for the suggesting the approach.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
3 months agoCODINGSTYLE: add raise missing from exception handling code
Rob Browning [Sat, 12 Sep 2020 22:04:46 +0000 (17:04 -0500)]
CODINGSTYLE: add raise missing from exception handling code

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
4 months agoUpdate base_version for 0.31+ development
Rob Browning [Sun, 20 Sep 2020 18:14:29 +0000 (13:14 -0500)]
Update base_version for 0.31+ development

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
4 months agohelpers.localtime: avoid floor-related deprecation warning
Johannes Berg [Tue, 26 May 2020 22:34:54 +0000 (00:34 +0200)]
helpers.localtime: avoid floor-related deprecation warning

floor() returns a float, even though it just rounded down,
so we need to convert to an int() to avoid a deprecation
warning here.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
4 months agobup: pass TTY width through to sub-command and remote
Johannes Berg [Thu, 27 Aug 2020 21:31:57 +0000 (23:31 +0200)]
bup: pass TTY width through to sub-command and remote

As we rewrite the sub-command's stderr, it cannot determine
the width. As a result, e.g. "bup ls" will always columnate
only to 70 columns, instead of using the entire width. Fix
this by passing the real width through in an environment
variable, just like we already pass through the fact that
it's a TTY.

Also pass it through on the ssh command line so that if you
run 'bup on ... ls' it actually behaves right.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
4 months agoio: fix path_msg on python2
Johannes Berg [Thu, 27 Aug 2020 20:25:16 +0000 (22:25 +0200)]
io: fix path_msg on python2

We get the binary representation here on both python 2 and 3,
but on 2 we really don't need to do any decoding since we must
print it as 'str'. Decoding also fails if the input is valid
UTF-8, which seems strange, but even if it worked we actually
don't want unicode objects here to print, we'd have to encode
them again to print.

Reported-by: Henninger Henningstone <henningstone@gmx.de>
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agoUpdate base_version for 0.31 release 0.31
Rob Browning [Sat, 22 Aug 2020 17:12:01 +0000 (12:12 -0500)]
Update base_version for 0.31 release

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agoUpdate HACKING and README for 0.31
Rob Browning [Sat, 22 Aug 2020 17:10:47 +0000 (12:10 -0500)]
Update HACKING and README for 0.31

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agoUpdate release notes for 0.31
Rob Browning [Sat, 22 Aug 2020 17:09:25 +0000 (12:09 -0500)]
Update release notes for 0.31

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agoupdate-checkout-info: compare physical paths (resolve symlinks)
Rob Browning [Sat, 22 Aug 2020 16:48:11 +0000 (11:48 -0500)]
update-checkout-info: compare physical paths (resolve symlinks)

Without this, symlinks in the parent path can cause erroneous
mismatches.

Thanks to Ralf Hemmecke for reporting the problem and help testing the
fix.

Reported-by: Ralf Hemmecke <ralf@hemmecke.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agoMakefile: remove vestigial checkout_info.py installation code
Rob Browning [Sat, 22 Aug 2020 16:46:41 +0000 (11:46 -0500)]
Makefile: remove vestigial checkout_info.py installation code

Reported-by: Ralf Hemmecke <ralf@hemmecke.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agotest.sh: test bup features python version
Rob Browning [Sat, 15 Aug 2020 17:10:14 +0000 (12:10 -0500)]
test.sh: test bup features python version

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agofeatures: show version number of the Python interpreter
Alexander Barton [Sun, 2 Aug 2020 18:10:29 +0000 (20:10 +0200)]
features: show version number of the Python interpreter

This can be handy for debiugging, especially if the interpreter used
(like /usr/bin/python3) is a symlink to the actual Python interpreter
(/usr/bin/python3.7) and/or doesn't include the "micro" version at all.

Signed-off-by: Alexander Barton <alex@barton.de>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agoMakefile: specify -Wformat=2 rather than error given -Werror
Rob Browning [Sat, 15 Aug 2020 01:13:15 +0000 (20:13 -0500)]
Makefile: specify -Wformat=2 rather than error given -Werror

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agotest-fsck: test invocation with pack arguments
Rob Browning [Sat, 15 Aug 2020 01:12:07 +0000 (20:12 -0500)]
test-fsck: test invocation with pack arguments

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agofsck: fix argv_bytes typo
Johannes Berg [Fri, 14 Aug 2020 17:43:58 +0000 (19:43 +0200)]
fsck: fix argv_bytes typo

Reported-by: gkonstandinos@gmail.com
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
5 months agotests: tclient: fix buffering behaviour in the test
Johannes Berg [Thu, 13 Aug 2020 13:20:33 +0000 (15:20 +0200)]
tests: tclient: fix buffering behaviour in the test

If the objects we write are small enough to be at least
partially buffered instead of written out to the server
connection, the test will hang in the loop as the data
hasn't made it to the server yet, which therefore can't
make a suggestion yet.

In normal non-testing scenarios this isn't an issue as
the connection is flushed before every new read(), and
thus if the client actually needs to wait for something
from the server, it'll have flushed before. Here, we're
directly poking at the internals by using has_input(),
and nothing causes a flush, causing the above scenario.

Fix this by some more ugly poking at the internals and
flush the connection before we go into the loop.

While at it, also avoid hanging there forever and break
out of the loop if one second passed without receiving
anything from the server - that should be long enough
for it to hash the two objects and respond. Also, add a
time.sleep() there to avoid busy spinning which takes
CPU cycles from the server that needs to be hashing the
object.

Reported-by: Robert Edmonds <edmonds@debian.org>
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
5 months agochecked_malloc: use %z to format size_t; enable -Wformat=2
Rob Browning [Sun, 9 Aug 2020 18:07:01 +0000 (13:07 -0500)]
checked_malloc: use %z to format size_t; enable -Wformat=2

Reported-by: Greg Troxel <gdt@lexort.com>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agotest-help: restrict MANPATH; only test manpages when available
Rob Browning [Sun, 9 Aug 2020 16:59:28 +0000 (11:59 -0500)]
test-help: restrict MANPATH; only test manpages when available

Reported-by: Robert Edmonds <edmonds@debian.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agoinstall-python-script: don't presume python exists
Rob Browning [Sat, 8 Aug 2020 18:05:39 +0000 (13:05 -0500)]
install-python-script: don't presume python exists

Reported-by: Robert Edmonds <edmonds@debian.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agobup_mincore: actually use size_t value (not ulonglong) in call
Rob Browning [Sat, 8 Aug 2020 17:27:51 +0000 (12:27 -0500)]
bup_mincore: actually use size_t value (not ulonglong) in call

We'd already done the correct conversion, but weren't using it.

Reported-by: Greg Troxel <gdt@lexort.com>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
5 months agosave: make the py3 save commit message args match py2
Johannes Berg [Tue, 4 Aug 2020 14:46:16 +0000 (16:46 +0200)]
save: make the py3 save commit message args match py2

We shouldn't use sys.argv, it's missing the actual arguments
now. Also, encode it but drop the "b" on the bytes (if present,
i.e. python3) to make this look the same as on python2.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
[rlb@defaultvalue.org: make formatting decision via py_maj; adjust
 commit message]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agoversion: fix --date argument for py3
Johannes Berg [Wed, 5 Aug 2020 19:17:13 +0000 (21:17 +0200)]
version: fix --date argument for py3

This has an issue with str vs. bytes, fix it.

Reported-by: Mark J Hewitt <mjh@idnet.com>
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agoAdd trivial tests for current help behavior
Rob Browning [Sat, 8 Aug 2020 16:40:54 +0000 (11:40 -0500)]
Add trivial tests for current help behavior

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
5 months agohelp: fix for python3
Johannes Berg [Wed, 5 Aug 2020 07:54:55 +0000 (09:54 +0200)]
help: fix for python3

Fix "bup subcommand --help" to properly compare and start
the man viewer.

Reported-by: Robert Edmonds <edmonds@debian.org>
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoftp: handle lack of readline in input
Johannes Berg [Tue, 28 Jul 2020 20:28:11 +0000 (22:28 +0200)]
ftp: handle lack of readline in input

If we don't have readline, print our prompt and read a line
from stdin instead.

Reported-by: Eric Waguespack <eric.w@guespack.com>
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
6 months agoftp: fix ls arguments for python3
Johannes Berg [Tue, 28 Jul 2020 21:28:46 +0000 (23:28 +0200)]
ftp: fix ls arguments for python3

During the conversion to python 3, ftp ls could no longer
get option arguments, because we're now passing *bytes* to
the option parser that still expects *str*.

Fix this by fsdecode()'ing the bytes into str, which then
causes the option parsing to work properly, and the result
will be fsencode()'d again via argv_bytes() for usage.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agohlinkdb: always load as bytes on python3
Johannes Berg [Mon, 27 Jul 2020 21:15:16 +0000 (23:15 +0200)]
hlinkdb: always load as bytes on python3

We really need this to be bytes, so always load as such.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: changed %s to %d]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agochange mktemp to be compatible with busybox
Brian Minton [Mon, 20 Jul 2020 18:55:11 +0000 (14:55 -0400)]
change mktemp to be compatible with busybox

Signed-off-by: Brian Minton <brian@minton.name>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agocirrus: make rdiff-backup completely optional for freebsd tests
Rob Browning [Thu, 23 Jul 2020 23:58:18 +0000 (18:58 -0500)]
cirrus: make rdiff-backup completely optional for freebsd tests

Thanks to Johannes Berg for reporting the problem and tracking down
the likely cause/fix.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
6 months agoEliminate redundant check of index start against ctime
Aidan Hobson Sayers [Fri, 17 Jul 2020 21:59:04 +0000 (22:59 +0100)]
Eliminate redundant check of index start against ctime

When (the first version of) this check was added 10 years ago in
b4b4ef116880, it was presumably to ensure that "index; save; touch;
index" in the same second would flag a file as needing saving.

Since then, tmax has been added in the indexing process to solve the
problem by capping timestamps stored in the index. This capping means
that a file with ctime in the same second as an indexing start will
be picked up on both the indexes in the example above - there's no need
to special case the check.

Signed-off-by: Aidan Hobson Sayers <aidanhs@cantab.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoweb: don't re-resolve item in listing
Johannes Berg [Fri, 17 Jul 2020 19:42:16 +0000 (21:42 +0200)]
web: don't re-resolve item in listing

We already have an item, we just need its metadata. There's no
need to re-resolve it. Somehow, resolving it again is also very
slow for large directories (perhaps re-reading metadata again
and again?), and this significantly speeds up things.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoAdd bup-features to report status; drop ACL warnings
Rob Browning [Sun, 19 Jul 2020 19:28:36 +0000 (14:28 -0500)]
Add bup-features to report status; drop ACL warnings

Add a new "bup features" command that reports information about the
status and capabilities of the current installation.

Given that, and the new summary at the end of the ./configure output,
drop the platform specific warnings, which were a bit odd anyway,
since they wouldn't warn you on say Linux if we found libacl, but it
didn't actually have all the features we required.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoconfigure: restore working dir after symlinking python
Rob Browning [Sun, 5 Jul 2020 20:04:04 +0000 (15:04 -0500)]
configure: restore working dir after symlinking python

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoAvoid varying git archive content for ref; rework versioning
Rob Browning [Sun, 5 Jul 2020 19:40:40 +0000 (14:40 -0500)]
Avoid varying git archive content for ref; rework versioning

Don't include the log --decorate names in the archive via export-subst
because they can change after the fact when new tags or branches are
added for a given hash -- for example, when we created the 0.30.x
branch after tagging 0.30.1.  Archives retrieved before the branch was
created would have a different set of NAMES in _release.py.

Move _release to source_info and add an optional checkout_info module.
source_info contains the (no longer variable) export-subst commit hash
and date values, and checkout_info contains the same data for a git
checkout.  Automatically update checkout_info whenever we're at the
top level of a git source tree, but don't include it in the archives.

Record the base version in version.py explicitly as either a release
version like 0.31, for an actual release (which must be committted
before tagging the release), or a development version like
0.31~ (indicating a version that's always less than 0.31).

Rework bup version to report the lib/version for an actual release, or
that version suffixed with the commit hash when running a non-release,
and add a "+" when uncommitted modifications are detected.  For
example:

      release: 0.31
  non-release: 0.31~4e4b9ba8689c93702743c8ecd49c5a7808a4d717
     modified: 0.31~4e4b9ba8689c93702743c8ecd49c5a7808a4d717+

Drop the --tag argument from bup version since the tags are variable,
and you can always ask git to describe the hash via

  git describe --always HASH

Add dev/refresh, similar to moreutils sponge, to handle file creation
safely, something we may want to deploy more widely (e.g. instead of
the $$/PID based tempfiles in the Makefile).

Thanks to Greg Troxel for reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoDrop pwd/gid reentrant function variants for now
Rob Browning [Tue, 30 Jun 2020 23:50:17 +0000 (18:50 -0500)]
Drop pwd/gid reentrant function variants for now

From the POSIX pages, it sounded like _SC_GETPW_R_SIZE_MAX, etc. were
upper limits, but other docs, and observed behavior (reported ERANGE
failures) indicate that it's more likely they're just suggestions.  As
a result, we'd either need larger hard-coded buffers, or a
reallocate/retry loop if we kept them.

Instead, since we're not at all thread-safe and aren't likely to be
anytime soon, the dynamic allocations are just less efficient and more
complex, so back off to the non-reentrant flavors for now.

Thanks to kd7spq for reporting the problem and Johannes Berg for
tracking down the cause and help settling on this solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agocirrus: fill out python 3 tests
Rob Browning [Sat, 27 Jun 2020 15:26:54 +0000 (10:26 -0500)]
cirrus: fill out python 3 tests

Test python 3 on all platforms and to the same extent as python 2.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
6 months agodrecurse: use portable S_ISDIR() instead of deriving S_IFMT
Rob Browning [Sat, 27 Jun 2020 16:47:31 +0000 (11:47 -0500)]
drecurse: use portable S_ISDIR() instead of deriving S_IFMT

In some simple testing, there didn't appear to be any notable,
consistent performance difference, and the S_IFMT(0xffffffff) call
fails with an OverflowError on at least macos.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoAdd portable dev/sort-z; link into t/bin/; use in test-meta.sh
Rob Browning [Thu, 25 Jun 2020 06:48:45 +0000 (01:48 -0500)]
Add portable dev/sort-z; link into t/bin/; use in test-meta.sh

Add a dev/sort-z wrapper to handle sorting null terminated lines
across platforms.  Apparently NetBSD supports "-R 000" instead of -z.

Add a t/bin for any commands we want to make available to all tests,
symlink sort-z there, and add it to the PATH in test-meta.sh.

Thanks to Greg Troxel for reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
6 months agoconfigure: test for functional readline more carefully
Rob Browning [Sun, 21 Jun 2020 17:46:50 +0000 (12:46 -0500)]
configure: test for functional readline more carefully

Apparently on (cirrus) macos, just testing for the ability to compile
readline.h isn't sufficient because configure ends up selecting the
built-in readline (which is insufficient) rather than the one we
specifically installed via brew.  More specifically, the built-in
readline has the wrong prototype for rl_completion_entry_function
which causes this error:

  _helpers.c:2096:38: error: incompatible function pointer types assigning to 'Function *' (aka 'int (*)(const char *, int)') from 'char *(const char *, int)' [-Werror,-Wincompatible-function-pointer-types]
        rl_completion_entry_function = on_completion_entry;
                                     ^ ~~~~~~~~~~~~~~~~~~~~

So change the test for an acceptable readline to check that
specifically, which is a better test for all the platforms.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoconfigure: check for <readline.h> vs <readline/readline.h>
Rob Browning [Sun, 21 Jun 2020 16:47:02 +0000 (11:47 -0500)]
configure: check for <readline.h> vs <readline/readline.h>

The readline docs (man and info pages) indicate that we should use

  #include <readline/readline.h>
  #include <readline/history.h>

Unfortunately, it appears that on a number of plaforms pkg-config
--cflags actually returns a -I value that requires

  #include <readline.h>
  #include <history.h>

So make an even bigger mess in config/configure to accomodate either
possibility.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
6 months agoAvoid readline.h -Wstrict-prototype induced failures
Rob Browning [Sun, 21 Jun 2020 16:50:30 +0000 (11:50 -0500)]
Avoid readline.h -Wstrict-prototype induced failures

On some platforms -Wstrict-prototype is now the default, and readline
includes prototypes like "int foo()" rather than "int foo(void)", so
for now, just suppress those warnings for the readline includes.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
6 months agoconfigure-sampledata: only create random paths if asked
Rob Browning [Sun, 21 Jun 2020 16:29:23 +0000 (11:29 -0500)]
configure-sampledata: only create random paths if asked

Stop creating randomized paths in t/sampledata/ by default.  I'd
originally just added this to allow some quick testing, and while it
now appears to be fine on Linux/ext4, it's too aggressive to be the
default, so hide it behind a BUP_TEST_RANDOMIZED_SAMPLEDATA_PATHS
environment variable.

Among other things, make-random-paths just crashes on cirrus macos,
and cirrus freebsd was having (different) trouble.  It might also have
been macos where test-import-duplicity.sh failed on compare-trees
mismatches.  Not sure whether that was an issue with bup, rsync, or
duplicity.

We'll want to restore broader randomized path testing, but likely via
a less blunt instrument, since placing the paths in t/sampledata
affects any test that relies on it, and existing testing constructs
like

  WVPASSEQ ... $(... | wc -l)

are completely incompatible.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
6 months agocirrus: ensure we use the real readline on macos
Rob Browning [Sat, 20 Jun 2020 19:19:21 +0000 (14:19 -0500)]
cirrus: ensure we use the real readline on macos

Signed-off-by: Rob Browning <rlb@defaultvalue.org>