Redirect to GNU make when possible

Rename Makefile to GNUmakefile and add a Makefile that for at least
FreeBSD make will redirect the build to GNU make, and don't fail if
make is not GNU make without checking gmake.

Thanks to Greg Troxel for a reminder about the issue.

Signed-off-by: Rob Browning <rlb@defaultvalue.org
Tested-by: Rob Browning <rlb@defaultvalue.org>

Move Makefile to GNUmakefile

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bup_py_bytes_main: add to support py 3.6 compat

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Move msg() to bup/io.[hc] in preparation for more sharing

We're going to need to share some python compatibility code between
dev/python and bup, etc., so create src/bup for the shared code, and
move msg() there.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Compute C dependencies automatically

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Rework compilation of binaries; prepare for automatic dependencies

Don't involve user-adjustable variables like CFLAGS until the final
compilation steps, in part so it's clearer where they fit in.  Handle
all of bup's flags independently, and split them into separate
library (helpers) and executable (bin/bup, etc.)  groups via
bup_shared_cflags, helpers_cflags, embed_cflags, etc.

Change the build rules to specify -fPIE for the executables, and
include all dependencies ($^) and use OUTPUT_OPTION in preparation for
automatic C dependency generation.

Add -Wno-unused-command-line-argument for clang.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Makefile: fix pytest invocation when xdist isn't available

Call "set -x", not "set-x".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

help: pass full bup path as argv[0] when running 'bup help'

Noticed this while tracking down a previous issue with our invocation
of strtok.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Makefile: don't depend on dev/python to clean

"make clean" should always work, even if we can't build dev/python.
Thanks to Greg Troxel for reporting the problem, and Johannes Berg for
suggesting additional improvements.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

configure: look for suitable pythonX.Y-config versions

Thanks to Greg Troxel for raising the issue.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bup.main: drop unicode-only getopt

It's relatively straightforward to replace it, and this will allow us
to avoid the unnecessary unicode round-trip.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Stash the env PYTHONPATH during startup and restore it in main

We have to set the PYTHONPATH from bup.c so that Py_Main will be able
to find bup.*, but we don't want that change to affect subprocesses,
so stash the original PYTHONPATH in bup_main.env_pythonpath, and
restore the original environment value once we've made it to the
python side (in bup.main).

Thanks to Johannes Berg for pointing out some issues with a previous
version of the changes.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Remove scaffolding needed during the bup.c conversion

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Convert top level executables to binaries and clean up clean

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Handle external commands directly rather than via renamed symlinks

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fuse: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fuse-cmd: copy to bup.cmd.fuse

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fsck: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fsck-cmd: copy to bup.cmd.fsck

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

get: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

get-cmd: copy to bup.cmd.get

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

restore: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

restore-cmd: copy to bup.cmd.restore

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

save: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

save-cmd: copy to bup.cmd.save

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on--server: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on--server-cmd: copy to bup.cmd.on__server

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on-cmd: copy to bup.cmd.on

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

import-duplicity: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

import-duplicity-cmd: copy to bup.cmd.import_duplicity

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

server: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

server-cmd: copy to bup.cmd.server

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

prune-older: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

prune-older-cmd: copy to bup.cmd.prune_older

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

mux: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

mux-cmd: copy to bup.cmd.mux

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

list-idx: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

list-idx-cmd: copy to bup.cmd.list_idx

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

drecurse: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

drecurse-cmd: copy to bup.cmd.drecurse

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

damage: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

damage-cmd: copy to bup.cmd.damage

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

ftp: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

ftp-cmd: copy to bup.cmd.ftp

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

web: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

web-cmd: copy to bup.cmd.web

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

daemon: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

daemon-cmd: copy to bup.cmd.daemon

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tick: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tick-cmd: copy to bup.cmd.tick

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tag: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tag-cmd: copy to bup.cmd.tag

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

midx: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

midx-cmd: copy to bup.cmd.midx

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

random: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

random-cmd: copy to bup.cmd.random

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

memtest: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

memtest-cmd: copy to bup.cmd.memtest

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

margin: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

margin-cmd: copy to bup.cmd.margin

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

features: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

features-cmd: copy to bup.cmd.features

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

split: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

split-cmd: copy to bup.cmd.split

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

init: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

init-cmd: copy to bup.cmd.init

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

gc: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

gc-cmd: copy to bup.cmd.gc

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

rm: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

rm-cmd: copy to bup.cmd.rm

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

index: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

index-cmd: copy to bup.cmd.index

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

join: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

join-cmd: copy to bup.cmd.join

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

help: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

help-cmd: copy to bup.cmd.help

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bloom: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bloom-cmd: copy to bup.cmd.bloom

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Makefile: check for bup version failures

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

version: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

version-cmd: copy to bup.cmd.version

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

xstat: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

xstat-cmd: copy to bup.cmd.xstat

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

meta: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

meta-cmd: copy to bup.cmd.meta

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

ls: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

ls-cmd: copy to bup.cmd.ls

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

cat-file-cmd: migrate to internal bup.cmd.cat_file

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

cat-file-cmd: copy to bup.cmd.cat_file

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Remove $(dirname "$0") from sys.path

Python adds $(dirname "$0") to the sys.path, which we definitely don't
want (why would we?), so remove it early in the startup process.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bup: include built-in subcommands in help

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bup: filter stdout/stderr via thread/pipe for internal subcommands

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

filter_output: handle multiple srcs/dests as lists

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

configure-sampledata: cp lib/bup/*.py, not cmd/*.py

So the tests won't depend on the location of the subcommands, in
preparation for their move to lib/bup/cmd.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Make it possible to close all catpipes

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

options: add parse_bytes

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

git: explicitly request that Popen close_fds

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Add --import-py-module

This can be used by tests (and will be used by test-save-errors) to
override functions for error injection, etc.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Add handle_ctrl_c to bup itself

Signed-off-by: Rob Browning <rlb@defaultvalue.org>