ftp: honour pwd for ls

Honour the current working directory for 'ls' by changing
ls.within_repo() to get the pwd, and using posixpath to
build the correct path to do the ls for.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>

git: remove get_commit_dates()

This function is unused.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: likely orphaned by vfs overhaul]

git: remove loose object handling

This isn't necessary - bup never writes loose objects, and to
read uses cat-file. Just remove the dead code.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>

git: remove all_packdirs()

This really never made sense - if you have a local repository
and want to run midx/bloom on it, then it doesn't make sense
to touch the index caches for other remote repositories.

And if you want to operate manually on the cache for some odd
reason (it's maintained automatically) then you anyway need to
(and should) pass the --dir option for it explicitly.

Remove this to make things easier to reason about.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
[rlb@defaultvalue.org: previous behavior also didn't respect the
 documentation in bup-bloom(1).]

.gitignore: update (include C .d files, etc.)

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

_helpers: remove empty PyArg_ParseTuple() from METH_NOARGS

With METH_NOARGS, args is NULL, and thus PyArg_ParseTuple()
will segfault. Remove it, since it's unnecessary anyway.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>

configure: fix readline.h detection

Unfortunately, readline.h requires stdio.h to be included first,
except where it has been patched (e.g. Debian). Do that in the
configure script so we correctly detect readline.h on systems
that have an unpatched readline version.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>

midx: correct path_msg() use in an error path

pathmsg() doesn't exist, it should be path_msg(). Fix that.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>

ftp: fix tab completion

We need the 'repo' variable in enter_completion() which assumes
it's global, so make it global.

Fixes: c4d10541ed1e ("ftp: convert to internal command")
Signed-off-by: Johannes Berg <johannes@sipsolutions.net>

helpers: adjust macos linkage

Apparently macos needs -bundle and -undefined dynamic_lookup since
around 2003.  cpython does something similar via its own sysconfig
LDSHARED command.

Thanks to Johannes Berg for finding the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Convert bup to binary executable and run python subcommands directly

Don't execute python subcommands in a child process, run them
directly.  Convert bup (and dev/python, etc.) to be C executables.
See the commit messages, comments, and doc changes (e.g. DESIGN) for
more information.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

wvtest-bup.sh: set _wvtop to the physical path

The previous fix was insufficient:
96da3f0142c8f41fc379db4e4e31df2426d466b5

The problem can be demonstrated via:

  ln -s bup-real bup
  cd bup
  make check

cf. 7a8f8d91501525d4710918a1638b5df1fd4c54a5

Thanks to Johannes Berg for tracking down the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

cirrus: drop py2 on freebsd

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

cirrus: drop py2 on macos since brew did

  Homebrew provided a python@2 formula until the end of 2019, at which
  point it was removed due to the Python 2 deprecation.

  -- https://docs.brew.sh/Homebrew-and-Python

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

id-other-than: pass strings to getpwnam/getgrnam

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bup: cat-file: fix error print

Some variable renaming and bytes issues, if a target cannot
be found in cat-file.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>

exe_parent_dir: allow arbitrary proc readlink path lengths

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

exe_parent_dir: allow arbritrary macos path lengths

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Redirect to GNU make when possible

Rename Makefile to GNUmakefile and add a Makefile that for at least
FreeBSD make will redirect the build to GNU make, and don't fail if
make is not GNU make without checking gmake.

Thanks to Greg Troxel for a reminder about the issue.

Signed-off-by: Rob Browning <rlb@defaultvalue.org
Tested-by: Rob Browning <rlb@defaultvalue.org>

Move Makefile to GNUmakefile

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bup_py_bytes_main: add to support py 3.6 compat

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Move msg() to bup/io.[hc] in preparation for more sharing

We're going to need to share some python compatibility code between
dev/python and bup, etc., so create src/bup for the shared code, and
move msg() there.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Compute C dependencies automatically

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Rework compilation of binaries; prepare for automatic dependencies

Don't involve user-adjustable variables like CFLAGS until the final
compilation steps, in part so it's clearer where they fit in.  Handle
all of bup's flags independently, and split them into separate
library (helpers) and executable (bin/bup, etc.)  groups via
bup_shared_cflags, helpers_cflags, embed_cflags, etc.

Change the build rules to specify -fPIE for the executables, and
include all dependencies ($^) and use OUTPUT_OPTION in preparation for
automatic C dependency generation.

Add -Wno-unused-command-line-argument for clang.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Makefile: fix pytest invocation when xdist isn't available

Call "set -x", not "set-x".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

help: pass full bup path as argv[0] when running 'bup help'

Noticed this while tracking down a previous issue with our invocation
of strtok.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Makefile: don't depend on dev/python to clean

"make clean" should always work, even if we can't build dev/python.
Thanks to Greg Troxel for reporting the problem, and Johannes Berg for
suggesting additional improvements.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

configure: look for suitable pythonX.Y-config versions

Thanks to Greg Troxel for raising the issue.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bup.main: drop unicode-only getopt

It's relatively straightforward to replace it, and this will allow us
to avoid the unnecessary unicode round-trip.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Stash the env PYTHONPATH during startup and restore it in main

We have to set the PYTHONPATH from bup.c so that Py_Main will be able
to find bup.*, but we don't want that change to affect subprocesses,
so stash the original PYTHONPATH in bup_main.env_pythonpath, and
restore the original environment value once we've made it to the
python side (in bup.main).

Thanks to Johannes Berg for pointing out some issues with a previous
version of the changes.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

Remove scaffolding needed during the bup.c conversion

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Convert top level executables to binaries and clean up clean

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Handle external commands directly rather than via renamed symlinks

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fuse: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fuse-cmd: copy to bup.cmd.fuse

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fsck: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

fsck-cmd: copy to bup.cmd.fsck

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

get: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

get-cmd: copy to bup.cmd.get

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

restore: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

restore-cmd: copy to bup.cmd.restore

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

save: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

save-cmd: copy to bup.cmd.save

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on--server: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on--server-cmd: copy to bup.cmd.on__server

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

on-cmd: copy to bup.cmd.on

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

import-duplicity: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

import-duplicity-cmd: copy to bup.cmd.import_duplicity

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

server: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

server-cmd: copy to bup.cmd.server

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

prune-older: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

prune-older-cmd: copy to bup.cmd.prune_older

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

mux: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

mux-cmd: copy to bup.cmd.mux

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

list-idx: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

list-idx-cmd: copy to bup.cmd.list_idx

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

drecurse: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

drecurse-cmd: copy to bup.cmd.drecurse

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

damage: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

damage-cmd: copy to bup.cmd.damage

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

ftp: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

ftp-cmd: copy to bup.cmd.ftp

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

web: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

web-cmd: copy to bup.cmd.web

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

daemon: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

daemon-cmd: copy to bup.cmd.daemon

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tick: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tick-cmd: copy to bup.cmd.tick

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tag: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

tag-cmd: copy to bup.cmd.tag

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

midx: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

midx-cmd: copy to bup.cmd.midx

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

random: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

random-cmd: copy to bup.cmd.random

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

memtest: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

memtest-cmd: copy to bup.cmd.memtest

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

margin: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

margin-cmd: copy to bup.cmd.margin

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

features: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

features-cmd: copy to bup.cmd.features

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

split: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

split-cmd: copy to bup.cmd.split

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

init: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

init-cmd: copy to bup.cmd.init

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

gc: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

gc-cmd: copy to bup.cmd.gc

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

rm: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

rm-cmd: copy to bup.cmd.rm

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

index: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

index-cmd: copy to bup.cmd.index

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

join: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

join-cmd: copy to bup.cmd.join

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

help: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

help-cmd: copy to bup.cmd.help

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bloom: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

bloom-cmd: copy to bup.cmd.bloom

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

Makefile: check for bup version failures

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

version: convert to internal command

Signed-off-by: Rob Browning <rlb@defaultvalue.org>

version-cmd: copy to bup.cmd.version

Signed-off-by: Rob Browning <rlb@defaultvalue.org>