From: Rob Browning Date: Tue, 15 Oct 2013 19:20:10 +0000 (-0500) Subject: Add --map-user --map-group --map-uid and --map-gid options to restore. X-Git-Tag: 0.25-rc4~16 X-Git-Url: https://arthur.barton.de/gitweb/?a=commitdiff_plain;h=f22c7343443d8af71dcac249cb21c7d7d2cc5686;p=bup.git Add --map-user --map-group --map-uid and --map-gid options to restore. Note that the usual metadata rules still appply, so a user or group entry will normally take precedence over a uid/gid unless --numeric-ids is specified. So if you want to map a uid, for example, you'll either need --numeric-ids, or you'll need to clear the user/group like this: bup restore ... --map-user rlb= --map-uid 1000=2000 These options should also make it possible to recover from archives that were broken as a result of our incorrect handling of signed/unsigned stat values (recently fixed). Signed-off-by: Rob Browning Reviewed-by: Gabriel Filion --- diff --git a/Documentation/bup-restore.md b/Documentation/bup-restore.md index 5208b30..78d2050 100644 --- a/Documentation/bup-restore.md +++ b/Documentation/bup-restore.md @@ -50,15 +50,24 @@ directory (or the `--outdir`). See the EXAMPLES section. Whenever path metadata is available, `bup restore` will attempt to restore it. When restoring ownership, bup implements tar/rsync-like -semantics. It will not try to restore the user unless running as -root, and it will fall back to the numeric uid or gid whenever the -metadata contains a user or group name that doesn't exist on the -current system. The use of user and group names can be disabled via -`--numeric-ids` (which can be important when restoring a chroot, for -example), and as a special case, a uid or gid of 0 will never be -remapped by name. Additionally, some systems don't allow setting a -uid/gid that doesn't correspond with a known user/group. On those -systems, bup will log an error for each relevant path. +semantics. It will normally prefer user and group names to uids and +gids when they're available, but it will not try to restore the user +unless running as root, and it will fall back to the numeric uid or +gid whenever the metadata contains a user or group name that doesn't +exist on the current system. The use of user and group names can be +disabled via `--numeric-ids` (which can be important when restoring a +chroot, for example), and as a special case, a uid or gid of 0 will +never be remapped by name. Additionally, some systems don't allow +setting a uid/gid that doesn't correspond with a known user/group. On +those systems, bup will log an error for each relevant path. + +The `--map-user`, `--map-group`, `--map-uid`, `--map-gid` options may +be used to adjust the available ownership information before any of +the rules above are applied, but note that due to those rules, +`--map-uid` and `--map-gid` will have no effect whenever a path has a +valid user or group. In those cases, either `--numeric-ids` must be +specified, or the user or group must be cleared by a suitable +`--map-user foo=` or `--map-group foo=`. Hardlinks will also be restored when possible, but at least currently, no links will be made to targets outside the restore tree, and if the @@ -108,6 +117,20 @@ See the EXAMPLES section for a demonstration. * '/foo/.' - exclude the content of any directory named foo * '^/tmp/.' - exclude root-level /tmp's content, but not /tmp itself +\--map-user *old*=*new* +: restore *old* user as *new* user. Specifying "" for *new* will + clear the user, i.e. `--map-user foo=`. + +\--map-group *old*=*new* +: restore *old* group as *new* group. Specifying "" for *new* will + clear the group, i.e. `--map-user foo=`. + +\--map-uid *old*=*new* +: restore *old* uid as *new* uid. + +\--map-gid *old*=*new* +: restore *old* gid as *new* gid. + -v, \--verbose : increase log output. Given once, prints every directory as it is restored; given twice, prints every @@ -177,6 +200,22 @@ Restore a tree without risk of unauthorized access: # rmdir restore-tmp +Restore a tree, remapping an old user and group to a new user and group: + + # bup restore -C dest --map-user foo=bar --map-group baz=bax /x/latest/y + Restoring: 42, done. + +Restore a tree, remapping an old uid to a new uid. Note that the old +user must be erased so that bup won't prefer it over the uid: + + # bup restore -C dest --map-user foo= --map-uid 1000=1042 /x/latest/y + Restoring: 97, done. + +An alternate way to do the same by quashing users/groups universally +with `--numeric-ids`: + + # bup restore -C dest --numeric-ids --map-uid 1000=1042 /x/latest/y + Restoring: 97, done. # SEE ALSO diff --git a/Makefile b/Makefile index cc01fcc..62f006c 100644 --- a/Makefile +++ b/Makefile @@ -89,6 +89,7 @@ runtests-cmdline: all t/test-cat-file.sh t/test-index-check-device.sh t/test-meta.sh + t/test-restore-map-owner.sh t/test-restore-single-file.sh t/test-rm-between-index-and-save.sh t/test-command-without-init-fails.sh diff --git a/cmd/restore-cmd.py b/cmd/restore-cmd.py index c18945b..59cdc73 100755 --- a/cmd/restore-cmd.py +++ b/cmd/restore-cmd.py @@ -1,5 +1,5 @@ #!/usr/bin/env python -import errno, sys, stat, re +import copy, errno, sys, stat, re from bup import options, git, metadata, vfs from bup.helpers import * @@ -10,6 +10,10 @@ C,outdir= change to given outdir before extracting files numeric-ids restore numeric IDs (user, group, etc.) rather than names exclude-rx= skip paths that match the unanchored regular expression v,verbose increase log output (can be used more than once) +map-user= given OLD=NEW, restore OLD user as NEW user +map-group= given OLD=NEW, restore OLD group as NEW group +map-uid= given OLD=NEW, restore OLD uid as NEW uid +map-gid= given OLD=NEW, restore OLD gid as NEW gid q,quiet don't show progress meter """ @@ -63,6 +67,38 @@ def create_path(n, fullname, meta): elif stat.S_ISLNK(n.mode): os.symlink(n.readlink(), fullname) + +def parse_owner_mappings(type, options, fatal): + """Traverse the options and parse all --map-TYPEs, or call Option.fatal().""" + opt_name = '--map-' + type + value_rx = r'^([^=]+)=([^=]*)$' + if type in ('uid', 'gid'): + value_rx = r'^(-?[0-9]+)=(-?[0-9]+)$' + owner_map = {} + for flag in options: + (option, parameter) = flag + if option != opt_name: + continue + match = re.match(value_rx, parameter) + if not match: + raise fatal("couldn't parse %s as %s mapping" % (parameter, type)) + old_id, new_id = match.groups() + if type in ('uid', 'gid'): + old_id = int(old_id) + new_id = int(new_id) + owner_map[old_id] = new_id + return owner_map + + +def apply_metadata(meta, name, restore_numeric_ids, owner_map): + m = copy.deepcopy(meta) + m.user = owner_map['user'].get(m.user, m.user) + m.group = owner_map['group'].get(m.group, m.group) + m.uid = owner_map['uid'].get(m.uid, m.uid) + m.gid = owner_map['gid'].get(m.gid, m.gid) + m.apply_to_path(name, restore_numeric_ids = restore_numeric_ids) + + # Track a list of (restore_path, vfs_path, meta) triples for each path # we've written for a given hardlink_target. This allows us to handle # the case where we restore a set of hardlinks out of order (with @@ -148,7 +184,7 @@ def find_dir_item_metadata_by_name(dir, name): meta_stream.close() -def do_root(n, restore_root_meta=True): +def do_root(n, owner_map, restore_root_meta = True): # Very similar to do_node(), except that this function doesn't # create a path for n's destination directory (and so ignores # n.fullname). It assumes the destination is '.', and restores @@ -170,15 +206,15 @@ def do_root(n, restore_root_meta=True): # Don't get metadata if this is a dir -- handled in sub do_node(). if meta_stream and not stat.S_ISDIR(sub.mode): m = metadata.Metadata.read(meta_stream) - do_node(n, sub, m) + do_node(n, sub, owner_map, meta = m) if root_meta and restore_root_meta: - root_meta.apply_to_path('.', restore_numeric_ids = opt.numeric_ids) + apply_metadata(root_meta, '.', opt.numeric_ids, owner_map) finally: if meta_stream: meta_stream.close() -def do_node(top, n, meta=None): +def do_node(top, n, owner_map, meta = None): # Create n.fullname(), relative to the current directory, and # restore all of its metadata, when available. The meta argument # will be None for dirs, or when there is no .bupm (i.e. no @@ -221,9 +257,9 @@ def do_node(top, n, meta=None): # Don't get metadata if this is a dir -- handled in sub do_node(). if meta_stream and not stat.S_ISDIR(sub.mode): m = metadata.Metadata.read(meta_stream) - do_node(top, sub, m) + do_node(top, sub, owner_map, meta = m) if meta and not created_hardlink: - meta.apply_to_path(fullname, restore_numeric_ids = opt.numeric_ids) + apply_metadata(meta, fullname, opt.numeric_ids, owner_map) finally: if meta_stream: meta_stream.close() @@ -242,6 +278,10 @@ if not extra: exclude_rxs = parse_rx_excludes(flags, o.fatal) +owner_map = {} +for map_type in ('user', 'group', 'uid', 'gid'): + owner_map[map_type] = parse_owner_mappings(map_type, flags, o.fatal) + if opt.outdir: mkdirp(opt.outdir) os.chdir(opt.outdir) @@ -266,7 +306,7 @@ for d in extra: if not isdir: add_error('%r: not a directory' % d) else: - do_root(n, restore_root_meta = (name == '.')) + do_root(n, owner_map, restore_root_meta = (name == '.')) else: # Source is /foo/what/ever -- extract ./ever to cwd. if isinstance(n, vfs.FakeSymlink): @@ -277,10 +317,10 @@ for d in extra: target = n.dereference() mkdirp(n.name) os.chdir(n.name) - do_root(target) + do_root(target, owner_map) else: # Not a directory or fake symlink. meta = find_dir_item_metadata_by_name(n.parent, n.name) - do_node(n.parent, n, meta=meta) + do_node(n.parent, n, owner_map, meta = meta) if not opt.quiet: progress('Restoring: %d, done.\n' % total_restored) diff --git a/t/test-restore-map-owner.sh b/t/test-restore-map-owner.sh new file mode 100755 index 0000000..6ebfdc5 --- /dev/null +++ b/t/test-restore-map-owner.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +. ./wvtest-bup.sh + +if [ $(t/root-status) != root ]; then + echo 'Not root: skipping restore --map-* tests.' + exit 0 # FIXME: add WVSKIP. +fi + +top="$(WVPASS pwd)" || exit $? +tmpdir="$(WVPASS wvmktempdir)" || exit $? +export BUP_DIR="$tmpdir/bup" + +bup() { "$top/bup" "$@"; } + +uid=$(WVPASS id -u) || exit $? +user=$(WVPASS id -un) || exit $? +gid=$(WVPASS id -g) || exit $? +group=$(WVPASS id -gn) || exit $? + +other_uinfo=$(WVPASS t/id-other-than --user "$user") || exit $? +other_user="${other_uinfo%%:*}" +other_uid="${other_uinfo##*:}" + +other_ginfo=$(WVPASS t/id-other-than --group "$group") || exit $? +other_group="${other_ginfo%%:*}" +other_gid="${other_ginfo##*:}" + +WVPASS bup init +WVPASS cd "$tmpdir" + +WVSTART "restore --map-user/group/uid/gid (control)" +WVPASS mkdir src +WVPASS touch src/foo +WVPASS bup index src +WVPASS bup save -n src src +WVPASS bup restore -C dest "src/latest/$(pwd)/src/" +WVPASS bup xstat dest/foo > foo-xstat +WVPASS grep -qE "^user: $user\$" foo-xstat +WVPASS grep -qE "^uid: $uid\$" foo-xstat +WVPASS grep -qE "^group: $group\$" foo-xstat +WVPASS grep -qE "^gid: $gid\$" foo-xstat + +WVSTART "restore --map-user/group/uid/gid (user/group)" +WVPASS rm -rf dest +# Have to remap uid/gid too because we're root and 0 would win). +WVPASS bup restore -C dest \ + --map-uid "$uid=$other_uid" --map-gid "$gid=$other_gid" \ + --map-user "$user=$other_user" --map-group "$group=$other_group" \ + "src/latest/$(pwd)/src/" +WVPASS bup xstat dest/foo > foo-xstat +WVPASS grep -qE "^user: $other_user\$" foo-xstat +WVPASS grep -qE "^uid: $other_uid\$" foo-xstat +WVPASS grep -qE "^group: $other_group\$" foo-xstat +WVPASS grep -qE "^gid: $other_gid\$" foo-xstat + +WVSTART "restore --map-user/group/uid/gid (user/group trumps uid/gid)" +WVPASS rm -rf dest +WVPASS bup restore -C dest \ + --map-uid "$uid=$other_uid" --map-gid "$gid=$other_gid" \ + "src/latest/$(pwd)/src/" +# Should be no changes. +WVPASS bup xstat dest/foo > foo-xstat +WVPASS grep -qE "^user: $user\$" foo-xstat +WVPASS grep -qE "^uid: $uid\$" foo-xstat +WVPASS grep -qE "^group: $group\$" foo-xstat +WVPASS grep -qE "^gid: $gid\$" foo-xstat + +WVSTART "restore --map-user/group/uid/gid (uid/gid)" +WVPASS rm -rf dest +WVPASS bup restore -C dest \ + --map-user "$user=" --map-group "$group=" \ + --map-uid "$uid=$other_uid" --map-gid "$gid=$other_gid" \ + "src/latest/$(pwd)/src/" +WVPASS bup xstat dest/foo > foo-xstat +WVPASS grep -qE "^user: $other_user\$" foo-xstat +WVPASS grep -qE "^uid: $other_uid\$" foo-xstat +WVPASS grep -qE "^group: $other_group\$" foo-xstat +WVPASS grep -qE "^gid: $other_gid\$" foo-xstat + +WVSTART "restore --map-user/group/uid/gid (zero uid/gid trumps all)" +WVPASS rm -rf dest +WVPASS bup restore -C dest \ + --map-user "$user=$other_user" --map-group "$group=$other_group" \ + --map-uid "$uid=0" --map-gid "$gid=0" \ + "src/latest/$(pwd)/src/" +WVPASS bup xstat dest/foo > foo-xstat +WVPASS grep -qE "^uid: 0\$" foo-xstat +WVPASS grep -qE "^gid: 0\$" foo-xstat + +WVPASS rm -rf "$tmpdir"