save: fix race in data vs. metadata size

Fix the race in data vs. metadata size by overriding the metadata
size with what we actually read and stored into the repository.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>

diff --git a/lib/bup/cmd/save.py b/lib/bup/cmd/save.py
index 32203db4b93da4b84b5171e875e521de36e877ae..21a49ad28b300d39d2ba533e3e49da2130b6eece 100755 (executable)
--- a/lib/bup/cmd/save.py
+++ b/lib/bup/cmd/save.py
@@ -434,9 +434,20 @@ def main(argv):
                 continue
             if stat.S_ISREG(ent.mode):
                 try:
+                    # If the file changes while we're reading it, then our reading
+                    # may stop at some point, but the stat() above may have gotten
+                    # a different size already. Recalculate the meta size so that
+                    # the repository records the accurate size in the metadata, even
+                    # if the other stat() data might be slightly older than the file
+                    # content (which we can't fix, this is inherently racy, but we
+                    # can prevent the size mismatch.)
+                    meta.size = 0
+                    def new_blob(data):
+                        meta.size += len(data)
+                        return w.new_blob(data)
                     with hashsplit.open_noatime(ent.name) as f:
                         (mode, id) = hashsplit.split_to_blob_or_tree(
-                                                w.new_blob, w.new_tree, [f],
+                                                new_blob, w.new_tree, [f],
                                                 keep_boundaries=False)
                 except (IOError, OSError) as e:
                     add_error('%s: %s' % (ent.name, e))