midx: close the mmap before erasing an midx file

Linux allows erasing a file even if it is still memory mapped, but
Windows does not. However, Cygwin tries to emulate Linux and, under
certain conditions, it silently moves the file to $RECYCLED.BIN/, but
not in a way where Windows takes care of it when we use the "empty the
recycled bin" command. The erased midx ends up comsuming more and more
disk space.

To solve this, we have to close the midx's mmap before erasing
the midx.

Cygwin users: you can use this command to clean up your
$RECYCLE.BIN/ directories:
find /cygdrive/?/\$RECYCLE.BIN/ -name "\.???[[:xdigit:]]*" -print -delete

Signed-off-by: Patrick Rouleau <prouleau72@gmail.com>
[rlb@defaultvalue.org: adjust commit message]

diff --git a/lib/bup/git.py b/lib/bup/git.py
index 59bc169a2e68508f23fe995b5888aea86f447926..826a2d1251d1f9d351004b22e3278e0e1d733501 100644 (file)
--- a/lib/bup/git.py
+++ b/lib/bup/git.py
@@ -468,6 +468,7 @@ class PackIdxList:
                                     '  used by %s\n') % (n, mxf))
                                 broken = True
                         if broken:
+                            mx.close()
                             del mx
                             unlink(full)
                         else:
@@ -489,6 +490,7 @@ class PackIdxList:
                     elif not ix.force_keep:
                         debug1('midx: removing redundant: %s\n'
                                % os.path.basename(ix.name))
+                        ix.close()
                         unlink(ix.name)
             for full in glob.glob(os.path.join(self.dir,'*.idx')):
                 if not d.get(full):

diff --git a/lib/bup/midx.py b/lib/bup/midx.py
index 3a06b733b9008affa934186ad497a8cc1fbcda30..57a478759297228622e32f102976150fe27d7525 100644 (file)
--- a/lib/bup/midx.py
+++ b/lib/bup/midx.py
@@ -18,6 +18,7 @@ class PackMidx:
     def __init__(self, filename):
         self.name = filename
         self.force_keep = False
+        self.map = None
         assert(filename.endswith('.midx'))
         self.map = mmap_read(open(filename))
         if str(self.map[0:4]) != 'MIDX':
@@ -46,6 +47,9 @@ class PackMidx:
         self.whichlist = buffer(self.map, self.which_ofs, nsha*4)
         self.idxnames = str(self.map[self.which_ofs + 4*nsha:]).split('\0')
 
+    def __del__(self):
+        self.close()
+
     def _init_failed(self):
         self.bits = 0
         self.entries = 1
@@ -67,6 +71,11 @@ class PackMidx:
     def _get_idxname(self, i):
         return self.idxnames[self._get_idx_i(i)]
 
+    def close(self):
+        if self.map is not None:
+            self.map.close()
+            self.map = None
+
     def exists(self, hash, want_source=False):
         """Return nonempty if the object exists in the index files."""
         global _total_searches, _total_steps