From: Johannes Berg Date: Fri, 1 May 2020 20:16:39 +0000 (+0200) Subject: mincore: fix reading information X-Git-Tag: 0.31~77 X-Git-Url: https://arthur.barton.de/cgi-bin/gitweb.cgi?p=bup.git;a=commitdiff_plain;h=2c1f4acab0355b28c4f8fb9f8d0329ec6b2c2afa mincore: fix reading information The _fmincore_chunk_size is typically set to 64MiB, which makes sense to avoid doing very large mmap() operations (to save already precious VM on 32-bit systems). However, since that's in bytes, we cannot divide a size in pages by it, and expect any useful outcome. Calculate the number of chunks (chunk_count) properly based on the size of the file, rather than its number of pages. Otherwise, chunk_count typically ends up just 1 even for a very large file (my test file was ~500MiB), and mincore() is run just once, so we fill the presence information only for the first 64MiB of the file, even if it was previously completely in RAM. Given a large enough test file (and enough RAM to keep it there), the following should print about the same times twice: cat test > /dev/null ; \ time cat test > /dev/null ; \ bup split --noop test ; \ time cat test >/dev/null Without the fix, it's evident that the file is evicted from RAM almost entirely (apart from the first 64MiB) even in this synthetic case. Signed-off-by: Johannes Berg Reviewed-by: Rob Browning Tested-by: Rob Browning --- diff --git a/lib/bup/helpers.py b/lib/bup/helpers.py index f384355..d988685 100644 --- a/lib/bup/helpers.py +++ b/lib/bup/helpers.py @@ -803,9 +803,7 @@ if _mincore: _set_fmincore_chunk_size() pages_per_chunk = _fmincore_chunk_size // sc_page_size; page_count = (st.st_size + sc_page_size - 1) // sc_page_size; - chunk_count = page_count // _fmincore_chunk_size - if chunk_count < 1: - chunk_count = 1 + chunk_count = (st.st_size + _fmincore_chunk_size - 1) // _fmincore_chunk_size result = bytearray(page_count) for ci in compat.range(chunk_count): pos = _fmincore_chunk_size * ci;