1 % bup-memtest(1) Bup %BUP_VERSION%
2 % Avery Pennarun <apenwarr@gmail.com>
7 bup-memtest - test bup memory usage statistics
11 bup memtest [options...]
15 `bup memtest` opens the list of pack indexes in your bup
16 repository, then searches the list for a series of
17 nonexistent objects, printing memory usage statistics after
20 Because of the way Unix systems work, the output will
21 usually show a large (and unchanging) value in the VmSize
22 column, because mapping the index files in the first place
23 takes a certain amount of virtual address space. However, this
24 virtual memory usage is entirely virtual; it doesn't take
25 any of your RAM. Over time, bup uses *parts* of the
26 indexes, which need to be loaded from disk, and this is
27 what causes an increase in the VmRSS column.
32 : set the number of objects to search for during each
33 cycle (ie. before printing a line of output)
36 : set the number of cycles (ie. the number of lines of
37 output after the first). The first line of output is
38 always 0 (ie. the baseline before searching for any
42 : ignore any `.midx` files created by `bup midx`. This
43 allows you to compare memory performance with and
49 $ bup memtest -n300 -c5
50 PackIdxList: using 1 index.
51 VmSize VmRSS VmData VmStk
52 0 20824 kB 4528 kB 1980 kB 84 kB
53 300 20828 kB 5828 kB 1984 kB 84 kB
54 600 20828 kB 6844 kB 1984 kB 84 kB
55 900 20828 kB 7836 kB 1984 kB 84 kB
56 1200 20828 kB 8736 kB 1984 kB 84 kB
57 1500 20828 kB 9452 kB 1984 kB 84 kB
59 $ bup memtest -n300 -c5 --ignore-midx
60 PackIdxList: using 361 indexes.
61 VmSize VmRSS VmData VmStk
62 0 27444 kB 6552 kB 2516 kB 84 kB
63 300 27448 kB 15832 kB 2520 kB 84 kB
64 600 27448 kB 17220 kB 2520 kB 84 kB
65 900 27448 kB 18012 kB 2520 kB 84 kB
66 1200 27448 kB 18388 kB 2520 kB 84 kB
67 1500 27448 kB 18556 kB 2520 kB 84 kB
72 When optimizing bup indexing, the first goal is to keep the
73 VmRSS reasonably low. However, it might eventually be
74 necessary to swap in all the indexes, simply because
75 you're searching for a lot of objects, and this will cause
76 your RSS to grow as large as VmSize eventually.
78 The key word here is *eventually*. As long as VmRSS grows
79 reasonably slowly, the amount of disk activity caused by
80 accessing pack indexes is reasonably small. If it grows
81 quickly, bup will probably spend most of its time swapping
82 index data from disk instead of actually running your
83 backup, so backups will run very slowly.
85 The purpose of `bup memtest` is to give you an idea of how
86 fast your memory usage is growing, and to help in
87 optimizing bup for better memory use. If you have memory
88 problems you might be asked to send the output of `bup
89 memtest` to help diagnose the problems.
91 Tip: try using `bup midx -a` or `bup midx -f` to see if it
92 helps reduce your memory usage.
94 Trivia: index memory usage in bup (or git) is only really a
95 problem when adding a large number of previously unseen
96 objects. This is because for each object, we need to
97 absolutely confirm that it isn't already in the database,
98 which requires us to search through *all* the existing pack
99 indexes to ensure that none of them contain the object in
100 question. In the more obvious case of searching for
101 objects that *do* exist, the objects being searched for are
102 typically related in some way, which means they probably
103 all exist in a small number of packfiles, so memory usage
104 will be constrained to just those packfile indexes.
106 Since git users typically don't add a lot of files in a
107 single run, git doesn't really need a program like `bup
108 midx`. bup, on the other hand, spends most of its time
109 backing up files it hasn't seen before, so its memory usage
110 patterns are different.
119 Part of the `bup`(1) suite.