Also add a 'help' command to ftp, and fix up some minor help messages.
# NAME
-bup drecurse - recursively list files in the filesystem
+bup drecurse - recursively list files in your filesystem
# SYNOPSIS
--- /dev/null
+% bup-ftp(1) Bup %BUP_VERSION%
+% Avery Pennarun <apenwarr@gmail.com>
+% %BUP_DATE%
+
+# NAME
+
+bup ftp - ftp-like client for navigating bup repositories
+
+# SYNOPSIS
+
+bup ftp
+
+# DESCRIPTION
+
+`bup ftp` is a command-line tool for navigating bup
+repositories. It has commands similar to the Unix `ftp`(1)
+command. The file hierarchy is the same as that shown by
+`bup-fuse`(1) and `bup-ls`(1).
+
+Note: if your system has the python-readline library
+installed, you can use the \<tab\> key to complete filenames
+while navigating your backup data. This will save you a
+lot of typing.
+
+
+# COMMANDS
+
+The following commands are available inside `bup ftp`:
+
+ls
+: print the contents of the current working directory
+
+cd *dirname*
+: change to a different working directory
+
+pwd
+: print the path of the current working directory
+
+cat *filenames...*
+: print the contents of one or more files to stdout
+
+get *filename* *localname*
+: download the contents of *filename* and save it to disk
+ as *localname*. If *localname* is omitted, uses
+ *filename* as the local name.
+
+mget *filenames...*
+: download the contents of the given *filenames* and
+ stores them to disk under the same names. The
+ filenames may contain Unix filename globs (`*`, `?`,
+ etc.)
+
+help
+: print a list of available commands
+
+quit
+: exit the `bup ftp` client
+
+
+# EXAMPLE
+
+ $ bup ftp
+ bup> ls
+ mybackup/
+ yourbackup/
+ bup> cd mybackup/
+ bup> ls
+ .2fe288dedbfab372c84b0502ee2bc1504270f3b3/
+ .ae760aa4cfc13b689b46e3d2ce5ae50e92299c72/
+ 2010-02-05-185507@
+ 2010-02-05-185508@
+ latest@
+ bup> cd latest/
+ bup> ls
+ (...etc...)
+ bup> get myfile
+ Saving 'myfile'
+ bup> quit
+
+
+# SEE ALSO
+
+`bup-join`(1), `bup-fuse`(1), `bup-ls`(1), `bup-save`(1), `git-show`(1)
+
+
+# BUP
+
+Part of the `bup`(1) suite.
# SEE ALSO
-`fuse`(7), `fusermount`(1), `bup-ls`(1)
+`fuse`(7), `fusermount`(1), `bup-ls`(1), `bup-ftp`(1)
# BUP
--- /dev/null
+% bup-help(1) Bup %BUP_VERSION%
+% Avery Pennarun <apenwarr@gmail.com>
+% %BUP_DATE%
+
+# NAME
+
+bup help - open the documentation for a given bup command
+
+# SYNOPSIS
+
+bup help <command>
+
+# DESCRIPTION
+
+`bup help <command>` opens the documentation for the given command.
+This is currently equivalent to typing `man bup-<command>`.
+
+
+# EXAMPLE
+
+ $ bup help help
+ (Imagine that this man page was pasted below,
+ recursively. Because that would cause an endless
+ we include this silly remark instead. Chicken.)
+
+# BUP
+
+Part of the `bup`(1) suite.
# DESCRIPTION
`bup ls` lists files and directories in your bup repository
-in the same layout as they would appear with `bup-fuse`(1).
+using the same directory hierarchy as they would have with
+`bup-fuse`(1).
The top level directory is the branch (corresponding to
the `-n` option in `bup save`), the next level is the date
# EXAMPLE
- bup ls /myserver/1999-01-01/etc/profile
+ bup ls /myserver/latest/etc/profile
# SEE ALSO
-`bup-join`(1), `bup-fuse`(1), `bup-save`(1), `git-show`(1)
+`bup-join`(1), `bup-fuse`(1), `bup-ftp`(1), `bup-save`(1), `git-show`(1)
# BUP
--- /dev/null
+% bup-memtest(1) Bup %BUP_VERSION%
+% Avery Pennarun <apenwarr@gmail.com>
+% %BUP_DATE%
+
+# NAME
+
+bup memtest - test bup memory usage statistics
+
+# SYNOPSIS
+
+bup memtest [options...]
+
+# DESCRIPTION
+
+`bup memtest` opens the list of pack indexes in your bup
+repository, then searches the list for a series of
+nonexistent objects, printing memory usage statistics after
+each cycle.
+
+Because of the way Unix systems work, the output will
+usually show a large (and unchanging) value in the VmSize
+column, because mapping the index files in the first place
+takes a certain amount of virtual address space. However, this
+virtual memory usage is entirely virtual; it doesn't take
+any of your RAM. Over time, bup uses *parts* of the
+indexes, which need to be loaded from disk, and this is
+what causes an increase in the VmRSS column.
+
+# OPTIONS
+
+-n, --number=*number*
+: set the number of objects to search for during each
+ cycle (ie. before printing a line of output)
+
+-c, --cycles=*cycles*
+: set the number of cycles (ie. the number of lines of
+ output after the first). The first line of output is
+ always 0 (ie. the baseline before searching for any
+ objects).
+
+--ignore-midx
+: ignore any `.midx` files created by `bup midx`. This
+ allows you to compare memory performance with and
+ without using midx.
+
+
+# EXAMPLE
+
+ $ bup memtest -n300 -c5
+ PackIdxList: using 1 index.
+ VmSize VmRSS VmData VmStk
+ 0 20824 kB 4528 kB 1980 kB 84 kB
+ 300 20828 kB 5828 kB 1984 kB 84 kB
+ 600 20828 kB 6844 kB 1984 kB 84 kB
+ 900 20828 kB 7836 kB 1984 kB 84 kB
+ 1200 20828 kB 8736 kB 1984 kB 84 kB
+ 1500 20828 kB 9452 kB 1984 kB 84 kB
+
+ $ bup memtest -n300 -c5 --ignore-midx
+ PackIdxList: using 361 indexes.
+ VmSize VmRSS VmData VmStk
+ 0 27444 kB 6552 kB 2516 kB 84 kB
+ 300 27448 kB 15832 kB 2520 kB 84 kB
+ 600 27448 kB 17220 kB 2520 kB 84 kB
+ 900 27448 kB 18012 kB 2520 kB 84 kB
+ 1200 27448 kB 18388 kB 2520 kB 84 kB
+ 1500 27448 kB 18556 kB 2520 kB 84 kB
+
+
+# DISCUSSION
+
+When optimizing bup indexing, the first goal is to keep the
+VmRSS reasonably low. However, it might eventually be
+necessary to swap in all the indexes, simply because
+you're searching for a lot of objects, and this will cause
+your RSS to grow as large as VmSize eventually.
+
+The key word here is *eventually*. As long as VmRSS grows
+reasonably slowly, the amount of disk activity caused by
+accessing pack indexes is reasonably small. If it grows
+quickly, bup will probably spend most of its time swapping
+index data from disk instead of actually running your
+backup, so backups will run very slowly.
+
+The purpose of `bup memtest` is to give you an idea of how
+fast your memory usage is growing, and to help in
+optimizing bup for better memory use. If you have memory
+problems you might be asked to send the output of `bup
+memtest` to help diagnose the problems.
+
+Tip: try using `bup midx -a` or `bup midx -f` to see if it
+helps reduce your memory usage.
+
+Trivia: index memory usage in bup (or git) is only really a
+problem when adding a large number of previously unseen
+objects. This is because for each object, we need to
+absolutely confirm that it isn't already in the database,
+which requires us to search through *all* the existing pack
+indexes to ensure that none of them contain the object in
+question. In the more obvious case of searching for
+objects that *do* exist, the objects being searched for are
+typically related in some way, which means they probably
+all exist in a small number of packfiles, so memory usage
+will be constrained to just those packfile indexes.
+
+Since git users typically don't add a lot of files in a
+single run, git doesn't really need a program like `bup
+midx`. bup, on the other hand, spends most of its time
+backing up files it hasn't seen before, so its memory usage
+patterns are different.
+
+
+# SEE ALSO
+
+`bup-midx`(1)
+
+# BUP
+
+Part of the `bup`(1) suite.
# SEE ALSO
-`bup-save`(1), `bup-margin`(1)
+`bup-save`(1), `bup-margin`(1), `bup-memtest`(1)
# BUP
--- /dev/null
+% bup-newliner(1) Bup %BUP_VERSION%
+% Avery Pennarun <apenwarr@gmail.com>
+% %BUP_DATE%
+
+# NAME
+
+bup newliner - make sure progress messages don't overlap with output
+
+# SYNOPSIS
+
+\<any command\> 2>&1 | bup newliner
+
+# DESCRIPTION
+
+`bup newliner` is run automatically by bup. You shouldn't
+need it unless you're using it in some other program.
+
+Progress messages emitted by bup (and some other tools) are
+of the form "Message ### content\r", that is, a status
+message containing a variable-length number, followed by a
+carriage return character and no newline. If these
+messages are printed more than once, they overwrite each
+other, so what the user sees is a single line with a
+continually-updating number.
+
+This works fine until some other message is printed. For
+example, progress messages are usually printed to stderr,
+but other program messages might be printed to stdout. If
+those messages are shorter than the progress message line,
+the screen will be left with weird looking artifacts as the
+two messages get mixed together.
+
+`bup newliner` prints extra space characters at the right
+time to make sure that doesn't happen.
+
+If you're running a program that has problems with these
+artifacts, you can usually fix them by piping its stdout
+*and* its stderr through bup newliner.
+
+# BUP
+
+Part of the `bup`(1) suite.
--- /dev/null
+% bup-random(1) Bup %BUP_VERSION%
+% Avery Pennarun <apenwarr@gmail.com>
+% %BUP_DATE%
+
+# NAME
+
+bup random - generate a stream of random output
+
+# SYNOPSIS
+
+bup random [-S seed] [-f] <numbytes>
+
+# DESCRIPTION
+
+`bup random` produces a stream of pseudorandom output bytes to
+stdout. Note: the bytes are *not* generated using a
+cryptographic algorithm and should never be used for
+security.
+
+Note that the stream of random bytes will be identical
+every time `bup random` is run, unless you provide a
+different `seed` value. This is intentional: the purpose
+of this program is to be able to run repeatable tests on
+large amounts of data, so we want identical data every
+time.
+
+`bup random` generates about 240 megabytes per second on a
+modern test system (Intel Core2), which is faster than you
+could achieve by reading data from most disks. Thus, it
+can be helpful when running microbenchmarks.
+
+# OPTIONS
+
+<numbytes>
+: the number of bytes of data to generate. Can be used
+ with the suffices `k`, `M`, or `G` to indicate
+ kilobytes, megabytes, or gigabytes, respectively.
+
+-S, --seed=*seed*
+: use the given value to seed the pseudorandom number
+ generator. The generated output stream will be
+ identical for every stream seeded with the same value.
+ The default seed is 1. A seed value of 0 is equivalent
+ to 1.
+
+-f, --force
+: generate output even if stdout is a tty. (Generating
+ random data to a tty is generally considered
+ ill-advised, but you can do if you really want.)
+
+# EXAMPLES
+
+ $ bup random 1k | sha1sum
+ 2108c55d0a2687c8dacf9192677c58437a55db71 -
+
+ $ bup random -S1 1k | sha1sum
+ 2108c55d0a2687c8dacf9192677c58437a55db71 -
+
+ $ bup random -S2 1k | sha1sum
+ f71acb90e135d98dad7efc136e8d2cc30573e71a -
+
+ $ time bup random 1G >/dev/null
+ Random: 1024 Mbytes, done.
+
+ real 0m4.261s
+ user 0m4.048s
+ sys 0m0.172s
+
+ $ bup random 1G | bup split -t --bench
+ Random: 1024 Mbytes, done.
+ bup: 1048576.00kbytes in 18.59 secs = 56417.78 kbytes/sec
+ 1092599b9c7b2909652ef1e6edac0796bfbfc573
+
+# BUP
+
+Part of the `bup`(1) suite.
# NAME
-bup server - the server side of a remote bup session
+bup server - the server side of the bup client-server relationship
# SYNOPSIS
# SYNOPSIS
-bup <command> [options...]
+bup \<command\> [options...]
# DESCRIPTION
# COMMONLY USED SUBCOMMANDS
-`bup-index`(1)
-: Manage the index of files to back up.
`bup-fsck`(1)
-: Verify or recover the bup repository.
+: Check backup sets for damage and add redundancy information
+`bup-ftp`(1)
+: Browse backup sets using an ftp-like client
`bup-fuse`(1)
-: Mount the bup repository as a filesystem.
-`bup-save`(1)
-: Back up the files in the index.
-`bup-split`(1)
-: Back up an individual file, such as a tarball.
+: Mount your backup sets as a filesystem
+`bup-help`(1)
+: Print detailed help for the given command
+`bup-index`(1)
+: Create or display the index of files to back up
`bup-join`(1)
-: Retrieve a file backed up using `bup-split`(1).
+: Retrieve a file backed up using `bup-split`(1)
+`bup-ls`(1)
+: Browse the files in your backup sets
`bup-midx`(1)
-: Make backups go faster by generating midx files.
+: Index objects to speed up future backups
+`bup-save`(1)
+: Save files into a backup set (note: run "bup index" first)
+`bup-split`(1)
+: Split a single file into its own backup set
# RARELY USED SUBCOMMANDS
`bup-damage`(1)
-: Deliberately destroy data.
+: Deliberately destroy data
`bup-drecurse`(1)
-: Recursively list files in your filesystem.
+: Recursively list files in your filesystem
`bup-init`(1)
-: Initialize a bup repository.
-`bup-ls`(1)
-: List the files in a bup repository.
+: Initialize a bup repository
`bup-margin`(1)
-: Determine how close your bup repository is to armageddon.
+: Determine how close your bup repository is to armageddon
+`bup-memtest`(1)
+: Test bup memory usage statistics
+`bup-newliner`(1)
+: Make sure progress messages don't overlap with output
+`bup-random`(1)
+: Generate a stream of random output
`bup-server`(1)
-: The server side of the bup client-server relationship.
+: The server side of the bup client-server relationship
`bup-tick`(1)
-: Sleep for up to one second.
+: Wait for up to one second.
# SEE ALSO
outf.close()
except Exception, e:
log(' error: %s\n' % e)
+ elif cmd == 'help' or cmd == '?':
+ log('Commands: ls cd pwd cat get mget help quit\n')
+ elif cmd == 'quit' or cmd == 'exit' or cmd == 'bye':
+ break
else:
raise Exception('no such command %r' % cmd)
except Exception, e:
# the wrapper program provides the default usage string
os.execvp('bup', ['bup'])
elif len(extra) == 1:
- docname = 'bup-%s' % extra[0]
+ docname = (extra[0]=='bup' and 'bup' or ('bup-%s' % extra[0]))
exe = sys.argv[0]
(exepath, exefile) = os.path.split(exe)
manpath = os.path.join(exepath, '../Documentation/' + docname + '.[1-9]')
optspec = """
-memtest [-n elements] [-c cycles]
+bup memtest [-n elements] [-c cycles]
--
n,number= number of objects per cycle
c,cycles= number of cycles to run
ignore-midx ignore .midx files, use only .idx files
"""
-o = options.Options(sys.argv[0], optspec)
+o = options.Options('bup memtest', optspec)
(opt, flags, extra) = o.parse(sys.argv[1:])
if extra:
ftp = 'Browse backup sets using an ftp-like client',
fsck = 'Check backup sets for damage and add redundancy information',
fuse = 'Mount your backup sets as a filesystem',
+ help = 'Print detailed help for the given command',
index = 'Create or display the index of files to back up',
- join = 'The reverse operation to "bup split"',
+ join = 'Retrieve a file backed up using "bup split"',
ls = 'Browse the files in your backup sets',
midx = 'Index objects to speed up future backups',
save = 'Save files into a backup set (note: run "bup index" first)',
projects / bup.git / commitdiff