The dyadic snapshot scheduler

dss is admin friendly: It is easy to configure and needs little attention once configured to run in daemon mode. It keeps track of the available disk space and removes snapshots if disk space becomes sparse or snapshots become older than the specified time.

dss is also user-friendly because users can browse the snapshot directories without admin intervention and see the contents of the file system at the various times a snapshot was taken. In particular, users can easily restore accidentally removed files by using their favorite file browser to simply copy files from the snapshot directory back to the live system.

dss gives your data an additional level of security besides the usual tape-based backups: If the file server goes down and all data is lost you can simply use the most recent snapshot as an immediate replacement -- no need for a restore from tape that takes days to complete.

Snapshot pruning takes place in a dyadic fashion: Many recent snapshots are available, but the number of snapshots per time interval decreases exponentially. For example, one can configure dss so that it keeps 16 snapshots not older than one week, 8 snapshots between one and two weeks old, 4 snapshots between two and three weeks old, and so on.

0.1.5 (to be announced)

- snapshot recycling: outdated, redundant and orphaned snapshots are

  reused as the basis for new snapshots.

- dss writes log messages to stderr rather than to the logfile unless

  running in daemon mode.

- Default hooks work also on systems where "/bin/true" does not exist,

  e.g. Mac OS.

0.1.4 (2010-11-08)

Support for resuming snapshot creation after restart has been added and snapshot removal is deferred until at least one current snapshot exists. The internal handling of the various hooks has been simplified and many error messages have been clarified. On SIGHUP, dss now writes its configuration and internal state to the log file.

0.1.3 (2009-06-06)

0.1.2 (2009-03-05)

0.1.1 (2008-11-13)

0.1.0 (2008-10-10)

Only the source code is available for download. Use git to clone the dss repository by executing

or grab the tarball of the current master branch. If you prefer to download the tarball of the latest release, select the corresponding snapshot link on the dss gitweb page

	make

in the dss source directory to build the dss executable and copy it to some directory that is included in your PATH, e.g. to $HOME/bin or to /usr/local/bin.

Note that you'll likely need a recent version of gnu gengetopt to compile dss.

Optionally, type

	 make man

to create the man page of dss. This invokes help2man so make sure that help2man is installed on your system. Note that the man page is just the nroff variant of the output of "dss --detailed-help".

dss is known to compile on Linux, MacOS, Solaris, FreeBSD and NetBSD. However, it is run-tested only on Linux.

Also make sure that rsync is installed on your system. Version 2.6.1 or newer is required.

Examples:

Suppose you'd like to create snapshots of the existing directory

	/foo/bar

in the directory

	/baz/qux.

Create the config file

	~/.dssrc

that contains the values for the source and the destination directories as follows:

	echo 'source-dir "/foo/bar"' > ~/.dssrc
	echo 'dest-dir "/baz/qux"' >> ~/.dssrc

Then execute the commands

	mkdir /baz/qux
	dss --run

In order to print the list of all snapshots created so far, use

	dss --ls

Yes, it's really that easy.

The second example involves a slightly more sophisticated config file. It instructs dss to exclude everything which matches at least one pattern of the given exclude file, prevents rsync from crossing file system boundaries and increases the number of snapshots.

	source-dir "/foo/bar"
	dest-dir "/baz/qux"
	# exclude files matching patterns in /etc/dss.exclude
	rsync-option "--exclude-from=/etc/dss.exclude"
	# don't cross filesystem boundaries
	rsync-option "--one-file-system"
	# maintain 2^6 - 1 = 63 snaphots
	num-intervals "6"

The /etc/dss.exclude file could look like this (see rsync(1) for more examples)

 
	 - /proc
	 - /**/tmp/

Note that dss supports many more features and config options such as taking snapshots from remote hosts and several hooks that are executed on certain events, for example whenever a snapshot was created successfully. Try

	dss -h

for an overview of all supported command line options or

	dss --detailed-help

for the full help text.

dss is open source software, licensed under the GNU General Public License, Version 2.

André Noll, maan@systemlinux.org

NAME

SYNOPSIS

DESCRIPTION

the dyadic snapshot scheduler

dss creates hardlink-based snapshots of a given directory on a remote or local host using rsync's link-dest feature.

-h, --help

Print help and exit

--detailed-help

Print help, including all details and hidden options, and exit

-V, --version

Print version and exit

General options:

-c, --config-file=filename

(default='~/.dssrc')

Options may be given at the command line or in the configuration file. As usual, if an option is given both at the command line and in the configuration file, the command line option takes precedence.

However, there is an important exception to this rule: If the --run option was given (see below) then dss honors SIGHUP and re-reads its configuration file whenever it receives this signal. In this case the options in the config file override any options that were previously given at the command line. This allows to change the configuration of a running dss process on the fly by sending SIGHUP.

-d, --daemon

Run as background daemon (default=off)

Note that dss refuses to start in daemon mode if no logfile was specified. This option is mostly useful in conjuction with the -R option described below.

Note that it is not possible to change whether dss runs as background daemon by sending SIGHUP.

-D, --dry-run

Only print what would be done (default=off)

This flag does not make sense for all commands. The run command refuses to start if this option was given. The ls command silently ignores this flag.

Logging:

-l, --loglevel=level

Set loglevel (0-6) (default=`3')

Lower values mean more verbose logging.

--logfile=filename

Logfile for the dss daemon process

This option is only honored if both --run and --daemon are given. Otherwise it is silently ignored and log output is written to stderr.

Commands:

Group: command

dss supports a couple of commands each of which corresponds to a different command line option. Exactly one of these options must be given.

-C, --create

Create a new snapshot

Execute the rsync command to create a new snapshot. Note that this command does not care about free disk space.

-P, --prune

Remove redundant and outdated snapshots

A snapshot is considered outdated if its interval number is greater or equal than the specified number of unit intervals. See the "Intervals" section below for the precise definition of these terms.

A snapshot is said to be redundant if it belongs to an interval that already contains more than the desired number of snapshots.

The prune command gets rid of both outdated and redundant snapshots.

-L, --ls

Print a list of all snapshots

The list will contain all snapshots no matter of their state, i. e. incomplete snapshots and snapshots being deleted will also be listed.

-R, --run

Start creating and pruning snapshots

This is the main mode of operation. Snapshots will be created in an endless loop as needed and pruned automatically. The loop only terminates on fatal errors or if a terminating signal was received. See also the --exit-hook option.

Rsync-related options:

-H, --remote-host=hostname

Remote host (default=`localhost')

If this option is given and its value differs from the local host, then rsync uses ssh. Make sure there is no password needed for the ssh connection. To achieve that, use public key authentication for ssh and, if needed, set the remote user name by using the --remote-user option.

-U, --remote-user=username

Remote user name (default: current user)

Set this if the user running dss is different from the user at the remote host when using ssh.

--source-dir=dirname

The data directory

The directory on the remote host from which snapshots are taken. Of course, the user specified as --remote-user must have read access to this directory.

--dest-dir=dirname

Snapshot dir

The destination directory on the local host where snapshots will be written. This must be writable by the user who runs dss.

--no-resume

Do not try to resume from previous runs (default=off)

Starting from version 0.1.4, dss tries to resume from a previously cancelled dss instance by default. It does so by looking at the status of the most recently created snapshot. If this snapshot status is incomplete, its directory is reused as the destination directory for a subsequent rsync run.

The --no-resume option deactivates this feature so that a new directory is always used as the rsync destination directory.

-O, --rsync-option=option

Further rsync options

This option may be given multiple times. The given argument is passed verbatim to the rsync command. Note that in order to use rsync options that require an argument, you have to specify the option and its argument as separate --rsync-options, like this:

--rsync-option --exclude --rsync-option /proc

Intervals:

-u, --unit-interval=days

The duration of a unit interval (default=`4')

dss snapshot aging is implemented in terms of intervals. There are two command line options related to intervals: the duration u of a "unit" interval and the number n of those unit intervals.

dss removes any snapshots older than n times u and tries to keep 2^(n - k - 1) snapshots in interval k, where the interval number k counts from zero, zero being the most recent unit interval.

In other words, the oldest snapshot will at most be u * n days (= 20 days if default values are used) old. Moreover, there are at most 2^n - 1 snapshots in total (i. e. 31 by default). Observe that you have to create at least 2^(n - 1) snapshots each interval for this to work out because that is the number of snapshots in interval zero.

-n, --num-intervals=num

The number of unit intervals (default=`5')

Hooks:

-r, --pre-create-hook=command Executed before snapshot creation

(default=`true')

Execute this command before trying to create a new snapshot. If this command returns with a non-zero exit status, no snapshot is being created and the operation is retried later.

For example, one might want to execute a script that checks whether all snapshot-related file systems are properly mounted.

Another possible application of this is to return non-zero during office hours in order to not slow down the file systems by taking snapshots.

-o, --post-create-hook=command

Executed after snapshot creation

(default=`true')

Execute this after a snapshot has successfully been created. The full path of the newly created snapshot is passed to the hook as the first argument. The exit code of this hook is ignored.

For instance this hook can be used to count the number of files per user and/or the disk usage patterns in order to store them in a database for further analysis.

--pre-remove-hook=command Executed before snapshot removal

(default=`true')

Execute this command before removing a snapshot. The full path to the snapshot about to be deleted is passed to the command as the first argument. If the command returns with a non-zero exit status, no snapshot is being removed and the operation is retried later.

For example, one might want to execute a script that checks whether the snapshot to be deleted is currently used by another process, e.g. by a tape-based backup system that runs concurrently to dss.

Another possible application of this is to record disk-usage patterns before and after snapshot removal.

--post-remove-hook=command

Executed after snapshot removal

(default=`true')

Execute this after a snapshot has successfully been removed. As for the pre-remove hook, the full path of the removed snapshot is passed to the hook as the first argument. The exit code of this hook is ignored.

-e, --exit-hook=command

Executed if run command exits (default=`true')

This hook is only used if the --run command was given which instructs dss to run in an endless loop. The exit-hook gets executed whenever this endless loop terminates. The reason for terminating the loop is passed as the first argument.

One possible application for this hook is to send email to the system administrator to let her know that no more snapshots are going to be created.

Disk space monitoring:

-m, --min-free-mb=megabytes

Minimal amount of free disk space (default=`100')

If disk space on the file system containing the destination directory gets low, "dss --run" will suspend the currently running rsync process and will start to remove snapshots in order to free disk space. This option specifies the minimal amount of free disk space. If less than the given number of megabytes is available, snapshots are being deleted. See also the --min_free_percent and the min-free-percent-inodes options.

A value of zero deactivates this check.

-p, --min-free-percent=percent

Minimal percent of free disk space

(default=`2')

See --min-free-mb. Note that it is not recommended to set both --min-free-mb and --min-free-percent to zero as this will cause your file system to fill up quickly.

-i, --min-free-percent-inodes=percent

Minimal percent of free inodes (default=`0')

Specify the minimum amount of free inodes on the file system containing the destination dir. If less than that many inodes are free, snapshot removal kicks in just as in case of low disk space.

Note that not every file system supports the concept of inodes. Moreover it is not possible to reliably detect whether this is the case. Therefore this feature is disabled by default. It's safe to enable it for ext2/ext3/ext4 file systems on linux though.

A value of zero (the default) deactivates this check.

-k, --keep-redundant

Prune by disk space only (default=off)

If this flag is not given dss removes redundant and outdated snapshots automatically.

Otherwise, this feature is deactivated so that snapshots are only being removed in case disk space or number of free inodes becomes low. Use this flag if the file system containing the destination directory is used for snapshots only.

SEE ALSO