Backing Up Files with rsync

A more feature-rich command for doing backups is rsync. What makes rsync so unique is the rsync algorithm, which compares the local and remote files one small block at a time using checksums, and only transfers the blocks that are different. This algorithm is so efficient that it has been reused in many backup products. The rsync command can work either on top of a remote shell (ssh), or by running an rsyncd daemon on the server end. The following example uses rsync over ssh to mirror a directory:

$ rsync -avz –delete giovanni@server1:/home/giovanni/pics/giovanni/pics/

The command just shown is intended to mirror the remote directory structure (/home/giovanni/pics/) on the local system. The -a says to run in archive mode (recursively copying all files from the remote directory), the -z option compresses the files, and –v makes the output verbose. The –delete tells rsync to delete any files on the local system that no longer exist on the remote system. For ongoing backups, you can have rsync do seven-day incremental backups. Here’s an example:

# mkdir /var/backups
# rsync –delete –backup \
–backup-dir=/var/backups/backup-`date +%A` \
-avz giovanni@server1:/home/giovanni/Personal/ \
/var/backups/current-backup/

When the command just shown runs, all the files from /home/giovanni/Personal on the remote system server1 are copied to the local directory /var/backups/current-backup. All files modified today are copied to a directory named after today’s day of the week, such as /var/backups/backup-Monday.

Over a week, seven directories will be created that reflect changes over each of the past seven days. Another trick for rotated backups is to use hard links instead of multiple copies of the files. This two-step process consists of rotating the files, then running rsync:

# rm -rf /var/backups/backup-old/
# mv /var/backups/backup-current/ /var/backups/backup-old/
# rsync –delete –link-dest=/var/backups/backup-old -avz \
giovanni@server1:/home/giovanni/Personal/ /var/backups/backup-current/

In the previous procedure, the existing backup-current directory replaces the backup-old directory, deleting the two-week-old full backup with last-week’s full backup. When the new full backup is run with rsync using the –link-dest option, if any of the files being backed up from the remote Personal directory on server1 existed during the previous backup (now in backup-old), a hard link is created between the file in the backup-current directory and backup-old directory.

You can save a lot of space by having hard links between files in your backup-old and backup-current directory. For example, if you had a file named file1.txt in both directories, you could check that both were the same physical file by listing the files’ inodes as follows:

$ ls -i /var/backups/backup*/file1.txt
260761 /var/backups/backup-current/file1.txt
260761 /var/backups/backup-old/file1.txt

Advertisements

About msotela

This blog is for anyone who wants to access the power of a Linux system as a systems administrator or user. You may be a Linux enthusiast, a Linux professional, or possibly a computer professional who is increasingly finding the Windows systems in your data center supplanted by Linux boxes.

Posted on September 23, 2009, in Unix/Linux. Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: