Rsync

General Information

Rsync can be used for local and over the network file copying. If the copy is interrupted, rsync will be able to only copy files and parts of files that it has not transferred yet.

Checklist

• Distro(s): Any

There are a lot of options to rsync. The ones that I actually use are:

• -a, –archive : archive mode, which is the same as
  • recursive (-r),
  • copy symlinks as symlinks (-l),
  • preserve permissions (-p),
  • preserve modification times (-t),
  • preserve group (-g),
  • preserve owner (-o),
  • preserve device and special files (-D).
• -h, –human-readable : output numbers in human readable format
• -v, –verbose : increase verbosity
• -P : keep partially transferred files (–partial) and show progress during transfer (–progress)
• –delete : delete files on the destination that are no longer on the source.
  • This option is useful if you are actually trying to keep two locations in sync and not trying to do a regular copy or backup.
• -exclude-from=FILE : do not copy files matching the patterns from FILE
  • This option is easier to show an example of instead of a description.

Perform a local file system copy from one folder to another. In this case, I would definitely want to use archive mode to preserve all of my file attributes. I would also normally want to see transfers in human readable format, details (verbose), and see the transfer progress.

rsync -ahvP /home/bill/scripts/ /backups/scripts/

Keep a destination in sync with whatever the source has. In this case, the delete option would be desired, so if a file is deleted from the source, it is also removed from the destination on the next sync. (Which is probably run as a cron job)

rsync -avhP --delete /home/bill/documents/shared/ /media/shared_docs/

Perform a folder sync, but exclude certain folders matching a pattern in a file.

rsync -avhP --delete -exclude-from=/home/bill/rsync_exceptions /home/bill/documents/shared/ /media/shared_docs/

The contents of rsync_exceptions could be:

-/records/
-projects
-*.jpg

Those two patterns mean:

• -/records/ : exclude (-) the records folder in the root of the source tree.
  • That means it would exclude /home/bill/documents/shared/records but would copy something like /home/bill/documents/shared/secretstuff/records.
• -projects : exclude any file, directory, or symlink that matches “projects” anywhere.
  • If the desire was to exclude any folder named “projects anywhere, then it should be ”-projects/“
• -*.jpg : exclude any file that ends in ”.jpg“

See the rsync man page and search for “FILTER RULES” to see more.

You can tunnel rsync in a SSH tunnel for over the internet transfers. It is much easier than it sounds, in fact..it is still a one line command.

rsync -ahvP -e "ssh" /home/bill/scripts/ bill@1.2.3.4:/backups/scripts/

The above opens a SSH tunnel to “bill@1.2.3.4” and sends normal rsync copy operations through the encrypted tunnel to the destination system.