RSync Examples

1.  Copying Local Data to a folder on Remote Machine

rsync -avz /MyLocalFolder/ root@myremotemachine:/MyBackUpFolder/

The above command will transfer all files in MyLocalFolder to MyBackUpFolder on a Remote machine named myremotemachine using user name as root. 

2.  If you want to Update the Remote machine folder so that any files that are moved from Local Folder are also deleted from the Remote machine then simply use --del. Here is an example:

rsync -avz --del /MyLocalFolder/ root@myremotemachine:/MyBackUpFolder/

3.  Copy not just the files under a directory but also the parent directory name should be copied and created on the server... To achieve this just drop the / after MyLocalFolder in example 1. Basically the command would look like:

rsync -avz /MyLocalFolder root@myremotemachine:/MyBackUpFolder/

The command will first create a folder named MyLocalFolder within MyBackUpFolder on remote machine and then transfer files.

4.  To Specify multiple directories that need to be transferred use it like:

rsync -avz /MyLocalFolder1 /MyLocalFolder2 root@myremotemachine:/MyBackUpFolder/

5.  To Transfer files from Remote Server to Local machine just change the order of Source and Destinations... See below:

rsync -avz root@myremotemachine:/MyBackUpFolder/ /MyLocalFolder 

Note:

r = recursive - means it copies directories and sub directories
v = verbose - means that it prints on the screen what is being copied
a = archive - means that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer
z = compress - compress file data

RSync at a Glance

What RSync Offers:


1. Differential Copy of Files: Only changed pieces of files are transferred and not complete files.
2. Compression: Before transfer the files are compressed to save transfer time
3. Secure Transfer: ssh is used. Even rsh can be used.


It is an excellent choice as a backup and mirroring tool.


Here is a very High level view of Rsync... It shows that once an rsync server has been setup any number of clients who has rsync installed can synchronize to/from this server.

Installing RSync Server:


1. Download RSync Server Binaries from here.
   
or

Debian or Ubuntu:
# apt-get install rsync
or
$ sudo apt-get install rsync

Red Hat Enterprise Linux (RHEL)
# up2date rsync

CentOS/Fedora Core Linux
# yum install rsync

2. Setup the RSync Configuration File. We will see shortly how the configuration file looks like and what changes we need to make.
3. Run RSync in Daemon mode.

 Here is the command to run rsync in Daemon Mode:

 rsync -v -v --no-detach --daemon


Dissecting the Configuration File:


So here is how the configuration file looks like.. It is located in /etc/rsyncd.conf... In not you should create one.


#/etc/rsyncd.conf

motd file = /etc/rsyncd.motd
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock

[my_path]
   path = /rsync_files_here
   comment = My Rsync Server
   uid = nobody
   gid = nobody
   read only = no
   list = yes
   auth users = username
   secrets file = /etc/rsyncd.scrt

   hosts allow = *.xyz.com

   hosts deny = * 
   list = false 

motd file: Is actually a Message of the day. Anything contained in this file will be displayed when clients connect to this machine. 

log file: Specifies Log file where diagnostic and runtime messages are sent.

pid file: Contains the Process ID number of the running rsync daemon. 

lock file: Ensures things run smoothly. 

All of the above 4 are global to the rsync daemon. 

Let's have a look at other options available: 

• path - Actual filesystem path where the files are rsync'ed from and/or to.
• comment - Any useful comment you wanna put.
• auth users - you really should put this in to restrict access to only a pre-defined user that you specify in the following secrets file - does not have to be a valid system user.
• secrets file - the file containing plaintext key/value pairs of usernames and passwords.
• hosts allow -  IPs or hostnames that you wish to specifically allow or deny!
• hosts deny -  IPs or hostnames that you wish to specifically allow or deny!

An Example of executing RSync Client to Transfer files:

rsync --verbose --progress --stats --compress --recursive --times --perms --links --delete /mysourcedir/* myservername:/mydestinationdir

--verbose : Tells that you want to see the progress display of what is being executed
--stats : Shows you the complete status after execution is finished
--recursive : Iterates through the directory and copies all files from/to within the directories
--perms :  Preserve permissions
--times: Preserver timestamps on the file
--links : Copy symbolic links with links
/mysourcedir/* : To copy all files from this source directory.
myservername:/mydestinationdir : Destination directory on the server where files will be transferred.

rsync common options

• --delete : delete files that don't exist on sender (system)
• -v : Verbose (try -vv for more detailed information)
• -e "ssh options" : specify the ssh as remote shell
• -a : archive mode
• -r : recurse into directories
• -z : compress file data

So that was the Basic Rsync Stuff... We will dig deeper into using RSync in upcoming articles.

Symbolic Links & Hard Links

Hard Link -> Where a file has two names and both the names points directly to the blocks on the disc that contain the data.

Suppose if a file file1.txt already exist and you want to give it another name file2.txt then use command :

ln file1.txt file2.txt

The above command creates a hard link.

The only way you should be able to distinguish whether there is a link or its a normal file is by doing ls -l. The second column tells you how many links are there for the particular block of data on disc.

To know what's linked to it use ls -i which will give you information of what is linked to what.

Symbolic or Soft Link -> Where a file has one main name and other names as well by which this can be referred. It is relatively slow than a hard link but more flexible.

You can setup symbolic links in the same way you setup Hard links.. the only difference is that you use a -s parameters to specific whether this is a symbolic link or not.

For e.g., ln -s file1.txt file2.txt

The above creates a symbolic link with file1.txt as the main file name. To know if it is a symbolic link or not do ls -l and if the first column displays an "l" you can be sure that it is a symbolic link.

Diagramatically Hard and Symbolic links can be explained as:-