What is Rsync?
Rsync or Remote Sync is a utility for Unix/Linux systems used to copy or synchronize the files and folders remotely or locally. This article contains rsync examples that will help you to understand how to mirror, copy or synchronize the data from one server to another server and even locally in Linux/Unix systems. Rsync was created as a replacement for rcp and scp.
Contents
- What is Rsync?
- Main Advantages
- The most useful rsync options are:
- Installing rsync
- 1) Copying and synchronizing data locally
- 2) Copy and synchronize data to or from a remote server
- 3) Using rsync on ssh
- 4) the progress line of the data transfer
- 5) The options -include and -exclude
- 6) -delete option
- 7) Maximum file size for transfer
- 8) Delete original data after transmission
- 9) Option Dry Run – “idle run”
- 10) Speed limitation I/O
- 11) Full data copying
Main Advantages
The main advantages of rsync are:
-Speed: Initially rsync replicates all content between source and destination (receiver). Further rsync only transmits the changed blocks or bits to the destination, which makes the synchronization really fast.
-Security: rsync includes data encryption for transfer using SSH protocol
-Small Load: rsync uses compression and decompression of data block by block on the transmitting and receiving sides, respectively. Therefore, the rsync bandwidth used is lower compared to other file transfer protocols.
Basic Syntax:-
1 |
rsync options source destination |
The most useful rsync options are:
-v-
detailed mode;
-r-
copy data recursively (but without saving the information about the time of file changes and access rights);
-a-
archiving mode, allows you to copy data recursively with saving symlinks, access rights to files/directories and other information);
-z-
Compression data;
-h-
output data in human-readable format.
Installing rsync
RedHat/CentOS :
1 |
yum -y install rsync |
Debian/Ubuntu :
1 |
apt-get -y install rsync |
1) Copying and synchronizing data locally
The following rsync example will copy or synchronize all files from one directory to another on the same host.
1 |
rsync -avh /root/firstdir /root/seconddir |
view the content of a directory
1 |
ls -l /root/seconddir |
2) Copy and synchronize data to or from a remote server
Copy local files to a remote host:
1 |
rsync -avzh /root/firstdir root@smashinglab:/root/ |
Both hosts must have rsync installed.
Copy from a remote host to a local host.
On the remote host, create the file:
1 |
touch firstdir/file2 |
Copy:
1 |
rsync -avzh root@smashinglab:/root/firstdir /root/ |
3) Using rsync on ssh
To specify rsync
which protocol to use – use the option -e
:
1 |
rsync -avzhv -e ssh /root/firstdir root@smashinglab:/root/ |
4) the progress line of the data transfer
With the help of the option, –progress you can display more information about how the transfer occurs (speed, number of transmitted and remaining bytes, time):
1 |
rsync -avzh -e ssh --progress /root/firstdir root@smashinglab:/root/ |
5) The options -include and -exclude
With the help of -–include and -–exclude you can specify which files and/or directories to include in the transfer, and which ones to exclude.
For example – we will only give file1
:
1 |
rsync -avzhe ssh --include 'file1' --exclude 'file2' /root/firstdir root@smashinglab:/root/ |
6) -delete option
–delete is used if there are files and/or directories in the destination directory that are not in the source, and they must be deleted.
For example, in the destination directory, create a file:
1 |
touch /root/firstdir/file3 |
Whereas in the source directory we will have:
1 |
ls -l /root/firstdir/ |
Now we are synchronizing with the option --delete
:
1 |
rsync -avzhe ssh --delete /root/firstdir root@smashinglab:/root/ |
deleting firstdir/file3 – the file has been deleted:
1 |
ls -l /root/firstdir/ |
7) Maximum file size for transfer
rsync can take an option –max-size, with which you can specify the maximum file size that will be transferred.
For example, create a 100MB file:
1 |
dd of=bigfilefile bs=1 count=0 seek=100M |
confirm content
1 |
ls -hl |
And we will perform the transfer, limiting the file size to 50MB:
1 |
rsync -avzhe ssh --max-size='50M' /root/firstdir root@smashinglab:/root/ |
8) Delete original data after transmission
With the option, –remove-source-files you can delete data after performing a copy or synchronization.
For example – we have a directory:
1 |
ls -l /root/firstdir/ |
We are copying:
1 |
rsync -avzhe ssh --remove-source-files /root/firstdir root@smashinglab:/root/ |
And we check the source directory:
1 |
ls -l /root/firstdir/ |
9) Option Dry Run – “idle run”
With the option, –dry-run you can only verify how the task will be executed, but without real data transfer.
This can be useful, for example, to check the difference between a local and a remote copy of the data.
For example:
1 |
rsync -avzhe ssh --dry-run /root/firstdir root@smashinglab:/root/ |
And check the remote directory:
1 |
ls -l /root/firstdir/ |
10) Speed limitation I/O
To limit the transfer spoofing – you can use the option –bwlimit that sets the read speed limit from the disk (and, accordingly, the transfer) in kilobytes/second.
For example, create a file:
1 |
dd of=bigfilefile bs=1 count=0 seek=10M |
And run the transfer:
1 |
rsync --bwlimit=100 --progress bigfilefile root@smashinglab:/root/ |
11) Full data copying
By default, rsync only copies changed data blocks.
To make a full copy – use the option -W:
1 |
rsync --progress bigfilefile root@smashinglab:/root/ |
Note the difference in the field sent between calls with and without an option -W.