- Exclude Multiple Directories Rsync
- Rsync Directory Sync
- Rsync Synchronize Directories One Way Transfer
Installing Rsync. Rsync comes pre-installed in Ubuntu 20.04 LTS. However, if it is accidentally. In order to demonstrate how rsync works in a simple way, we'll construct two directories of files on the same machine, and use rsync to synchronize them. Here's an outline of the two directories. We can imagine that in the future, one directory will be on a desktop machine and the other will be on a laptop or netbook. Rsync -avv left/ right/ will not sync anything in right to left. It will, as @atbg says, only sync left to right. Rsync is not a bi-directional syncer.
- Rsync is a useful Linux command-line tool that syncs and copies files and directories. You can use the tool to synchronize data locally between directories and drives, or between two remote machines. The basic rsync commands in Linux sync everything from a location you specify. In many backup scenarios, you may want to exclude specific files, directories, or file types.
- Rsync (Remote Sync) is a synchronization tool for copying files and directories in a system or between systems. The single biggest advantage of rsync is that it copies only the changed files and thus reduces the CPU consumption and saves the bandwidth and time while copying files. Advantages of rsync over scp.
Now I have backups working between my Synology Diskstation NAS and a storage account in Microsoft Azure (with over half a TB of photos so far backed up in the cloud), the next stage is to consolidate some more images into the folder that the backup works from.
I don’t want to remove them from their source (which in this case is the copy of my OneDrive data on my home drive) but I do want to archive all of the iPhone images I have there to the master photos folder so they are included in the backup.
Reading around the Synology forums suggests that this is not as straightforward as one might think. It appears there’s no easy way to synchronise two folders on the same NAS within the DSM software; but then I stumbled across Zarino Zappia (@zarino)’s post about a Synology-flavoured rsync backup script.
By following Zarino’s advice and using
ssh
to connect to the box as admin, I was able to achieve what I wanted with the following command:rsync --itemize-changes --archive --progress --verbose --inplace --exclude '*@SynoResource' --exclude '@eaDir' --exclude '*.vsmeta' --exclude '.DS_Store' --exclude 'Thumbs.db' /volume1/homes/mark/OneDrive/iPhone Photos/ /volume1/photos/Digital Photos (Master)/Mark's iPhone/
(BTW, right click is the way to paste text to the command line in PuTTY!)
Some people on the Synology forums had suggested synchronising via another computer on the network would be fast! That sounds strange to me – logically a copy will always be faster on a single device with no network in between. For reference, it took about 20 minutes to
rsync
32GB of images/videos on my on my DS916+.Incidentally, the Error 23 in the screen shots was actually a typo in my command (missing space before one of the –exclude options). I re-ran with –dry-run to see which files were not transferred…
The next step will be to script this and get it running as a scheduled task but that can wait for another day…
How To Synchronize Directories with Rsync
Today I’d like to show you the basic usage of rsync – a wonderful, old and reliable tool for incremental data transfers and synchronization of local directories or even data between different Unix systems.
rsync is quite a complicated command, so don’t expect this first post to explain everything and cover every possibility. Like I said, this is only the beginning.
What is rsync?
Conference call transcription services. rsync (stands for remote synchronization) is an open source tool for data transfers between Unix systems.
In simplest form, it’s just a Unix command you run locally to synchronize two directories. But the real power of rsync is when you need to synchronize directories between remote systems. rsync relies on ssh protocol for transferring the data between Unix systems, but earlier versions used rsh. Advanced deployments imply using rsync server in addition to simply running the command – this is basically the same command but running in a stand-by daemon mode.
rsync can easily be found or installed in any modern Unix-like OS, but it’s always best to check the official website for latest developments around this tool: rsync website.
What does rsync do?
rsync synchronizes directories – makes one directory look (contain the same files and subdirectories) exactly like another one. rsync works by getting a list of files in your source and destination directories, comparing them as per specified criteria (file size, creation/modification date or checksum) and then making the destination directory reflect all the changes which happened to the source since the last synchronization session.
Basic rsync usage
Just to show you how it works, I’m going to create two directories with a few files in them. /tmp/dir1 in my examples will be a source directory (original dataset), while /tmp/dir2 will be a destination directory – to be made the same as /tmp/dir1 as the result of running rsync.
Exclude Multiple Directories Rsync
So that’s how I set up directories and files:
That’s how our directories and files look now, so dir2 contains a copy of file1:
Now it’s time to run your first ever rsync. There’s two ways of specifying options for the command, a full option name starting with — and usually having a meaningful name, or a short option name – starting with – and having short meaningless names (usually one-letter ones) for each option.
The last two parameters in an rsync command line should be the source and the destination directories.
In this example below, we’re using the following options:
-avz – a for archive mode (preserve all the attributes of each file and directory – ownership, permissions, etc), v for verbose mode (report a list of files processed by rsync) and z for data compression to speed transfers up.
–stats – this option shows a summary at the end of rsync’ing process to highlight the main stats of the job
Stats are self-explanatory, and you can see that although there were 4 files found in source directory /tmp/dir1, only 2 files were transferred into /tmp/dir2 because /tmp/dir2 already had one of the files.
Rsync Directory Sync
That’s all I have for you today, in the next post on rsync I’ll show you some more advanced uses of this command. For the time being, read man rsync or even rsync –help on your system to get an idea of how really powerful this tool is.
Rsync Synchronize Directories One Way Transfer
Until next time – good luck with your Unix experiments!
See also
- How to compare directories in Unix
- List subdirectories of a Unix directory
- Finding large files and directories