Filesystem Syncronization with Lsyncd
Lsyncd monitors a local filesystem for changes and mirrors those changes to a filesystem on a remote server. The advantage of lsyncd over rsync is the former’s ability to detect filesystem changes without having to re-scan the entire source and target directory structures. This makes lsyncd faster and more efficient. And you can still use rsync with lsyncd to handle the actual data transfer.
Here’s a quick example of installing and using lsyncd. In this example the source_server and target_server are both RHEL 6.4 64-bit VMs running on ESX host. To make things simple for myself, I was using a root account. However, this is not recommended, unless the filesystems you are synchronizing require root access.
Configure key-based SSH authentication
If using the root account, you must allow direct root SSH on the target server.
Edit target_server:/etc/ssh/sshd_config and set: PermitRootLogin yes Save the file and restart SSH: service sshd restart
Generate SSH key-pair for your account and send the public key to the target_server.
on the source_server: ssh-keygen -t rsa ssh-copy-id target_server test ssh connection: ssh -qt target_server "hostname"
Install Lua and Lsyncd
yum -y install lua* yum -y install lsyncd
NOTE: the version of lsyncd currently (Nov 2014) available from RHEL/CentOS repos is 2.1.4. Version 2.1.5 brings back a potentially very valuable functionality of initiating file sync after file attribute change (i.e. file ownership, permissions, etc). If that’s something you may need, here’s how to get lsyncd 2.1.5
cd /tmp wget http://mirrors.htbindustries.org/CentOS/6/x86_64/lsyncd-2.1.5-1.el6.x86_64.rpm yum -y install /tmp/lsyncd-2.1.5-1.el6.x86_64.rpm
Before continuing, this would be a good time to do a quick test.
on source_server generate some folders and file for testing: mkdir /tmp/test1 for i in `seq -w 1 10` ; do mkdir /tmp/test1/dir${i} ; for j in `seq -w 1 10` ; do touch /tmp/test1/dir${i}/file${j} ; done ; done on the target_server create the base folder: mkdir /tmp/test1 from source_server run the sync: lsyncd -rsyncssh /tmp/test1/ root@target_server /tmp/test1/
The next step is to create a configuration file and run lsyncd as a service. I would recommend removing the default configuration file – it’s causes problems with the /etc/init.d/lsyncd start-up script:
rm /etc/sysconfig/lsyncd
Here’s an example configuration file (/etc/lsyncd.conf) for the example above:
settings { logfile = "/var/log/lsyncd/lsyncd.log", statusFile = "/var/log/lsyncd/lsyncd.status", nodaemon = false, maxDelays = 3, onStartup = true, onAttrib = true, onCreate = true, onDelete = true, onModify = true, onMove = true, delete = true, } sync { default.rsyncssh, source = "/tmp/test1", host = "target_server", targetdir = "/tmp/test1", delay=3, rsync = { owner = true, perms = true, checksum = true, compress = false, acls = true, verbose = true, _extra = {"-aKx"}, } }
The final steps are to create the log directory and activate the lsyncd service.
mkdir -p /var/log/lsyncd chkconfig lsyncd on service lsyncd start service lsyncd status
Lsyncd can be a bit fiddly to configure, but it is well worth the effort. Hopefully, these instructions will make it a bit easier for you.