Networking

Unix and Linux network configuration. Multiple network interfaces. Bridged NICs. High-availability network configurations.

Applications

Reviews of latest Unix and Linux software. Helpful tips for application support admins. Automating application support.

Data

Disk partitioning, filesystems, directories, and files. Volume management, logical volumes, HA filesystems. Backups and disaster recovery.

Monitoring

Distributed server monitoring. Server performance and capacity planning. Monitoring applications, network status and user activity.

Commands & Shells

Cool Unix shell commands and options. Command-line tools and application. Things every Unix sysadmin needs to know.

Home » Backups, Commands & Shells

Using rsync to copy files

Submitted by on November 21, 2005 – 4:00 pm

In the following example we need to make sure that /export/home on server host2 looks exactly like /export/home on host1. Thus, our SOURCE is host1:/export/home and our TARGET is host2:/export/home

There are two ways of using rsync to copy files. One method is to install rsync on both servers and to configure both rsyncs to communicate with each other. The other method it to NFS-mount the SOURCE directory on the TARGET server or to mount the TARGET directory on the SOURCE server. Then you run rsync on the server with the NFS mount. It’s a good idea to mount the remote filesystem and to run rsync on the faster of the two servers.

The following example is applicable to Solaris

Step 1: Share /export/home from host1 to host2

On host1 vi /etc/dfs/dfstab and add the following line:

share -F nfs -o root=host2,rw=host2 /export/home

Save the dfstab file and execute shareall

Step 2: Mount host1:/export/home on host2

On host2 mkdir /host1_export_home

vi /etc/vfstab and add the following line:

host1:/export/home – /host1_export_home nfs 2 yes –

Save the vfstab file and execute mountall

Now you have /export/home from host1 mounted under /host1_export_home on host2.

Step 3: Create rsync script on host2

Create script called /usr/local/bin/dirsync.ksh (or whatever else you want to call it):

#!/bin/ksh

date=$(date +'%Y-%m-%d')

RECEPIENTS="your_email@domain.com"

SOURCE="/host1_export/home/"
TARGET="/export/home/"

if [ ! -d $SOURCE ] || [ ! -d $TARGET ]
then
        echo "Souce or destination not found! Exiting..." | /usr/bin/mailx -s "`hostname` sync FAILED" $RECEPIENTS
        exit 1
fi

/usr/local/bin/rsync -a -v -u --delete "$SOURCE" "$TARGET"

(( MINS = SECONDS / 60 ))

echo "host1 --> `hostname` synchronization complete n

        Date: `date` n
        Total copy time: $MINS minutes." | /usr/bin/mailx -s "`hostname` sync complete for $date" $RECEPIENTS

Whatever stupid mistakes you plan on making, NEVER confuse SOURCE and TARGET parameters! And always make sure to backup all data before running rsync for the first time.

Add the rsync script to cron if you need it to run regularly:

15 22 * * * /usr/local/bin/dirsync.ksh > /dev/null 2>&1

Here’s a more elaborate version of the backup script above:

#!/bin/sh

#---------------------------------------------------------
# Configure script
#---------------------------------------------------------

date=$(date +‘%Y-%m-%d’)

LOG="/var/log/backup.log"

if [ `wc -l "$LOG" | awk '{print $1}'` -gt 1000 ]
then
	tail -500 "$LOG" > /tmp/backup.log.tmp
	mv /tmp/backup.log.tmp "$LOG"
fi

ENTRY=0
log() {
	echo "$MSG" >> "$LOG"
	(( ENTRY = ENTRY + 1 ))
}

MSG=$(echo "----------------------------------------"); log
MSG=$(echo "`date`	`hostname`	Backup started"); log

SCRIPT="/var/adm/bin/backup.sh"
BACKUPSDIR="/WD160GB_03/backups"

if [ ! -x "$SCRIPT" ] || [ ! -d "$BACKUPSDIR" ]
then
	MSG=$(echo "Script $SCRIPT or directory $BACKUPSDIR not found. Exiting..."); log
	echo "$MSG"
	exit 1
fi

RECEPIENTS="your_email@domain.com"
SUBJECT="Backups on `hostname`"

#---------------------------------------------------------
# Backups list
#---------------------------------------------------------

#BACKUPDEF	/WD120GB_01/htdocs/
#BACKUPDEF	/WD120GB_02/htdocs/
#BACKUPDEF	/var/lib/mysql/
#BACKUPDEF	/var/adm/bin/
#BACKUPDEF	/etc/

cat "$SCRIPT" | grep BACKUPDEF | grep -v grep | while read LINE
do
	STATUS=0
	SOURCE=$(echo "$LINE" | awk '{print $2}')
	DIR=$(echo "$SOURCE" | sed 's+^/++g' | sed 's+/$++g' | sed 's+/+_+g')
	TARGET=$(echo "${BACKUPSDIR}/${DIR}/")

	MSG=$(echo "Backing up $SOURCE to $TARGET"); log

	if [ ! -d "$SOURCE" ]
	then
        	MSG=$(echo "Source directory $SOURCE not found!"); log
		STATUS=1
	elif [ ! -d "$TARGET" ]
	then
		mkdir "$TARGET"
		if [ $? -ne 0 ]
		then
			MSG=$(echo "Failed to create target directory $TARGET"); log
			STATUS=1
		fi
	fi

	if [ $STATUS -eq 0 ]
	then
		/usr/bin/rsync -a -v -u –delete "$SOURCE" "$TARGET"
	fi
done

(( MINS = SECONDS / 60 ))

MSG=$(echo "Backups completed in $MINS minutes"); log

tail -$ENTRY "$LOG" | mailx -s "$SUBJECT" "$RECEPIENTS"
Print Friendly, PDF & Email

Leave a Reply