Tuesday, November 19, 2013

Sync script

The script in this post handles synchronization of files and folders. You can specify source and destination file. It's using timestamp of file and md5 sum to check if the file was modified.
I've created this script because I needed to sync several files/folders with owncloud folder.

How does it work:
Firs you have to specify which file or folder you want to sync, and where should it be synced (this is specified in variable LIST). Then it handles paths if necessary, creates folder structure and list of files in the specified folder, this is handled in function syncfind(). The main function sync() copies file (from source or destination) if it does not exist or if it does, it's comparing timestamps of files and md5 sums and then it copies latest file.

This is how it looks:
#!/bin/bash

# KrisKo 2013
#
# synchronization script (synchronizes files and folders)
#
# USAGE:
# in LIST variable fill:
# /my/file.txt:/destination/file.txt
# /other/folder:/destination/folder

LIST="
/home/krisko/linux/git howto.txt:/home/krisko/ownCloud/Notes/git howto.txt
/home/krisko/linux/git_mini_howto:/home/krisko/ownCloud/Notes/git_mini_howto
/home/krisko/linux/ArchLinux Installation:/home/krisko/ownCloud/Notes/ArchLinux Installation
/home/krisko/folder/rPI documentation:/home/krisko/ownCloud/rPI documentation
"

sync(){
 SRC=$1
 DST=$2

 if [ ! -f "$SRC" ] && [ -f "$DST" ]; then
  echo INFO: File \"$SRC\" doesn\'t exist. Copying \"$DST\".
  cp "$DST" "$SRC"
  return 0
 elif [ ! -f "$DST" ] && [ -f "$SRC" ]; then
  echo INFO: File \"$DST\" doesn\'t exist. Copying \"$SRC\".
  cp "$SRC" "$DST"
  return 0
 elif [ ! -f "$SRC" ] && [ ! -f "$DST" ];then
  echo WARNING: No file was found \"$SRC\" or \"$DST\"
  return 0
 fi

 if [ "$SRC" -nt "$DST" ]; then
  #compare files, if differ sync with newest version
  SUMO=`md5sum "$SRC" | cut -d " " -f 1`
  SUMS=`md5sum "$DST" | cut -d " " -f 1`
  if [ $SUMO != $SUMS ]; then
   echo INFO: Copying \"$SRC\" to \"$DST\"
   cp "$SRC" "$DST"
  fi
 elif [ "$DST" -nt "$SRC" ]; then
  #compare files, if differ sync with newest version
  SUMO=`md5sum "$SRC" | cut -d " " -f 1`
  SUMS=`md5sum "$DST" | cut -d " " -f 1`
  if [ $SUMO != $SUMS ]; then
   echo Copying \"$DST\" to \"$SRC\"
   cp "$DST" "$SRC"
  fi
 fi
}

syncfind(){
 SRCf=$1
 DSTf=$2

 #create folder structure in DSTf
 SRCfind=$(echo -e "$(cd $SRCf; find . -mindepth 1 -type d | sed -e 's# ./#\n./#g' | sort | uniq)")
 for SRCffolder in $SRCfind; do
  if [ ! -d "$DSTf/$SRCffolder" ]; then
   echo INFO: mkdir \"$DSTf/$SRCffolder\"
   mkdir -p "$DSTf/$SRCffolder"
  fi
 done

 #create folder structure in SRCf
 DSTfind=$(echo -e "$(cd $DSTf; find . -mindepth 1 -type d | sed -e 's# ./#\n./#g' | sort | uniq)")
 for DSTffolder in $DSTfind; do
  if [ ! -d "$SRCf/$DSTffolder" ]; then
   echo INFO: mkdir \"$SRCf/$DSTffolder\"
   mkdir -p "$SRCf/$DSTffolder"
  fi
 done

 #get files to sync
 # SAVEIFS=$IFS
 # IFS=$(echo -en "\n\b")
 ARR=$(echo -e "$(cd "$SRCf"; find . -mindepth 1 -type f) $(cd "$DSTf"; find . -mindepth 1 -type f)" | sed -e 's# ./#\n./#g' | sort | uniq)
 for file in $ARR; do 
  #echo FILE: $file
  sync "$SRCf/$file" "$DSTf/$file"
  #echo sync "$SRCf/$file" "$DSTf/$file"
 done
}

#set IFS to handle spaces in names
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

for file in $LIST; do
 # echo PRINT: $file
 SRC=$(echo $file | cut -d: -f1)
 DST=$(echo $file | cut -d: -f2)

 #test for file, create paths if necessary and sync
 if [ -f $SRC ]; then
  if [ ! -d ${DST%/*} ]; then
   echo INFO: Creating path ${DST%/*}
   mkdir -p "${DST%/*}" || echo ERROR: could not create \"${DST%/*}\"
  fi
  sync "$SRC" "$DST"
 elif [ -f $DST ]; then
  echo INFO: Creating path ${SRC%/*}
  mkdir -p "${SRC%/*}" || echo ERROR: could not create \"${SRC%/*}\"
  sync "$DST" "$SRC"
 elif [ -d $SRC ]; then
  if [ ! -d $DST ]; then
   echo INFO: Creating path "$DST"
   mkdir -p "$DST" || echo ERROR: could not create \"$DST\"
  fi
  syncfind "$SRC" "$DST"
 elif [ -d $DST ]; then
  echo INFO: Creating path "$SRC"
  mkdir -p "$SRC" || echo ERROR: could not create \"$SRC\"
  syncfind "$SRC" "$DST"
 else
  echo WARNING: \"$(basename "$SRC")\" not found in "$SRC" or "$DST"
 fi

done

IFS=$SAVEIFS

exit 0

No comments:

Post a Comment