Thursday, January 23, 2014

Script for unmounting NFS mounts

I encountered an issue when shutting down or rebooting system with mounted NFS exports. It seems that systemd shuts down network service before all NFS mounts are unmounted and the system will hang, waiting for these mount points to be unmounted.
To prevent this, I've composed short script, which handles NFS unmounting before shutdown.


In general, this script has to be called before poweroff or reboot. It needs to be executed with root rights, so either you can execute it as root or you can add entry to sudoers.

Short description:
First step is to get all nfs mounted filesystems from mount command. Then it tries to unmount them, if it fails
fuser command is used to get PIDs of processes holding this mount point and ask then to end, following with kill if necessary. In the end it stops php-fpm service.

#!/bin/bash
################
# krisko 2014
# Simple script to umount all nfs mounts and stop php-fpm
# this should be used before system shutdown/reboot, because of hanging nfs mounts
#
# it needs sudoers entry with NOPASSWD for user krisko (i3 is running under krisko)
################

# set default values
EXIT=0

# end process and umount
endproc(){
 PID=$1
 MOUNT=$2
 NAME=$(cat /proc/$PID/status | grep "Name:" | awk '{print $2}')

 # send SIGTERM and wait for proccess to end
 echo INFO: asking to quit $NAME, PID: $PID
 kill -15 $PID && sleep 1
 
 # if the process is still running, send SIGKILL
 if [ -d /proc/$PID ]; then
  echo INFO: killing $NAME, PID: $PID
  kill -9 $PID && sleep 0.5
  # if it is still here, show error and set exit status to 1
  if [ -d /proc/$PID ]; then
            echo ERROR: $NAME, PID: $PID still running...
   EXIT=1
  else
   echo -n "INFO: umounting \"$MOUNT\"... "
   if [ $(umount $MOUNT 1>/dev/null 2>&1; echo $?) -eq 0 ]; then
    echo DONE
   else
    echo FAILED
    EXIT=1
   fi
  fi
 else
  echo -n "INFO: umounting \"$MOUNT\"... "
  if [ $(umount $MOUNT 1>/dev/null 2>&1; echo $?) -eq 0 ]; then
   echo DONE
  else
   echo FAILED
   EXIT=1
  fi
 fi

}

# do the umounting...
nfsumount(){
 # get process numbers holding for mountpoint
 MOUNT=$1
 PIDs=$(fuser $MOUNT 2>/dev/null | cut -d: -f2)

 # if fuser does not find blocking process, try lsof
 if [ -z "$PIDs" ]; then
  echo INFO: no blocking process found, trying lsof...
  PIDs=$(lsof $MOUNT 2>/dev/null | awk '{print $2}' | grep -v PID)
 fi

 # if there is some process blocking umount, end it
 if [ ! -z "$PIDs" ]; then
  for PID in $PIDs; do 
   endproc $PID $MOUNT;
  done
 else
  echo ERROR: no blocking process found...
  EXIT=1
 fi
}

phpstop(){
 #stop php-fpm
 if [ $(pgrep -c php-fpm) -gt 0 ]; then
     echo INFO: stopping php-fpm
     systemctl stop php-fpm
 else
     echo INFO: php-fpm not running
 fi
}

# # # # # # #
# BEGINNING #
# # # # # # #

#we need to have privileges
if [ "$UID" != 0 ]; then
 exec sudo $0
fi

# get all nfs mounts
MOUNTS=$(\mount | grep "type nfs" | awk '{print $3}')

echo HINT: Use -f option if NFS server is gone...
if [ x$1 == "x-f" ]; then
 if [ ! -z "$MOUNTS" ]; then
  echo -n "INFO: umount -f -l $MOUNTS... "
  umount -f -l $MOUNTS
  phpstop
  echo DONE
  exit 0
 else
  echo INFO: no nfs mounts
  phpstop
  exit 0
 fi
fi

if [ ! -z "$MOUNTS" ]; then
 for MOUNT in $MOUNTS; do
  echo -n "INFO: umounting \"$MOUNT\"... "
  if [ $(umount $MOUNT 1>/dev/null 2>&1; echo $?) -eq 0 ]; then
   echo DONE
  else
   echo ENDING PROCESSES...
   nfsumount $MOUNT
  fi
 done
else
    echo INFO: no nfs mounts
fi

#stop php-fpm
phpstop

# exit
if [ $EXIT -eq 1 ]; then
 exit 1
else
 exit 0
fi

##################
# ChangeLog/TODO #
##################
# 1.0.0 20140316   Added changelog/todo section
##################

1 comment:

  1. Krisko thanks script works well here on 5.15.73-1-lts. Had 4 of 6 nfs shares mounted and all umounted fine and quick. Non expert or professional here - just personal use.

    ReplyDelete