Adding Counter in shell script

I have below code in my shell script which will keep on sleeping if it doesn't finds any file. And it sleeps for half an hour but currently I don't have any counter like only execute the below code 20 times and then exit the program if the files are still are not there (means don't do anything after 20 checks and exit the full script).

What's the best way to do this problem? So that I am also aware by looking at the emails that it has tried 20 times.

Hope I am clear enough.

while true; do
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       break
  else
       echo "Sleeping for half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
       sleep 1800
  fi
done

Here's how you might implement a counter:

counter=0
while true; do
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       exit 0
  elif [[ "$counter" -gt 20 ]]; then
       echo "Counter: $counter times reached; Exiting loop!"
       exit 1
  else
       counter=$((counter+1))
       echo "Counter: $counter time(s); Sleeping for another half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
       sleep 1800
  fi
done

Some Explanations:

• counter=$((counter+1)) - this is how you can increment a counter. The $ for counter is optional inside the double parentheses in this case.
• elif [[ "$counter" -gt 20 ]]; then - this checks whether $counter is not greater than 20. If so, it outputs the appropriate message and breaks out of your while loop.

Try this:

counter=0
while true; do
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       break
  elif [[ "$counter" -gt 20 ]]; then
       echo "Counter limit reached, exit script."
       exit 1
  else
       let counter++
       echo "Sleeping for another half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
       sleep 1800
  fi
done

Explanation

• break - if files are present, it will break and allow the script to process the files.
• [[ "$counter" -gt 20 ]] - if the counter variable is greater than 20, the script will exit.
• let counter++ - increments the counter by 1 at each pass.

You may do this with a for loop instead of a while:

max_loop=20
for ((count = 0; count < max_loop; count++)); do
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       break
  else
       echo "Sleeping for half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
       sleep 1800
  fi
done

if [ "$count" -eq "$max_loop" ]; then
  echo "Maximum number of trials reached" >&2
  exit 1
fi

#!/usr/bin/env bash
counter=0
while [[ "$counter" -lt 20 ]] 
do
  if [[ $counter -gt 0 ]] ; then
    echo "Counter: $counter time(s); Sleeping for another half an hour" | mailx -s "Time to Sleep Now"  -r [email protected] [email protected]
    sleep 1800  
  fi
  if /home/hadoop/latest/bin/hadoop fs -ls /apps/hdtech/bds/quality-rt/dt=$DATE_YEST_FORMAT2 ; then
       echo "Files Present" | mailx -s "File Present"  -r [email protected] [email protected]
       exit 0
  fi
  : $((counter++))
done

echo "Counter: $counter times reached; Exiting loop!"
exit 1

Here's another take on this. I've added this bit of code to illustrate these points:

1. The while can take your counter as the condition - this removes the need to check the variable value in the loop.
2. Since the if branches cause the loop to end anyway ( via break or exit ) there is no need for the final else.
3. I've provided a different syntax for increasing the counter. To be honest, I've never seen koola's version (let counter++) but I like it a lot (it's simpler to me).
4. I'm not 100% sure that some of the while loops provide as answers actually loop 20 times. Rather, I think some of them loop 21 times.
5. The solutions presented actually email the admin an extra time - there is no need to tell the admin that you are sleeping for a half an hour if you are going to be exiting anyway. I've added an additional if at the beginning of the while loop to avoid the extra email (and sleep).

Here's the output for a run that fails to find files 20 times:

Counter: 1 time(s); Sleeping for another half an hour
Counter: 2 time(s); Sleeping for another half an hour
...
Counter: 19 time(s); Sleeping for another half an hour
Counter: 20 times reached; Exiting loop!

The short of if is that we check for files 20 times, but only sleep 19 times.