Issues while Executing ssh command from within Bash Script

I was trying to execute ssh command from bash script which will: 

1. Connect to the Host using ssh. 

2.  Execute the date command on it.

3.  Print the output of step 2 on screen. 

So basically I have several servers running on Amazon's EC2 and I want to actually see what's the date/time on all those servers for the reasons that I've seen some time drift on Amazon Instances. I have set up password less entry to remote host and also added host key to remote host.

Here is the code that I'm executing: 

inputfile="myfile.csv"

OLDIFS="$IFS"

IFS=","

while read IP; do

        echo "Instance $IP Date/Time is:" `ssh ubuntu@$IP date`

done < "$inputfile"

The problem is: When I execute this script it returns the date time for a few servers like sometimes 1, sometimes 2 or 4 and so on and then exits. 

After having spent hours on this problem finally I figured that ssh was consuming stdin. After adding a -n option to ssh all worked great. 

So here is how I should have actually used it: 

        echo "Instance $IP Date/Time is:" `ssh -n ubuntu@$IP date`

Easy.. Isn't It?

Amazon EC2 Instances Times differ Considerably

So, I'm running uBuntu instnaces on Amazon's EC2 for a time critical application. When these were first started they had the same time... But after few days there has been more than 5 minutes time difference between both the instances.


This happened because by default these instances have not been configured to update time from NTP Servers. So to get around this problem I have to either manually run the command: 

sudo ntpdate ntp.ubuntu.com

which will actually sync up the instance time with a network Time Server. 

To answer from where did the time difference came - Actually when rebooting the instance the time it took to reboot it each time got accumulated and hence resulted into a total 5 min difference in my case. So each time you reboot your instance time would get back by that amount. 

To permanently fix this problem we would need to sync up times on every restart.

How to remove last 3 character of a string in Bash

The below example will: 


1.  Remove/Trim last 3 characters of a string variable.
2.  It does it in one go... 


VAR = "ap-southeast-1a"
VAR=${VAR%%???}


echo $VAR


This will output: 
ap-southeast


Simple... Isn't It?

Find out how many concurrent connections Amazon RDS can support

So you have a running RDS (mysql) instance for which you want to find out:


1. Number of Concurrent connections this RDS can support. 


    From command line login to the RDS DB. 
    Now run the command: 


    show variables like '%conn%';


      This command will list down the max concurrent sessions that can be made to this RDS DB.
      This is quite useful when you want to see and tune your applications connection pools for DB. 


2. List down the current concurrent connections. 


    Try: 


    show processlist;


    This lists down various processes that are connected to this RDS currently. 

How to trim leading whitespaces from a variable in Bash

Suppose your variable is PDNS where 


PDNS=" test " then to remove leading whitespaces from it try:


PDNS="${PDNS#"${PDNS%%[![:space:]]*}"}"
echo $PDNS


This will trim the leading whitespaces.


To remove trailing whitespaces from this variable try: 


PDNS="${PDNS%"${PDNS##*[![:space:]]}"}"

Cron Jobs and common issues with cron jobs

Cron jobs are commands that are run at a specified interval and hence they are difficult to troubleshoot. Cron searches its spool area (/var/spool/cron/crontabs) for crontab files (which are named after accounts in /etc/passwd); crontabs found are loaded into memory. Note that crontabs in this directory should not be accessed directly - the crontab command should be used to access and update them. The cron daemon runs every minute and checks for stored crontabs and commands to see if there are any that need to be run in that minute. If it finds one it is run. 


Common issues: 


1. If you are running a script via cron job which uses relative paths then your script won't be able to process those relative paths. You need to use complete path's. For e.g., if your script is trying to open a file abc.txt you should modify your script so that the file path is complete like: /home/myuser/abc.txt otherwise your script will not be able to find this file. 


2. Strict Permissions - Make sure all scripts/files/folders are given execute permissions. Use command chmod +x <filename> to give write permissions. 


3. To troubleshoot cron job you can modify your crontab file to include MAILTO=email@email.com with your email address. When cron job is executed the result/errors etc are sent to this email id and hence they are useful in troubleshooting. 


4. When cron job is run from the users crontab it is executed as that user. It does not however source any files in the users home directory like their .cshrc or .bashrc or any other file. If you need cron to source (read) any file that your script will need you should do it from the script cron is calling. Setting paths, sourcing files, setting environment variables, etc. need to be done from your script itself. 


5. If the users account has a crontab but no useable shell in /etc/passwd then the cronjob will not run. You will have to give the account a shell for the crontab to run.


6. If your cronjobs are not running check if the cron deamon is running. Then remember to check /etc/cron.allow and /etc/cron.deny files. If they exist then the user you want to be able to run jobs must be in /etc/cron.allow. You also might want to check if the /etc/security/access.conf file exists. You might need to add your user in there.


7. Cron does not deal with seconds so you can't have cronjob's going off in any time period dealing with seconds. Like a cronjob going off every 30 seconds.

Calculate difference between two dates in bash?

The easiest way is to convert both the dates to unix timestamp (which is the number of seconds from Jan 1, 1970). Once done simply substract them and divide by 86400. 


For e.g., date --date "2012-01-06" +%s Converts the date to number of seconds..



d1=`date --date "2012-01-06" +%s`
d2=`date +%Y-%m-%d`
d2=`date --date "$d2" +%s`


diff=$((d2-d1))
days=`echo $((diff/86400))`

The variables days will contain the difference in days between d1 and d2. Where, d1 is 2012-01-06 (yyyy-mm-dd format) and d2 is current date. 

Enjoy.