One thing that often confuses new users to the Unix / Linux shell, is
how to do (even very simple) maths. In most languages, x = x + 1 (or
even x++) does exactly what you would expect. The Unix/Linux shell is
different,however. It doesn’t have any built-in mathematical operators
for variables. It can do comparisons, but maths isn’t supported, not
even simple addition.
Shell script variables are by default treated as strings,not numbers,
which adds some complexity to doing math in shell script. To keep with
script programming paradigm and allow for better math support, langua-
ges such Perl or Python would be better suited when math is desired.
However, it is possible to do math with shell script.In fact, over the
years, multiple facilities have been added to Unix to support working
with numbers.
You can do maths using any one of the following methods.
1. Using expr command
2 Using $(()) construct.
3 Using let command
4 Using bc command.
5 Using $[] construct.
expr command:
expr command performs arithmetic operations on integers. It can
perform the four basic arithmetic operations, as well as the modulus
(remainder function).
$ expr 5 + 10
15
$ a=10 b=5
$ expr $a + $b
15
$ expr $a / $b
2
$ expr $a * $b
expr: syntax error
$ expr $a \* $b
50
The operand, be it +,-,* etc., must be enclosed on either side by
whitespace. Observe that the multiplication operand (*) has to be
escaped to prevent the shell from interpreting it as the filename meta
character. Since expr can handle only integers, division yields only
the integral part.
expr is often used with command substitution to assign a variable.
For example, you can set a variable x to the sum of two numbers:
$ x=`expr $a + $b`
$ echo $x
15
Note: As you can see, for expr, you must put spaces around the
arguments: "expr 123+456" doesn’t work. "expr 123 + 456" works.
With double parentheses: $(()) and (())
In bash version 3.2 and later you can (and should) use $(()) and (())
for integer arithmetic expressions. You may have may not have spaces
around the operators, but you must not have spaces around the equal
sign, as with any bash variable assignment.
$ c=$(($a+9))
$ echo $c
19
$ c=$((a+9)) #Also correct, no need of $ sign.
$ c=$((a + 9)) #Also correct, no restriction on spaces.
$ c= $((a + b)) #Incorrect, space after assignment operator.
You may also use operations within double parentheses without
assignment.
$ ((a++))
$ echo $a
11
$ ((a+=1)) ; echo $a
12
$ ((d=a+b+9)) ; echo $d
26
$ ((a+=$b)) #Correct
$ (($a+=1)) #Incorrect
let command:
The let command carries out arithmetic operations on variables. In
many cases, it functions as a less complex version of expr command.
As you can see, it is also a little picky about spaces, but it wants
the opposite of what expr wanted. let also relaxes the normal rule of
needing a $ in front of variables to be read.
$ let a=10 # Same as 'a=11'
$ let a=a+5 # Equivalent to let "a = a + 5"
# Quotes permit the use of spaces in variable assignment. (Double
# quotes and spaces make it more readable.)
$ let a=$a + 5 # Without quotes spaces not allowed.
bash: let: +: syntax error: operand expected (error token is "+")
You need to use quotes if you want to use spaces between tokens of
the expression, for example
$ let "a = a + 5";echo $a
20
The only construct that is more convenient to use with let is incre-
ment such as
$ let a++ ; echo $a # as well as to ((i++))
16
bc: THE CALCULATOR
Bash doesn't support floating point arithmetic. The obvious candidate
for adding floating point capabilities to bash is bc. bc is not only a
command, it also a pseudo-programming language featuring arrays,funct-
ions,conditional(if) and loops(for and while). It also comes with a
library for performing scientific calculations. It can handle very,
very large numbers. If a computation results in a 900 digit number, bc
will show each and every digit. But you have to treat the variables as
strings.
Here is what happens when we try to do floating point math with the
shell:
$ let a=12.5
bash: let: a=12.5: syntax error: invalid arithmetic operator (error
token is ".5")
$ ((b=1*0.5))
bash: ((: b=1*0.5: syntax error: invalid arithmetic operator (error
token is ".5")
I can't explain everything about bc here, it needs another post.
But I will give some examples here.
Most of the bellow examples follow a simple formula:
$ echo '57+43' | bc
100
$ echo '57*43' | bc
2451
$ echo '6^6' | bc # Power
46656
$ echo '1.5*5'|bc # Allows floating point math.
7.5
$[] construct:
$ x=85
$ y=15
$ echo $[x+y]
100
$ echo $[x/y]
5
$ c=$[x*y]
$ echo $c
1275
Working of above methods shell dependent. Bash shell supports all 5
methods. Following shell script demonstrates above methods.
#!/bin/bash
# SCRIPT: basicmath.sh
# USAGE: basicmath.sh
# PURPOSE: Addition, Subtraction, Division and Multiplication of
# two numbers.
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
#####################################################################
# Variable Declaration #
#####################################################################
clear #Clears Screen
Bold="\033[1m" #Storing escape sequences in a variable.
Normal="\033[0m"
echo -e "$Bold Basic mathematics using bash script $Normal\n"
items="1. ADDITTION
2. SUBTRACTION
3. MULTIPLICATION
4. DIVISION
5. EXIT"
choice=
#####################################################################
# Define Functions Here #
#####################################################################
# If didn't understand these functions, simply remove functions and
# its entries from main script.
exit_function()
{
clear
exit
}
#Function enter is used to go back to menu and clears screen
enter()
{
unset num1 num2
ans=
echo ""
echo -e "Do you want to continue(y/n):\c"
stty -icanon min 0 time 0
# When -icanon is set then one character has been received.
# min 0 means that read should read 0 characters.
# time 0 ensures that read is terminated the moment one character
# is hit.
while [ -z "$ans" ]
do
read ans
done
#The while loop ensures that so long as at least one character is
# not received read continue to get executed
if [ "$ans" = "y" -o "$ans" = "Y" ]
then
stty sane # Restoring terminal settings
clear
else
stty sane
exit_function
fi
}
#####################################################################
# Main Starts #
#####################################################################
while true
do
echo -e "$Bold \tPROGRAM MENU $Normal\n"
echo -e "\t$items \n"
echo -n "Enter your choice : "
read choice
case $choice in
1) clear
echo "Enter two numbers for Addition : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
echo "$num1 + $num2 = `expr $num1 + $num2`"
enter ;;
2) clear
echo "Enter two numbers for Subtraction : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
echo "$num1 - $num2 = $((num1-num2))"
enter ;;
3) clear
echo "Enter two numbers for Multiplication : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
echo "$num1 * $num2 = `echo "$num1*$num2"|bc`"
enter ;;
4) clear
echo "Enter two numbers for Division : "
echo -n "Number1: "
read num1
echo -n "Number2: "
read num2
let div=num1/num2
echo "$num1 / $num2 = $div"
enter ;;
5) exit_function ;;
*) echo "You entered wrong option, Please enter 1,2,3,4 or 5"
echo "Press enter to continue"
read
clear
esac
done
OUTPUT:
[venu@localhost ~]$ sh basicmath.sh
Basic mathematics using bash script
PROGRAM MENU
1. ADDITTION
2. SUBTRACTION
3. MULTIPLICATION
4. DIVISION
5. EXIT
Enter your choice : 1
Enter two numbers for Addition :
Number1: 123
Number2: 456
123 + 456 = 579
Do you want to continue(y/n):y
PROGRAM MENU
1. ADDITTION
2. SUBTRACTION
3. MULTIPLICATION
4. DIVISION
5. EXIT
Enter your choice : 3
Enter two numbers for Multiplication :
Number1: 12.5
Number2: 2
12.5 * 2 = 25.0
Do you want to continue(y/n):n
Sunday, December 26, 2010
310 comments | 7:36 AM
Wednesday, December 22, 2010
39 comments | 12:38 PM
By definition in mathematics, the Fibonacci Numbers are the numbers
in the below sequence:
0,1,1,2,3,5,8,13,21,34,55,89,144, ......
By definition, the first two Fibonacci numbers are 0 and 1, and each
subsequent number is the sum of the previous two. Some sources omit
the initial 0, instead beginning the sequence with two 1s.
In mathematical terms, the sequence Fn of Fibonacci numbers is defi-
ned by the recurrence relation
Fn = Fn-1 + Fn-2,
with seed values
F0 = 0 and F1 = 1.
Iterative Method:
#!/bin/bash#!/bin/bash
# SCRIPT: fibo_iterative.sh
# USAGE: fibo_iterative.sh [Number]
# PURPOSE: Generate Fibonacci sequence.
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
#####################################################################
# Script Starts Here #
#####################################################################
if [ $# -eq 1 ]
then
Num=$1
else
echo -n "Enter a Number :"
read Num
fi
f1=0
f2=1
echo "The Fibonacci sequence for the number $Num is : "
for (( i=0;i<=Num;i++ ))
do
echo -n "$f1 "
fn=$((f1+f2))
f1=$f2
f2=$fn
done
echo
OUTPUT:
# sh fibo_iterative.sh 18
The Fibonacci sequence for the number 18 is :
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584
# sh fibo_iterative.sh
Enter a Number :20
The Fibonacci sequence for the number 20 is :
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765
Recursive Method:
#!/bin/bash
# SCRIPT: fibo_recursive.sh
# USAGE: fibo_recursive.sh [Number]
# PURPOSE: Generate Fibonacci sequence.
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
#####################################################################
# Arguments Checking #
#####################################################################
if [ $# -eq 1 ]
then
Num=$1
else
echo -n "Enter a Number : "
read Num
fi
#####################################################################
# Define Functions Here #
#####################################################################
Fibonacci()
{
case $1 in
0|1) printf "$1 " ;;
*) echo -n "$(( $(Fibonacci $(($1-2)))+$(Fibonacci $(($1-1))) )) ";;
esac
#$(( )) construct is used instead of expr command for doing addition.
#$( ) constrict is used instead of back ticks.
}
#####################################################################
# Main Script Starts Here #
#####################################################################
echo "The Fibonacci sequence for the number $Num is : "
for (( i=0; i<=$Num; i++ ))
do
Fibonacci $i #Calling function Fibonacci
done
echo
OUTPUT:
# sh fibo_recursive.sh
Enter a Number : 11
The Fibonacci sequence for the number 11 is :
0 1 1 2 3 5 8 13 21 34 55 89
# sh fibo_recursive.sh 13
The Fibonacci sequence for the number 13 is :
0 1 1 2 3 5 8 13 21 34 55 89 144 233
Be aware that recursion is resource-intensive and executes slowly,
and is therefore generally not appropriate to use in a script.
[root@localhost shell]# time ./fibo_iterative.sh 15
The Fibonacci sequence for the number 15 is :
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610
real 0m0.008s
user 0m0.008s
sys 0m0.000s
[root@localhost shell]# time ./fibo_recursive.sh 15
The Fibonacci sequence for the number 15 is :
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610
real 0m7.875s
user 0m0.908s
sys 0m5.188s
Too many levels of recursion may crash a script with a segfault.
Monday, December 20, 2010
16 comments | 12:08 PM
#!/bin/bash
# SCRIPT: armstrong_bw_range.sh
# USAGE: armstrong_bw_range.sh
# PURPOSE: Finding Armstrong numbers between given range.
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#Armstrong numbers are the sum of their own digits to the power of
#the number of digits. As that is a slightly brief wording, let me
#give an example:
#153 = 1³ + 5³ + 3³
#Each digit is raised to the power three because 153 has three
#digits. They are totalled and we get the original number again!
#Notice that Armstrong numbers are base dependent,but we'll mainly be
#dealing with base 10 examples.The Armstrong numbers up to 5 digits
#are 1 to 9,153, 370, 371, 407, 1634, 8208, 9474, 54748, 92727, 93084
#
#####################################################################
# Script Starts Here #
#####################################################################
echo -n "Enter the Lower Limit : "
read Start
echo -n "Enter the Upper Limit : "
read Ending
echo "Armstrong Numbers between $Start and $Ending are: "
while [ $Start -le $Ending ]
do
Number=$Start
Length=${#Number}
Sum=0
OldNumber=$Number
while [ $Number -ne 0 ]
do
Rem=$((Number%10))
Number=$((Number/10))
Power=$(echo "$Rem ^ $Length" | bc )
Sum=$((Sum+Power))
done
if [ $Sum -eq $OldNumber ]
then
echo -n "$OldNumber "
fi
let Start++
done
echo
OUTPUT:
# sh armstrong_bw_range.sh
Enter the Lower Limit : 1
Enter the Upper Limit : 500
Armstrong Numbers between 1 and 500 are:
1 2 3 4 5 6 7 8 9 153 370 371 407
# sh armstrong_bw_range.sh
Enter the Lower Limit : 1000
Enter the Upper Limit : 10000
Armstrong Numbers between 1000 and 10000 are:
1634 8208 9474
20 comments | 12:00 PM
#!/bin/bash
# SCRIPT: armstrongnumber.sh
# USAGE: armstrongnumber.sh
# PURPOSE: Check if the given number is Armstrong number ?
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
# Armstrong numbers are the sum of their own digits to the power of
# the number of digits. As that is a slightly brief wording, let me
# give an example:
# 153 = 1³ + 5³ + 3³
# Each digit is raised to the power three because 153 has three
# digits. They are totalled and we get the original number again!
#Notice that Armstrong numbers are base dependent,but we'll mainly be
# dealing with base 10 examples.The Armstrong numbers up to 5 digits
# are 1 to 9,153, 370, 371, 407, 1634, 8208, 9474, 54748, 92727,93084
#
#####################################################################
# Script Starts Here #
#####################################################################
echo -n "Enter the number: "
read Number
Length=${#Number}
Sum=0
OldNumber=$Number
while [ $Number -ne 0 ]
do
Rem=$((Number%10))
Number=$((Number/10))
Power=$(echo "$Rem ^ $Length" | bc )
Sum=$((Sum+$Power))
done
if [ $Sum -eq $OldNumber ]
then
echo "$OldNumber is an Armstrong number"
else
echo "$OldNumber is not an Armstrong number"
fi
OUTPUT:
# sh armstrongnumber.sh
Enter the number: 8208
8208 is an Armstrong number
# sh armstrongnumber.sh
Enter the number: 4210818
4210818 is an Armstrong number
# sh armstrongnumber.sh
Enter the number: 4376541
4376541 is not an Armstrong number
10 comments | 6:28 AM
#!/bin/bash
# SCRIPT: dec2binary.sh
# USAGE: dec2binary.sh Decimal_Number(s)
# PURPOSE: Decimal to Binary Conversion. Takes input as command line
# arguments.
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
#####################################################################
# Script Starts Here #
#####################################################################
if [ $# -eq 0 ]
then
echo "Argument(s) not supplied "
echo "Usage: dec2binary.sh Decimal_number(s)"
else
echo -e "\033[1mDECIMAL \t\t BINARY\033[0m"
while [ $# -ne 0 ]
do
DecNum=$1
Binary=
Number=$DecNum
while [ $DecNum -ne 0 ]
do
Bit=$(expr $DecNum % 2)
Binary=$Bit$Binary
DecNum=$(expr $DecNum / 2)
done
echo -e "$Number \t\t $Binary"
shift
# Shifts command line arguments one step.Now $1 holds second argument
unset Binary
done
fi
#NOTE: Using bc command you can directly get output at command line.
# $ echo 'obase=2;15' | bc
# 1111
# $ echo 'obase=2;1023' | bc
# 1111111111
# $ echo 'obase=2;1024' | bc
# 10000000000
# No need of ibase=10, because default ibase is 10.
OUTPUT:
# sh dec2binary.sh 7 16 255 256 1023 1024
DECIMAL BINARY
7 111
16 10000
255 11111111
256 100000000
1023 1111111111
1024 10000000000
# sh dec2binary.sh 9223372036854775807
DECIMAL BINARY
9223372036854775807 1111111111111111111111111111111111111
11111111111111111111111111
#sh dec2binary.sh 9223372036854775808
DECIMAL BINARY
dec2binary.sh:line 13:[:9223372036854775808:integer expression expected
9223372036854775808
NOTE:My bash can handle upto 9223372036854775807.
But with bc command you can do more.
# echo 'obase=2;9223372036854775808'|bc
1000000000000000000000000000000000000000000000000000000000000000
10 comments | 6:22 AM
#!/bin/bash
# SCRIPT: binary2dec.sh
# USAGE: binary2dec.sh Binary_Numbers(s)
# PURPOSE: Binary to Decimal Conversion. Takes input as command line
# arguments.
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
#####################################################################
# Script Starts Here #
#####################################################################
if [ $# -eq 0 ]
then
echo "Argument(s) not supplied "
echo "Usage: binary2dec.sh Binary_Number(s)"
else
echo -e "\033[1mBINARY \t\t DECIMAL\033[0m"
while [ $# -ne 0 ]
do
Binary=$1
Bnumber=$Binary
Decimal=0
power=1
while [ $Binary -ne 0 ]
do
rem=$(expr $Binary % 10 )
Decimal=$((Decimal+(rem*power)))
power=$((power*2))
Binary=$(expr $Binary / 10)
done
echo -e "$Bnumber \t\t $Decimal"
shift
done
fi
#NOTE: Using bc command you can directly get output at command line.
# Here we're converting the binary number 111 to a base 10 (decimal)
# number.
#
# $ echo 'ibase=2;obase=A;111' | bc
# 7
#
#Note that the obase is "A" and not "10".The reason for this is you've
# set the ibase to 2, so if you now had tried to use "10" as the value
# for the obase, it would stay as "2", because "10" in base 2 is "2".
# So you need to use hex to break out of binary mode or use obase=1010
#
# $ echo 'ibase=2;obase=1010;10000' | bc
# 16
# $ echo 'ibase=2;obase=1010;1000001' | bc
# 65
#
# You can omit obase=A, because default obase is 10(decimal).
# $ echo 'ibase=2;1000001' | bc
# 65
OUTPUT:
# sh binary2dec.sh 100 1111 1111111 1000000
BINARY DECIMAL
100 4
1111 15
1111111 127
1000000 64
# sh binary2dec.sh 1111111111111111111
BINARY DECIMAL
1111111111111111111 524287
# sh binary2dec.sh 10000000000000000000
BINARY DECIMAL
binary2dec.sh:line 28:[:10000000000000000000:integer expression expected
10000000000000000000
NOTE:My bash can handle upto 9223372036854775807.
But with bc command you can do more.
# echo 'ibase=2;10000000000000000000'|bc
524288
# echo 'ibase=2;110000000000000000000'|bc
1572864
Sunday, December 19, 2010
82 comments | 11:25 AM
Method 1:
#!/bin/bash
# SCRIPT: validinteger.sh
# USAGE: validinteger.sh [ Input value to be validated ]
# PURPOSE: validate integer input, allow negative integers also
#
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
# Using the $? exit status variable or integer comparison operators, a
# script may test if a parameter contains only digits, so it can be
# treated as an integer.
#
#####################################################################
# Arguments Checking #
#####################################################################
if [ $# -eq 0 ]
then
echo -n "Enter input to test: "
read Number
else
Number=$1
fi
# You can also use bellow one liner
#[ $# -eq 0 ] && { echo -n "Enter input:";read Number; } || Number=$1
#####################################################################
# Main Script Starts Here #
#####################################################################
# Check the input is an integer or not
if [ $Number -ne 0 -o $Number -eq 0 2>/dev/null ]
then
# An integer is either equal to 0 or not equal to 0.
# 2>/dev/null suppresses error message.
echo "Supplied Input $Number is an Integer"
else
echo "Supplied Input $Number is not an Integer."
fi
# You can compare number itself also
# if [ $Number -eq $Number 2> /dev/null ]
# then
# echo "Supplied Input $Number is an Integer"
# else
# echo "Supplied Input $Number is not an Integer."
# fi
OUTPUT:
# chmod 755 validinteger.sh
# ./validinteger.sh -12345
Supplied Input -12345 is an Integer
# ./validinteger.sh 12345A
Supplied Input 12345A is not an Integer.
# ./validinteger.sh
Enter input to test: 876549
Supplied Input 876549 is an Integer
# ./validinteger.sh
Enter input to test: -345k123
Supplied Input -345k123 is not an Integer.
Method 2 :
#!/bin/bash
# SCRIPT: validinteger2.sh
# USAGE: validinteger2.sh [ Input value to be validated ]
# PURPOSE: validate integer input, allow negative integers also
#
#
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
# In this method if you want to allow negative integers, it gets a
# tiny bit more complicated. You must have basic knowledge of string
# manipulation methods.
#
#####################################################################
# Arguments Checking #
#####################################################################
if [ $# -eq 0 ]
then
echo -n "Enter input to test: "
read Number
else
Number=$1
fi
#####################################################################
# Main Script Starts Here #
#####################################################################
# Check first character is -
if [ "${Number:0:1}" = "-" ] #extract first character
then
RemNumber="${Number:1}" #extract except first char
else
RemNumber="$Number"
fi
# You can extract substring using bellow method also
#if [ "$(echo $Number|cut -c 1)" = "-" ]
#then
# RemNumber=$(echo $Number|cut -c 2-)
#else
# RemNumber="$Number"
#fi
if [ -z "$RemNumber" ]
then
echo "Supplied Input $Number is not an Integer"
else
NoDigits="$(echo $RemNumber | sed 's/[[:digit:]]//g')"
if [ -z "$NoDigits" ]
then
echo "Supplied Input $Number is an Integer"
else
echo "Supplied Input $Number is not an Integer."
fi
fi
OUTPUT:
# chmod 755 validinteger2.sh
# ./validinteger2.sh -987612
Supplied Input -987612 is an Integer
# ./validinteger2.sh -A987612
Supplied Input -A987612 is not an Integer.
# ./validinteger2.sh
Enter input to test: -123456789
Supplied Input -123456789 is an Integer
# ./validinteger2.sh
Enter input to test: A234K78
Supplied Input A234K78 is not an Integer.
Method 3:
#!/bin/bash
# SCRIPT: validinteger3.sh
# USAGE: validinteger3.sh [ Input value to be validated ]
# PURPOSE: validate integer input, allow negative integers also
#
# \\\\ ////
# \\ - - //
# @ @
# ---oOOo-( )-oOOo---
#
# In this method input value validated using ASCII value range of
# numbers (48 - 57).
# Sample script provided by Bond, Thank you Bond.
#
#####################################################################
# Arguments Checking #
#####################################################################
if [ $# -eq 0 ]
then
echo -n "Enter input to test: "
read Number
else
Number=$1
fi
# You can also use bellow one liner
#[ $# -eq 0 ] && { echo -n "Enter input:";read Number; } || Number=$1
#####################################################################
# Main Script Starts Here #
#####################################################################
# Check first character is - ?
# This is another method to extract substring using Substring Removal
# method.
if [ "${Number%${Number#?}}" = "-" ] #extract first char
then
RemNumber="${Number#?}" #extract all except first char
else
RemNumber="$Number"
fi
if [ -z "$RemNumber" ]
then
echo "Supplied Input $Number is not an Integer"
else
Length=${#RemNumber} # Counting length of the string
for ((i=0; i < Length; i++))
do
char=${RemNumber:$i:1} # Reads character one by one
code=`printf '%d' "'$char"` # Get ASCII value of char
# Compare code with ASCII value range of Intergers ( 48 - 57 )
if [ $code -lt 48 ] || [ $code -gt 57 ]
then
echo "Supplied Input $Number is not an Integer"
exit 1
fi
done
echo "Supplied Input $Number is an Integer."
fi
OUTPUT:
$ sh validinteger3.sh -1234
Supplied Input -1234 is an Integer.
$ sh validinteger3.sh -12-12
Supplied Input -12-12 is not an Integer
$ sh validinteger3.sh
Enter input to test: -
Supplied Input - is not an Integer
$ sh validinteger3.sh
Enter input to test: 123a456
Supplied Input 123a456 is not an Integer
Saturday, July 31, 2010
14 comments | 4:18 PM
Palindrome: A palindrome is a word, phrase, number or other sequence
of units that can be read the same way in either direction (the adjus-
tment of punctuation and spaces between words is generally permitted).
Examples:
Phrases: Dammit, I'm mad!
Quotations: Able was I ere I saw Elba.
Madam, I'm Adam.
Names: Some people have names that are palindromes.Lon Nol (1913-
1985) was Prime Minister of Cambodia.
Palindromic names are very common in Finland. Examples
include Emma Lamme,Sanna Rannas, Anni Linna and Asko Oksa.
Words: civic,radar,level,rotator,rececar,reviver.
The command "Level, madam, level!", composed only of words
that are themselves palindromes, is both a character-by-
character and a word-by-word palindrome.
Numbers: 5335, 123454321
Dates: 01/02/2010 (dd/mm/yyyy format)
Method 1:
#!/bin/bash
# SCRIPT: palindrome1.sh
# USAGE: palindrome.sh or palindrome.sh STRING
# PURPOSE: Script to test if a given string is a palindrome.
#
# In this script I uses the well known method, compare first character
# with last character, up to middle of the string. One mismatch in the
# scanning leads to immediate termination of the scanning as it is
# not a palindrome. To extract character from string, I will use cut
# command with the -c option with the position number.
#
#####################################################################
# Arguments Checking #
#####################################################################
if [ $# -eq 0 ]
then
echo -n "Enter a String: "
read orgstr
else
orgstr=$*
fi
# You can also use single statement
#[ $# -eq 0 ] && (echo -n "Enter a String:"; read String) || String=$*
#####################################################################
# Variable Initialization #
#####################################################################
# Remove all punctuations from input string and convert upper case to
# lower or lower case to upper.
String="$(echo $orgstr | sed 's/[^[:alnum:]]//g' | \
tr '[:upper:]' '[:lower:]')"
Flag=0
# Find length of the string.
len=${#String}
#You can also calculate string length using bellow commands.
#len=`echo $str | wc -c`
#len=$((len-1))
#get the mid value up to which the comparison would be done.
mid=$((len/2))
#####################################################################
# Main Script Starts Here #
#####################################################################
for ((i=1;i<=mid;i++))
do
c1=`echo $String|cut -c$i` # extracts from beginning
c2=`echo $String|cut -c$len` # extracts from last
if [ $c1 != $c2 ]
then
Flag=1
break 2 # break N breaks out of N levels of loop.
fi
let len--
done
if [ $Flag -eq 0 ]
then
echo "\"$orgstr\" is a Palindrome"
else
echo "\"$orgstr\" is not a Palindrome"
fi
OUTPUT:
[root@www ]# ./palindrome1.sh Dammit, I\'m mad!
"Dammit, I'm mad!" is a Palindrome
[root@www ]# ./palindrome1.sh
Enter a String: 01/02/2010
"01/02/2010" is a Palindrome
[root@www ]# ./palindrome1.sh Hello world
"Hello world" is not a Palindrome
Method 2:
#!/bin/bash
# SCRIPT: palindrome2.sh
# USAGE: palindrome.sh or palindrome.sh STRING
# PURPOSE: Script to test if a given string is a palindrome.
#
# In this script I uses the well known method, compare first character
# with last character, up to middle of the string. One mismatch in the
# scanning leads to immediate termination of the scanning as it is
# not a palindrome. To extract a character from the string, I will use
# string manipulation operations.So you need to know how to manipulate
# strings to understand this script. I will give little bit of explan-
# tion at the end of this script.
#
#####################################################################
# Arguments Checking #
#####################################################################
[ $# -eq 0 ] && { echo -n "Enter a String: "; read orgstr ;} || \
orgstr=$*
#####################################################################
# Variable Initialization #
#####################################################################
# Remove all punctuations from input string and convert upper case to
# lower or lower case to upper.
String="$(echo $orgstr | sed 's/[^[:alnum:]]//g' | \
tr '[:upper:]' '[:lower:]')"
# Find length of the string.
len=${#String}
#get the mid value up to which the comparison would be done
mid=$(($len/2))
i=0
Flag=0
#####################################################################
# Main Script Starts Here #
#####################################################################
while [ $i -lt $mid ]
do
fchar=${String:$i:1}
let i++
bchar=${String: -$i:1}
if [ "$fchar" != $bchar ]
then
Flag=1
break 2 # break N breaks out of N levels of loop.
fi
done
if [ $Flag -eq 0 ]
then
echo "\"$orgstr\" is a Palindrome"
else
echo "\"$orgstr\" is not a Palindrome"
fi
Substring Extraction:
${string:position}
Extracts substring from $string at $position.
${string:position:length}
Extracts $length characters of substring from $string at $position
Bash numbers first character of string as '0'.
${string: 0: 1} will extracts one character from the 0th character of
the string, ie it will only get the 0th character. ${string: 2: 1}
will get the third character. Also ${string: -1: 1} will extracts the
last one character, ${string: -3:1} will get the third last character.
Note: ${string: -1:1} in this construct don't forget to give space
before -1, otherwise you will get full string.
For example
[root@localhost www]# tempvar=madam
[root@localhost www]# echo ${tempvar: -1:1}
m
[root@localhost www]# echo ${tempvar:-1:1}
madam
You can also use following command
[root@localhost www]# echo ${tempvar:(-1):1}
m
OUTPUT:
[root@www ]# ./palindrome2.sh Able was I ere I saw Elba
"Able was I ere I saw Elba" is a Palindrome
[root@www ]# ./palindrome2.sh 123454321
"123454321" is a Palindrome
[root@www ]# ./palindrome2.sh
Enter a String: 12345654321
"12345654321" is a Palindrome
[root@www ]# ./palindrome2.sh
Enter a String: 1234564321
"1234564321" is not a Palindrome
Method 3:
#!/bin/bash
# SCRIPT: palindrome3.sh
# USAGE: palindrome.sh or palindrome.sh STRING
# PURPOSE: Script to test if a given string is a palindrome.
#
# This simply uses the 'rev' utility which is used to reverse lines of
# a file. Then check if the reverse of the string is same as the
# original.rev command is part of util-linux-ng or util-linux package.
#
if `which rev &>/dev/null` # Checks rev command exist or not
then
[ $# -eq 0 ] && { echo -n "Enter a String: "; read orgstr ;} || \
orgstr=$*
String="$(echo $orgstr | sed 's/[^[:alnum:]]//g' | \
tr '[:upper:]' '[:lower:]')"
if [ "$(echo $String | rev)" = "$String" ]
then
echo "\"$orgstr\" is a palindrome"
else
echo "\"$orgstr\" is not a palindrome"
fi
else
echo "Install util-linux or util-linux-ng package"
fi
OUTPUT:
[root@www ]# ./palindrome3.sh 01/02/2010
"01/02/2010" is a palindrome
[root@www ]# ./palindrome3.sh 01/03/2010
"01/03/2010" is not a palindrome
[root@www ]# ./palindrome3.sh
Enter a String: Hello World
"Hello World" is not a palindrome
[root@www ]# ./palindrome3.sh
Enter a String: rotator
"rotator" is a palindrome
Method 4:
#!/bin/bash
# SCRIPT: palindrome4.sh
# USAGE: palindrome.sh or palindrome.sh STRING
# PURPOSE: Script to test if a given string is a palindrome.
#
# In this method we are not using 'rev' command to reverse the string.
# Using Substring Removal method or Substring Extraction method we
# will reverse the string, then compare it with oldstring.
#
#####################################################################
# Arguments Checking #
#####################################################################
[ $# -eq 0 ] && { echo -n "Enter a String: "; read orgstr ;} || \
orgstr=$*
#####################################################################
# Variable Initialization #
#####################################################################
String="$(echo $orgstr | sed 's/[^[:alnum:]]//g' | \
tr '[:upper:]' '[:lower:]')"
oldstring=$String
newstring=
#####################################################################
# Main Script Starts Here #
#####################################################################
while [ -n "$String" ]
do
temp=${String#?}
letter=${String%"$temp"}
String=$temp
newstring=${letter}${newstring}
done
if [ "$oldstring" = "$newstring" ]
then
echo "\"$orgstr\" is a palindrome"
else
echo "\"$orgstr\" is not a palindrome"
fi
# ${string#substring} is a Substing Removal operation. If you want to
# use Substring Extraction method, use bellow code.
#i=0
#while [ $i -lt ${#String} ]
#do
# letter=${String:$i:1}
# newstring=${letter}${newstring}
# let i++;
#done
#
#if [ "$String" = "$newstring" ]
#then
# echo "\"$orgstr\" is a palindrome"
#else
# echo "\"$orgstr\" is not a palindrome"
#fi
Substring Removal:
${string#substring}
Strips shortest match of $substring from front of $string.
Example:
[root@www]# tempvar=madam
[root@www]# echo ${tempvar#m}
adam
[root@www]# echo ${tempvar#ma}
dam
[root@www]# echo ${tempvar#?}
adam
${string%substring}
Strips shortest match of $substring from back of $string.
Example:
[root@www]# temp=${tempvar#?}
[root@www]# echo $temp
adam
[root@www]# echo ${tempvar%$temp}
m
OUTPUT:
[root@www ]# ./palindrome4.sh Madam, I\'m Adam
"Madam, I'm Adam" is a palindrome
[root@www ]# ./palindrome4.sh Madam I Adam
"Madam I Adam" is not a palindrome
[root@www ]# ./palindrome4.sh
Enter a String: 123454321
"123454321" is a palindrome
[root@www ]# ./palindrome4.sh
Enter a String: 1234564321
"1234564321" is not a palindrome
Thursday, July 29, 2010
7 comments | 9:34 AM
#!/bin/bash#!/bin/bash
# SCRIPT: insertionsort.sh
#
# LOGIC: Here, sorting takes place by inserting a particular element
# at the appropriate position, that’s why the name insertion sorting.
# In the First iteration, second element ARRAY[1] is compared with
# the first element ARRAY[0]. In the second iteration third element
# is compared with first and second element. In general, in every
# iteration an element is compared with all the elements before it.
# While comparing if it is found that the element can be inserted at
# a suitable position, then space is created for it by shifting the
# other elements one position up and inserts the desired element at
# the suitable position. This procedure is repeated for all the
# elements in the list.
#
#####################################################################
# Define Functions Here #
#####################################################################
printnumbers()
{
echo ${ARRAY[*]}
}
sortnumbers()
{
for((i=1;i<count;i++))
do
Temp=${ARRAY[i]}
j=$((i-1))
while [ $Temp -lt ${ARRAY[j]} ]
do
ARRAY[j+1]=${ARRAY[j]}
let j--
if [ $j == -1 ]
then
break
fi
done
ARRAY[j+1]=$Temp
done
}
#####################################################################
# Variable Initialization #
#####################################################################
echo "Enter numbers to be sorted"
read -a ARRAY
count=${#ARRAY[@]}
#####################################################################
# Main Script Starts Here #
#####################################################################
echo "---------------------------------------------------------------"
echo "Numbers Before Sort:"
printnumbers
sortnumbers
echo "Numbers After Sort: "
printnumbers
echo "---------------------------------------------------------------"
OUTPUT:
[root@www blog]# sh insertionsort.sh
Enter Numbers to be Sorted :
12 76 34 -34 67 9 -56 5 99 -3 17
---------------------------------------------------------------
Numbers Before Sort:
12 76 34 -34 67 9 -56 5 99 -3 17
Numbers After Sort:
-56 -34 -3 5 9 12 17 34 67 76 99
---------------------------------------------------------------
NOTE: If we complement the while condition in this program, it will
give out the sorted array in descending order.
9 comments | 9:29 AM
#!/bin/bash#!/bin/bash
# SCRIPT: selectionsort.sh
#
# LOGIC : Here, to sort the data in ascending order, the first element
# ARRAY[0] is compared with all the other elements till the end of the
# array. If it is greater than any other the elements then they are
# interchanged. So after the first iteration of the outer for loop
# smallest element will be placed at the first position. The same pro-
# cedure is repeated for the other elements too.
#
#####################################################################
# Define Functions Here #
#####################################################################
printnumbers()
{
echo ${ARRAY[*]}
}
swap()
{
temp=${ARRAY[$1]}
ARRAY[$1]=${ARRAY[$2]}
ARRAY[$2]=$temp
}
sortnumbers()
{
for ((i=0;i<count;i++))
do
min=$i
for ((j=i+1;j<count;j++))
do
if [ ${ARRAY[j]} -lt ${ARRAY[min]} ]
then
min=$j
fi
done
swap $i $min
done
}
#####################################################################
# Variable Initialization #
#####################################################################
echo "Enter Numbers to be Sorted : "
read -a ARRAY
count=${#ARRAY[@]}
#####################################################################
# Main Script Starts Here #
#####################################################################
echo "---------------------------------------------------------------"
echo "Numbers Before Sort:"
printnumbers
sortnumbers
echo "Numbers After Sort: "
printnumbers
echo "---------------------------------------------------------------"
OUTPUT:
[root@www blog]# sh selectionsort.sh
Enter Numbers to be Sorted :
34 76 -8 12 23 5 9 -2 88 41 62
---------------------------------------------------------------
Numbers Before Sort:
34 76 -8 12 23 5 9 -2 88 41 62
Numbers After Sort:
-8 -2 5 9 12 23 34 41 62 76 88
---------------------------------------------------------------
NOTE: If we complement the if condition in this program, it will give
out the sorted array in descending order.
14 comments | 9:23 AM
#!/bin/bash
# SCRIPT: bubblesort.sh
# LOGIC:
# Bubble sort is a simple sorting, it works by repeatedly stepping
# through the list to be sorted, comparing two items at a time and
# swapping them if they are in the wrong order. If you are sorting
# the data in Ascending order, at the end of the first pass, the
# "heaviest" element has move to bottom. In the second pass, the
# comparisons are made till the last but one position and now second
# largest element is placed at the last but one position. And so
# forth.
#
#####################################################################
# Define Functions Here #
#####################################################################
printnumbers()
{
echo ${ARRAY[*]}
#You can also use bellow code
#for ((i=0;i<count;i++))
#do
#echo -n " ${ARRAY[i]} "
#done
}
exchange()
{
temp=${ARRAY[$1]}
ARRAY[$1]=${ARRAY[$2]}
ARRAY[$2]=$temp
}
sortnumbers()
{
for (( last=count-1;last>0;last--))
do
for((i=0;i<last;i++))
do
j=$((i+1))
if [ ${ARRAY[i]} -gt ${ARRAY[j]} ]
then
exchange $i $j
fi
done
done
}
#####################################################################
# Variable Initialization #
#####################################################################
echo "Enter Numbers to be Sorted"
read -a ARRAY
count=${#ARRAY[@]}
#####################################################################
# Main Script Starts Here #
#####################################################################
echo "--------------------------------------------------------------"
echo "Numbers Before Sort:"
printnumbers
echo
sortnumbers
echo "Numbers After Sort: "
printnumbers
echo "--------------------------------------------------------------"
OUTPUT:
[root@www blog]# sh bubblesort.sh
Enter Numbers to be Sorted :
78 34 12 98 21 8 36 98 12 88 7 5 61 -12 62 -1 77 -46
------------------------------------------------------
Numbers Before Sort:
78 34 12 98 21 8 36 98 12 88 7 5 61 -12 62 -1 77 -46
Numbers After Sort:
-46 -12 -1 5 7 8 12 12 21 34 36 61 62 77 78 88 98 98
------------------------------------------------------
NOTE: If we complement the if condition in this program, it will give
out the sorted array in descending order.
Method2: Without Using Arrays
#!/bin/bash
# SCRIPT: bubblesort2.sh
# Without using arrays
#
#####################################################################
# Define Functions Here #
#####################################################################
printnumbers()
{
k=1
while [ $k -le $max ]
do
eval echo -n "\$x$k"
echo -n " "
let k++
done
echo
}
#####################################################################
# Variable Initialization #
#####################################################################
echo -n "Enter Total Numbers to be Sorted : "
read max
count=1
while [ $count -le $max ]
do
echo -n "Enter number $count: "
read x$count
let count++
done
#####################################################################
# Main Script Starts Here #
#####################################################################
echo -e "\nElements Before Sort"
printnumbers
for (( last=count-1;last>0;last--))
do
for ((i=1;i<last;i++))
do
j=$((i+1))
eval sval=\$x$i
eval nval=\$x$j
#The eval command evaluates the command line to complete any shell
#substitutions necessary and then executes the command. So $i and $j
#substituted first then $x1 and $x2 evaluated.
if [ $sval -gt $nval ]
then
eval x$i=$nval
eval x$j=$sval
fi
done
done
echo "Elements After Sort: "
printnumbers
OUTPUT:
[root@www shell]# sh bubblesort2.sh
Enter Total Numbers to be Sorted : 6
Enter number 1: 12
Enter number 2: -4
Enter number 3: 6
Enter number 4: -11
Enter number 5: 43
Enter number 6: 9
Elements Before Sort
12 -4 6 -11 43 9
Elements After Sort:
-11 -4 6 9 12 43
Friday, May 14, 2010
68 comments | 11:45 PM
There are many ways to handle any task on a Unix platform, but some
techniques that are used to process a file waste a lot of CPU time.
Most of the wasted time is spent in unnecessary variable assignment and
continuously opening and closing the same file over and over. Using a
pipe also has a negative impact on the timing.
In this article I will explain various techniques for parsing a file
line by line. Some techniques are very fast and some make you wait for
half a day. The techniques used in this article are measurable, and I
tested each technique with time command so that you can see which tec-
hniques suits your needs.
I don't explain in depth every thing, but if you know basic shell
scripting, I hope you can understand easily.
I extracted last five lines from my /etc/passwd file, and stored in a
file "file_passwd".
[root@www blog]# tail -5 /etc/passwd > file_passwd
[root@www blog]# cat file_passwd
venu:x:500:500:venu madhav:/home/venu:/bin/bash
padmin:x:501:501:Project Admin:/home/project:/bin/bash
king:x:502:503:king:/home/project:/bin/bash
user1:x:503:501::/home/project/:/bin/bash
user2:x:504:501::/home/project/:/bin/bash
I use this file whenever a sample file required.
Method 1: PIPED while-read loop
#!/bin/bash
# SCRIPT: method1.sh
# PURPOSE: Process a file line by line with PIPED while-read loop.
FILENAME=$1
count=0
cat $FILENAME | while read LINE
do
let count++
echo "$count $LINE"
done
echo -e "\nTotal $count Lines read"
With catting a file and piping the file output to a while read loop a
single line of text is read into a variable named LINE on each loop
iteration. This continuous loop will run until all of the lines in the
file have been processed one at a time.
Bash can sometimes start a subshell in a PIPED "while-read" loop. So
the variable set within the loop will be lost (unset) outside of the
loop. Therefore, $count would return 0, the initialized value outside
the loop.
Output:
[root@www blog]# sh method1.sh file_passwd
1 venu:x:500:500:venu madhav:/home/venu:/bin/bash
2 padmin:x:501:501:Project Admin:/home/project:/bin/bash
3 king:x:502:503:king:/home/project:/bin/bash
4 user1:x:503:501::/home/project/:/bin/bash
5 user2:x:504:501::/home/project/:/bin/bash
Total 0 Lines read
Method 2: Redirected "while-read" loop
#!/bin/bash
#SCRIPT: method2.sh
#PURPOSE: Process a file line by line with redirected while-read loop.
FILENAME=$1
count=0
while read LINE
do
let count++
echo "$count $LINE"
done < $FILENAME
echo -e "\nTotal $count Lines read"
We still use the while read LINE syntax, but this time we feed the
loop from the bottom (using file redirection) instead of using a pipe.
You will find that this is one of the fastest ways to process each
line of a file. The first time you see this it looks a little unusual,
but it works very well.
Unlike method 1, with method 2 you will get total number of lines out
side of the loop.
Output:
[root@www blog]# sh method2.sh file_passwd
1 venu:x:500:500:venu madhav:/home/venu:/bin/bash
2 padmin:x:501:501:Project Admin:/home/project:/bin/bash
3 king:x:502:503:king:/home/project:/bin/bash
4 user1:x:503:501::/home/project/:/bin/bash
5 user2:x:504:501::/home/project/:/bin/bash
Total 5 Lines read
Note: In some older shell scripting languages, the redirected loop
would also return as a subshell.
Method 3:while read LINE Using File Descriptors
A file descriptor is simply a number that the operating system assigns
to an open file to keep track of it. Consider it a simplified version
of a file pointer. It is analogous to a file handle in C.
There are always three default "files" open, stdin (the keyboard),
stdout (the screen), and stderr (error messages output to the screen).
These, and any other open files, can be redirected. Redirection simply
means capturing output from a file, command, program, script, or even
code block within a script and sending it as input to another file,
command, program, or script.
Each open file gets assigned a file descriptor. The file descriptors
for stdin,stdout, and stderr are 0,1, and 2, respectively. For opening
additional files, there remain descriptors 3 to 9 (may be vary depend-
ing on OS). It is sometimes useful to assign one of these additional
file descriptors to stdin, stdout, or stderr as a temporary duplicate
link. This simplifies restoration to normal after complex redirection
and reshuffling .
There are two steps in the method we are going to use. The first step
is to close file descriptor 0 by redirecting everything to our new file
descriptor 3. We use the following syntax for this step:
exec 3<&0
Now all of the keyboard and mouse input is going to our new file des-
criptor 3. The second step is to send our input file, specified by the
variable $FILENAME, into file descriptor 0 (zero), which is standard
input. This second step is done using the following syntax:
exec 0<$FILENAME
At this point any command requiring input will receive the input from
the $FILENAME file. Now is a good time for an example.
#!/bin/bash
#SCRIPT: method3.sh
#PURPOSE: Process a file line by line with while read LINE Using
#File Descriptors
FILENAME=$1
count0=
exec 3<&0
exec 0< $FILENAME
while read LINE
do
let count++
echo "$count $LINE"
done
exec 0<&3
echo -e "\nTotal $count Lines read"
while loop reads one line of text at a time.But the beginning of this
script does a little file descriptor redirection. The first exec comm-
and redirects stdin to file descriptor 3. The second exec command red-
irects the $FILENAME file into stdin, which is file descriptor 0. Now
the while loop can just execute without our having to worry about how
we assign a line of text to the LINE variable. When the while loop
exits we redirect the previously reassigned stdin, which was sent to
file descriptor 3, back to its original file descriptor 0.
exec 0<&3
In other words we set it back to the system’s default value.
Output:
[root@www tempdir]# sh method3.sh file_passwd
1 venu:x:500:500:venu madhav:/home/venu:/bin/bash
2 padmin:x:501:501:Project Admin:/home/project:/bin/bash
3 king:x:502:503:king:/home/project:/bin/bash
4 user1:x:503:501::/home/project/:/bin/bash
5 user2:x:504:501::/home/project/:/bin/bash
Total 5 Lines read
Method 4: Process file line by line using awk
awk is pattern scanning and text processing language. It is useful
for manipulation of data files, text retrieval and processing. Good
for manipulating and/or extracting fields (columns) in structured
text files.
Its name comes from the surnames of its authors: Alfred Aho, Peter
Weinberger, and Brian Kernighan.
I am not going to explain everything here.To know more about awk just
Google it.
At the command line, enter the following command:
$ awk '{ print }' /etc/passwd
You should see the contents of your /etc/passwd file appear before
your eyes.Now, for an explanation of what awk did. When we called awk,
we specified /etc/passwd as our input file. When we executed awk, it
evaluated the print command for each line in /etc/passwd, in order.All
output is sent to stdout, and we get a result identical to catting
/etc/passwd. Now, for an explanation of the { print } code block. In
awk, curly braces are used to group blocks of code together, similar
to C. Inside our block of code,we have a single print command. In awk,
when a print command appears by itself, the full contents of the curr-
ent line are printed.
Here is another awk example that does exactly the same thing:
$ awk '{ print $0 }' /etc/passwd
In awk, the $0 variable represents the entire current line, so print
and print $0 do exactly the same thing. Now is a good time for an
example.
#!/bin/bash
#SCRIPT: method4.sh
#PURPOSE: Process a file line by line with awk
FILENAME=$1
awk '{kount++;print kount, $0}
END{print "\nTotal " kount " lines read"}' $FILENAME
Output:
[root@www blog]# sh method4.sh file_passwd
1 venu:x:500:500:venu madhav:/home/venu:/bin/bash
2 padmin:x:501:501:Project Admin:/home/project:/bin/bash
3 king:x:502:503:king:/home/project:/bin/bash
4 user1:x:503:501::/home/project/:/bin/bash
5 user2:x:504:501::/home/project/:/bin/bash
Total 5 lines read
Awk is really good at handling text that has been broken into multiple
logical fields, and allows you to effortlessly reference each individ-
ual field from inside your awk script. The following script will print
out a list of all user accounts on your system:
awk -F":" '{ print $1 "\t " $3 }' /etc/passwd
Above, when we called awk, we use the -F option to specify ":" as the
field separator. By default white space (blank line) act as filed sep-
arator. You can set new filed separator with -F option. When awk proc-
esses the print $1 "\t " $3 command, it will print out the first and
third fields that appears on each line in the input file. "\t" is used
to separate field with tab.
Method 5: Little tricky with head and tail
commands
#!/bin/bash
#SCRIPT: method5.sh
#PURPOSE: Process a file line by line with head and tail commands
FILENAME=$1
Lines=`wc -l < $FILENAME`
count=0
while [ $count -lt $Lines ]
do
let count++
LINE=`head -n $count $FILENAME | tail -1`
echo "$count $LINE"
done
echo -e "\nTotal $count lines read"
On each iteration head command extracts top $count lines, then tail
command extracts bottom line from that lines. A very stupid method,
but some people still using it.
Output:
[root@www blog]# sh method5.sh file_passwd
1 venu:x:500:500:venu madhav:/home/venu:/bin/bash
2 padmin:x:501:501:Project Admin:/home/project:/bin/bash
3 king:x:502:503:king:/home/project:/bin/bash
4 user1:x:503:501::/home/project/:/bin/bash
5 user2:x:504:501::/home/project/:/bin/bash
Total 5 lines read
Time Comparison for the Five Methods
Now take a long breath, we are going test each technique. Before you
get into test each method of parsing a file line by line create a large
file that has the exact number of lines that you want to process.
Use bigfile.sh script to create a large file.
$ sh bigfile.sh 900000
bigfile.sh with 900000 lines as an argument,it has taken more than two
hours to generate bigfile.4227. I don't know exactly how much time it
has taken. This file is extremely large to parse a file line by line,
but I needed a large file to get the timing data greater than zero.
[root@www blog]# du -h bigfile.4227
70M bigfile.4227
[root@www blog]# wc -l bigfile.4227
900000 bigfile.4227
[root@www blog]# time ./method1.sh bigfile.4227 >/dev/null
real 6m2.911s
user 2m58.207s
sys 2m58.811s
[root@www blog]# time ./method2.sh bigfile.4227 > /dev/null
real 2m48.394s
user 2m39.714s
sys 0m8.089s
[root@www blog]# time ./method3.sh bigfile.4227 > /dev/null
real 2m48.218s
user 2m39.322s
sys 0m8.161s
[root@www blog]# time ./method4.sh bigfile.4227 > /dev/null
real 0m2.054s
user 0m1.924s
sys 0m0.120s
[root@www blog]# time ./method5.sh bigfile.4227 > /dev/null
I waited more than half day, still i didn't get result, then I created
a 10000-line file to test this method.
[root@www tempdir]# time ./method5.sh file.10000 > /dev/null
real 2m25.739s
user 0m21.857s
sys 1m12.705s
Method 4 came in first place,it has taken very less time 2.05 seconds,
but we can't compare Method 4 with other methods, because awk is not
just a command, but a programming language too.
Method 2 and method 3 are tied for second place, they produce mostly
the same real execution time at 2 minutes and 48 seconds . Method 1
came in third at 6 minutes and 2.9 seconds.
Method 5 has taken more than half a day. 2 minutes 25 seconds to pro-
cess just a 10000 line file, how stupid it is.
Note: If file contain escape characters, use read -r instead of read,
then Backslash does not act as an escape character. The back-slash is
considered to be part of the line. In particular, a backslash-newline
pair may not be used as a line continuation.
Tuesday, April 20, 2010
24 comments | 6:05 AM
Using tput command:
#!/bin/bash
# SCRIPT: digclock.sh
# USAGE: ./digiclock &
# PURPOSE: Displays time and date in the top right corner of the
# screen using tput command.
# To stop this digclock use command "kill pid"
################################################################
####################### VARIABLE DECLARATION ###################
# To place the clock on the appropriate column, subtract the
# length of $Time and $Date, which is 22, from the total number
# of columns
Columns=$(tput cols)
Startpoint=$(($Columns-22))
# If you're in an X Window System terminal,you can resize the
# window, and the clock will adjust its position because it is
# displayed at the last column minus 22 characters.
Color1=`tput setab 2` # Green background color for time
Color2=`tput setab 6` # Cyan background color for date
Normal=`tput sgr0` # back to normal screen colors
####################### MAIN PROGRAM ###########################
# The script is executed inside a while without conditions
while :
do
Time=`date +%r`
Date=`date +"%d-%m-%Y"`
tput sc #Save the cursor position&attributes
tput cup 0 $Startpoint
# You can also use bellow one liner
# tput cup 0 $((`tput cols`-22))
# But it is not efficient to calculate cursor position for each
# iteration. That's why I placed variable assignment before
# beginning of the loop.
# print time and date in the top right corner of the screen.
echo -n $Color1$Time $Color2$Date$Normal
# restore the cursor to whatever was its previous position
tput rc
# Delay for 1 second
sleep 1
done
Using ANSI escape sequences:
The ANSI escape sequences don't work in all terminal emulators, but
they do fine in xterm. Here's the script:
#!/bin/bash
# SCRIPT: digclock.sh
# USAGE: ./digiclock &
# PURPOSE: Displays time and date in the top right corner of the
# screen using ANSI escape sequences.
# To stop this digclock use command kill pid.
################################################################
#################### VARIABLE DECLARATION ######################
# To place the clock on the appropriate column, subtract the
# length of $Time and $Date, which is 22, from the total number
# of columns
Columns=$(tput cols)
Startpoint=$(($Columns-22))
# If you're in an X Window System terminal,you can resize the
# window, and the clock will adjust its position because it is
# displayed at the last column minus 22 characters.
########################### MAIN PROGRAM #######################
# The script is executed inside a while without conditions.
while :
do
Time=`date +%r`
Date=`date +"%d-%m-%Y"`
echo -en "\033[s" #save current screen position & attributes
tput cup 0 $Startpoint
# You can also use bellow one liner.
# tput cup 0 $((`tput cols`-22))
# But it is not efficient to calculate cursor position for each
# iteration. That's why I placed variable assignment before
# beginning of the loop
# print time and date in the top right corner of the screen
echo -en "\033[42m$Time \033[46m$Date\033[0m"
#restore current screen position & attributes
echo -e -n "\033[u"
#Delay for 1 second
sleep 1
done
Save the script as digclock.sh,change permissions to 755 using chmod,
and run it with ./digclock.sh & or . digclock.sh & or sh digclock &.
The time and date should now appear at the top right of your screen.
Output:
When you run digclock.sh, the terminal will return the job number and
process identifier (PID) of the digclock.sh process. From above output
you can find job number is "1" and PID is "15800".
You can end the execution of the script by two ways:
1. Using the kill command and specifying the job number or process ID.
If you don't remember job number or PID, you can get job number by
running jobs command and PID by ps command.
$ jobs
[1]+ Running ./digclock.sh &
$ shell]# ps | grep digclock
15800 pts/1 00:00:00 digclock.sh
To kill this job/process, either kill %1 or kill 15800 works.
$ kill %1
$
[1]+ Terminated ./digclock.sh
or
$ kill 15800
$
[1]+ Terminated ./digclock.sh
2. Using the fg command. The fg command switches a job running in the
background into the foreground, Then press Ctrl+c to terminate the
job. If no job number is specified, then fg command acts upon the
currently running job.
$ fg 1
./digclock.sh
Ctrl+c
$
With this script, you can display not only a clock, but other useful
information as well. For example,monitoring free space with df command
or CPU's load average with uptime command. Samba, Apache, and many
other servers have status commands where you can extract pieces of
information to show this way.
Sunday, April 18, 2010
12 comments | 8:19 AM
Using ANSI escape sequences :
ANSI escape sequences or tput allow you to move the cursor around the
screen at will. This is more useful for full screen user interfaces
generated by shell scripts, but can also be used in prompts.
The movement escape sequences are as follows:
- Position the Cursor:
\033[<L>;<C>H
Or
\033[<L>;<C>f
puts the cursor at line L and column C.
- Move the cursor up N lines:
\033[<N>A
- Move the cursor down N lines:
\033[<N>B
- Move the cursor forward N columns:
\033[<N>C
- Move the cursor backward N columns:
\033[<N>D
- Clear the screen, move to (0,0):
\033[2J
- Erase to end of line:
\033[K
- Save cursor position:
\033[s
- Restore cursor position:
\033[u
The latter two codes are NOT honored by many terminal emulators. The
only ones that I'm aware of that do are xterm and nxterm - even though
the majority of terminal emulators are based on xterm code. As far as
I can tell, rxvt, kvt, xiterm, and Eterm do not support them. They are
supported on the console.
$ echo -en "\033[s\033[7B\033[1;34m BASH BASH\033[u\033[0m"
Above line saves the current cursor position(\033[s), then move the
cursor seven lines down the screen(\033[7B),print the word "BASH BASH"
in dark blue color(\033[1;34m), and then return to where it started to
produce a normal prompt(\033[u), and also back to the normal color
(\033[0m).
Example:
$ echo -en "\033[s\033[7B\033[1;34m BASH BASH\033[u\033[0m"
$
BASH BASH
Using tput command:
As with so many things in Unix, there is more than one way to achieve
the same ends.A utility called tput can also be used to move the cursor
around the screen, get back information about the status of the termi-
nal, or set colors.
man tput doesn't go into much detail about the available commands,but
man terminfo will give you a huge list of capabilities, many of which
are device independent, and therefore better than the escape sequences
previously mentioned.
Here is some useful tput capabilities:
tput Cursor Movement Capabilities:
tput cup Y X
Move cursor to screen location X,Y (top left is 0,0)
tput sc
Save the cursor position
tput rc
Restore the cursor position
tput lines
Output the number of lines of the terminal
tput cols
Output the number of columns of the terminal
tput cub N
Move N characters left
tput cuf N
Move N characters right
tput cuu N
up N lines
tput cud N
down N lines
tput Colour Capabilities :
tput setab [1-7]
Set a background colour using ANSI escape
tput setb [1-7]
Set a background colour
tput setaf [1-7]
Set a foreground colour using ANSI escape
tput setf [1-7]
Set a foreground colour
tput Text Mode Capabilities:
tput bold
Set bold mode
tput dim
turn on half-bright mode
tput smul
begin underline mode
tput rmul
exit underline mode
tput rev
Turn on reverse mode
tput smso
Enter standout mode (bold on rxvt)
tput rmso
Exit standout mode
tput sgr0
Turn off all attributes (doesn't work quite as expected).
tput Clear and Insert Capabilities :
tput ech N
Erase N characters
tput clear
clear screen and home cursor
tput el1
Clear to beginning of line
tput el
clear to end of line
tput ed
clear to end of screen
tput ich N
insert N characters (moves rest of line forward!)
tput il N
insert N lines
This is by no means a complete list of what terminfo and tput allow,
in fact it's only the beginning. man tput and man terminfo if you want
to know more.
Example:
$ tput sc;tput cup 4 35;tput setaf 2;echo "Hello World";\
> tput rc;tput sgr0
$
Hello World
Bellow example shows two different ways to achieve same task:
$ tput sc;tput cud 7;tput setaf 4;echo BASH BASH; tput rc;\
> tput sgr0
$ echo -en "\033[s\033[7B\033[0;34m BASH BASH\033[u\033[0m"
Sunday, March 28, 2010
6 comments | 11:26 AM
To understand this article you should to be familiar with Linux adva-
nced file permissions, otherwise go throw bellow link before following
this article.
Advanced File Permissions
My Obejective:
I have a folder which I want to share with "rw" permissions for a
selected group of users. Let's say the folder is /home/project and I
want to share it with the group development. What I want is not only
having users accessing files in /home/project with rw access, but also
to ensure that all files created in /home/project will have ownership
username:development and permissions -rw-rw-r--.
Sharing a directory among users in same group is one of the essential
tasks.To let a group of users work on a set of files without infringing
on security, you'll have to do this:
1.Create a common group for these users in /etc/group
# groupadd development
check group created or not
# tail -1 /etc/group
development:x:501:
2.Add user project administrator (padmin) and setup password
# useradd -g development -d /home/project -c "Project Admin" \
-m padmin
# tail -1 /etc/passwd
padmin:x:501:501:Project Admin:/home/project:/bin/bash
#passwd padmin
3.Create separate user accounts for the rest of users but specify the
same home directory.
# useradd -d /home/project/ -g development user1
# passwd user1
Add another user:
# useradd -d /home/project/ -g development user2
# passwd user2
Create as many user accounts as you want.
4.Make sure the home directory and all subdirectories are not owned by
any of the users. Use chown to surrender ownership to padmin.
# chown padmin:development /home/project/
# ls -ld /home/project/
drwxrwxr-x 18 padmin development 4096 Mar 28 16:18 /home/project/
5.Make the directories group-writable and set their SGID and Sticky
Bits with chmod 3775 (1 for sticky and 2 for SGID).
# chmod -R 3775 /home/project/
# ls -ld /home/project/
drwxrwsr-t 18 padmin development 4096 Mar 28 18:22 /home/project/
In this scenario, every user of the group has write permission on the
directory and can create files and directories, but can only delete
those he owns. SGID bit ensures that all files created in
/home/project will have ownership username:development and Sticky bit
ensures that only owner can delete files those he owns
Note that setting the SGID permission on a directory only affects the
groupID of new files and subdirectories created after the SGID bit is
set, and is not applied to existing entities. Setting the setgid bit
on existing subdirectories must be done manually.
Can You Inherit File Permissions?
When you create a file or directories under a directory the default
permission for them will be determined by your umask, files or
directories won't inherit parent directory permissions, only SGID bit
inherited by newly created directories under it. So even your shared
directory has group writable, you can't edit other users files.
Login as user1 and create a temp file.
# su – user1
$ touch temp ; ls -l temp
-rw-r--r-- 1 user1 development 0 Mar 28 18:54 temp
Now logins as user2 and try to edit temp file.
# su - user2
$ cat > temp
-bash: temp: Permission denied
There is no way to inherit permissions from a directory, it's contro-
lled by the process's umask. But there is a way to make file permissi-
ons group writable when it is created.Add umask 002 command to .bashrc
file if it exist, otherwise create it and add the command.
# su - padmin
$ ls -l .bashrc
-rwxrwsr-t 1 padmin development 124 Mar 28 13:05 .bashrc
$ cat >> .bashrc
umask 002
Ctrl+d
Now login as user1 and create a temp file.
# su - user1
$ touch temp ; ls -l temp
-rw-rw-r-- 1 user1 development 0 Mar 28 19:38 temp
File created with default group writable permissions.
Note: Inform already logged in users to logout and login again.
Have you thought about using ACLs? They will give you much finer
grained control over the permissions you can set on files and directo-
ries. ACLs will also allow you to set a default mask for any given
directory.
To know more about ACLs(Access Control Lists) Google it as Linux acls
or wait for my next article.
Saturday, March 27, 2010
84 comments | 10:51 AM
After you have worked for a while with Linux you discover probably
that there is much more to file permissions than just the "rwx" bits.
When you look around in your file system you will see "s" and "t"
$ ls -ld /tmp
drwxrwxrwt 29 root root 36864 Mar 21 19:49 /tmp
$ which passwd
/usr/bin/passwd
$ ls -l /usr/bin/passwd
-rwsr-xr-x 1 root root 22984 Jan 6 2007 /usr/bin/passwd
What is this "s" and "t" bit? The vector of permission bits is really
4 * 3 bits long. Yes there are 12 permission bits,not just 9.The first
three bits are special and are frequently zero. And you almost always
learn about the trailing 9 bits first.Some people stop there and never
learn those first three bits.
The forth permission bit is used only when a special mode of a file
needs to be set. It has the value 4 for SUID, 2 for SGID and 1 for the
sticky bit. The other 3 bits have their usual significance.
Here we will discuss about the 3 special attributes other than the
common read/write/execute:
1.Set-User-Id (SUID)
2.Set-Group-Id (SGID)
3.Sticky Bit
Set-User_Id (SUID): Power for a Moment:
By default, when a user executes a file, the process which results in
this execution has the same permissions as those of the user. In fact,
the process inherits his default group and user identification.
If you set the SUID attribute on an executable file, the process res-
ulting in its execution doesn't use the user's identification but the
user identification of the file owner.
The SUID mechanism,invented by Dennis Ritchie,is a potential security
hazard. It lets a user acquire hidden powers by running such a file
owned by root.
$ ls -l /etc/passwd /etc/shadow /usr/bin/passwd
-rw-r--r-- 1 root root 2232 Mar 15 00:26 /etc/passwd
-r-------- 1 root root 1447 Mar 19 19:01 /etc/shadow
The listing shows that passwd is readable by all, but shadow is unre-
adable by group and others. When a user running the program belongs to
one of these two categories (probably, others), so access fails in the
read test on shadow. suppose normal user wants to change his password,
How can he do that? He can do that by running /usr/bin/passwd. Many
UNIX/Linux programs have a special permission mode that lets users
update sensitive system files –like /etc/shadow --something they can't
do directly with an editor. This is true of the passwd program.
$ ls -l /usr/bin/passwd
-rwsr-xr-x 1 root root 22984 Jan 6 2007 /usr/bin/passwd
The s letter in the user category of the permission field represents a
special mode known as the set-user-id (SUID). This mode lets a process
have the privileges of the owner of the file during the instance of
the program. Thus when a non privileged user executes passwd, the eff-
ective UID of the process is not the user's, but of root's – the owner
of the program. This SUID privilege is then used by passwd to edit
/etc/shadow.
What is effective user-id:
Every process really has two user IDs: the effective user ID and the
real user ID. (Of course, there's also an effective group ID and real
group ID.Just about everything that's true about user IDs is also true
about group IDs) Most of the time,the kernel checks only the effective
user ID. For example, if a process tries to open a file, the kernel
checks the effective user ID when deciding whether to let the process
access the file.
Save the following script under the name reids.pl and make it
executable (chmod 755 reids.pl).
#!/usr/bin/perl
# print real UID
print "Real UID: $<\n";
# print real GID
print "Real GID: $(\n";
# print effective UID
print "Effective UID: $>\n";
# print effective GID
print "Effective GID: $)\n";
check file permissions:
$ ls -l reids.pl
-rwxr-xr-x 1 venu venu 203 Mar 24 10:40 reids.pl
Note: For security reasons the s-bit works only when used on binaries
(compiled code) and not on scripts (an exception are perl scripts).
Scripts,i.e. programs that cannot be executed by the kernel directory
but need an interpreter such as the Bourne shell or Java,can have
their setuid bit set, but it doesn't have any effect. There are some
platforms that honor the s bits even on scripts ( some System V vari-
ants, for example), but most systems don't because it has proven such
a security headache - most interpreters simply aren't written with
much security in mind. Set the SUID bit on shell script is useless,
that's why I am using perl script here.
When you run the script you will see that the process that runs it
gets your user-ID and your group-ID:
$ ./reids.pl
Real UID: 500
Real GID: 500 500
Effective UID: 500
Effective GID: 500 500
Note: If you get an error like this:
Can't do setuid (cannot exec sperl)
In Debian install perl-suid using following command:
apt-get install perl-suid
In Centos install perl-suidperl using following command:
yum install perl-suidperl
Now change owner ship to another user (Do it as an administrator).
# chown king /home/venu/reids.pl
# ls -l /home/venu/reids.pl
-rwxr-xr-x 1 king venu 203 Mar 24 10:40 /home/venu/reids.pl
Now run the script again.
$ ./reids.pl
Real UID: 500
Real GID: 500 500
Effective UID: 500
Effective GID: 500 500
What you observed, the output of the program depends only on the user
that runs it and not the one who owns the file.
How to assign SUID permission:
The SUID for any file can be set (mostly by the superuser) with a
special syntax of the chmod command. This syntax uses the character s
as the permission. Now add SUID permission to the script reids.pl :
# chmod u+s /home/venu/reids.pl (Do it from root account)
Now return from the super user mode to the usual non privileged mode.
$ ls -l reids.pl
-rwsr-xr-x 1 king venu 203 Mar 24 10:40 reids.pl
To assign SUID in an absolute manner, simply prefix 4 to whatever
octal string you would otherwise use (like 4755 instead of 755).
The file reids.pl is owned by king and has the s-bit set where norma-
lly the x is for the owner of the file. This causes the file to be
executed under the user-ID of the user that owns the file rather than
the user that executes the file. If venu runs the program then this
looks as follows:
$ perl reids.pl
Real UID: 500
Real GID: 500 500
Effective UID: 503
Effective GID: 500 500
Effective user id of process is 503, this is not the venu's , but of
king's - the owner of the program. As you can see this is a very powe-
rful feature especially if root owns the file with s-bit set. Any user
can then do things that normally only root can do.
Caution: When you write a SUID program then you must make sure that
it can only be used for the purpose that you intended it to be used.
As administrator, you must keep track of all SUID programs owned by
root that a user may try to create or copy. The find command easily
locate them:
# find /home -perm -4000 -print | mail root
The extra octal bit (4) signifies the SUID mode, but find treats the
"–" before 4000 as representing any other permissions.
Set-Group_Id (SGID):
The set-group-id (SGID) is similar to SUID except that a program with
SGID set allows the user to have the same power as the group which
owns the program. The SGID bit is 2,and some typical examples could be
chmod g+s reids.pl or chmod 2755 reids.pl.
You can remove SGID bit using following commands:
$ chmod g-s reids.pl
$ chmod 755 reids.pl (Absolute manner)
It is really useful in case you have a real multi-user setup where
users access each others files. As a single homeuser I haven't really
found a lot of use for SGID. But the basic concept is the same as the
SUID,Similar to SUID, SGID also grants privileges and access rights to
the process running the command, but instead of receiving those of the
file's owner it receives those of the file's group. In other words,the
process group owner will be set to the file's group.
I explain it with an example. I have created two user accounts king
and venu with same home directory project. king belongs to king and
development groups, venu belongs to venu and development groups.
# groups king venu
king : king development
venu : venu development
venu's default group is venu and king's default group is king.
Login as king and create reids.pl file again and make it executable
(using chmod 755 reids.pl) .
$ id
uid=503(king) gid=503(king) groups=501(development),503(king)
$ ls -l reids.pl
-rwxr-xr-x 1 king development 203 Mar 25 19:00 reids.pl
Now login as venu and run the program:
$ id
uid=501(venu) gid=504(venu) groups=501(development),504(venu)
$ perl reids.pl
Real UID: 501
Real GID: 504 504 501
Effective UID: 501
Effective GID: 504 504 501
The effective GID of the process is the venu's,but not of the king's
-the owner of the program.
Now login as king and assign SGID bit to reids.pl program:
$ chmod 2755 reids.pl; ls -l reids.pl
-rwxr-sr-x 1 king development 203 Mar 25 19:00 reids.pl
Now login as venu and run the reids.pl program:
$ perl reids.pl
Real UID: 501
Real GID: 504 504 501
Effective UID: 501
Effective GID: 501 504 501
Real GID and Effective GID are different,here Effective GID is the
king's - the owner of the program.
Set SGID on a directory:
When SGID is set on a directory it has a special meaning. Files crea-
ted in a directory with SGID set will inherit the same group ownership
as the directory itself,not the group of the user who created the file.
If the SGID is not set the file's group ownership corresponds to the
user's default group.
In order to set the SGID on a directory or to remove it, use the
following commands:
$ chmod g+s directory or $ chmod 2755 directory
$ chmod g-s directory or $ chmod 755 directory
As I mentioned earlier venu and king's home directory is same that is
/home/project. I changed group ownership of /home/project directory
to development.
# ls -ld /home/project/
drwxrwxr-x 16 root development 4096 Mar 26 00:22 /home/project/
Now login as king and create a temp file.
$ whoami
king
$ pwd
/home/project/
$ touch temp; ls -l temp
-rw-r--r-- 1 king king 0 Mar 26 12:34 temp
You can see from the ls output that the group owner for project is
development, and that the SGID bit has not been set on the directory
yet. When king creates a file in project, the group for the file is
king (king's primary gid).
Set SGID bit on project directory. For that login as administrator
and set SGID bit using following command:
# chmod g+s /home/project/
# ls -ld /home/project/
drwxrwsr-x 15 root development 4096 Mar 26 12:34 /home/project/
From the ls output above, you know the SGID bit is set because of the
s in the third position of the group permission set,which replaces the
x in the group permissions.
Now login as king and create temp2 file.
$ whoami
king
$ touch temp2; ls -l temp2
-rw-r--r-- 1 king development 0 Mar 26 13:49 temp2
Notice the group ownership for temp2 file. It inherits group permiss-
ion from the parent directory.
Enabling SGID on a directory is extremely useful when you have a
group of users with different primary groups working on the same set
of files.
For system security reasons it is not a good idea to set many
program's set user or group ID bits any more than necessary,since this
can allow an unauthorized user privileges in sensitive system areas.If
the program has a flaw that allows the user to break out of the inten-
ded use of the program, then the system can be compromised.
Sticky bit:
The sticky bit(also called the saved text bit) is the last permission
bit remaining to be discussed. It applies to both regular files and
directories. When applied to a regular file, it ensures that the text
image of a program with the bit set is permanently kept in the swap
area so that it can be reloaded quickly when the program's turn to use
the CPU arrives. Previously, it made sense to have this bit set for
programs like vi and emacs. Today,machines with ultra-fast disk drives
and lots of cheap memory don't need this bit for ordinary files and
that is also useless.
However, the sticky bit become a useful security feature when used
with a directory. The UNIX/Linux system allows users to create files
in /tmp, but none can delete files not owned by him. That's possible
because sticky bit set for /tmp directory.
The /tmp directory is typically world-writable and looks like this
in a listing:
# ls -ld /tmp
drwxrwxrwt 32 root root 36864 Mar 27 12:38 /tmp
Everyone can read,write and access the directory.The t indicates that
only the user (root and owner of the directory,of course) that created
a file in this directory can delete that file.
In order to set or to remove the sticky bit, use the following
commands:
$ chmod +t directory or $ chmod 1754 directory
$ chmod -t directory or $ chmod 754 directory
Note: 754 permissions for a directory are powerful enough to guard
your directories from intruders with malicious intentions, that's why
I used 754 as default,if yow want you can change it.
Example:
I logged in as king and created a temp file.
$ whoami
king
$ pwd
/home/project/
$ touch temp; ls -l
-rw-r--r-- 1 king king 0 Mar 27 13:44 temp
Now logged in as venu and try to delete temp file.
$ whoami
venu
$ rm temp
rm: remove write-protected regular empty file `temp'? Y
$ ls temp
ls: temp: No such file or directory
So what happened? venu deleted file owned by king.
Assign sticky bit to the project directory.As a owner of the directory
or administrator.
# chmod +t /home/project
# ls -ld /home/project/
drwxrwxr-t 15 root development 4096 Mar 27 13:46 /home/project/
From the ls output above, you know the sticky bit is set because of
the t in the third position of the other permission set,which replaces
the x in the other permissions.
Now repeat same steps again,then you get the following message:
$ whoami
venu
$ ls -l temp
-rw-r--r-- 1 king king 0 Mar 27 17:36 temp
$ rm temp
rm: remove write-protected regular empty file `temp'? y
rm: cannot remove `temp': Operation not permitted
Observation: Login as normal user and create a file.
[venu@localhost ~]$ touch sample
[venu@localhost ~]$ ls -l sample
-rw-rw-r-- 1 venu venu 0 Dec 21 03:41 sample
Now change permissions to 644
[venu@localhost ~]$ chmod 644 sample
[venu@localhost ~]$ ls -l sample
-rw-r--r-- 1 venu venu 0 Dec 21 03:41 sample
Now assign SUID permission.
[venu@localhost ~]$ chmod u+s sample
[venu@localhost ~]$ ls -l sample
-rwSr--r-- 1 venu venu 0 Dec 21 03:41 sample
After setting SUID, if you see 'S' then it means that the file has no
executable permissions for that user.
Now remove SUID permission and change permissions to 744. Then assign
SUID permission. You should see a smaller 's' in the executable permi-
ssion position.
[venu@localhost ~]$ chmod u-s sample
[venu@localhost ~]$ chmod 744 sample
[venu@localhost ~]$ chmod u+s sample
[venu@localhost ~]$ ls -l sample
-rwsr--r-- 1 venu venu 0 Dec 21 03:41 sample
Same is applicable for SGID and Stickybit.
Subscribe to:
Posts (Atom)
