| | | | |
You can put any sequence of Unix commands into a file and turn that file into a command. Such a file is called a script. For example, suppose that you are working on a program myprog and have several files of test data that you run through it each time you make a change. Create a file dotest1 with the following lines:
Most shells provide special facilities for use in scripts. Since these differ from one shell to another, it's a good idea to tell Unix which shell to use when running the script. You do this by placing the command #!/bin/csh in the first line of the script.
In fact, you can list any program there, not just /bin/csh, and Unix will use that program to process the remainder of the lines in the script.
We can pass parameters to shell scripts from the command line. For example, suppose we wanted a script to execute a single test like this:
In csh, we use the symbol $k to stand for the kth argument given to the script. So we can write our script dotest2, as follows:
Of course, scripts can have more than one parameter. For example, try saving this file as mcdonald
Shells feature control flow based on “status codes” returned by programs to indicate whether the program execution “succeeded” or “failed” . For example, this script tests to see if the gcc compiler has successfully compiled a file.
| csh, tcsh | sh, bash |
|---|---|
|
if ( { gcc -c expr.c } ) then
echo All is well.
else
echo Did not compile!
endif
|
if gcc -c expr.c;
then
echo All is well.
else
echo Did not compile!
fi
|
In sh (and its relatives bsh and bash), the if is followed by a list of commands, each terminated by a ';'. The status code of the last command is used as the if condition. In csh (and its relative, tcsh), the condition must be placed inside parentheses. To execute a command as part of the condition, place that inside {} brackets (with white space to either side of each bracket).
This idea of returning a status code explains why, in C and C++, the function main is always supposed to return an int. The returned value is the status code. You are supposed to return a zero when your program works normally, but you are supposed to return a non-zero “error code” if your program terminates abnormally (e.g., if it discovers that a file it needs is missing).
The shell if statements will take the “then” branch when a command returns a status code of 0, but any non-zero value denotes a failure and will cause the script to follow the “else” branch. Most Unix commands will list the status codes they return on their man pages53
The pages you get when running the command
Looping is also available in the shells. One of the most commonly used loop forms is that of visiting every item in some list. The list is often all files satisfying some wildcard pattern:
| csh, tcsh | sh, bash |
|---|---|
foreach file (*.txt)
echo $file
end
|
for file in *.txt
do
echo $file
done
|
Another common kind of loop visits every parameter passed to the script
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
foreach file ($*)
echo $file
end
|
#!/bin/sh
for x in $*
do
echo $x
done
|
Another common scripting pattern is a script that has few special parameters at the beginning, followed by an arbitrary number of remaining parameters (often filenames). The shift command helps us to handle these. Each call to shift removes he first element of the $* list. These scripts, stored as testp2:
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set p1=$1
shift
set p2=$1
shift
echo The first two parameters are\
$p1 and $p2
echo The remaining parameters are
foreach file ($*)
echo $file
end
|
#!/bin/sh
p1=$1
shift
p2=$1
shift
echo The first two parameters are\
$p1 and $p2
echo The remaining parameters are
for x in $*
do
echo $x
done
|
A while loop is also available.
| csh, tcsh | sh, bash |
|---|---|
while ( condition )
commands
end
|
while condition;
do
commands
done
|
One thing that becomes obvious quickly is that the status code based testing is limited. Often we want to
The solution is to use a program whose job is to
| csh, tcsh | sh, bash |
|---|---|
gcc -c expr.c
if (-r expr.o) then
echo All is well.
else
echo Did not compile!
endif
|
gcc -c expr.c
if test -r expr.o;
then
echo All is well.
else
echo Did not compile!
fi
|
test takes a bewildering variety of possible parameters. You can see the whole list by giveng the command man test. Many of these are used for checking the status of various files. The -r expr.o in the above shells checks to see if a file named expr.o exists and if we have permission to read it. Some common file tests are:
| test | is true if |
|---|---|
| -r file | file exists and is readable |
| -w file | file exists and is writable |
| -x file | file exists and is executable |
| -d file | file exists and is a directory |
Strings are compared with =~ and !~ in csh, = and != in sh.
| csh, tcsh | sh, bash |
|---|---|
if ( $USER =~ zeil ) then
echo Nice guy!
else
echo Who are you?
endif
|
if test $USER = zeil;
then
echo Nice guy!
else
echo Who are you?
fi
|
Numbers are compared with the conventional relational operators (== != < > <= >=) in csh, but have more clumsy equivalents in sh (-eq -ne -lt -gt -le -ge):
| csh, tcsh | sh, bash |
|---|---|
if ( $count == 0 ) then
echo zero
else
echo non-zero
endif
|
if test $count -eq 0;
then
echo zero
else
echo non-zero
fi
|
But where do numbers come from if all the variables contain strings?
From yet another program, of course. The shells themselves have no built-in numeric capability. Calculations can be performed by the expr program. This program treats its arguments as an arithmetic expression, evaluates that expression, and prints the result on standard output. For example:
Now, how do we get the output of an expr program evaluation into a variable or a script expression where it can do some good? For this we use the convention that the backwards apostrophes, `, when used to quote a string, mean “execute this string as a shell command and put its standard output right here in this command” .
For example:
With these two ideas, we can now do numerical calculations in our scripts:
| csh, tcsh | sh, bash |
|---|---|
set count=0
foreach file (*)
set count=`expr $count + 1`
end
echo There are $count files\
in this directory.
|
count=0
for file in *
do
count=`expr $count + 1`
done
echo There are $count files\
in this directory.
|
As an example of how to bring all those scripting details together, let's look at some scripts to aid in testing programs. Many programming students wind up adopting a hit-and-miss approach to testing their code, in part because they don't set themselves up with an easy way to repeat a large number of tests every time they make a change to their programs.
What we'd like to end up with is a simple system for testing non-GUI programs. The idea is that the student designs a number of tests and can then issue a command to run all or a selected subset of those tests. The command should run those tests, capturing the program outputs in files that the student can examine later. Furthermore, we can save the student a bit of time by letting him or her know if the program output has changed on any of those tests.
So, suppose the student is working on a program named myProg and has designed 20 test cases. Once the program compiles successfully, the student could say
Encouraged, the student launches the whole test set once again
As we have envisioned it, our runTests script takes three parameters:
So we start our script by gathering those three parameters:
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set programName=$1
set firstTest=$2
set lastTest=$3
|
#!/bin/sh
programName=$1
firstTest=$2
lastTest=$3
|
Clearly, the main control flow here will be a loop going through the requested test numbers.
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set programName=$1
set firstTest=$2
set lastTest=$3
#
# Loop through all tests
set testNum=$firstTest
while ($testNum <= $lastTest)
|
#!/bin/sh
programName=$1
firstTest=$2
lastTest=$3
#
# Loop through all tests
testNum=$firstTest
while test $testNum -le $lastTest;
do
|
For each test, we will eventually want to compare the output from this test, stored in testOuti.txt to the output from the previous test, which we will assume is stored in testOuti.old.txt. The cmp command lets us compare two files to see if their contents are identical.
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set programName=$1
set firstTest=$2
set lastTest=$3
#
# Loop through all tests
set testNum=$firstTest
while ($testNum <= $lastTest)
|
#!/bin/sh
programName=$1
firstTest=$2
lastTest=$3
#
# Loop through all tests
testNum=$firstTest
while test $testNum -le $lastTest;
do
|
To set this up, we must determine just where the .old.txt files come from. They are simply the previous version of the test output files (if that particular test has ever been run).
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set programName=$1
set firstTest=$2
set lastTest=$3
#
# Loop through all tests
set testNum=$firstTest
while ($testNum <= $lastTest)
# Save the previous output from this test
if (-r testOut$testNum.txt) then
/bin/mv testOut$testNum.txt testOut$testNum.old.txt
endif
|
#!/bin/sh
programName=$1
firstTest=$2
lastTest=$3
#
# Loop through all tests
testNum=$firstTest
while test $testNum -le $lastTest;
do
# Save the previous output from this test
if test -r testOut$testNum.txt;
then
/bin/mv testOut$testNum.txt testOut$testNum.old.txt
fi
|
Finally, we come to the heart of the matter. We need to actually execute the program, saving the output in the appropriate testOut… file. Exactly how we want to execute the program depends upon how the program gets its input data. I'm going to assume, for the moment, that this program reads its input data from the standard input stream, and that the student saves the input test cases in testIn1.txt, testIn2.txt, ….
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set programName=$1
set firstTest=$2
set lastTest=$3
#
# Loop through all tests
set testNum=$firstTest
while ($testNum <= $lastTest)
#
# Save the previous output from this test
if (-r testOut$testNum.txt) then
/bin/mv testOut$testNum.txt testOut$testNum.old.txt
endif
# Run the test!
echo Starting test $testNum...
$programName < testIn$testNum.txt >& testOut$testNum.txt
#
# Has the output changed?
if (-r testOut$testNum.old.txt) then
if ( ! { cmp testOut$testNum.old.txt testOut$testNum.txt } ) then
echo \*\* Test $testNum output has changed
endif
endif
set testNum=`expr $testNum + 1`
end
|
#!/bin/sh
programName=$1
firstTest=$2
lastTest=$3
#
# Loop through all tests
testNum=$firstTest
while test $testNum -le $lastTest;
do
#
# Save the previous output from this test
if test -r testOut$testNum.txt;
then
/bin/mv testOut$testNum.txt testOut$testNum.old.txt
fi
# Run the test!
echo Starting test $testNum...
$programName < testIn$testNum.txt 2>&1 > testOut$testNum.txt
#
# Has the output changed?
if test -r testOut$testNum.old.txt;
then
if cmp testOut$testNum.old.txt testOut$testNum.txt;
then
donothing=0
else
echo \*\* Test $testNum output has changed
fi
fi
testNum=`expr $testNum + 1`
done
|
By making minor changes to the way the program is run, we can accommodate a number of different possible program styles. How would you change these scripts for a program that read no inputs at all, but could be invoked with different command-line parameters?
There are a number of possibilities, but I would put the various parameters into the testIn… files, and run them this way:
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
|
#!/bin/sh
|
Scripting is just another form of programming. Just like programs that you write in C++ or other “traditional” programming languages, you need to test your scripts to see if they work, and, just like those other programs, your scripts are probably not going to work correctly on the first try.
Debugging your scripts is not all that different from debugging traditional programs, either. Many of the basic techniques are similar:
If you have a script that does several things in succession, try to determine how far along that sequence you have been getting before things go wrong. The whole reasoning process that we call “debugging” is a lot easier if you know where to look.
You can try to do this by examining any intermediate results to see if they are correct. In particular, if your script produces some temporary or working files, look at those. If your script rms those files when it's done with them, try commenting out the rm statements (put a # in front of them) until you're sure everything is working. Another way to examine intermediate results is by adding debugging output to print them out (see below).
If that doesn't work, you can try “shortening” your script by commenting out the later steps and seeing if the partial script seems to work, then uncommenting the next step and testing the script again, and so on.
Just as you might add extra output statements to a C++ program to reveal the values of selected importatnt variables, you can do the same thing with scripts. The echo command is used for this purpose. For example, if we were having problems with this script:
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set count=0
foreach file (*)
set count=`expr $count + 1`
end
echo There are $count files\
in this directory.
|
#!/bin/sh
count=0
for file in *
do
count=`expr $count + 1`
done
echo There are $count files\
in this directory.
|
| csh, tcsh | sh, bash |
|---|---|
#!/bin/csh
set count=0
foreach file (*)
echo Looking at $file.
set count=`expr $count + 1`
echo Count is $count.
end
echo There are $count files\
in this directory.
|
#!/bin/sh
count=0
for file in *
do
echo Looking at $file.
count=`expr $count + 1`
echo Count is $count.
done
echo There are $count files\
in this directory.
|
Because echo can print pretty much anything you give it as an argument, it's particularly useful when you have a script that runs other programs and suspect that your script may not be invoking those programs correctly. For example, if you have a script huntFor:
You might find it valuable to actually create the above script, storing it in a file named huntFor, and try testing it like this:
The use of echo to print out entire commands does carry a bit of a risk if the commands involve output redirection or pipes. For example, if the original command is redirecting its output into a file:
In cases like that, you can obtain the same information more easily by our next technique, tracing the execution.
If you were working with a C++ program, you could use gdb or ddd to step through the program and see exactly which statements were being executed, in which order.
Shells provide a more primitive, but still useful, tracing facility. The “-x” option, supplied to sh or csh, asks it to list each command before it executes it (and after replacing any variables by their values).
So, if you have written a shell script named “myScript” with the first line:
| | | | |