Difference between pages "Set up a Webcam with Linux" and "BaBE - Bash By Examples"

From Linuxintro
(Difference between pages)
 
 
Line 1: Line 1:
<metadesc>Here is how you set up your webcam with Linux, capture videos and images, do VoIP calls and automate recording. Tested with SUSE Linux, Ubuntu and Debian.</metadesc>
+
<metadesc>BaBE - Bash by Examples; Your significant Linux scripting tutorial: Redirection, Piping, conditional execution, user dialogs, loops, process management, backticks, string replacement, pitfalls and much more.</metadesc>
[[File:Set-up-a-webcam.png]]
 
You want to set up your webcam with Linux, see a video stream from it and learn which applications you can use with it, right? Then this article is for you.
 
  
= Try it =
+
BaBE - Bash By Examples; Your significant Linux scripting tutorial;;
First let's try if it works out of the box, so, connect your webcam (if it's not inbuilt), [[open a terminal]] and start the application '''''cheese''''':
 
cheese
 
  
This should switch on your webcam and you should see what it's recording.
+
<pic src="http://www.linuxintro.org/images/Bash-scripting-mindmap.jpg" width=70% align=right caption="MindMap of what you can learn about Linux scripting" />
  
Ok, you are reading on, so there is still something left, maybe:
+
= Hello world =
* terminal says "command not found", you need to install cheese. Read on at [[#Install_cheese|Install Cheese]].
+
The easiest way to get your feet wet with a programming language is to start with a program that simply outputs a trivial text, the so-called hello-world-example. Here it is for bash:
* cheese does not show the camera's content -> read on at [[#Check the Setup|Check the Setup]].
+
* create a file named hello in your home directory with the following content:
* you want to [[#record_a_video]] with your webcam.
+
#!/bin/bash
 +
echo "Hello, world!"
 +
* [[open a console]], make the file executable:
 +
chmod +x hello
 +
* now you can execute your file like this:
 +
./hello
 +
Hello, world!
 +
* or like this:
 +
bash hello
 +
Hello, world!
 +
You see - the output of your shell [[program]] is the same as if you had entered the commands into a console.
  
= Install cheese =  
+
= calling commands =
To install cheese, you need to know your distribution. Here is how to [[find out your distribution]].
+
In your shell script you can call every command that you can call when [[opening a console]]:
* for Debian, Raspbian, Ubuntu, Kubuntu, Flubuntu, Xubuntu, GEUbuntu, Edubuntu, ...:
+
echo "This is a directory listing, latest modified files at the bottom:"
  sudo apt-get install cheese
+
  ls -ltr
* for SUSE:
+
echo "Now calling a browser"
  yast -i cheese
+
firefox
 +
  echo "Continuing with the script"
  
= Check the Setup =
+
= variables =
If cheese starts, but does not find a webcam, you need to dig deeper.
+
== input ==
* [[open a console]] and list your video4linux devices:
+
To show you how to deal with variables, we will now write a script that asks for your name and greets you:
 +
echo "what is your name? "
 +
read name
 +
echo "hello $name"
 +
 
 +
You see that the name is stored in a variable $name. Note the quotation marks '''"''' around "hello $name". By using these you say that you want variables to be replaced by their content. If you were to use apostrophes, the name would not be printed, but $name instead.
 +
 
 +
== ${} ==
 +
The ${} operator stands for the variable, there is no difference if you write
 +
echo "$name"
 +
or
 +
echo "${name}"
 +
So what is the sense of this? Imagine you want to echo a string directly, without any blank, after the content of a variable:
 +
<source>
 +
echo "if I add the syllable owa to your name it will be ${name}owa"
 +
</source>
 +
 
 +
== common mistakes ==
 +
Note that the variable is called $name, however the correct statement to read it from the keyboard is
 +
read name
 +
It is a common mistake to write
 +
read $name
 +
which means "read a string and store it into the variable whose name is stored in $name"
 +
 
 +
= parameters =
 +
 
 +
echo "Here are all parameters you called this script with: $@"
 +
echo "Here is parameter 1: $1"
 +
echo "Which parameter do you want to be shown? "
 +
read number
 +
args=("$@")
 +
echo "${args[$number-1]}"
 +
 
 +
= return codes =
 +
Every bash script can communicate with the rest of the system by
 +
* sending data to [[stdout]]
 +
* sending data to [[stderr]]
 +
* delivering a return code
 +
The return code is 0 if everything worked well. You can query it for the most recent command using $?:
 +
bootstick@bootstick:~$ echo "hello world"; echo $?
 +
hello world
 +
0
 +
bootstick@bootstick:~$ echo "hello world">/proc/cmdline; echo $?
 +
bash: /proc/cmdline: Permission denied
 +
1
 +
 
 +
In bash, true is 0 and false is any value but 0. There exist two commands, true and false that deliver true or false, respectively:
 +
bootstick@bootstick:~$ true; echo $?
 +
0
 +
bootstick@bootstick:~$ false; echo $?
 +
1
 +
 
 +
= line feeds =
 +
In bash you can use line feeds or semicolons to separate commands. For example:
 +
<source>
 +
read name
 +
if [ $name = "Thorsten" ]; then echo "I know you"; fi
 +
</source>
 +
Instead of a semicolon you can write a line feed like this:
 +
<source>
 +
read name
 +
if [ $name = "Thorsten" ]
 +
  then echo "I know you"
 +
fi
 +
</source>
 +
 
 +
So we could put everything into one line and separate the commands by semicolons:
 +
<source>
 +
read name; if [ $name = "Thorsten" ]; then echo "I know you"; fi
 +
</source>
 +
 
 +
If you want to insert a line feed where you do not need one, e.g. to make the code better readable, just put a backslash at the line's end to indicate it will continue:
 +
<source>
 +
read \
 +
  name
 +
if [ $name = "Thorsten" ]
 +
  then \
 +
    echo "I know you"
 +
fi
 +
</source>
 +
 
 +
= storing a command's output =
 +
To read a command's output into a variable use $(), backticks or [[piping]].
 +
 
 +
== $() ==
 +
arch=$(uname -m)
 +
echo "Your system is a $arch system."
 +
 
 +
== backticks ==
 +
arch=`uname -m`
 +
echo "Your system is a $arch system."
 +
 
 +
== piping ==
 +
[[Piping]] is a very elegant concept in the Linux world. It allows you to take one command's output and use it as input for the next command. Now you can divide tasks into smaller tasks. For example instead of having a program counting all files in a directory you use one program (ls) to ''list'' all files in a directory and one program (wc) to count the lines:
 +
<source>
 +
ls | wc -l
 +
</source>
 +
You can also put the output into a variable, in this case $arch:
 +
<source>
 +
uname -m | while read arch; do echo "Your system is a $arch system."; done
 +
</source>
 +
 
 +
== comparison ==
 +
The advantage of using backticks over $() is that backticks also work in the sh shell. The advantage of using $() over backticks is that you can cascade them. In the example below we use this possibility to get a list of all files installed with rpm on the system:
 +
filelist=$([[rpm]] -ql $(rpm -qa))
 +
You can use the piping approach if you need to cascade in sh, but this is not focus of this bash tutorial.
 +
 
 +
== common mistakes ==
 +
Usually unexperienced programmers try something like
 +
uname -m | read arch
 +
which [http://mywiki.wooledge.org/BashFAQ/024 does not work]. You must embed the read into a while loop.
 +
 
 +
= conditions =
 +
The easiest form of a condition in bash is this '''if''' example:
 +
<source>
 +
echo "what is your name? "
 +
read name
 +
if [ $name = "Thorsten" ]; then echo "I know you"; fi
 +
</source>
 +
Now let's look closer at this, why does it work? Why is there a blank needed behind the [ sign? The answer is that [ is just an ordinary [[command]] in the shell. It delivers a return code for the expression that follows till the ] sign. To prove this we can write a script:
 +
<source>
 +
if true; then echo "the command following if true is being executed"; fi
 +
if false; then echo "this will not be shown"; fi
 +
</source>
 +
 
 +
== empty strings ==
 +
An empty string evaluates to false inside the [ ] operators so it is possible to check if a string ''result'' is empty like this:
 +
# result=
 +
# if [ $result ]; then echo success; fi
 +
# result=good
 +
# if [ $result ]; then echo success; fi
 +
success
 +
 
 +
== arithmetic expressions ==
 +
You can compare integer numbers like this:
 
<source>
 
<source>
  ls -ltr /dev/video*
+
  echo "what is your age? "
 +
read age
 +
if (( $age >= 21 )); then echo "Let's talk about sex."; fi
 
</source>
 
</source>
* you will find an output like this:
+
However bash does not understand floating point numbers. To compare floating numbers you will use external programs such as bc:
 
<source>
 
<source>
  crw-rw----+ 1 root video 81, 0 Nov 11 09:06 /dev/video0
+
  $ if [ $(echo "2.1<2.2"|bc) = 1 ]; then echo "correct"; else echo "wrong"; fi
 +
correct
 +
$ if [ $(echo "2.1>2.2"|bc) = 1 ]; then echo "correct"; else echo "wrong"; fi
 +
wrong
 
</source>
 
</source>
In this example your webcam device is named ''/dev/video0''. If you have no /dev/video file, read [[#Troubleshooting]]. If you have several video4linux devices, for example a [[tv card]] your webcam may show up as /dev/video1 or whatever. But the time (in this case Nov 11 09:06) will be the time when you plugged it in.
 
  
You can also test your webcam the following ways:
+
== not equal ==
* if you have [[vlc]] installed you can start it and choose Media -> Open Capture Device -> Video device name = ''/dev/video0'' -> Play
+
To check if a variable is NOT equal to whatever, use !=:
* if you have mplayer installed you can use the command
 
 
<source>
 
<source>
  mplayer tv:// -tv driver=v4l2:width=640:height=480:device=/dev/video0 -fps 30
+
  if [ "$LANG" != "C" ]; then echo "please set your system lanugage to C"; fi
 
</source>
 
</source>
  
= Troubleshooting =
+
== common mistakes ==
Troubleshooting heavily depends on the distribution and version you are using. If you have done cabling correctly and a device file /dev/video* does not appear, your kernel probably does not know the hardware. In this case you may have to install the device driver separately because it may not be part of the kernel.
+
Common mistakes are:
 +
* to forget the blank behind/before the [ or ] character
 +
* to forget the blank behind/before the equal sign
 +
* see [[what does "unary operator expected" mean]]
 +
 
 +
= Redirections =
 +
To redirect the output of a [[command]] to a file you have to consider that there are two output streams in UNIX, [[stdout,stderr and stdin|stdout and stderr]].
 +
 
 +
= filling files =
 +
To create a file, probably the easiest way is to use [[cat]]. The following example writes text into README till a line occurs that only contains the string "EOF":
 +
cat >README<<EOF
 +
This is line 1
 +
This is line 2
 +
This is the last line
 +
EOF
 +
Afterwards, README will contain the 3 lines below the cat command and above the line with EOF.
 +
 
 +
= loops =
 +
 
 +
== for loops ==
 +
Here is an example for a for-loop. It makes a [[backup]] of all text files:
 +
for i in *.txt; do [[cp]] $i $i.bak; done
 +
The above command takes each .txt file in the current directory, stores it in the [[variable]] $i and copies it to $i.bak. So ''file''.txt gets copied to ''file''.txt.bat.
 +
 
 +
You can also use subsequent numbers as a for loop using the command seq like this:
 +
for i in $([[seq]] 1 1 3); do [[echo]] $i; done
 +
 
 +
== while loops ==
 +
$ while true; do read line; done
  
== SUSE Linux 11.0 and earlier ==
+
= negations =
This has been tested with SUSE Linux 11.0 x64 but should work with any earlier SUSE version. You will need to log in as user root.
+
You can negate a result with the ! operator. $? is the last command's return code:
To find out what driver you need, [[open a console]] and call
+
# true
  [[hwinfo]] --usb
+
  # echo $?
If a Logitech Quickcam Messenger is plugged in the answer will be like:
+
0
<source highlight=23>
+
# false
06: USB 00.2: 0000 Unclassified device
+
# echo $?
  [Created at usb.122]               
+
1
  UDI: /org/freedesktop/Hal/devices/usb_device_46d_8da_noserial_if2
+
# ! true
  Unique ID: Eopr.vE+cdFBwClB                                     
+
# echo $?
  Parent ID: uIhY.uOe2OKugI8D                                     
+
1
  SysFS ID: /devices/pci0000:00/0000:00:1a.2/usb3/3-1/3-1:1.2     
+
# ! false
  SysFS BusID: 3-1:1.2                                           
+
# echo $?
  Hardware Class: unknown                                         
+
0
  Model: "Logitech QuickCam Messanger"                           
+
So you get an endless loop out of:
  Hotplug: USB                                                   
+
while ! false; do echo hallo; done
  Vendor: usb 0x046d "Logitech, Inc."                             
+
 
  Device: usb 0x08da "QuickCam Messanger"                        
+
The following code checks the file /tmp/success to contain "success". As long as this is ''not'' the case it continues checking:
  Revision: "1.00"                                               
+
while ! (grep "success" /tmp/success)
  Driver: "snd-usb-audio"                                         
+
do
  Driver Modules: "snd_usb_audio"                                
+
   sleep 30
  Speed: 12 Mbps                                                 
+
done
  Module Alias: "usb:v046Dp08DAd0100dc00dsc00dp00ic01isc02ip00"    
+
 
  Driver Info #0:                                                
+
The following code checks if the file dblog.log exists. As long as this is not the case it tries to download it via ftp:
    Driver Status: quickcam_messenger is active                   
+
<source>
    Driver Activation Cmd: "modprobe quickcam_messenger"
+
while ! (test -e dblog.log); do
  Driver Info #1:                                                
+
  ftp -p ftp://user:password@server/tmp/dblog.log >/dev/null
    Driver Status: gspca is active                               
+
  echo -en "."
    Driver Activation Cmd: "modprobe gspca"
+
  sleep 1
  Config Status: cfg=new, avail=yes, need=no, active=unknown     
+
done
  Attached to: #20 (Hub)
 
 
</source>
 
</source>
This means you can install and load the webcam driver like this:
 
yast -i gspcav-kmp-default
 
modprobe gspca
 
Now you should see a video device:
 
ls /dev/video*
 
/dev/video  /dev/video0
 
That means you can install and start your webcam-viewer-software. We choose gqcam:
 
yast -i gqcam
 
gqcam
 
It works. You see a video what from what is going on in front of your webcam.
 
  
== Ubuntu ==
+
== common mistakes ==
This has been tested with Ubuntu 8.10 x32 but should work with any Ubuntu version.
+
* bash is very picky regarding spaces. There MUST be a space after the ! if it means negation.
Find out the driver activation command of your webcam. For this, first install the software '''hwinfo'''. [[Open a console]] and type:
 
sudo apt-get [[install]] [[hwinfo]]
 
Then call hwinfo:
 
hwinfo --usb
 
If a Logitech Quickcam Messenger is plugged in the response will be like:
 
<source highlight=20>
 
04: USB 00.2: 0000 Unclassified device
 
  [Created at usb.122]
 
  UDI: /org/freedesktop/Hal/devices/usb_device_46d_8da_noserial_if2
 
  Unique ID: 4ajv.vE+cdFBwClB
 
  Parent ID: k4bc._Mkd+LmXb03
 
  SysFS ID: /devices/pci0000:00/0000:00:11.0/0000:02:00.0/usb1/1-1/1-1:1.2
 
  SysFS BusID: 1-1:1.2
 
  Hardware Class: unknown
 
  Model: "Logitech QuickCam Messanger"
 
  Hotplug: USB
 
  Vendor: usb 0x046d "Logitech, Inc."
 
  Device: usb 0x08da "QuickCam Messanger"
 
  Revision: "1.00"
 
  Driver: "snd-usb-audio"
 
  Driver Modules: "snd_usb_audio"
 
  Speed: 12 Mbps
 
  Module Alias: "usb:v046Dp08DAd0100dc00dsc00dp00ic01isc02ip00"
 
  Driver Info #0:
 
    Driver Status: gspca_zc3xx is active
 
    Driver Activation Cmd: "modprobe gspca_zc3xx"
 
  Config Status: cfg=new, avail=yes, need=no, active=unknown
 
  Attached to: #8 (Hub)</source>
 
Activate the driver:
 
sudo modprobe gspca_zc3xx
 
Now you should be able to see the video device:
 
[[ls]] /dev/video*
 
/dev/video0
 
You can now test your webcam using the software cheese:
 
sudo apt-get install cheese
 
cheese
 
  
== Other webcams ==
+
= sending a process to the background =
If you have another webcam, try the above nevertheless. If it does not work, exchange the driver gspca against uvcvideo:
+
To send a process to the [[background]], use the ampersand sign (&):
  yast -i uvcvideo_kmp_default
+
  firefox & echo "Firefox has been started"
modprobe uvcvideo
+
You see a newline is not needed after the &
and start gqcam again.
 
  
= Use it =
+
= forking a process =
 +
You can build a process chain using parantheses. This is useful if you want to have two instruction streams being executed in parallel:
 +
<source>
 +
(find -iname "helloworld.txt") & (sleep 5; echo "Timeout exceeded, killing process"; kill $!)
 +
</source>
  
== record video ==
+
= functions =
to capture video streams you can use [http://en.wikipedia.org/wiki/Cheese_%28software%29 cheese], a nice program with a graphical user interface. It allows you to watch and record your camera output at the same time. It stores in a strange format (.webm), but [[vlc]] can play this.
+
To define a function in bash, use a non-keyword and append opening and closing parentheses. Here a function greet is defined and it prints "Hello, world!". Then it is called:
 +
# greet()
 +
{
 +
    echo "Hello, world"
 +
}
 +
# greet
  
You can also automate video recording so you can capture the camera stream with sitting in front of the computer. To do this you can
+
If you hand over parameters you can greet any planet you like:
* use the software ''mencoder'':
+
# greet()
 +
{
 +
    echo "Hello, $1"
 +
}
 +
# greet Mars
 +
Hello, Mars
 +
# greet World
 +
Hello, World
 +
 
 +
= react on CTRL_C =
 +
The command trap allows you to trap CTRL_C keystrokes so your script will not be aborted
 
<source>
 
<source>
mencoder tv:// -tv driver=v4l2:width=320:height=240:device=/dev/video0 -nosound -ovc lavc -o myvideo.avi
+
#!/bin/bash
 +
 
 +
trap shelltrap INT
 +
 
 +
shelltrap()
 +
{
 +
    echo "You pressed CTRL_C, but I don't let you escape"
 +
}
 +
 
 +
while true; do read line; done
 
</source>
 
</source>
* or use the software ''streamer'', example:
+
 
 +
;Note: You can still ''pause'' your script by pressing CTRL_Z, send it to the [[background]] and kill it there. To catch CTRL_Z, replace INT by TSTP in the above example. To get an overview of all signals that you might be able to trap, [[open a console]] and enter
 +
kill -l
 +
 
 +
= helpful programs =
 +
== awk: read a specific column ==
 +
[[awk]] is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can extract columns from a text. Let's imagine you want to use the [[program]] [[vmstat]] to find out how high the CPU user load was. Here is the output from vmstat:
 +
 
 +
<pic src="http://www.linuxintro.org/images/vmstat.png" align=text width=100% caption=VmStat  />
 +
 
 +
We see the user load is in colum 13, and we only want to print this column. We do it with the following command:
 +
vmstat 5 | awk '{print $13}'
 +
This will print a line from vmstat all 5 seconds and only write the column 13. It looks like this:
 +
# vmstat 5 | awk '{print $13}'
 +
 +
us
 +
1
 +
1
 +
0
 +
1
 +
To store the CPU user load into a variable we use
 
<source>
 
<source>
  streamer -c /dev/video0 -f jpeg -F stereo -o myvideo.avi -t 0:05
+
  load=$(vmstat 1 2 | tail -n 1 | awk '{print $13}')
 
</source>
 
</source>
 +
What happens here? First vmstat outputs some data in its first line. The data about CPU load can only be rubbish because it did not have any time to measure it. So we let it output 2 lines and wait 1 second between them ( => vmstat 1 2 ). From this command we only read the last line ( => tail -n 1 ). From this line we only print column 13 ( => awk '{print $13}' ). This output is stored into the variable $load ( => load=$(...) ).
  
== video conferencing ==
+
== grep: search a string ==
* to do video conferencing, [[use skype under Linux]].
+
[[grep]] is a [[program]] that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can extract lines that contain a string or match a [[regex]] pattern. Let's imagine you want all external links from www.linuxintro.org's main page:
 +
<source>
 +
wget -O linuxintro.txt http://www.linuxintro.org
 +
grep "http:" linuxintro.txt
 +
</source>
 +
 
 +
== sed: replace a string ==
 +
[[sed]] is a [[program]] that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can replace a string by another one. Let's imagine you want to print your distribution's name, but lsd_release outputs too much:
 +
# lsb_release -d
 +
Description:    openSUSE 12.1 (x86_64)
 +
You want to remove this string "Description" so you replace it by nothing:
 +
lsb_release -d | sed "s/Description:\t//"
 +
openSUSE 12.1 (x86_64)
 +
 
 +
Once you understand [[regular expressions]] you can use sed with them:
 +
 
 +
* to replace protocol names for a given port (in this case 3200) in /etc/services:
 +
<source>
 +
sed -ri "s/.{16}3200/sapdp00 3200/" /etc/services
 +
</source>
 +
* if you have an [[apache]] [[web server]] here's how you get the latest websites that have been requested:
 +
<source>
 +
cat /var/log/apache2/access_log | sed ";.*\(GET [^\"]*\).*;\1;"
 +
</source>
  
== watch it ==
+
== tr: replace linebreaks ==
To watch your camera's input, use ''cheese'' or mplayer:
+
[[sed]] is a [[program]] that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can replace a character by another one, even over line breaks.
 +
For example here is how you remove all empty lines from your processor information:
 
<source>
 
<source>
  mplayer -fps 30 -cache 128 -tv driver=v4l2:width=640:height=480:device=/dev/video0 tv://
+
  # cat /proc/cpuinfo | while read a; do ar=$(echo -n $a|tr '\n' ';')
 +
if [ "$ar" <> ";" ]; then echo "$ar"; fi; done
 
</source>
 
</source>
Or use [[vlc]]. You can [[run vlc as root]] by the way. To watch your webcam /dev/video0, start vlc and select Media -> Open Capture Device -> Video device name = /dev/video0 -> Play
 
  
= Testbed =
+
== wc: count ==
The following webcams have been found working with this tutorial:
+
With the command wc you can count words, characters and lines. wc -l counts lines. For example to find out how many semicolons are in a line, use the following statement:
* Logitech Quickcam messenger
+
<source>
* Philips Webcam SPC220NC
+
while read line
 +
do echo "$line" | tr '\n' ' ' | sed "s/;/\n/g" | wc -l
 +
done
 +
</source>
 +
It lets you input a line of text, counts the semicolons in it and outputs the number.
  
A general list of working webcams can be found at http://mxhaard.free.fr/spca5xx.html.
+
How does it do this?
  
The guide has been tested with SUSE Linux 11.4 till 13.2 and Ubuntu.
+
It reads lines from your keyboard (while read line). It outputs the line (echo "$line"), but it does not output it in the console. The pipe (|) redirects the output to the input stream of the command tr. The command tr replaces the ENTER ('\
 +
') by a space (' '). The pipe (|) redirects the output to the input stream of sed. sed substitutes ("s/) the semicolon (;) by (/) a linefeed (\
 +
), globally (/g"). The pipe redirects the output to the input stream of the wc -l command that outputs the count of lines.
 +
 
 +
== dialog: create dialogs ==
 +
Dialog is a command that helps you creating dialogs in the shell. The answers given by the user are send to [[stderr]] and/or influence the command's return code. For example if you run this script:
 +
<source>
 +
#!/bin/bash
 +
if (dialog --title "Message"  --yesno "Are you having fun?" 6 25)
 +
then echo "glad you have fun"
 +
else echo "sad you don't have fun"
 +
fi
 +
</source>
 +
It will display this dialog:
 +
 
 +
<pic src=http://www.linuxintro.org/images/Snapshot-dialog.png />
 +
 
 +
This has been taken from http://www.linuxjournal.com/article/2807. Read there for more info.
  
 
= See also =
 
= See also =
* [[hardware]]
+
* [[bash]]
* http://en.opensuse.org/Webcam
+
* [[shell]]
* http://www.linux.com/feature/126186
+
* [[scripting]]
* http://ubuntulinuxhelp.com/linux-driver-for-quickcam-usb-cameras-logitech-quickcam-fusion/
+
* [[bash operators]]
* http://www.goldmann.de/webcam-linux_tipp_408.html
+
* http://en.wikibooks.org/wiki/Bash_Shell_Scripting
* http://wiki.ubuntuusers.de/Webcam
+
* http://ryanstutorials.net/linuxtutorial/scripting.php
 +
* http://wiki.linuxquestions.org/wiki/Bash_tips
 +
* http://wiki.linuxquestions.org/wiki/Bash
 +
* http://linuxconfig.org/Bash_scripting_Tutorial
 +
* http://steve-parker.org/sh/intro.shtml
 +
* http://mywiki.wooledge.org/BashFAQ
 +
* http://mywiki.wooledge.org/BashPitfalls
 +
 
 +
[http://www.facebook.com/sharer.php?u=http%3A%2F%2Fwww.linuxintro.org%2Fwiki%2FShell_scripting_tutorial&t=Shell%20scripting%20Tutorial&src=sp Share on Facebook]
  
 
<stumbleuponbutton />
 
<stumbleuponbutton />
 +
 +
[[Category:Mindmap]]
 +
[[Category:Learning]]

Revision as of 12:45, 27 December 2020


BaBE - Bash By Examples; Your significant Linux scripting tutorial;;

MindMap of what you can learn about Linux scripting

Hello world

The easiest way to get your feet wet with a programming language is to start with a program that simply outputs a trivial text, the so-called hello-world-example. Here it is for bash:

  • create a file named hello in your home directory with the following content:
#!/bin/bash
echo "Hello, world!"
chmod +x hello
  • now you can execute your file like this:
./hello
Hello, world!
  • or like this:
bash hello
Hello, world!

You see - the output of your shell program is the same as if you had entered the commands into a console.

calling commands

In your shell script you can call every command that you can call when opening a console:

echo "This is a directory listing, latest modified files at the bottom:"
ls -ltr
echo "Now calling a browser"
firefox
echo "Continuing with the script"

variables

input

To show you how to deal with variables, we will now write a script that asks for your name and greets you:

echo "what is your name? "
read name
echo "hello $name"

You see that the name is stored in a variable $name. Note the quotation marks " around "hello $name". By using these you say that you want variables to be replaced by their content. If you were to use apostrophes, the name would not be printed, but $name instead.

${}

The ${} operator stands for the variable, there is no difference if you write

echo "$name"

or

echo "${name}"

So what is the sense of this? Imagine you want to echo a string directly, without any blank, after the content of a variable: <source>

echo "if I add the syllable owa to your name it will be ${name}owa"

</source>

common mistakes

Note that the variable is called $name, however the correct statement to read it from the keyboard is

read name

It is a common mistake to write

read $name

which means "read a string and store it into the variable whose name is stored in $name"

parameters

echo "Here are all parameters you called this script with: $@"
echo "Here is parameter 1: $1"
echo "Which parameter do you want to be shown? "
read number
args=("$@")
echo "${args[$number-1]}"

return codes

Every bash script can communicate with the rest of the system by

  • sending data to stdout
  • sending data to stderr
  • delivering a return code

The return code is 0 if everything worked well. You can query it for the most recent command using $?:

bootstick@bootstick:~$ echo "hello world"; echo $?
hello world
0
bootstick@bootstick:~$ echo "hello world">/proc/cmdline; echo $?
bash: /proc/cmdline: Permission denied
1

In bash, true is 0 and false is any value but 0. There exist two commands, true and false that deliver true or false, respectively:

bootstick@bootstick:~$ true; echo $?
0
bootstick@bootstick:~$ false; echo $?
1

line feeds

In bash you can use line feeds or semicolons to separate commands. For example: <source>

read name
if [ $name = "Thorsten" ]; then echo "I know you"; fi

</source> Instead of a semicolon you can write a line feed like this: <source>

read name
if [ $name = "Thorsten" ]
  then echo "I know you"
fi

</source>

So we could put everything into one line and separate the commands by semicolons: <source>

read name; if [ $name = "Thorsten" ]; then echo "I know you"; fi

</source>

If you want to insert a line feed where you do not need one, e.g. to make the code better readable, just put a backslash at the line's end to indicate it will continue: <source>

read \
  name
if [ $name = "Thorsten" ]
  then \
    echo "I know you"
fi

</source>

storing a command's output

To read a command's output into a variable use $(), backticks or piping.

$()

arch=$(uname -m)
echo "Your system is a $arch system."

backticks

arch=`uname -m`
echo "Your system is a $arch system."

piping

Piping is a very elegant concept in the Linux world. It allows you to take one command's output and use it as input for the next command. Now you can divide tasks into smaller tasks. For example instead of having a program counting all files in a directory you use one program (ls) to list all files in a directory and one program (wc) to count the lines: <source>

ls | wc -l

</source> You can also put the output into a variable, in this case $arch: <source>

uname -m | while read arch; do echo "Your system is a $arch system."; done

</source>

comparison

The advantage of using backticks over $() is that backticks also work in the sh shell. The advantage of using $() over backticks is that you can cascade them. In the example below we use this possibility to get a list of all files installed with rpm on the system:

filelist=$(rpm -ql $(rpm -qa))

You can use the piping approach if you need to cascade in sh, but this is not focus of this bash tutorial.

common mistakes

Usually unexperienced programmers try something like

uname -m | read arch

which does not work. You must embed the read into a while loop.

conditions

The easiest form of a condition in bash is this if example: <source>

echo "what is your name? "
read name
if [ $name = "Thorsten" ]; then echo "I know you"; fi

</source> Now let's look closer at this, why does it work? Why is there a blank needed behind the [ sign? The answer is that [ is just an ordinary command in the shell. It delivers a return code for the expression that follows till the ] sign. To prove this we can write a script: <source>

if true; then echo "the command following if true is being executed"; fi
if false; then echo "this will not be shown"; fi

</source>

empty strings

An empty string evaluates to false inside the [ ] operators so it is possible to check if a string result is empty like this:

# result=
# if [ $result ]; then echo success; fi
# result=good
# if [ $result ]; then echo success; fi
success

arithmetic expressions

You can compare integer numbers like this: <source>

echo "what is your age? "
read age
if (( $age >= 21 )); then echo "Let's talk about sex."; fi

</source> However bash does not understand floating point numbers. To compare floating numbers you will use external programs such as bc: <source>

$ if [ $(echo "2.1<2.2"|bc) = 1 ]; then echo "correct"; else echo "wrong"; fi
correct
$ if [ $(echo "2.1>2.2"|bc) = 1 ]; then echo "correct"; else echo "wrong"; fi
wrong

</source>

not equal

To check if a variable is NOT equal to whatever, use !=: <source>

if [ "$LANG" != "C" ]; then echo "please set your system lanugage to C"; fi

</source>

common mistakes

Common mistakes are:

Redirections

To redirect the output of a command to a file you have to consider that there are two output streams in UNIX, stdout and stderr.

filling files

To create a file, probably the easiest way is to use cat. The following example writes text into README till a line occurs that only contains the string "EOF":

cat >README<<EOF
This is line 1
This is line 2
This is the last line
EOF

Afterwards, README will contain the 3 lines below the cat command and above the line with EOF.

loops

for loops

Here is an example for a for-loop. It makes a backup of all text files:

for i in *.txt; do cp $i $i.bak; done

The above command takes each .txt file in the current directory, stores it in the variable $i and copies it to $i.bak. So file.txt gets copied to file.txt.bat.

You can also use subsequent numbers as a for loop using the command seq like this:

for i in $(seq 1 1 3); do echo $i; done

while loops

$ while true; do read line; done

negations

You can negate a result with the ! operator. $? is the last command's return code:

# true
# echo $?
0
# false
# echo $?
1
# ! true
# echo $?
1
# ! false
# echo $?
0

So you get an endless loop out of:

while ! false; do echo hallo; done

The following code checks the file /tmp/success to contain "success". As long as this is not the case it continues checking:

while ! (grep "success" /tmp/success)
do
  sleep 30
done

The following code checks if the file dblog.log exists. As long as this is not the case it tries to download it via ftp: <source>

while ! (test -e dblog.log); do
  ftp -p ftp://user:password@server/tmp/dblog.log >/dev/null
  echo -en "."
  sleep 1
done

</source>

common mistakes

  • bash is very picky regarding spaces. There MUST be a space after the ! if it means negation.

sending a process to the background

To send a process to the background, use the ampersand sign (&):

firefox & echo "Firefox has been started"

You see a newline is not needed after the &

forking a process

You can build a process chain using parantheses. This is useful if you want to have two instruction streams being executed in parallel: <source>

(find -iname "helloworld.txt") & (sleep 5; echo "Timeout exceeded, killing process"; kill $!)

</source>

functions

To define a function in bash, use a non-keyword and append opening and closing parentheses. Here a function greet is defined and it prints "Hello, world!". Then it is called:

# greet()
{
    echo "Hello, world"
}
# greet

If you hand over parameters you can greet any planet you like:

# greet()
{
    echo "Hello, $1"
}
# greet Mars
Hello, Mars
# greet World
Hello, World

react on CTRL_C

The command trap allows you to trap CTRL_C keystrokes so your script will not be aborted <source>

  1. !/bin/bash

trap shelltrap INT

shelltrap() {

   echo "You pressed CTRL_C, but I don't let you escape"

}

while true; do read line; done </source>

Note
You can still pause your script by pressing CTRL_Z, send it to the background and kill it there. To catch CTRL_Z, replace INT by TSTP in the above example. To get an overview of all signals that you might be able to trap, open a console and enter
kill -l

helpful programs

awk: read a specific column

awk is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can extract columns from a text. Let's imagine you want to use the program vmstat to find out how high the CPU user load was. Here is the output from vmstat:

VmStat

We see the user load is in colum 13, and we only want to print this column. We do it with the following command:

vmstat 5 | awk '{print $13}'

This will print a line from vmstat all 5 seconds and only write the column 13. It looks like this:

# vmstat 5 | awk '{print $13}'

us
1
1
0
1

To store the CPU user load into a variable we use <source>

load=$(vmstat 1 2 | tail -n 1 | awk '{print $13}')

</source> What happens here? First vmstat outputs some data in its first line. The data about CPU load can only be rubbish because it did not have any time to measure it. So we let it output 2 lines and wait 1 second between them ( => vmstat 1 2 ). From this command we only read the last line ( => tail -n 1 ). From this line we only print column 13 ( => awk '{print $13}' ). This output is stored into the variable $load ( => load=$(...) ).

grep: search a string

grep is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can extract lines that contain a string or match a regex pattern. Let's imagine you want all external links from www.linuxintro.org's main page: <source>

wget -O linuxintro.txt http://www.linuxintro.org
grep "http:" linuxintro.txt

</source>

sed: replace a string

sed is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can replace a string by another one. Let's imagine you want to print your distribution's name, but lsd_release outputs too much:

# lsb_release -d
Description:    openSUSE 12.1 (x86_64)

You want to remove this string "Description" so you replace it by nothing:

lsb_release -d | sed "s/Description:\t//"
openSUSE 12.1 (x86_64)

Once you understand regular expressions you can use sed with them:

  • to replace protocol names for a given port (in this case 3200) in /etc/services:

<source>

sed -ri "s/.{16}3200/sapdp00 3200/" /etc/services

</source>

  • if you have an apache web server here's how you get the latest websites that have been requested:

<source>

cat /var/log/apache2/access_log | sed ";.*\(GET [^\"]*\).*;\1;"

</source>

tr: replace linebreaks

sed is a program that is installed on almost all Linux distributions. It is a good helper for text stream processing. It can replace a character by another one, even over line breaks. For example here is how you remove all empty lines from your processor information: <source>

# cat /proc/cpuinfo | while read a; do ar=$(echo -n $a|tr '\n' ';')
if [ "$ar" <> ";" ]; then echo "$ar"; fi; done

</source>

wc: count

With the command wc you can count words, characters and lines. wc -l counts lines. For example to find out how many semicolons are in a line, use the following statement: <source>

while read line
do echo "$line" | tr '\n' ' ' | sed "s/;/\n/g" | wc -l
done

</source> It lets you input a line of text, counts the semicolons in it and outputs the number.

How does it do this?

It reads lines from your keyboard (while read line). It outputs the line (echo "$line"), but it does not output it in the console. The pipe (|) redirects the output to the input stream of the command tr. The command tr replaces the ENTER ('\ ') by a space (' '). The pipe (|) redirects the output to the input stream of sed. sed substitutes ("s/) the semicolon (;) by (/) a linefeed (\ ), globally (/g"). The pipe redirects the output to the input stream of the wc -l command that outputs the count of lines.

dialog: create dialogs

Dialog is a command that helps you creating dialogs in the shell. The answers given by the user are send to stderr and/or influence the command's return code. For example if you run this script: <source>

#!/bin/bash
if (dialog --title "Message"  --yesno "Are you having fun?" 6 25)
then echo "glad you have fun"
else echo "sad you don't have fun"
fi

</source> It will display this dialog:

This has been taken from http://www.linuxjournal.com/article/2807. Read there for more info.

See also

Share on Facebook