Friday, October 16, 2009
Linux - Changing Directories with the 'cd' Command
Sunday, October 11, 2009
Linux - File Permissions and the 'chmod' Command
Linux – File Permissions and the ‘chmod’ Command
Linux was designed to be a multi-user operating system. So
there must be a way to limit who has access to what files, and how much access.
There are three sets of users in linux,
· User-this is an individual, such as yourself, that has login credentials to the system. Each user has their own home directory where they have full access to the files.
· Group-this is a set of multiple users. A group is typically created by the system admin. For instance, at a university, there could be a group setup for each course, where the professor is the group leader, and students are group members.
· Other-this refers to individuals that are not the user and not in the group being discussed.
Linux uses file permission attributes to set who has access to what files. Type in 'ls -l' in any directory with at least one file in it. This will show you all the information about each file/directory.
$ ls
-l
drwxr-x--- 1 jdoe jane 4096 Dec 28 04:09 tmp
-rw-r--r-- 1 jdoe jdoe 969 Dec 21 02:32 foo
|
Column |
Value |
Meaning |
|
1 |
drwxr-x--- |
File Permissions |
|
2 |
1 |
Forget this one! |
|
3 |
Jdoe |
Owner |
|
4 |
Jane |
Group owner |
|
5 |
4096 |
Size in bytes |
|
6 |
Dec |
Last Modified Month |
|
7 |
28 |
Last Modified Day |
|
8 |
04:09 |
Last Modified Time |
|
9 |
tmp |
File Name |
File Permission Attribute
Let's break down the 10 character file permission attribute in Column 1:
|
Special |
User |
Group |
other |
|
- |
--- |
--- |
--- |
· Special attribute- This is blank(-) for normal files, but it is set to d for directories. This can be other letters for other devices etc, but do not worry about that now.
· 'user' permission- The user refers to the person who owns the file. So if you own the file, these characters apply to you.
· 'group' permissions- These apply to any user who is in the group owned by the group owner.
· 'other' permissions- These apply to any one who is not the owner, and who is not in the group owned by the group owner.
Read, Write and Execute
The user, group, and other permissions can have read(r),
write(w), and or execute(x) permissions. The order of the permissions is 'rwx'
for each permission set. Each permission attribute can either be on(r,w, or x)
or off(-).
What does read, write and execute mean?
· Read(r)-the contents of the file/directory can be viewed.
· Write(w)-the file can be modified or deleted.
· Execute(x)-the file can be executed, or run. This only makes sense if the file is a program or script that does something. Also, having the execute permission for a directory means the directory can be cd'd into(eg. cd directoryName)
Examples
-rwxrwxrwx
All users have full access to this file.
-rw-r--r--
Owner has read, write access to the file. All other users have read access.
drwx------
This is a directory that the user has full access, while all other users have no access.
Changing the Permissions with 'chmod' Command
There are several ways to change the attributes with the chmod command. In my opinion, the easiest way to use chmod is as follows (you would insert file permissions after the equals sign, see examples below),
chmod u=,g=,o= fileName
Fittingly, u, g and o are as follows,
· u-user
· g-group
· o-other
If you leave out u,g or o, that attribute will not be modified.
chmod u=,g= fileName
Note: An error will occur if you put spaces between u=,g=.
Examples
$ chmod u=rwx,g=r,o=w foo
$ ls -l
-rwxr---w- jdoe jdoe 969 Dec 21 02:32 foo
Not including g will keep it the same as before,
$ chmod u=x,o=r foo
$ ls -l
---xr--r-- jdoe jdoe 969 Dec 21 02:32 foo
A blank after o will clear all attributes,
$ chmod u=rx,g=w,o= foo
$ ls -l
-r-x-w---- jdoe jdoe 969 Dec 21 02:32 foo
Using + and – to add and remove attributes
You can also use + and – to add and subtract attributes.
Examples
$ chmod u=wx,g=w,o= foo
$ ls -l
--wx-w---- jdoe jdoe 969 Dec 21 02:32 foo
$ chmod u+r-w,g-w+r,o+r foo
$ ls -l
-r-xr--r-- jdoe jdoe 969 Dec 21 02:32 foo
Using 3-digit octal value to set attributes
Lastly, you can specify the attributes with a 3 digit octal. Each of the user, group and other permissions have 3 attributes(rwx) that can be on or off, and as such can be thought of as a 3 bit word which is easily converted to an octal digit(0-7),
chmod (u#)(g#)(o#) fileName
· 0='---'
· 1='--x'
· 2='-w-'
· 3='-wx'
· 4='r--'
· 5='r-x'
· 6='rw-'
· 7='rwx'
Example
$ chmod 764 foo
$ ls -l
-rwxrw-r-- jdoe jdoe 969 Dec 21 02:32 foo
Previous - Finding A File | Next - Creating and Deleting Files and Directories
Friday, October 9, 2009
Linux - Bash Shell Scripting - Expressions and Substrings
Linux - Bash Shell Scripting - Expressions and Substrings
Expressions
Sometimes in a script, you would like to evaluate an expression. For example, to find the sum of two variables. The expr command can be used to do this,
$ cat testsum.sh
#!/bin/bash
v1=12
v2=39
v3=`expr $v1 + $v2`
echo $v3
$ ./testsum.sh
51
Note that the expr command must be enclosed in backquotes `…` for this to properly work. Typing ‘man expr’ will display a help screen about the expr command.
Substrings
The expr substr command can be used to extract a substring from a string variable, e.g., this program prints the individual characters of a string stored in a variable v,
v="foobar"
len=`expr length $v`
for i in `seq 1 1 $len`; do
echo -n `expr substr $v $i 1`
done
echo
The syntax of expr substr is, 'expr substr string pos len' which will extract len characters from the string string starting at position pos. Note that the first character of the string string is at position 1.
Previous - For and While Loops
Linux - Bash Shell Scripting - For and While Loops
For Loop
The basic syntax of the for loop is:
for var in list; do
some commands
done
Note the semicolon; it is important to not omit it. The commands are executed once for every item in the list. The current item is accessed through the variable $var, e.g.,
Use VI to edit for.sh
--------------------------------
for i in 1 2 3 4 5 6 7 8 9 10; do
echo -n "$i "
done
echo
--------------------------------
$ chmod 755 for.sh
$ ./for.sh
1 2 3 4 5 6 7 8 9 10
The option -n to echo means do not print a newline character after outputting the text.
Another example,
for i in $(ls); do
echo $i
done
This script display the name of each of the files in the current directory one per output line. Another example,
Use VI to edit ls-dirs.sh
--------------------------------
for i in *; do
if [ -d "$i" ]; then
echo "$i [dir]"
else
echo $i
fi
done
--------------------------------
$ mkdir foo1
$ mkdir foo2
$ chmod 755 ls-dirs.sh
$ alias ls-dirs=./ls-dirs.sh
$ ls
foo1 foo2 for.sh if.sh ls-dirs.sh
$ ls-dirs
foo1 [dir]
foo2 [dir]
The seq command (type man seq for help) can be used to generate an integer sequence. The syntax is, seq first increment last which will generate an integer sequence starting at first, incrementing by increment, and stopping at last. For example, Here is a loop which displays the first 100 odd integers,
for i in `seq 1 2 100`; do
echo -n $i " "
done
Note that the seq command must be enclosed in backquotes `seq…` so the shell will execute it to generate the desired output. This loop prints the first 100 odd integers in reverse order,
for i in `seq 99 -2 1`; do
echo -n $i " "
done
While Loops
The basic syntax of the while loop is,
while [ conditional-expression ]; do
statements
done
I have ommitted examples as the while loop is used in a similar context to the for loop.
Linux - Bash Shell Scripting - If Statements
if [ conditional-expression ]; then
or
if [ conditional-expression ]; then
some commands
else
some other commands
fi
The character "[" actually refers to a built-in Bash command—it is a synonym for another command named test. The result of the conditional expression is 0 or nonzero with 0 being false and nonzero being true. Some examples,
Use VI to edit if.sh
--------------------------------
today=0
if [ $today = 1 ]; then
echo "today = 1"
else
echo "today <> 1"
fi
--------------------------------
$ chmod 755 if.sh
$ ./if.sh
today <> 1
Since using [ ] is equivalent to using test,
today=0
if test $today = 1; then
echo "today = 1"
else
echo "today <> 1"
fi
The test command has command line options for checking the status of files, e.g.,
test -e file
True if file exists
Multiple conditions can be checked using -a (and) and -o (or), e.g.,
if test -e if.sh -a -x if.sh; then
echo "if.sh is an executable"
else
echo "if.sh either does not exist or it is not an executable"
fi
In a schell script, the ! symbol is used as the NOT logical operator, e.g.,
echo '"'src'"' does not exist, creating directory.
mkdir src
fi
To learn more about conditional expressions I suggest you try $ man test.
Linux - Bash Shell Scripting - Variables in Output
VarName=$(command)
#!/bin/bash
LS_OUT=$(ls)
echo $LS_OUT
Or
#!/bin/bash
#the 'date' command retrieves the current date(many options)
DOW=$(date +%a) #get day of week
MONTH=$(date +%b)
DAY=$(date +%d)
YEAR=$(date +%Y)
echo "Today is ${DOW} ${MONTH}-${DAY}-${YEAR}"
$ alias today="./today.sh"
$ today
Today is Wed Oct-17-2007
Linux - Bash Shell Scripting - Reading From the Keyboard
#!/bin/bash
echo "Enter your name: "
read name
echo "Pleased to meet you $name."
Linux - Bash Shell Scripting - A Simple Backup Script
#!/bin/bash
# usage: bu dir
tar cvf $1.tar $1a
bzip2 $1.tar
mv $1.tar.bz2 mybackups
echo "$1 backed up."
$ bu src
src backed up.
$ ls mybackups
src.tar.bz2
Linux - Bash Shell Scripting - Displaying Text from a Shell Script
#!/bin/bash
# usage: rmwild dir pattern
dirname=$1
pattern=$2
echo "Removing all $2 files from $1..."
cd $dirname
rm $pattern 2> /dev/null
echo "Done."
$ ls tmp
file1.c file1.o file2.c file2.o
$ rmwild tmp *.o
Removing all *.o files from tmp...
Done.
$ ls tmp
file1.c file2.c
Previous - Aliases | Next - A Simple Backup Script
Linux - Bash Shell Scripting - Aliases
$ alias rmwild="./rmwild"
$ rmwild tmp *.o
$ alias gohome=cd ~
$ pwd
/usr/bin
$ gohome
$ pwd
/home/fredf
Aliases defined on the command line only exist for the current Bash session. If you want the alias name to be available every time you log in, define the alias in your .bashrc file. To delete an alias that was defined at the command prompt, use the unalias command, e.g., unalias rmwild. To delete all of your aliases, type unalias -a.
Previous - Command Line Arguments | Next - Displaying Text from Shell Script
Linux - Bash Shell Scripting - Command-Line Arguments
Linux - Bash Shell Scripting - An Example
#!/bin/bash
# usage: rmo
mv tmp
rm *.o
$ ./rmo.sh
./rmo.sh: Permission denied
The shell script failed to run because shell scripts are not executable files by default. To make the shell script file executable, you must change the permissions using the chmod command to add x,
$ ls -l rmo.sh
-rw-r--r-- 1 jdoe jdoe 122 Oct 17 11:35 rmo.sh
$ chmod 744 rmo.sh
$ ls -l rmo.sh
-rwxr--r-- 1 jdoe jdoe 122 Oct 17 11:35 rmo.sh
$ ./rmo.sh
#!/bin/bash
# usage: rmo
dirname=tmp
pattern=*.o
cd $dirname
rm $pattern
Here dirname and pattern are variables. Note that when the variable is assigned to (using =), we do not put a $ in front of the variable name, but we do use a $ when referring to the variable later on. If you don't, the shell will think that the variable name is the filename of a program to be executed and it will probably fail with an error message.
Previous - Arithmetic and Relational Operators | Next - Command Line Arguments
Linux - Bash Shell Scripting - Arithmetic and Relational Operators
Within an expression, the normal precedence rules are applied; in the table precedence is from highest (unary minus, plus) to lowest (logical or).
- +
Unary minus, plus
! ~
Logical negation, bitwise negation
**
Exponentiation
* / %
Multiplication, division, remainder
+ -
Addition, Subtraction
<< >>
Left and right bitwise shifts
> < >= <= Greater than, less than, greater than or equal, less than or equal
== !=
Equal, not equal
&
Bitwise and
^
Bitwise exclusive or
|
Bitwise or
&&
Logical and
||
Logical or
Variables can be assigned integer values using normal assignment, or using let, e.g.,
The last statement will cause the shell script to fail; there should be no spaces to
the left and right of the = operator. Expressions are evaluated by enclosing them
in $[ … ], e.g.,
The above displays 2 (note integer division is performed).
Linux - Bash Shell Scripting - Variables
LONG_NAME="Flintstone Fred"
SHORTNAME=Fred
Note in the first case that quotation marks are required because the string contains spaces. The backquote operator will execute the enclosed command and assign the output to the designated variable, e.g.,
LS_OUT=`ls`
When referring to the variable, its name must be preceded by $, e.g.,
You can declare numerical values too,
intA=5
Summary
- Declare string
- string1="string text"
- Declare command
- command1='ls'
- Declare numerical value
- intA=5
- floatB=.5
Linux - Bash Shell Scripting - Introduction
Complete books have been written about shell scripting, and it is not something we can cover in a few blog posts. This will be a basic introduction, and you are referred to the online guides and tutorials for more information.
A script file is a text file, and the first line is usually supposed to be
if the shell script is designed to be executed by the Bash shell. If you are using a different shell to write your script (e.g., the Korn shell), then set the first line to point to the Korn shell's binary image (usually it is in the /bin directory; e.g., /bin/ksh). This will ensure that the correct shell is loaded and used to execute the shell script regardless of what particular shell the user may be running.
Note that in shell scripts, # lines are comments, just as // are comments in C++. Any character from the # symbol to the end of the line is ignored by the shell interpreter. Typically shell scripts are given a .sh extension to let the user know it is a shell script, but this is not necessary for it to run.
Summary
- Shell scripts are text files that automate tasks. You can put any command in it that you can type in the shell prompt.
- The commands will execute sequentially, just as in most programming languages.
- All shell scripts must start with the location of the bash install, "#!/usr/bin/env bash, as the first line.
- Lines beginning with # are comments.
- Typically shell scripts are given a .sh extension to let the user know it is a shell script, but this is not necessary for it to run.
Thursday, October 8, 2009
Linux - Editing Files - Painless Introduction to VIM (vi)
Starting VI
At the Bash command prompt, type vim or vim some-filename. VI is a modal editor in which you are either in editing mode(--insert--) or command mode().
$ vi filename
Getting out of VI
This is the most important thing you need to know about VI: how to get out of it. If you are in editing mode, hit ESC to enter command mode. Enter :wq to write your file and quit. To quit without writing the file, hit :q. If you really want to quit without saving the file you have made changes to hit :q!. To save the file under a different name, try :wq new-filename. You can also press ctrl-z to suspend the process
Switching Between Editing Mode and Command Mode
If you are in editing mode, hit ESC to enter command mode. If you are in command mode hit i to enter insert (editing) mode. If you cannot tell which mode you are in, trying hitting ESC several times to make sure you are in command mode, then type :set showmode. This may tell VI to let you know when you are in insert
mode (it depends on the version of VI you are using).
Other Useful Settings
All of these are recommended in addition to :set showmode. In command mode, type :set nocompatible to enable advanced VIM features. Type :set ruler to tell VI to display your current cursor position. Type :set number to display line numbers. It would be preferable if I were also on fire and being eaten alive by rabid hyenas at the same time. That would still be more pleasurable than using VI.
Moving Around a Character at a Time
The original VI was written by Bill Joy (at UC Berkeley) using a computer system and keyboard that lacked arrow, page up, and page down keys. This is the reason for the stupid assignment of the keys for moving around: h (move left one character), j (move down one line), k (move up one line), and l (move right one character). On Joy's terminal, these four keys also had arrow keys on them and that is the historical reason they are still mapped that way today.
Moving Around a Word at a Time
Forget about your Del and Backspace keys. They may or may not work. Switch to command mode. Press w to move forward one word. Press b to move backward one word. Press nw to move forward n words, e.g., 3w to move forward three words. Press nw to move backward n words.
Moving Around the File
Switch to command mode. To move to the end of the line, hit $. To move to the beginning, hit 0. Hit 1G to go the first line of text. hit nG to go line number n. Hit G to go to the end of file. To display line numbers, in command mode type :set number. Note that :13 is equivalent to, e.g., 13G. To page down, hit Ctrl+F. To
page up hit Ctrl+B.
Deleting Characters
Switch to command mode, move to the character you want to delete, and hit x to delete that character. To delete n characters hit nx, e.g., 17x to delete 17 characters. To delete all the characters from the cursor position to the end of the line hit D.
Copying a Line of Text
To copy a line of text from one place in the file to another, switch to command mode. Move to the line you want to copy. Hit yy (yank something or other, I dunno). Move the cursor to the location where you want to make the copy. Hit p (lowercase p) to put the copied line after the current line. Hit P (uppercase P) to put
the copied line before the current line.
Moving a Line of Text
To move a line of text from one place in the file to another, switch to commandmode. Move to the line you want to copy. Hit dd. Move the cursor to the location where you want to make the move. Hit p (lowercase p) to move the line after the current line. Hit P (uppercase P) to move the line before the current line.
Deleting a Line of Text
Switch to command mode. Move to the line you want to delete. Hit dd.
Cutting, Copying, Pasting Multiple Lines of Text
Use nyy to copy n lines of text to the copy buffer. Move to where you want the lines to be and hit p or P to copy them. Use ndd to move n lines of text somewhere. Hit ndd to delete n lines of text, e.g., 3dd will delete the line of text the cursor is on and the next two lines.
Moving and Copying Text in Visual Mode
To cut, copy, and paste text that is not an entire line, enter visual mode by hitting v in command mode. Use the h, j, k, l keys to move around and select the text you want to copy or move. Once selected, hit y to copy the text to a buffer or d to delete the text to the buffer. Once copied (with y) or deleted with (with d) move to
where you want the text to be. Hit p to paste the text.
Finding Text
Switch to command mode. Hit / (forward slash) and enter the string you are searching for, e.g., /cookies. The cursor will be placed on the first occurrence of cookies following the current location. Hitting n over-and-over will continue searching.
Find and Replace
Switch to command mode. To replace all occurrences of homer by bart, try something like :1,100 s/homer/bart/g. The 1,100 part means search from line 1 to line 100 inclusive (I think 1,$ will search the entire file). The s stands for search. We are searching for homer and replacing all occurrences by bart. The g stands for global which means it will replace all occurrences without prompting you for each one. If you want to be prompted for each one, use c instead (for confirmation).
Undo and Redo
Since you will use it the most because of VI's horrible user interface the most useful command in VI (other than :q!) is the undo command. Hitting u will undo your last change. Hitting u multiple times will undo all of your changes back to the last time you saved the file (only if you are in :set nocompatible mode). Redo
is Ctrl+R.
More Help
Let's say you're either insane or a masochist and want to learn more about VI. In command mode, hit :h to bring up help. The most useful thing you need to know here is how to get out of help, hit :q (like in less).
Linux - Processes
Every process has a system-wide unique identifier called the process id, or pid; it is just an integer. The process is owned by the user who launched it. The ps command will show you a list of the running processes of which you are the owner.
$ ps
PID TTY TIME CMD
17776 pts/18 00:00:00 bash
19041 pts/18 00:00:00 ps
I am running two programs right now: bash and the ps program itself. The PID of each process is shown. The time column is the cumulative amount of CPU time the process has consumed—this is not the same as wall time, i.e., how many seconds it has been since you ran the program. To see all of the processes running on the system, type ps -A or ps -e. The command ps -f will display full process information including the user id of the process owner. It can be combined with -A or -e. To sort this list by user id and view it in less, use
$ ps -ef | sort | less.
When you run a program (i.e., a process) you can ask the OS to run it in the background. Do this by putting a & on the end of the command line,
$ some-program &
$ now-you-can-type-another-command
Now some-program will run in the background and you can get back to work typing commands. To bring a running program back into the foreground, type the fg command,
$ fg
Now some-program would be running and if it is displaying output, the output would be going to the console window. To put the program back into the background, hit Ctrl+Z to pause it, and then type bg to send it back to the background.
To see a list of your processes running in the background use the jobs command. If you have more than one background job they will be numbered with a job number 1, 2, 3, .... To bring job number 3 to the foreground, type fg %3. To send this job back to the background, hit Ctrl+Z, and type bg.
To terminate a runaway process, use the kill command. The syntax is kill pidnumber. You cannot kill processes that you do not own. To kill a process running in the background, issue a jobs command to see your running processes and the determine the job number. Then use kill job number to kill the job; e.g., kill %3 to kill job number 3.
The top command will display a table of the processes which are currently using the most CPU time. It is a highly configurable program; try man top. (Be aware that top is a fairly resource-intensive program; it's not cool or nice to run top alot).
The nice command was designed to allow a user to run a program with lower than normal priority, so it will not hog the CPU as much. The idea is that other people may be peeved if you're sucking up too much CPU time, so you can be nice about it and still run your program. The syntax is nice -n adjust command where command is the program you want to run (e.g., it could be a big make process). Adjust is an integer which determines the adjusted process priority: -20 is highest, to 19 lowest. No one ever uses nice.
Linux - Piping
Linux - Piping
To connect the stdout of one program to the stdin of another program, in *nix we use a pipe (the output flows out one program through the pipe to the input of the other program). The pipe symbol is | and the way to send the output of one program to the input of another program is program1 | program2. For example, to display a long listing of the files in a directory and sort them based on the time the file was last modified,
$ ls -l | sort +7
The program sort sorts its input and sends the sorted output to stdout. The option +7 tells sort to sort the input file based on the data in field 7 which in the ls -l output is the column which contains the timestamp. To sort on size try ls -l | sort + 4. To display the contents of a directory and display the output using less,
$ ls -l | less
As you can see, this is similar to I/O redirection, but we are not dealing with reading and writing to files, but rather with programs that interact in some way. IO redirects and piping can be used in conjunction in the same command as well.
$ func1 < file1.txt | less
The above command will take file1.txt as input to func1 and then pipe the output of func1 to less to display to the screen.
Previous - Concatenating and Appending Files | Next - Info Command