PATTERN
SEARCHING AND PERMISSIONS
The Unix command language contains a number of utilities for
patten/field matching and substitution. The common utilities available
on all Unix flavors are
- sed
- awk and nawk
- find
- grep, fgre and egrep
- locate
ATTENTION!
create a lab7 directory;
submit all downloads into lab7
dir
|
GREP
The grep command is consists of grep, egrep and fgrep. The purpose of
the grep command is to globally search for regular expressions in
files and standard outputs them to the screen with line number and
selected file.
Basic pattern Searching
cd into your $HOME dir and find the word alias |
| <bash>
grep alias * |
finds all
files containing the word alias in $HOME
|
| <bash>
grep -n alias * |
prints the
line number along the file name
|
| <bash>
grep -n --color=auto alias * |
prints the
line number and highlights the selected word
|
<bash>
grep -n -i Alias *
|
makes grep
case insensitive
|
NOTE to search for patterns in startup files the wild card needs to
follow a period, "."
|
PATTERN SEARCHING OF EXPRESSIONS
cd into a directory src/ that contains f90 source files e.g.
cd into $HOME/aero361/proj1/src
|
<bash>
grep -n -i --color=auto "implicit none" *
|
when spaces
are involved enclose the expression in single or double quotes
|
<bash>
grep -n -i -I --color=auto
"implicit none" *
|
do not process
the binary files
|
|
RECURSIVE PATTERN SEARCHING
cd back into the aero361/ directory and do a recursive search
|
<bash>
grep -n -i -I -r --color=auto
"implicit none" *
|
PIPE SYMBOL
The
| symbol allows the standard output of one command to become the
standard
input to another command in the same line e.g.
|
- <bash>ps
-ef | grep --color=auto <your username> <enter>
- ps -ef will display all the jobs and spit
them on the screen, but
what happens with the pipe command is what was supposed to go to the
screen
now becomes the input for grep
- so grep searches for your username as the
pattern in the ps -ef output
- <bash>ps
-ef | grep --color=auto matlab
- would only display the system calls for
matlab and nothing else. You need to have matlab running prrior to this
command.
|
SEARCHING AND REPLACING IN vi
| Searching
for a word |
| <SHIFT><ESC>:/word |
searches
and highlights the word
|
n
|
searches the
next instance of the word
|
|
Search
for word at
the end of the line
|
<SHIFT><ESC>:/word$ |
Search
for word at
the beginning of the line
|
<SHIFT><ESC>:/^word |
Search
for word
which has a meta character e.g. $,!,:, etc
|
<SHIFT><ESC>:/word\<meta character> |
SEARCH AND REPLACE IN Vi
I use this command very often!
|
<SHIFT><ESC>:%
|
consider the
complete file.
|
<SHIFT><ESC>:s/word/new
word
|
to substitute
a string locally
|
| <SHIFT><ESC>:%s/word/new
word/g |
to substitute
the string globally
|
| <SHIFT><ESC>:%s/^word/new
word |
word at the
start of line
|
| <SHIFT><ESC>:%s/word$/new
word |
word at the
end of line
|
<SHIFT><ESC>:64,102
s/^/!
|
comment f90 file
from line 64 to 102. The "^" is for 1st column in line
|
<SHIFT><ESC>:%s/word/
/g
|
replace the
word with a space
|
|
Download the following f77 file and perform
some tasks
conmin.f
|
|
convert
the "C" comment symbol to !
|
<SHIFT><ESC>:%s/^C/\!/g
|
substitute all
occurence of "C" at the start of a line with "!"
|
convert
"SUBROUTINE" to subroutine
|
<SHIFT><ESC>:%s/SUBROUTINE/subroutine/g
|
converts all
instances of SUBROUTINE with subroutine
|
Sed is a stream
line editor that performs the tasks similar to vi. The "sed" command
works from a shell and uses the similar syntax to the stream line
editor in vi. This will be tackled later with the awk program.
INTRODUCTION TO AWK
AWK PROGRAMMING
|
| '..........................'
|
single quotes
to start the awk field
|
'{print
.........................}'
|
all awk
commands are enclosed in braces inside the single quotes
|
'{print $1
"\t" $2}'
|
fields are
preceded with $ and tabs are enclosed in " "
|
<bash>
awk '{print $1 "\t" $2}' file
|
print fields 1
and 2 separated by a tab
|
|
Download the following data file
naca747a314
=file
|
|
|
<bash>awk
'{print $1}' file
|
will print only
field one of the file
|
|
<bash>awk
'{print "\t field one:\t" $1}'
file
|
will print field one
with text and tabbed over
|
|
<bash>awk
'{print "\t field one:\t"
$1}'
file > outfile
|
will print field one with text and tabbed
over and output to file
|
|
<bash>awk
'{print "\t field one:\t" $1"\t
field two:\t" $2}' file
|
will print field one
and field 2 with text
|
POSSIBLE EXAM QUESTION
|
<bash>ps
-euf | awk '{print $1 "\t" $2
"\t" $????}'
|
print the required
fields from the ps command where ??? is a field that you define
|
FINDING FILES AND TYPES OF FILES and USING THE exec
COMMAND
The "find" command is commonly used to locate certain types of files
and operate on them. Once you are done with aero361 you may want to
delete all the code.x and object files since they do nothing but take
up space. Instead of going through each and every directory and
removing the redundant files the user can find these type of files and
remove them automatically. The "find" command will search through all
the directories starting from the selected directory looking for the
desired files. Once the files are found they will be stored in some
buffer {} created by the exec command. The exec command allows the
shell to continue with another operation along the find command.
do a <bash>man
find or <bash>gman
find
|
|
<bash>find
. -name "*.o"
|
finds
all files with suffix .o starting from the present working directory
"." The find command will search for the .o files through all dirs
below the present working directory. If the find command was issued in
$HOME then it will go through all the directories in $HOME.
-name
pattern
Base of file name (the path with
the leading directories
removed) matches shell pattern
pattern. The metacharacters
(`*', `?', and `[]') do not match a `.' at the start of the base
name. To ignore a directory and the files under it, use -prune;
see an example in the description of -path.
|
| <bash>find
$HOME/aero361 -name "*.o" |
The find command
will look for the ".o" files only in dirs and places inside the aero361
directory structure and not outside it
|
<bash>find
-atime 1 -name "*.f90" -type f
|
File with -name
*.f90 last accessed exactly n*24 hrs ago where n here is 1. For a
ballpark reference use + or - n. The -type option is when the user only
wants find to locate files in this case. if -type d were used then find
would only try to match directories.
|
- <bash>:find
-size +1000
- <bash>:find
-size -1000
|
- find all files that are more than 1000*512
bytes big.
- find all files that are less than 1000*512
bytes big.
|
<bash>find
. -name "*.o" -exec rm -i {}
\;
|
Find all object
files and remove them using the exec command
|
| <bash>find
. -name "core" -exec rm -i {}
\; |
Find all instances of the
core file and remove them
|
FILE PERMISSIONS
Files and directories in a unix system have associated ownership. Some
belong
to
root, system, and others belong to you. You need to understand how to
manage permissions.
The permissions also allow you lock people out of directories and
files. If
you
want
your classmate to only read a certain file, but not change anything
then
you need to apply the proper permissions on the file. The same can be
done
for directories.
<bash>
chmod 755 file
|
user
had read-write-execute permissions; group and others have read and
execute only ----> for WWW dir
|
<bash>
chmod 711 file
|
user
has read-write-execute permissions; group and others have only
|
<bash>
chmod 777 file
|
user-group
and others are given read-write-execute permissions
|
<bash>
chmod go-rwx file
|
group
and user lose read-write-execute permissions
|
<bash>
chmod a=rw file
|
all
are given read-write permissions |
<bash>
chmod -R go+xr
/remote/scratch/joeusr
|
Set
permissions for read and execute for group/others on the scratch
directory and all its contents. The "-R" is the recursive flag
|
<bash>
chown joeusr2 file
|
ATTENTION DO NOT DO THIS!!
to change owner ship of the file to another user. Not to use it while
file or dir is in your account. Used when depositing files and dirs in
some other users account e.g. scratch space. That way that user can
delete it when they want to
|
LETTER
|
PERMISSION
|
DECIMAL VALUE
|
---
|
No permissions
|
0
|
--x
|
execute
permissions only |
1
|
-w-
|
write
permissions only
|
2
|
-wx
|
write
and execute permissions
|
3
|
r--
|
read only
|
4
|
r-x
|
read
and execute permissions
|
5
|
rw-
|
read and write permissions
|
6
|
rwx
|
read,write and execute
permissions
|
7
|
PERMISSIONS
|
NUMBER FORM
|
SYMBOLIC FORM
|
WHAT
JUST HAPPENED
|
EXAMPLE
OF PERMISSIONS USING ls -l $HOME ON viper
|
-rw-------
|
600
|
u=rw
|
Owner has read and write
permissions. Set for most
files
|
 |
-rw-r--r--
|
644
|
u=rw,g=r,o=r
|
Owner has read and write
permissions; group has
read permissions and other have read permissions
|
-rw-rw-rw-
|
666
|
a=rw
|
all have read write
permissions
|
-rwx------
|
700
|
u=rwx
|
only owner has read,
write and execute permissions
|
-rwxr-xr-x
|
755
|
u=rwx,go=rx
|
Owner has read,write and
execute; group has read
and execute; other has read and execute
|
-rwxrwxrwx
|
777
|
a=rwx
|
all have read write
permissions
|
-rwx-x--x
|
711
|
u=rwx,go=x
|
Owner read, write and
execute permissions; group
and other have execute permissions
|
SHARING DIRECTORIES AND FILES
In aero461 the projects will be completed by groups of 3 or 4 students.
File sharing will be common for the different stages of the project
ranging from code development to the final report. One member of the
group will be in charge of code development while another member may be
in charge of post processing-processing and the third member will write
the report. So, files, data and figures will have to be shared among
the members. One easy, but insecure way to do this is to use a
designated scratch space which will act as the main repository of
the project. The other members can then copy items out of the main
repository to their respective accounts. By default the scratch space
is only accessible by its owner. To make sure that group members can
access the scratch space the permissions associated with the "others"
field will require an execute, "x" permission. A directory can only be accessed
if the execute permission is set
<bash>
chmod o+x /remote/scratch/joeusr
|
CREATING YOUR mail.eng.iastate account for mail
It is important that you create and setup your native
mail.eng.iastate.edu account. When using the linux machines you are on
the engineering server and if you need to mail attachments that are in
your engineering account it comes in handy when you have a
mail.eng.iastate.edu account
The password required to access the mail.eng.iastate server will be
your engineering password.
Account name:
- joeusr@mail.eng.iastate.edu
|
|
SERVER SETTINGS
- IMAP MAIL SERVER
- SERVER NAME: mail.eng.iastate.edu
- USER NAME: joeusr
|
|
OUTGOING SERVER (SMTP)
- SERVER NAME: mailhub.iastate.edu
|
|
|
