Differences

This shows you the differences between two versions of the page.

--- wiki:cli:scripts [2014/08/10 17:53] – Add scripting lesson body jensv
+++ wiki:cli:scripts [2022/07/21 06:59] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
+===Objectives===
+  *Write a shell script that runs a command or series of commands for a fixed set of files.
+  *Run a shell script from the command line.
+  *Write a shell script that operates on a set of files defined by the user on the command line.
+We are finally ready to see what makes the shell such a powerful programming environment.
+We are going to take the commands we repeat frequently and save them in files
+so that we can re-run all those operations again later by typing a single command.
+For historical reasons,
+a bunch of commands saved in a file is usually called a shell script,
+but make no mistake:
+these are actually small programs.
+Let's start by putting the following lines in the file ''sorted_lengths.sh'':
+    #! /bin/bash
+    wc -l *.pdb | sort -n > sorted_lengths.txt
+The first line is a "shebang" or "hash bang" it tells the shell which program to run the script with. In this case it is a bash script. For a python script one would add the path for python.
+The second line is our pipe & filter command for generating a sorted list of file line counts.
+We can try to run the script:
+     $./sorted_lengths.sh
+     bash: ./sorted_lengths.sh: Permission denied
+Not quite what we expected. We don't have permission to execute / run the file.
+Unix controls who can read, modify, and run files using *permissions*.
+Users can belong to any number of groups,
+each of which has a unique group name
+and numeric group ID
+The list of who's in what group is usually stored in the file ''/etc/group''.
+Now let's look at files and directories.
+Every file and directory on a Unix computer belongs to one owner and one group.
+Along with each file's content,
+the operating system stores the numeric IDs of the user and group that own it.
+The user-and-group model means that
+for each file
+every user on the system falls into one of three categories:
+the owner of the file,
+someone in the file's group,
+and everyone else.
+For each of these three categories,
+the computer keeps track of
+whether people in that category can read the file,
+write to the file,
+or execute the file
+(i.e., run it if it is a program).
+^         ^ user  ^ group ^ all ^
+| read    | yes   | yes   | yes |
+| write   | yes   | no    | no  |
+| execute | no    | non   | no  |
+it would mean that:
+  *the file's owner can read and write it, but not run it;
+  *other people in the file's group can read it, but not modify it or run it; and
+  *everybody else can do nothing with it at all.
+Let's look at this model in action.
+Let's run ''ls -l''
+   $ ls -l
+   -rw--r--r 1 jens jens 1178 Aug 10 14:31 cubane.pdb
+   -rw--r--r 1 jens jens  634 Aug 10 14:31 ethane.pdb
+   -rw--r--r 1 jens jens  431 Aug 10 14:31 methane.pdb
+   -rw--rw-r 1 jens jens   57 Aug 10 14:31 sorted_lengths.sh
+The ''-l'' flag tells ''ls'' to give us a long-form listing.
+It's a lot of information, so let's go through the columns in turn.
+On the right side, we have the files'  names.
+Next to them,
+moving left,
+are the times and dates they were last modified.
+Backup systems and other tools use this information in a variety of ways,
+but you can use it to tell when you (or anyone else with permission)
+last changed a file.
+Next to the modification time is the file's size in bytes
+and the names of the user and group that owns it
+(in this case, ''jens'' and ''jens'' respectively).
+We'll skip over the second column for now
+(the one showing ''1'' for each file)
+because it's the first column that we care about most.
+This shows the file's permissions, i.e., who can read, write, or execute it.
+Let's have a closer look at one of those permission strings:
+''-rwxr-xr-x''.
+The first character tells us what type of thing this is:
+'-' means it's a regular file,
+while 'd' means it's a directory,
+and other characters mean more esoteric things.
+The next three characters tell us what permissions the file's owner has.
+Here, the owner can read, write, and execute the file: ''rwx''.
+The middle triplet shows us the group's permissions.
+If the permission is turned off, we see a dash, so ''r-x'' means "read and execute, but not write".
+The final triplet shows us what everyone who isn't the file's owner, or in the file's group, can do.
+In this case, it's 'r-x' again, so everyone on the system can look at the file's contents and run it.
+To change permissions, we use the ''chmod'' command
+(whose name stands for "change mode").
+    $ chmod u+x sorted_lengths.sh
+The 'u' signals that we're changing the privileges
+of the user (i.e., the file's owner),
+''+'' that we are adding permissions,
+and ''rw'' is the new set of permissions.
+A quick ''ls -l'' shows us that it worked,
+because the owner's permissions are now set to read and write:
+    $ ls -l
+    -rw--r--r 1 jens jens 1178 Aug 10 14:31 cubane.pdb
+    -rw--r--r 1 jens jens  634 Aug 10 14:31 ethane.pdb
+    -rw--r--r 1 jens jens  431 Aug 10 14:31 methane.pdb
+    -rwx-rw-r 1 jens jens   57 Aug 10 14:31 sorted_lengths.sh
+Now we can finally run our script.
+    $ ./sorted_lengths.sh
+We check the result with ''ls'' and ''cat''.
+    $ ls
+    cubane.pdb           ethane.pdb           methane.pdb
+    sorted_lengths.sh    sorted_lengths.txt
+    $ cat sorted_lengths.txt
+methane.pdb
+ethane.pdb
+propane.pdb
+cubane.pdb
+pentane.pdb
+octane.pdb
+total
+Maybe we want to make the input more flexible so that we can tell the script which files to run on.
+Let's edit the file:
+    #! /bin/bash
+    wc -l $* | sort -n > sorted_lengths.txt
+''$*'' means "All of the command-line parameters to the shell script."
+Before we try this out let's add a comment:
+   #! /bin/bash
+   # Write sorted file length (lines) list in sorted_lengths
+   # Usage: sorted_lengths.sh filenames
+   wc -l $* | sort -n > sorted_lengths.txt
+A comment starts with a ''#'' character and runs to the end of the line. The computer ignores comments, but they're invaluable for helping people understand and use scripts.
+We can run the script specifying files
+    $ ./sorted_lengths.sh methane.pdb ethane.pdb propane.pdb
+or using the wild-card:
+    $ ./sorted_lengths.sh *.pdb
+Let's make another script:
+    #! /bin/bash
+    # Select lines from the middle of a file
+    # Usage: middle.sh filename -end_line -num_lines
+    head $2 $1 | tail $3
+Inside a shell script, ''$1'' means "the first filename (or other parameter) on the command line". ''$2'' and ''$3'' mean the "second parameter" and "third parameter", respectively.
+What does this script do?
+Now we have two scripts we can run in succession. When we make a small change to one of the input file is there a quick way to rerun the whole analysis?
+===Key Points===
+  *Save commands in files (usually called shell scripts) for re-use.
+  *Setting permissions of a file.
+  *''$*'' refers to all of a shell script's command-line parameters.
+  *''$1'', ''$2'', etc., refer to specified command-line parameters.
+  *Letting users decide what files to process is more flexible and more consistent with built-in  Unix commands.