Homework 1: Unix Commands and Beginning Shell Scripts

Due September 30, 1997

This assignment has two parts. The purpose of the first part is to point you at several Unix commands that I have found necessary or extremely useful, and to get you to find out about these commands and try them yourself. The purpose of the second part of the assignment is to introduce you to shell programming by writing some CGI scripts to display classified advertisements stored as files on the disk. In a subsequent assignment, you will greatly expand these scripts to allow users to post advertisements as well.

What to Submit

You will submit your work electronically, as described here. The easiest way to hand in all these things at once is probably as follows. First, make a directory cse230 in a convenient place. Go to that directory, and create the HW1_WRITEUP file. Then execute the following commands: cp -Rp ~/public_html/classifieds ./classifieds ~cse230/bin/handin HW1 HW1_WRITEUP classifieds The first command will copy the entire contents of the classifieds directory you made for Part II to your cse230 directory. The second command will hand in the HW1_WRITEUP file, together with the entire contents of the classifieds directory.

Part I: Some Useful Unix Commands

Work your way through the material below, answering the questions that are posed there. Place your answers to these questions in a file HW1_WRITEUP and submit it, together with your scripts for Part II below, as assignment HW1. Numbers in parentheses refer to section numbers in Unix for the Impatient. Don't forget to read online manual pages using man, too!

This part of the assignment, doesn't require an extensive amount of hand-in. However, you shouldn't take a minimalist attitude toward it. Try to explore beyond the specific tasks I have given below. If you are already familiar with most of what is covered, treat the assignment as an opportunity to expand your knowledge.

Session Initialization

Here are some things to try having to do with logging in and initializing your working environment:
Changing your password (5.4.4)
If you are a new user, you should change your password right away. This is done with the passwd command. Read about this command and use it.

C Shell initialization files (.login and .cshrc)
The files .login and .cshrc in your home directory are used by the C shell to initialize your working environment. Check whether you have these files using ``ls -a''. (The -a option is required to get ls to list files starting with ``.'', which it normally does not list.) If you are a new user, you won't have these files. Instead, you will automatically be using the system defaults, which are in the files /etc/csh.cshrc /etc/csh.login

NOTE: If you have copied .cshrc or .login files from some other system, I highly recommend that you remove them and start afresh. Many ``problems'' that are reported to me are in fact caused by people blindly copying .cshrc and .login files from another system. The system defaults on the UG lab are set up so that they are reasonable starting points for most users. Do not add anything to your .cshrc or .login file that you do not understand.

When you log in, the C shell executes these files in the following order: first /etc/csh.cshrc, then /etc/csh.login, then .cshrc in your home directory (if it exists), and finally .login in your home directory (if it exists). If the C shell is not running as a ``top-level'' login shell, then it only runs the /etc/csh.cshrc and .cshrc files. Although there is a certain component of taste involved in what you put in .cshrc and what you put in .login, in general you have to put things that are not propagated to subshells as part of the environment in .cshrc. All other stuff should generally go in .login.

Put a line that reads simply:

msgs -q in your .login, so that when you log in you will be informed if there are any new messages to read.

Environment Variables
The operation of many programs is affected by the settings of various environment variables. The current environment is passed by the shell to any command you execute. Many mysterious problems turn out to be explained by improper settings of environment variables. Generally, environment variables are set from your .login file, using the setenv command of the C shell.

The following is an example of commands to set some important environment variables. These commands (or your own customized versions) could be put in your .login file:

setenv PATH .:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin setenv EDITOR xemacs setenv PAGER less The extremely important PATH environment variable controls where the shell looks to try to find binary executables for commands you type. The EDITOR variable states which editor you prefer, and is consulted by some programs (such as mailers) that start an editor. If you don't set this, you will probably end up in vi (the default editor) at some point. The PAGER variable determines which program you prefer for paginating output to be viewed on a screen. This is important for the correct operation of the man command. The default pager is more. I prefer less. (See section 3.6).

Other important environment variables are TERM, which tells the official name of the type of terminal you are using, and DISPLAY, which is used by X Windows programs to determine which server (screen) to use as the display device.

QUESTION: From shell command level, execute the command:

setenv PATH /usr/bin Now try running the command ls. Now try /bin/ls. In your writeup, explain the results. You can recover from this situation by logging out and logging back in again. In your writeup, describe a way to recover that does not involve logging out.

Using the command which, find out the location (full pathname) of the xterm command. Put it in your writeup.

Using the printenv command of the shell, find out what values you have set for the TERM and the DISPLAY environment variables. Put them in your writeup.

The umask command:
The umask command controls the default permissions that files or directories will have when you create them. By default, your umask is set to 022, which means that write permissions are masked off for everyone but you. However, this means that by default everyone on the system will be able to read your files unless you take steps to prevent them. Another common umask is 077, which by default masks off all permissions for anyone other than you. I recommend that you put the command: umask 077 into your .login file.

Try setting your umask to various values. Then, create files or directories and use ls -l and ls -ld to list them. Briefly explain the results.

Disk Space Management

The du command:
The du command can be used to find out how much disk space you are currently using. Read the man page and try it out.

QUESTION: How can one cause du to display its information in units of kilobytes? How about in units of 512-byte disk blocks? In megabytes?

Disk quotas:
The ``Hints for Newcomers'' item under the UG lab home page contains information on how to find out what your disk quota is. Read it.

QUESTION: What is your disk quota? Attempt to exceed your disk quota by making a number of copies of a large file /usr/share/dict/web2 is convenient. in your home directory. What happened to the last file you tried to write? Now delete all the junk files from your directory.

Useful Commands

The following exercises are supposed to get you to find out about some of the most useful and powerful Unix commands. Find the best solutions you can, and put them in your writeup.
The ls command (3.2):
Read about the ls command. Find the simplest way you can to use ls to determine all the files under your home directory and any subdirectories that have been modified in the last day.

The cp command (3.4.3):
The cp command is used to copy files from one place to another on the system. Find out how to use cp to copy the entire hierarchy of files under your current directory to /tmp.
The grep command (4.2):
The grep command is used to search for patterns in files. Read about grep. Then find all the files in the directory /usr/include (the C system header file directory) that have occurrences of the string PATH_MAX in them.

To take advantage of the full power of grep, you need to understand about regular expressions (2.17). For now, though, you can limit your attention to using it to search for fixed-string patterns.

The find command (3.8.1):
The find command is extremely useful and powerful, but has a syntax that is somewhat difficult for the neophyte. Read about it. Then, construct a find command to change the mode (protection) of all the regular files under your home directory to mode 600 (read/write by you, no permissions to others). Construct a find command to change the mode of all the directories under your home directory to 700 (read/write/execute by you, no permissions to others). Leave your home directory with mode 755, otherwise X Windows won't work.

If the above commands were too easy, try to to construct a "find" command to find all the regular files (not directories) under the directory /var that have not been modified in one week, and for each one to prompt you and ask if you want to view that file using "more".

The gzip and gunzip commands (3.13.2):
The gzip command is used to reduce the space occupied by a file by performing data compression on it. The gunzip command uncompresses a previously compressed file. Read about gzip, then try it out on various kinds of files, including text files and binary executable files. In general, what are the compression ratios for the various kinds of files you tried? Make a copy of a text file, compress it, then uncompress it, and use the diff command (3.10.2) to check whether the result is identical to the original file.

Part II: Simple CGI Scripts

In this section, you will write some simple POSIX (/bin/sh) shell scripts to be invoked by the World Wide Web server on www.ug.cs.sunysb.edu in response to requests from clients on the Internet. You will use the Common Gateway Interface (CGI) standard for passing data from the client to the shell script.

Your objective in this assignment and the next will be to construct a CGI-based system that permits clients on the Internet to read and post classified advertisements. For the moment, you will concentrate on displaying advertisements already online.

Setting up your Web Directory

The first thing you need to do, if haven't done it already, is to create a subdirectory public_html of your home directory, and to make sure that you have set ``world read and execute'' permissions on it. If your user ID is fred, then when somebody on the Internet attempts to access the URL of the form http://www.ug.cs.sunysb.edu/~fred/yourfile.html what happens is the Web server on www.ug.cs.sunysb.edu (actually the same machine as public.ug.cs.sunysb.edu) will respond by sending the file: http://www.ug.cs.sunysb.edu/~fred/public_html/yourfile.html

All files you wish to make available on the Web must have ``world read'' permission set. In addition, all directories the Web server must search to find these files must have ``world execute'' permission set.

To test out whether the Web server can access your public_html directory, you may copy this file and put it as Welcome.html under your public_html directory. Also create a subdirectory classifieds under your public_html directory, and create a subdirectory ads of classifieds. Using a Web browser, access the URL:

http://www.ug.cs.sunysb.edu/~youruserid/ You should see a simple ``Welcome to my home page'' message, with a single button, like this. Clicking on the button should for the moment give you an error message.

In case something went wrong, you can likely get some information about what happened by looking at the Web server error logs. To view these, you have to log in on www.ug.cs.sunysb.edu and look at the files

/A/www/logs/httpd-access_log /A/www/logs/httpd-error_log These are logs of all the Web accesses and error messages produced by the Web server. The server is very active, serving a ``hit'' every few seconds, so it is best to use a pager program such as less to view these files. You'll have to view near the end of the file to see recent messages.

Setting up Skeleton CGI Scripts

To make things a little easier for you, I have created some skeleton HTML files and CGI scripts which you should copy into your newly created ``classifieds'' directory. Make sure that ``world execute'' permission is set on both of the above shell scripts. If you have done this step properly, when you click on the button leading to your classified ads project, instead of seeing a directory listing, you will now see a form that looks like this. If you click on either READ or POST, then if you have installed the select.cgi script properly, you will see a message indicating that you have successfully invoked the select.cgi script, and it will display for you the arguments that were passed from the form.

Basically what happens when you click on the ENTER button, is the Web server invokes the main.cgi shell script. This script consists essentially of a bunch of echo commands, which generate HTML code telling your Web browser to display the form you see. When you click on the READ or POST button on this form, the Web server invokes the select.cgi script, which again consists of echo commands to output HTML code telling your browser to produce the display you see. This time, though, the script uses the cgiparse command to obtain information passed from the form and assign it to shell variables. For example, within the select.cgi script, the shell variable FORM_category variable has as its value the menu item that was selected in the form.

Displaying a List of Advertisement Categories

Now, you get to make some simple modifications to the select.cgi script. For this assignment, you are to do two things:
  1. Modify the main.cgi script so that, instead of displaying a fixed list of categories in the menu, it instead obtains the names of all subdirectories of the directory classifieds/ads and uses those names as the menu items instead. Thus, if classifieds/ads has subdirectories Automobiles and Computers, these are the menu items that will be displayed in the form.

  2. Modify the select.cgi script so that, instead of displaying the CGI arguments to the form, it interprets the value of the FORM_category variable as the name of a subdirectory of classifieds/ads, it obtains a list of all the files in that subdirectory, and then constructs and displays a list of hyperlinks to those files.

    For example, if the FORM_category variable has the value Automobiles, and the directory classifieds/ads/Automobiles contained files 001, 002, and 003, then your script should output HTML code like this:

    <UL> <LI><A HREF="ads/Automobiles/001">001</A>< <LI><A HREF="ads/Automobiles/002">002</A>< <LI><A HREF="ads/Automobiles/003">003</A>< </UL> so that when the page is displayed it will look like this: and so that clicking on a link will cause the contents of that file to be displayed.

Eugene W. Stark