Frequently Asked Questions about UNIX and the DCI Michael A. Dickerson | 15 Nov 2005 This document is available in PDF and HTML from http://www.dci.pomona.edu/docs.

______________________________________________________________________________________________________________________________________________________________________________

Contents

1 What can you do for me?

1.1 For whom was this written?

This FAQ was written for students, faculty, and staff of Pomona College in Claremont, California. The natural sciences departments at Pomona operate a diverse group of Solaris, IRIX, Linux, and FreeBSD computers that we call the “distributing computing infrastructure” or DCI. If you are not associated with Pomona College, but you found this document in some other way, many of the answers will still be helpful, but if specific instructions do not work, it is probably because they were written for our environment.

1.2 What does DCI stand for?

Truth. Justice. Liberty. Equality. Or, if you mean, what is it an acronym for, it’s “Distributed Computing Infrastructure,” a euphemism for the collection of assorted Unix workstations, servers, and other hardware that I maintain for the natural sciences departments at Pomona.¹

There are a few core services that unite these many scattered computers into something that might deserve to be called a “distributed computer.” I use “DCI” as an adjective to describe a computer which is integrated with these services. They are:

Authentication:

the same accounts can be used across all DCI hosts

File storage:

the same home directory is presented when you log into any DCI host, except for the SGI IRIX workstations

Backups

are centrally managed for most DCI hosts.

Printing

uses the same centrally managed queues and accounting on all DCI hosts

Firewall:

DCI hosts are gathered on a private subnet with filtered traffic, so that they can enjoy all these shared services that would, in some cases, be unacceptably insecure.

1.3 What operating systems do you support?

The following operating systems are actively in use and supported by me on DCI hosts:

• Debian GNU/Linux 3.1 (sarge)
• FreeBSD 4
• FreeBSD 5
• Solaris 9 SPARC
• Red Hat Enterprise Linux 3 (when under RHN support contract)
• Apple OS X 10.4 (as it relates to Unix: I set up authentication, home directories, the X server, and things like that. I do not help with Microsoft Office, Safari, iTunes, and ordinary desktop stuff.)

The following are operating systems that I have some experience with, but I do not claim to support:

• Red Hat Linux 7.3
• Red Hat Linux 8 (All versions of Red Hat Linux have been discontinued and are no longer supported by Red Hat.)
• Debian GNU/Linux sid/unstable
• Solaris 8 on SPARC
• Apple OS X 10.2/Jaguar, 10.3/Panther (We are considering this in the CS lab and presently investigating how well it can be integrated with the established DCI services.)
• Silicon Graphics IRIX 6.5 (does not support DCI home directories, and has kludged-in, limited support for DCI accounts)

1.4 Whom do you support?

I am an employee of the mathematics, physics, and chemistry departments of Pomona College. This means that I have to pay attention to questions and requests from faculty in the math, computer science, physics, astronomy, and chemistry programs before anything else. When there is time left over, I try to help any other students, staff, or faculty at Pomona College that are interested in Unix or open source.

If you are none of the above (this includes alumni and former employees), I try to help you when there is time left over from all of those people. Do not be surprised if this does not happen very often (see 1.5).

1.5 Why is it taking so long for you to help me?

First ask yourself if you are a member of the group of people that I am paid to help, as described in 1.4. If not, I have no choice but to leave your request at a low priority.

Then ask yourself whether your computer runs an operating system that I support, as described in 1.3. This does not include Red Hat installations where you have chosen not to renew your Red Hat support contract. If not, I have to leave your request at a low priority.

My situation being what it is, in 2005, means that you should expect “low priority” requests to get done some time between “several weeks from now” and “never.” I am not kidding about that.

I do wish it had not come to this, but my “high priority” work has increased dramatically since 2002, while at the same time I have lost the ability to hire assistants. There is currently no reason to think that this situation will improve, and that I will be able to again spend weeks fiddling with your Web page next month, next semester, or next year.

2 Your DCI account

2.1 Can I have an account?

Assuming you are somehow affiliated with Pomona College, yes. I am happy to give you access to at least the core DCI servers, which will allow you to experiment with Linux, use email, and host web pages. Please go to http://www.dci.pomona.edu and click the “Request Account” link.

Whether or not you may use the Unix servers belonging to each academic department depends on the specific policies of that department. At the time of this writing, the math, CS, and physics departments seem generally willing to grant access to anyone that is a student, former student, or sometimes-student in their department. The chemistry workstations are generally only allowed to students currently engaged in an active research project with a faculty member.

2.2 How do I change my password?

Don’t try to run passwd, the usual system password changing utility. It will almost certainly not work with our distributed account system. Instead, run dpasswd, which is a custom program installed on many hosts, or use the Web interface at http://www.dci.pomona.edu.

3 Loving the command line

3.1 Why are my commands not found?

Probably because they are not in your $PATH. Try the following:

The $PATH variable is a list of directories that your shell will search whenever you type a command name without a path. In this example, when you type ls, the shell will look for a program called /usr/dt/bin/ls, then /usr/bin/ls, and so on. If it falls off the end of the list without finding a program named ls, the response is “command not found.” Hence it is convenient to have your $PATH contain all the directories with programs that you frequently want to run.

If you don’t know how to change environment variables, see the question numbered 3.3, “How do I change my PATH?”

Beware! You may be tempted to put a “.” in your PATH. This produces the DOS-like behavior where you may run any program in the current directory by typing its name. This is considered harmful. The reason is that a devious person could go to a location such as /tmp and create a shell script named “ls.” Then, if you happen to be in /tmp and “.” appears early in your PATH, you could be tricked into running the evil script, with unfortunate consequences. Therefore it is generally thought better to leave “.” out of the PATH, and explicitly run any programs in the current directory by typing a name of the form ./my_script.pl.

3.2 What shell am I running?

I don’t know. Examine the $SHELL environment variable:

$ echo $SHELL  
/bin/shells/bash

If you want to run a different shell, just start it like any other program (e.g. say bash or tcsh). Don’t modify the $SHELL environment variable; that will only cause confusion.

If you want to change your “login shell,” which is the shell that gets started when you log in, go to the DCI web page at http://www.dci.pomona.edu and click “Update user information.”

3.3 How do I change my PATH, or another interesting environment variable?

You can make a one-time change to the PATH by doing something like

But, what you probably want is to change the path that takes effect each time you log in. The way to do this is to put the command into one of your shell’s startup files, such as .cshrc (for csh or tcsh) or .profile (for sh or bash). For example:

You can modify any other interesting or uninteresting environment variable in the same way.

3.4 How do I convert text files from DOS format?

Plain ASCII text files can be transferred from DOS systems, or systems running graphical DOS shells, such as Windows. They will be mostly readable, but DOS inserts a carriage return character at the end of each line, which Unix considers superfluous. Some editors will display these characters as ^M, and you can see them with cat -v:

You can strip out the extra ^M characters with the tr command, like this:

3.5 How do I convert text files from Macintosh format?

Macintosh text files will sometimes contain carriage return characters (ASCII 13) at the end of each line instead of line feeds (ASCII 9). You can use the tr program to translate all appearances of one character to another:

or

3.6 How can I ssh from one machine to another without a password?

Use a feature of ssh called public key authentication, which is capable of substituting for the standard password authentication. First you have to create a key pair:

You can just press enter to accept the default file name and no passphrase. As the program says, you have now created an asymmetric encryption key, which has two parts. The private key, which ssh calls your “identification” or “identity file,” is written in ~/.ssh/id_dsa. This part is already in the right place, and you should leave it there, but make sure that it is readable only to yourself. The corresponding public key is written in ~/.ssh/id_dsa.pub. You need to copy the contents of this file into ~/.ssh/authorized_keys on any machine you want to ssh to. For example, if I wanted to use the key I just created to be able to log in to redlance, I would say:

As you can see, after I put the public key in authorized_keys on redlance, ssh magically let me in without a password.

A particularly relevant special case is when your home directory on the source machine is the same on the destination machine, which is true of the many DCI machines that automount your home directory from a file server. The procedure is the same but we can short-cut it by putting the public key directly in authorized_keys:

You now have an interesting situation, which is that you can ssh from any machine (with the DCI home directory) to any machine (with the DCI home directory) without being asked for a password.

I do not think there is any particular security risk to doing this,² because anyone that gains access to your home directory on one DCI server has no particular motivation to break into the others, since it is the same files and same privileges anywhere. I would not use this same key to give access to non-DCI systems, though.

3.7 What does the “find” command do? It clearly doesn’t find things.

The find command does find things, if you ask nicely. In its simplest form, you can use it to search for files whose names match a given expression:

$ find /usr/src -name '*.c'

The /usr/src is the point in the filesystem you want find to start looking. This argument is required, but it can be . to indicate the current directory. The -name argument tells find to search the names of the files. Note that it is necessary to put quotes around an expression containing shell metacharacters like *, or else the shell will try to interpret them and find will not behave correctly.

find is capable of doing many weird and wonderful things; one feature that is especially useful is the ability to execute an arbitrary command on the files it finds:

$ find /usr/src -name '*.c' -exec grep -l 'panic' {} \;

Like the last example, find will look at all the files under /usr/src, and select the ones whose names match '*.c'. But, instead of printing the matching file names, which is the default action, it will execute the command grep -l 'panic' (file) once for each file. The {} is a magic word meaning “name of file,” and the
; is required to tell find that it has reached the end of the command to exec. Again, you must escape the semicolon with a backslash or else the shell will interpret it, and find will not see it.

Putting it all together, since we know grep -l 'panic' just prints the names of files that contain the string “panic”, this find command will search the contents of all the C files in /usr/src, and print the names of those that contain the word “panic,” thus roughly approximating the Windows “Find in files” function. Hopefully this gives you a hint of the stupendous power of find.

4 Home directories and storage

4.1 What is my disk quota? How can I see it?

Quotas are only imposed on the large RAID volume that contains all of the home directories for all of the faculty and students in math, CS, physics, and chemistry. This is the disk that holds your home directory, obviously, but it is also the same disk that holds /common. So, moving files from /home/mikey to /common/cs (for example) would make no difference to my quota.

You can see how much of your quota is used at any time by saying quota on any of the DCI machines. The output will look like this:

linus ~ $ quota  
Disk quotas for user mikey (uid 24047):  
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace  
savah.dci.pomona.edu:/raid/common  
                3494030  4000000 5000000           26279   40000   50000  
savah.dci.pomona.edu:/raid/home/mikey  
                3494030  4000000 5000000           26279   40000   50000

There are two quotas to worry about. One of them limits the amount of space you can use (measured in “blocks” of 1kB), and the other limits the number of files you can create. In each case, you have a quota that may be different from the hard limit (shown as “limit” above). It is possible to exceed your quotas for a limited time, called the “grace period,” but it is generally impossible to exceed the hard limits, even temporarily. If you remain over quota until the grace period expires, the quota is then treated like a hard limit, and you cannot create any more files or write any more bytes.

If you reach your hard limit (or let the grace period expire while you are over quota), you will not be able to log in to the GNOME desktop. You will need to log in through ssh and delete enough files to get back under quota.

4.2 Why do I have a quota when there is so much space available?

Indeed there is a huge amount of storage (about 1 TB) available on the main DCI disk server, thanks to the generosity of ITS. However, the capacity of the backup system (35G per volume) is much, much less. Given the current state of technology, it is far more expensive to arrange for proper backups of your disk space than it is to get that disk space in the first place.

Another reason is the fact that many computer users will make no effort whatsoever to control their disk usage unless they are forced to. Only after the limit has been reached will they delete some unneeded trash and continue working. If this sort of person has a 100M quota, he or she will accumulate 50M of needed information and 50M of trash. Give the same person a 500M quota, and eventually they will have the same 50M of live data and 450M of trash. This is a sucker’s game from the perspective of the system administrator. Since I don’t know what is important and what is trash, we end up backing up tens of gigabytes of core files and “temporary” data, not to mention PDFs, MP3s, and video files that were downloaded and used once.

4.3 What is this garbage-collection nonsense?

As noted in 4.2, many people do not like to clean up their old files, yet I have to observe certain upper bounds on the amount of space consumed to keep the backup system running happily. Hoping to help this situation, I added a basic “garbage collection” function in the summer of 2004.

The garbage collector is a script that runs every night on savah, which is where your home directory is physically stored. There are four independent algorithms that select files for removal; they are all designed not to remove anything that might possibly be important. The four algorithms are:

core

removes any files whose names match *core that are more than 3 days old.

emacs

removes any files whose names match *~ that are more than 7 days old.

tex

searches for files named *.tex, and for each one that is found, removes any TEX output files that are older than the .tex file. TEX output files are those that have the extensions aux, log, 4ct, 4tc, dvi, idv, lg, tmp, toc, xref, or pdf.

tmp

removes any files that are in the directory ~/tmp and are more than 7 days old.

If you do not like one or more of the garbage collection algorithms, you can “opt out” of them by putting the words nocore, noemacs, notmp, or notex in a file named .trashrc in your home directory.

Anything removed by the garbage collector is archived for at least thirty days, so please ask for assistance if a file you want has disappeared and you suspect it is in the trash. As mentioned, the algorithms have been designed to make this event extremely unlikely.

4.4 How are my files backed up?

We run an automatic tape backup scheduler called AMANDA, which dumps most disks on most DCI machines every weekday night between about 2 and 6 A.M. (Technically, that is early mornings Tuesday to Saturday.) This includes your home directory, everything in /common, and most other disk partitions except the ones mounted on /tmp. The AMANDA dump cycle is about three weeks long, and each night’s dump runs at about 50% of capacity. This means that it is usually possible to recover any version of a file that was backed up in the last three to six weeks.

The backups are stored on industrial tapes with 30 to 50 year readable lifespans, which are themselves kept in fire-resistant storage, so your data should survive any catastrophe up to the magnitude of a building fire that destroys Millikan and Andrew. If a catastrophe of greater magnitude should transpire, it seems unlikely that either you or I will care about what happens to the backups.

If your important files seem to have disappeared, either because of your error or for mysterious reasons, don’t panic. If they mysteriously disappeared, they may have been reaped by the garbage collector (see 4.3). No matter what the cause, files that existed long enough to be backed up can be recovered if you ask me within about six weeks.

4.5 How can I transfer files to and from Unix?

If you used Unix in the olden days, you may remember using ftp or possibly rcp to move files between computers. We can’t allow these protocols anymore, because they are unsafe, but their descendants sftp and scp are available.

If you are using Windows, you need a program called WinSCP: http://winscp.net.

If you are using Apple OS X, you can already run the standard sftp and scp programs inside your Terminal, but you can also get a GUI version called Fugu: http://rsug.itd.umich.edu/software/fugu/.

With either of these programs, or any other sftp client, just connect to whatever DCI server you have been using. If you are a loss, use savah.dci.pomona.edu. Once connected, give your DCI username and password and you should be ready to go.

5 Graphical desktops, GNOME, X11

5.1 How can I improve GNOME performance?

Here are ways to cut out some of the bloat of GNOME 2, which should result in a more responsive desktop:

Kill Nautilus

Nautilus is the default GNOME file manager, and it is nice for people that are highly dependent on things like Windows Explorer. But, for people that never use it, it wastes an enormous amount of memory only to draw the “Trash” and “Home” icons on the desktop. On the Applications menu, click Desktop Preferences, Advanced, Session. Click the Current Session tab, remove Nautilus from the list of programs, and click Apply.

Use Firefox instead of Mozilla

Your Internet menu should have both Mozilla and Mozilla Firefox.³ Firefox is a much lighter web browser based on the same “Gecko” rendering engine. It starts much faster than Mozilla and consumes less memory. Firefox still has the most popular Mozilla features, like pop-up blocking and tabs.⁴

There is an even faster and lighter Mozilla-based browser called Epiphany, which may suit you if you don’t use fancy features. It seems to get buried under Debian Menu, Apps, Net, or you can start it from the command line as “epiphany.” It is also sometimes called “Web Browser” on the Debian menu.

Don’t use gnome-terminal

The scrolling and redrawing speed of gnome-terminal is unbelievably bad. Here is an unscientific comparison of the time various terminals take to cat a 20kb text file:

aterm	.023s
xgterm	.079s
xterm	.152s
gnome-terminal	.753s

Consider using one of the faster terminals. aterm is a small xterm replacement with a scroll bar and a few more features. xgterm is frequently used with IRAF, an astronomy package; I am not sure how it is different from xterm.

Use a lightweight window manager theme

Specifically, avoid window manager themes that require pixmap graphics. These require many extra X11 graphics calls to draw window frames. An example of a fast theme is the “Simple” theme installed with Metacity.

5.2 How do I run a remote desktop session?

It is possible with X11 to get what Windows or Apple would call a remote desktop, but it’s done using something called XDMCP (X display manager control protocol), which we generally avoid because it’s insecure in the same way that telnet and ftp are.

XDMCP isn’t used much anymore, because it is possible to run graphical programs through an ssh connection, assuming that you have an X11 server listening on your local display (e.g. the Apple X11 server, or Xwin32 on Windows). This is because ssh is actually a general purpose encrypted tunnel that we just happen to use for a shell most of the time. If you start ssh with the -X option, such as:

. . . then you can check whether X forwarding was set up correctly like this:

If $DISPLAY is set to localhost:xx.x then you should be able to run any X program such as:

and it will open its own window on your screen.

5.3 Why do my X programs crash on Mac OS X 10.4?

I don’t know. Something changed in the X server that comes with Apple OS X 10.4 (Tiger), such that when you ssh with the -X option as described above, the X programs work for a while and then die with a mysterious “X Window System error.”

You can fix this by adding an extra -Y option to the ssh command, such as: ssh -XY linus.cs.pomona.edu. The -Y option is magic; it was not documented anywhere when I looked; it is just part of the oral tradition.

5.4 Can I make the mouse pointer move faster?

Try saying xset m 5. This turns on “mouse acceleration” which makes it much easier to move the mouse.

5.5 How do I make a screenshot?

Use the import command, which is part of the ImageMagick package that should be installed on your machine. Usually you really only want an image of a particular window, such as a figure displayed in IRAF or SYBYL. In that case, just go to a terminal window and say import figure.png. The mouse pointer will turn to a little crosshairs, which you click on the window that you want to capture. That’s it; a file named figure.png will magickally appear. Note that you can save in whatever format you want by giving the filename as figure.jpeg or figure.bmp or whatever.

If you want an image of the entire screen, add the -window root option:

$ import -window root fullscreen.png

There are other programs that can take screenshots, such as the GIMP and the GNOME screenshot taker that might be on your “Actions” menu.

6 Email Questions

6.1 Can I have an email account?

If you have a DCI account, such that you can log in to any of the Unix computers, then you already have an email account. The address is yourusername@dci.pomona.edu. Thus you are probably now interested in either getting an account (see 2.1), or learning how to access that email (see 6.2).

6.2 How can I access my DCI email?

There are several ways:

PINE

, after logging in through ssh, is the most straightforward and portable way to read and send mail. There is nothing that you need to configure since it is done by me, and you only have to find an ssh client whereever you may be. PINE is available on suntop.dci.pomona.edu, which everyone has access to; also on linus and galileo and possibly other systems.

IMAP

clients should be pointed at imap.dci.pomona.edu, port 993, with SSL enabled.

For an outgoing mail server, use smtp.dci.pomona.edu, on port 25. You must enable TLS and provide your DCI username and password.

If you are using a Microsoft program such as Outlook: Do not enable the option [mis-]labeled “secure password authentication.” If you do not have the TLS option, you can select SSL and set the port to 465.

If you are using Apple OS X Mail (aka Mail.app): As of February 2005, Mail.app does not work with the TLS system on the outgoing mail relay (smtp.dci.pomona.edu). Attempting to send mail will cause Mail.app to hang and eventually crash. This seems to be a bug in Mail.app, which I have asked Apple to look into. In the meantime, you can work around this problem by using a different outgoing mail relay or using Mozilla Thunderbird.

Webmail

is available at https://webmail.dci.pomona.edu.

If you use a lot of different methods to access your email, particularly if you use different IMAP clients at different times, you should be prepared for folder naming schemes and things like that to be inconsistent. I have done what I can to make sure that Pine and IMAP see the same set of folders (that is, ~/mail), but other things, such as where to put sent mail, are client-side behaviour that the server can’t do anything about. See also 6.11.

6.3 How is incoming mail processed?

Postfix is the mail transfer agent (MTA) on suntop. Mail that comes in locally or from the Internet is passed from Postfix to procmail. Procmail is configured to pipe every message through the SpamAssassin and Anomy sanitizer filters before reading your personal .procmailrc file for further instructions. If your .procmailrc does not redirect the mail somewhere else, procmail will deliver to the file named .mail in your home directory.

6.4 How do I disable spam filtering on my DCI mail?

Edit the file .spamassassin/user_prefs in your home directory. You will see a line that defines the number of “required hits.” Uncomment this line if necessary (delete the # character at the beginning), then set the value to some improbably high threshold such as 999. SpamAssassin will continue to rate your messages and assign spamminess scores, but they will only be visible in the headers, and no messages will be rewritten to say, “This message is probably spam,” etc.

6.5 How do I customize or strengthen spam filtering on my DCI mail?

Examine the file .spamassassin/user_prefs in your home directory. If you want to tighten the spam filter generally, edit the definition for required_hits to be some lower value (2.5 works pretty well for me). If you want to alter the scores assigned to individual tests, append a line of the form score TEST_NAME 1.23. The various possible values for TEST_NAME are listed in the headers of messages not considered spam, or in the body of messages marked as spam. If you want to see the default score assignments, have a look at the files in /usr/share/spamassassin on suntop.

For example, any message to me that contains Windows programs is either spam, a virus, or from somebody being stupid, so my user_prefs contains:

6.6 How do I redirect spam to a different folder?

Once SpamAssassin is doing a good job discriminating between spam and ham, it might be nice to have the spam automatically routed to a “junk mail” folder and not to your inbox. This is easily done with a procmail recipe, which you should put in the .procmailrc file in your home directory:

This tells procmail to deliver any message that contains the header “X-Spam-Status: Yes” (which will be set by SpamAssassin) to the mail folder named “junk.”

6.7 What is a Bayesian spam classifier?

So-called “Bayesian filtering,” which is a hot topic in spam filtering research at the moment, is based on statistical analysis of a large number of known spam and non-spam (“ham”) messages. This technique was first made popular by Paul Graham’s paper “A Plan for Spam” which was written in 2002.⁵

____________________________________________________________________________________________________________________________

The name comes from the “Bayes rule” which is a simple formula for calculating the probability that E is true if you already know that A and B are true, and you know that A implies E with some probability a and B implies E with some probability b. Then In the case of spam, we are trying to figure out if a given message is spam (E) given that we know it contains the words A and B, and we know that word A means spam with probability a and word B means spam with probability b. (Note: The formula will be garbled if you are viewing this page in HTML; please look for the PDF form if you are interested.)

It is possible for a Bayesian filter to become extrememly accurate at separating your spam from your ham, if you train it with a large enough body of spam and ham that you have collected yourself. Other peoples’ spam might look like your ham and vice versa, so a Bayesian filter that you haven’t trained yourself is much less effective.

Besides its potential for great accuracy, another advantage to the Bayesian filter is that it continues to adapt to new and evolving spam messages, if you continue to provide it with feedback. A disadvantage is that it is very slow and expensive compared to simpler mail filters, so it has the potential to put a heavy load on a mail server if a large number of messages are being subjected to Bayesian analysis.

6.8 Can I use a Bayesian classifier to identify spam?

Yes. One way is to use an end-user client that includes a Bayes analysis engine. At the time of this writing, these include Mozilla 1.6, Thunderbird 0.5, and the Apple OS X Mail programs.

You can also have Bayesian filtering applied on the server, since SpamAssassin includes a Bayesian analysis engine. It is turned off by default, since (as noted in 6.7) it is slow, and not very effective unless you train it to recognize your individual spam and ham mail.

Before you can train the Bayesian filter, you will need a reasonably large collection (maybe a couple thousand messages) of mail that you have personally received and identified as spam, and another large collection of mail you have identified as ham. Most people already have a large collection of non-spam mail they have saved, but you will need to deliberately save spam for a few months to build up a large enough sample.

Now, assuming you have a file in the standard mbox format named spam-samples and another file named ham-samples, you can activate and train the Bayesian classifier by logging in to suntop.dci.pomona.edu (where SpamAssassin is installed) and doing the following:

• Put the line use_bayes 1 in the user_prefs file which you will find in your home directory under .spamassassin.
• Run the command

  suntop ~ $ sa-learn --mbox --spam --showdots spam-samples

  which will tell the filter to “learn” the messages in spam-samples as examples of spam. This will take a while, so I recommend the --showdots option which gives you some reassurance that something is happening.

• Run the command

  suntop ~ $ sa-learn --mbox --ham --showdots ham-samples

  and the filter will learn the messages in ham-samples as examples of ham. Again, expect this to take maybe 15 minutes per 1000 messages.

SpamAssassin will now apply Bayesian analysis to each incoming message and assign appropriate points. If you would now like to have messages tagged as spam diverted to a folder that is not your inbox, see 6.6.

You can retrain the filter at any time by doing the same thing. It is not necessary to worry about whether you have already fed a given message to the trainer, since it keeps a signature of each message that it has “learned” and will ignore any duplicates you give it.

6.9 What is this “DEFANGED” stuff that keeps happening to my messages?

All DCI mail is piped through a script called the Anomy Sanitizer, which neutralizes (“defangs”) potentially dangerous content such as executable attachments, active HTML, MIME boundary overflow attacks, and so forth.

Some types of attachments (such as Windows executables) are so dangerous and useless that they will never be delivered, and will be “quarantined” instead. If this happens to you, the message will contain instructions for how to obtain your quarantined file. Other types of attachments that are dangerous but also potentially useful, such as Word documents, may be renamed to prevent a stupid email program from opening them automatically. This all happens before the messages are ever delivered to your mailbox, so you are protected from many classes of present and future email threats (e.g. viruses).

Although spam processing can be disabled if you desire (see 6.4), the sanitizer is mandatory. We hope you will recognize this as a feature.

If you would like more details on the Anomy sanitizer, see http://mailtools.anomy.net.

6.10 I need this “winmail.dat” attachment that was dropped.

You really don’t, I promise. winmail.dat is how Microsoft Exchange mail programs attach extra information that is not part of the email, such as whether the sender requested a return receipt. For some reason Exchange does this with a binary attachment encoded in a proprietary format, instead of using a header, like every other email system ever devised. This extra junk information is not useful to you, and can be dangerous to receive, since viruses and worms often transmit themselves this way.

6.11 Why can’t I create subfolders with my IMAP client?

The mail on the DCI server is in mbox format, which means that a mailbox is just a text file containing the concatenated mail messages. The IMAP server will see subdirectories that you create, and mbox files in those subdirectories, but you can’t have a “folder” that contains both messages, and other folders (since that would make no sense in terms of the underlying filesystem).

Different IMAP clients (e.g. webmail, Netscape, Mac OS X Mail) will react to this in different ways; some might let you choose whether to create a “folder” as a directory or an mbox file, and others will just report errors if you try to create a folder inside a folder.

What I would do to minimize ambiguity is create the directory and file structure you want by hand (e.g. mkdir  /mail/OWA; touch  /mail/OWA/mailbox) and then subscribe your various mail clients to the folders once they exist. But there is no general solution to the problem of different IMAP clients not behaving consistently, since the IMAP server acts more or less like a file server and clients make up their own conventions.

6.12 Can I have my ITS Exchange mail forwarded to the Unix system?

Yes. But, proper mail forwarding from the Pomona Exchange system requires that your Exchange mailbox be deleted and replaced with something called a “custom recipient.” I am supposed to warn you that if you do this, you will give up access to Exchange public folders, calendaring, and any other gee-whiz gimcrackery that Microsoft has thought up. Futhermore, the only persons that can help you with any mail problems will be me and any assistants I might or might not have.

If this doesn’t scare you, ask me to have your Exchange mail forwarded and I will ask ITS to create a custom recipient for you. (You can ask ITS yourself, but they might assume you want Exchange-rule forwarding instead of system-level forwarding, which you don’t.) You will want to copy all saved messages out of all folders into your Unix mail account or into local storage, since your Exchange mailbox will be deleted.

6.13 Can I forward my DCI mail to someplace else?

I suppose. All you need to do is put the other email address in a file named .forward in your home directory, such as:

$ echo mikey@gmail.com >~/.forward

When you change your mind, you can un-forward your mail by deleting the .forward file.

7 Web Server (Apache)

7.1 Can I host a Web page?

Yes. From your home directory create a subdirectory named www, if it doesn’t already exist. Make sure it is world-readable, then create any files you want available to the Web, usually starting with index.html:

All files and directory structure that you create under www will be served from the URL http://www.dci.pomona.edu/~username. If a web browser requests a directory name, Apache will look for a file named “index.html” to return by default.

The Apache web server runs from suntop.dci.pomona.edu.

7.2 Can I run CGI scripts?

Yes. At present, Apache will execute any file that it finds in your www space whose name ends in .pl. You can run Perl-based message boards and so on without any special assistance. Remember that if there are any data files that your CGI scripts need to be able to read and write, the web server process will need permission to read or write them. The server runs on suntop as the user and group named “www-data” (uid/gid 33). This probably means your data files will need to be world-writable, or you can ask a system administrator to assign them to the www-data group.

7.3 How can I create a Web page that requires a password?

Create a directory for the files you want to restrict, and create a file named .htaccess in that directory, which contains the following:

You must then use the htpasswd command to create valid user accounts in the .htusers file. The htpasswd command is present on suntop.dci.pomona.edu, but might not be installed anywhere else. This is done in the following way:

A web browser should now require a correct username and password before it can display any files in your restricted directory. (If it does not, Apache is probably not configured to allow authentication and you must ask your systems administrator.) Beware that this sort of Web browser authentication is not at all secure, and should not be used to protect sensitive information. It would be well within the resources of an interested student (or College employee) to intercept the usernames and passwords and gain access to your site.

7.4 Where are the web server logs?

The Apache logs are on suntop.dci.pomona.edu, under /var/log/apache. There is a “combined” format access log named access.log, and the standard server error log is named error.log. The error log is the place to look if one of your CGI scripts is giving the “500 Internal Server Error” message or similar.

8 Frequently Asked Questions that Shouldn’t Be

8.1 I’m using Windows and...

I am sorry to hear about your Windows grievances, and would be happy to help. The first step is to erase your hard drive and install a modern operating system. Please contact me for more information, or feel free to pick up a Knoppix CD from the terminal room outside my office.