7.6. Information exchange services at user level

7.6.1. The mail transport agent (MTA)

An mail transport agent (MTA) is responsible for sending/receiving mails from an e-mail server to/from Internet, implementing the simple mail transfer protocol (SMTP). By default, Debian uses exim, because it is easier to configure than other MTA packages, such as smail or sendmail (the latter is one of the precursors). exim offers advanced features such as rejecting known SPAM site connections, it has defences against junk mail or mail bombing and is extremely efficient at processing large amounts of mail. It is run through inetd on a line in the configuration file /etc/inetd.conf with parameters for normal configurations (or xinetd).

exim uses a configuration file in /etc/exim/exim.conf, which can be modified manually, but it is advisable to do so using a shell script called eximconfig, in order to be able to configure exim interactively. The configuration values will depend on the machine's situation; however, its connection is extremely easy, since the script itself suggests the default values. Nonetheless, in /usr/doc/exim we can find examples of typical configurations.

We can test whether the configuration is valid with exim-bV and, if there are errors in the configuration file, the program will show them on screen or, if everything is correct, it will simply indicate the version and date. To test if it can recognise a local mailbox, use:

exim -v -bt local_user

Which will show the layers of transport used and the user's local address. We can also do the following test with a remote user by replacing local user with a remote address to see how it behaves. Then try sending a local mail message and remotely, passing the messages directly to exim (without using an agent, for example, mailx), by keying in for example (all together):

exim postmaster@OurDomain
From: user@domain
To: postmaster@OurDomain
Subject: Test Exim
Test message
^D

Next, we can analyse the mainlog and paniclog track files in /var/log/exim/ to see its behaviour and see what error messages have been generated. Obviously, we can also connect to the system as the postmaster user (or as the user to which the mail has been sent) and read the mail messages to see if everything is correct. The other way consists of running it in debug mode using -dNro as a parameter, where Nro is the debug level (1-9). The normal parameter with which we should boot it is exim -bs, whether by inetd or xinetd. It is also possible to run it as a daemon through /etc/init.d/exim start in systems that require a high mail processing capacity. See the documentation (included in Debian in the exim-doc-html package) in order to configure filters, verification of hosts, senders etc. It is also interesting to install the eximon package, which is an exim monitor that allows the administrator to see the queue of mail messages and logs and to act on messages in the queue in order to distribute them (freezing, bouncing, thawing...).

The latest version of exim is exim4 (it can be installed with apt-get install exim4-daemon-heavy (and also install exim4-config which will help to configure exim4) – bear in mind that there are different packages with different possibilities but exim4-daemon-heavy is the most complete. We recommend reading /usr/share/doc/exim/README.Debian.gz and update-exim4.conf(8). For further information, see the HowTo section http://www.exim.org/docs.html. One of the small differences to consider in the configuration is that instead of having a single configuration exim.conf (the default option if we install exim from the sources) the package exim4-config (it is advisable to install it) uses small configuration files instead of a single one and that these will be in /etc/exim4/conf.d/* and will be chained into a single file (/var/lib/exim4/config.autogenerated by default) by update-exim4.conf.

7.6.2. Internet message access protocol (IMAP)

This service allows access to mail messages stored in a single server through a mail client such as Thunderbird or the Seamonkey mail client (both in mozilla.org). This service supported by the imapd daemon (the current ones support the IMAP4rev1 protocol) allows an electronic mail file that is on a remote machine. The imapd service is offered through the 143 (imap2) or 993 (when SSL encryption is supported) (imaps) ports. If we use inetd, this server is booted through a line in /etc/inetd.conf as:

imap2 stream tcp nowait root /usr/sbin/tcpd /usr/sbin/imapd
imap3 stream tcp nowait root /usr/sbin/tcpd /usr/sbin/imapd

In this example, the tcpd wrapper is called, which functions with hosts.allow and hosts.deny in order to increase security. The most popular applications are uw-imapd (University of Washington and installed by default in Debian) or its secure version uw-imapd-ssl, but also cyrus-imap or courier-imap. To test that the imap server functions, we could use a client, such as seamonkey -mail and create an account for a local user and configure it appropriately so that it connects over the local machine, verifying that imap works correctly.

On Debian, the imap version has been compiled to support MD5 as the method for authenticating remote users, for encrypting connection passwords and to avoid replaced identities by sniffing on the network (the client used to connect to the imap server must also support the MD5 authentication method). The method is very simple and secure, but the server must know the passwords in plain text of the mail users, meaning that it is advisable to use the version of imapd over SSL which functions over port 993. Like ssh, the imaps protocol is based on encrypting the communication through a host certificate (the client used for connecting to the server must also support this connection method, for example thunderbird or seamonkey -mail). To configure the imaps server, install the Debian package uw-imap-dsslwhich is the imap server with SSL support.

The installation will generate an auto-signed certificate valid for one year and stored in /etc/ssl/certs/imapd.pem. This certificate can be replaced by one signed by a certifying company or can generate its own one using OpenSSL. It is advisable to leave just the imaps entry in the file /etc/inetd.conf and to remove the imap2 and imap3 entries if we want the access to imap to be only by SSL.

Another protocol with similar characteristics which has been very popular in the past but that has been overtaken now by IMAP, is the post office protocol (POP) version 2 and 3. It is installed and booted in the same way as IMAP. There are numerous POP servers, but the most common ones are courier-pop, cyrus-pop3d, ipopd (University of Washington), qpopper, solid-pop3d.

7.6.2.1. Complementary aspects

Let's suppose that as users we have 4 email accounts on different servers and that we would like all email messages that are sent to these accounts to be gathered into a single one; to access that account externally and for it also to have an anti-spam filter.

First, we will have to install exim + Imap and check that they work. We need to take into account that if we install courier-imap (which according to some authors is better than uw-imapd) it functions over a mail format called Maildir, that exim will also have to be configured to run over maildir with the following configuration in /etc/exim/exim.conf (or the corresponding one if we have exim4), changing the option mail_dir format = true (the mails will be saved in the local user account in a directory called Maildir). Then we will have to reinitiate the exim server with /etc/init.d/exim restart, repeat the operational test by sending us an email message and read it with a client that supports maildir (for example mutt -mailx does not support it – see http://www.mutt.org).

To fetch the mail from the different accounts we will use fetchmail, (which is installed with apt-get install fetchmail). Next, we will have to create the .fetchmailrc file in our $HOME (we can also use the fetchmailconf tool) which will have to contain something like:

set postmaster "pirulo"
set bouncemail
set no spambounce
set flush

poll pop.domain.com proto pop3
user 'user1' there with password 'secret' is pirulo here

poll mail.domain2.com
user 'user5' there with password 'secret2' is 'pirulo' here
user 'user7' there with password 'secret3' is 'pirulo' here

The action set tells Fetchmail that this line contains a global option (error sending, delete mail from servers...). Next, we will specify the mail servers: one for checking if there is mail with the POP3 protocol and another for testing the use of several protocols to find one that works. We check the mail of two users with the second server option, but all mail found is sent to pirulo's mail spool. This allows us to check several mailboxes of different servers as if they were a single MUA mailbox. The specific information of each user starts with the action user. The fetchmail can be put in the cron (for example in /var/spool/cron/crontabs/pirulo adding 1 * * * * /usr/bin/fetchmail -s), so that it runs automatically or can be run in daemon mode (put set daemon 60 in .fetchmailrc and run it once for example in Autostart of Gnome/KDE or in .bashrc – it will run every 60 seconds).

To remove junk mail we will use SpamAssassin (apt-get install spamassassin) and we can configure Kmail or Evolution (check the bibliography to see how to configure it) for them to run it. In this configuration we will use Procmail, which is a very powerful tool (it allows mail distribution, filtering, automatic resending...). Once installed (apt-get install procmail), we need to create a file called .procmailrc in each user's home which will call the Spamassassin:

  • Setyesfor functioning or debugging messages
    VERBOSE=no

  • We suppose that the mails are in "~/.Maildir", change if it is another PATH=/usr/bin:/bin:/usr/local/bin:

    MAILDIR=$HOME/Maildir
    DEFAULT=$MAILDIR/
    # Directory for storing the files
    PMDIR=$HOME/.procmail
    # Comment if we do not want a log of Procmail
    LOGFILE=$PMDIR/log
    # Spam filter
    INCLUDERC=$PMDIR/spam.rc

The file ~/.procmail/spam.rc contains:

# If the spamassassin is not on the PATH
# add the directory to the PATH variable:
# 0fw: spamassassin.lock|
| spamassassin - a

# The three following lines will move
# Spam mail to a directory called
# "spam-folder". If we want to save it in the Inbox, so that
# it can be filtered later with the client, comment the three lines.

:0:
* ^X-Spam-Status: Yes
spam-folder

The file ~/.spamassassin/user_prefs contains some useful configurations for spamassassin (see the bibliography):

#User preferences file. Ver man
#Mail::SpamAssassin::Conf
#Threshold for recognising a Spam: #Default 5, but with 4 it works a bit better
required_hits 4
# Sites we will never consider Spam to
#come from
whitelist_from root @debian.org
whitelist_from *@uoc.edu
#Sites SPAM always comes from
#(separated by commas)
blacklist_from viagra@domain.com
#Addresses on Whitelist and blacklist are
#global patterns such
#as:"friend@place.com", "*@isp.net", or
#"*.domain.com".
#Insert the word "[SPAM]" in the subject
#(to make filtering easier).
#If we do not wish to comment the line.
subject_tag [SPAM]

This will generate a X-Spam-Status tag: Yes in the message heading if it believes that the message is Spam. Then we will have to filter these and put them in another file or to delete them directly. We can use procmail to filter mails from domains, users etc. For further information, visit http://www.debian-administration.org/articles/242. Finally, we can install a mail client and configure filters so that it selects all email messages with X-Spam-Status: Yes and deletes them or sends them to a directory where we will later verify false positives (mails identified as junk but that are not). A complementary aspect of this installation is if we wish to have a mail server through webmail (in other words, to be able to check the mails from a server through a navigator without having to install or configure a client – like consulting a gmail or hotmail account) it is possible to install Squirrelmail (apt-get install squirrelmail) in order to offer this service. For Debian visit http://www.debian-administration.org/articles/200.

There are other possibilities as discussed at http://www.debian-administration.org/articles/364 installing MailDrop instead of Procmail, Postfix instead of Exim, or including Clamav/Amavisd as an antivirus (Amavisd allows postfix to be linked with spamassassin and clamav).

7.6.3. News

The news or discussion groups are supported through the Network News Transfer Protocol (NNTP). Installing a news server is necessary if we wish to read news offline, if we wish to have a repeater of the central servers or if we wish to have our own news master server. The most common servers are INN or CNEWS, but they are complex packages designed for large servers. Leafnode is a USENET package that implements a TNP server, especially suited for sites with small groups of users but from which we wish to access a large number of news groups. This server is installed in the basic Debian configuration and can be reconfigured with dpkg-reconfigure leafnode for all parameters such as central servers, type of connection etc. This daemon starts up from inetd in a similar way as imap (or with xinetd). Leafnode supports filters through regular indicated expressions (of the type ^Newsgroups:. * [,] alt.flame$) in /etc/news/leafnode/filters, where for each message the heading is compared to the regular expression and if there is a match, the message is rejected.

This server is simple to configure and all the files must be the property of a news user with authorisation to write (check that this owner exists in /etc/passwd). All control, news and configuration files are found in /var/spool/news except for the configuration of the server itself which is in the /etc/news/leafnode/config file. The configuration has some obligatory parameters that must be configured (for example, so that the server can connect to the master servers). They are server (news server from which the news will be obtained and sent) and expire (number of days that a thread or session has been read and will be deleted). Likewise, we have a set of optional parameters of a general or specific nature to the server that can be configured. For further information, see the documentation (leafnode man or /usr/doc/leafnode/README.Debian).

To check the server performance, we can run:

telnet localhost nntp

and if everything works correctly, it will show the server identification and will wait for a command, as a test, we can enter help [to abort, Ctrl+ (and then Quit)].

7.6.4. World Wide Web (httpd)

Apache is one of the most popular servers with the best capabilities in terms of hypertext transfer protocol (HTTP). Apache has a modular design and supports dynamic module extensions during its execution. It is highly configurable in the number of servers and available modules and supports various mechanisms of authentication, access control, metafiles, proxy caching, virtual servers etc. With modules (included in Debian) it is possible to have PHP3, Perl, Java Servlets, SSL and other extensions (see the documentation in http://www.apache.org).

Apache is designed to be executed as a daemon standalone process. This way it creates a set of subsidiary processes that will handle entry requests. It can also be executed as an Internet daemon through inetd, meaning that it will start up every time it receives a request. The server's configuration can be extremely complex depending on the requirements (check the documentation), however, here we can see a minimum acceptable configuration. The configuration files are in /etc/apache and are httpd.conf (main configuration file), srm.conf, access.conf (these last two are maintained for compatibility), mime.conf (MIME formats) and magic (file identification number). The log files are in /var/log/apache and are error.log (registers the errors in the server requests), access.log (register of who has accessed what) and apache.pid (process identifier).

Apache boots from the start up script /etc/init.d/apache and /etc/rcX.d, but can be controlled manually through the apachectl command. The apacheconfig command can also be used in order to configure the server. The default directories (in Debian) are:

The default file that is read from each directory is index.html. After installing the apache and apache-common packages, Debian basically configures the server and initiates it. We can check that it functions by opening a browser (for example, the Konqueror, and typing "http://localhost" in the URL bar, which will load the page /var/www/index.html).

7.6.4.1. Manual (minimum) configuration of httpd.conf

Let's look at some of the most important parameters to be configured in Apache (the example is taken from Apache version 1.X and there are some minor changes if we use version 2).

7.6.4.2. Apache 2.2 + SSL + PHP + MySQL

An important aspect of dynamic web servers is making the most of the advantages of Apache in secure mode (SSL), PHP (is programming language generally used to create web site content) and MySQL+PHPAdmin (database that will be discussed in later chapters and graphic interface for managing it) all working in combination. We will start by installing it on a Debian Sarge, but not through the deb packages but rather from the software downloaded from the relevant sites, this way we can repeat the experience with other distributions. Obviously, afterwards it will not be possible to control these packages using apt or another package manager. We need to take care with the versions, which can change, and not to install the package over already installed packages.

a) Download the necessary files (for example within the directory /root -> cd /root):

1) Apache: from http://httpd.apache.org/download.cgi: httpd-2.2.4.tar.bz2

2) PHP: from http://www.php.net/downloads.php PHP 5.2.1 (tar.bz2)

3) MySQL from http://mysql.org/get/Downloads/MySQL-4.1/mysql-standard-4.1.21-pc-linux-gnu-i686.tar.gz/from/pick

4) PHPAdmin from http://prdownloads.sourceforge.net/phpmyadmin/phpMyAdmin-2.9.1-all-languages.tar.bz2?download

b) Utilities: bzip2 libssl-dev openssl gcc g++ cpp make (verify that they are not installed or otherwise, run apt-get install bzip2 libssl-dev openssl gcc g++ cpp make.

c) Apache:

cd /root tar jxvf httpd-2.2.4.tar.bz2 cd httpd-2.2.4

With prefix, we indicate that we will install for example /usr/local/apache2 ./configure --prefix=/usr/local/apache2 \ –with ssl=/usr/include/openssl \ --enable-ssl make make install

We modify the configuration file /usr/local/apache2/conf/httpd.conf andchange the user and workgroup for www-data:

User www-data
Group www-data

We change the owner and group of the data directory to

www-data:chown -R www-data:www-data /usr/local/apache2/htdocs

We modify the user www-data to change its home directory in /etc/passwd:

www-data:x:33:33:www-data:/usr/local/apache2/htdocs:/bin/sh

Apache server installed. To initiate it (to stop it, change start for stop):

/usr/local/apache2/bin/apachectl start

We can place a script to start up the apache server upon booting.

ln -s /usr/local/apache2/bin/apachectl /etc/rcS.d/S99apache chmod 755 /etc/rcS.d/S99apache

d) SSL:

In /usr/local/apache2/conf/httpd.conf we remove the comment from the line
Include conf/extra/httpd-ssl.conf

The files are generated with the keys for the secure server, in /root we run (adjust the versions to the ones that have been downloaded) – the first openssl command is a long line and ends with 1024:

openssl genrsa -rand ../httpd-2.2.4.tar.bz2:../php-5.2.1.tar.bz2:../phpMyAdmin-2.9.1-all-languages.tar.bz2 -out server.key 1024
openssl rsa -in server.key -out server.pem
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 720 -in server.csr -signkey server.key -out server.crt

We copy the files...

cp server.crt /usr/local/apache2/conf/
cp server.key /usr/local/apache2/conf/

We restart the server...

/usr/local/apache2/bin/apachectl restart

We can check how to add the SSL module to a server that does not have it installed at http://www.debian-administration.org/articles/349.

e) MySQL (for more information see module 8):

We create a group and a user for MySQL if it does not exist.

groupadd mysql
useradd -g mysql mysql

In the directory where we will install MySQL (/usr/local/) we type:
cd /usr/local/ gunzip < /root/mysql-standard-4.1.21-pc-linux-gnu-i686.tar.gz | tar xvf - ln -s mysql-standard-4.1.21-pc-linux-gnu-i686 mysql cd mysql

We create a database and change the permissions
scripts/mysql_install_db --user=mysql chown -R root. chown -R mysql data chgrp -R mysql.

We can place a script for initiating the mySQL server.
ln -s /usr/local/mysql/support-files/mysql.server /etc/rcS.d/S99mysql.server chmod 755 /etc/rcS.d/S99mysql.server

We start the server
/etc/rcS.d/S99mysql.server start

We can enter the database and change the password of the root user for security (consulthttp://dev.mysql.com/doc/refman/5.0/en/index.htmlfor the syntax)
/usr/local/mysql/bin/mysql

Inside, we can type:
        USE mysql

We place the password pirulo on the user root
        UPDATE user SET Password=PASSWORD('pirulo') WHERE User='root';
        FLUSH privileges;

To enter MySQL we will have to type
/usr/local/mysql/bin/mysql -u root -ppirulo

f) PHP (replace with the appropriate versions):

Necessary utilities:

apt-get install libxml2-dev curl \ libcurl3-dev libjpeg-mmx-dev zlib1g-dev \ libpng12-dev

With the Apache server stopped we can type:

cd /root tar jxvf php-5.2.0.tar.bz2 cd php-5.2.0

With the prefix we can indicate where we want to install it (all on one line):

./configure --prefix=/usr/local/php5 --enable-mbstring --with-apxs2=/usr/local/apache2/bin/apxs --with-mysql=/usr/local/mysql --with-curl=/usr/include/curl --with-jpeg-dir=/usr/include --with-zlib-dir=/usr/include --with-gd --with-xml --enable-ftp --enable-bcmath

make make install cp php.ini-dist /usr/local/php5/lib/php.ini

We modify Apache (/usr/local/apache2/conf/httpd.conf) in the indicated part:

<IfModule mime_module>
        AddType application/x-httpd-php .php .phtml
         AddType application/x-httpd-php-source .phps

And also:

DirectoryIndex index.php index.html

We restart the server.

g) PHPAdmin

cd /usr/local/apache2/

The phpmyadmin is decompressed in the apache2 directory (be careful with the versions).

tar jxvf /root/phpMyAdmin-2.9.1-all-languages.tar.bz2 mv phpMyAdmin-2.9.1-all-languages phpmyadmin cd phpmyadmin cp config.sample.inc.php config.inc.php

We need to modify the configuration file (config.inc.php):

$cfg['blowfish_secret'] = 'pirulo';

We remove the user and user password by default two quotation marks (') one after the other:

$cfg['Servers'][$i]['controluser'] = '';
$cfg['Servers'][$i]['controlpass'] = '';

We change apache (/usr/local/apache2/conf/httpd.conf) adding in <IfModule alias_module>

<IfModule alias_module>
        Alias /phpmyadmin "/usr/local/apache2/phpmyadmin/"
<Directory "/usr/local/apache2/phpmyadmin/">
        Order allow, deny
        Allow from all
</Directory>

We reinitiate the server and we can it call with http://localhost/phpadmin

Further information can be obtained from the respective websites of each application and in LWP.