108.3 Lesson 1

Traditional user accounts in network connected machines make up the simplest email exchange scenario, where every network node runs its own MTA daemon. No software other than the MTA is required to send and to receive email messages. In practice, however, it is more common to use a remote email account and not have an active local MTA service (i.e. instead using an email client application to access the remote account).

Unlike local accounts, a remote email account — also called remote mailbox — requires user authentication to grant access to the user’s mailbox and to the remote MTA (in this case, simply called the SMTP server). While the user interacting with a local inbox and MTA is already identified by the system, a remote system must verify the user’s identity before handling its messages through IMAP or POP3.

When an MTA daemon is running on the local system, local users can send an email to other local users or to users on a remote machine, provided their system also has an MTA service that is accepting network connections. TCP port 25 is the standard port for SMTP communication, but other ports may be used as well, depending on the authentication and/or encryption schema being used (if any).

Leaving aside topologies involving the access to remote mailboxes, an email exchange network between ordinary Linux user accounts can be implemented as long as all network nodes have an active MTA that is able to perform the following tasks:

Normally, email addresses specify a domain name as the location, e.g. lpi.org in info@lpi.org. When this is the case the sender’s MTA will query the DNS service for the corresponding MX record. The DNS MX record contains the IP address of the MTA handling the email for that domain. If the same domain has more than one MX record specified in the DNS, the MTA should try to contact them according to their priority values. If the recipient’s address does not specify a domain name or the domain does not have an MX record, then the part after the @ symbol will be treated as the host of the destination MTA.

Security aspects must be considered if the MTA hosts will be visible to hosts on the Internet. For instance, it is possible for an unknown user to use the local MTA to impersonate another user and send potentially harmful emails. An MTA that blindly relays an email is known as an open relay, when it can be used as an intermediary to potentially disguise the true sender of the message. To prevent these misuses, the recommendation is to accept connections from authorized domains only and to implement a secure authentication schema.

In addition to that, there are many different MTA implementations for Linux, each one focusing on specific aspects like compatibility, performance, security, etc. Nevertheless, all MTAs will follow the same basic principles and provide similar features.

The traditional MTA available for Linux systems is Sendmail, a very flexible general purpose MTA used by many Unix-like operating systems. Some of the other common MTAs are Postfix, qmail and Exim. The main reason to choose an alternative MTA is to implement advanced features more easily, as configuring custom email servers in Sendmail can be a complicated task. Also, each distribution may have its preferred MTA, with predefined settings appropriate for most common setups. All MTAs intend to be drop-in Sendmail replacements, so all Sendmail-compatible applications should work regardless of what MTA is being used.

If the MTA is running but not accepting network connections, it will only be able to deliver email messages on the local machine. For the sendmail MTA, the /etc/mail/sendmail.mc file should be modified to accept non-local connections. To do so, the entry

should be modified to the correct network address and the service should be restarted. Some Linux distributions, such as Debian, may offer configuration tools to help bring up the email server with a predefined set of commonly used features.

Once the MTA is running and accepting connections from the network, new email messages are passed to it with SMTP commands that are sent through a TCP connection. The command nc — a networking utility which reads and writes generic data across the network — can be used to send SMTP commands directly to the MTA. If the command nc is not available, it will be installed with the ncat or nmap-ncat package, depending on the package management system in use. Writing SMTP commands directly to the MTA will help you understand the protocol and other general email concepts better, but it can also help to diagnose problems in the mail delivery process.

If, for example, user emma at the lab1.campus host wants to send a message to user dave at the lab2.campus host, then she can use the nc command to directly connect to the lab2.campus MTA, assuming it is listening on the TCP port 25:

Once the connection is established, the remote MTA identifies itself and then it’s ready to receive SMTP commands. The first SMTP command in the example, HELO lab1.campus, indicates lab1.campus as the exchange initiator. The next two commands, MAIL FROM: emma@lab1.campus and RCPT TO: dave@lab2.campus, indicate the sender and the recipient. The proper email message starts after the DATA command and ends with a period on a line by itself. To add a subject field to the email, it should be in the first line after the DATA command, as shown in the example. When the subject field is used, there must be an empty line separating it from the email content. The QUIT command finishes the connection with the MTA at the lab2.campus host.

On the lab2.campus host, user dave will receive a message similar to You have new mail in /var/spool/mail/dave as soon as he enters a shell session. This file will contain the raw email message sent by emma as well as the headers added by the MTA:

The header Received: shows that the message from lab1.campus was received directly by lab2.campus. By default, MTAs will only accept messages to local recipients. The following error will probably occur if user emma tries to send an email to user henry at the lab3.campus host, but using the lab2.campus MTA instead of the proper lab3.campus MTA:

SMTP response numbers beginning with 5, such as the Relaying denied message, indicate an error. There are legitimate situations when relaying is desirable, like when the hosts sending and receiving emails are not connected all the time: an intermediate MTA can be configured to accept emails intended to other hosts, acting as a relay SMTP server that can forward messages between MTAs.

The ability to route email traffic through intermediate SMTP servers discourages the attempt to connect directly to the host indicated by the recipient’s email address, as shown in the previous examples. Moreover, email addresses often have a domain name as the location (after the @), so the actual name of the corresponding MTA host must be retrieved through DNS. Therefore, it is recommended to delegate the task of identifying the appropriate destination host to the local MTA or to the remote SMTP server, when remote mailboxes are used.

Sendmail provides sendmail command to perform many email-related operations, including assisting in composing new messages. It also requires the user to type the email headers by hand, but in a friendlier way than using SMTP commands directly. So a more adequate method for user emma@lab1.campus to send an email message to dave@lab2.campus would be:

Here again, the dot in a line by itself finishes the message. The message should be immediately sent to the recipient, unless the local MTA was unable to contact the remote MTA. The command mailq, if executed by root, will show all non-delivered messages. If, for example, the MTA at lab2.campus did not respond, then command mailq will list the undelivered message and the cause of the failure:

The default location for the outbox queue is /var/spool/mqueue/, but different MTAs may use different locations in the /var/spool/ directory. Postfix, for instance, will create a directory tree under /var/spool/postfix/ to manage the queue. The command mailq is equivalent to sendmail -bp, and they should be present regardless of the MTA installed in the system. To ensure backward compatibility, most MTAs provide these traditional mail administration commands.

If the primary email destination host — when it is provided from an MX DNS record for the domain — is unreachable, the MTA will try to contact the entries with lower priority (if there are any specified). If none of them are reachable, the message will stay in the local outbox queue to be sent later. If configured to do so, the MTA can periodically check the availability of the remote hosts and perform a new delivery attempt. If using a Sendmail-compatible MTA, a new attempt will immediately take place with the command sendmail -q.

Sendmail will store incoming messages in a file named after the corresponding inbox owner, for example /var/spool/mail/dave. Other MTAs, like Postfix, may store the incoming email messages in locations like /var/mail/dave, but the file content is the same. In the example, the command sendmail was used in the sender’s host to compose the message, so the raw message headers show that the email took extra steps before reaching the final destination:

From bottom to top, the lines starting with Received: show the route taken by the message. The message was submitted by user emma with the command sendmail dave@lab2.campus issued on lab1.campus, as stated by the first Received: header. Then, still on lab1.campus, the MTA uses ESMTPS — a superset of the SMTP, which adds encryption extensions — to send the message to the MTA at lab2.campus, as stated by the last (top) Received: header.

The MTA finishes its job after the message is saved in the user’s inbox. It is common to perform some kind of email filtering, such as spam blockers or the enforcement of user-defined filtering rules. These tasks are executed by third-party applications, working together with the MTA. The MTA could, for example, call the SpamAssassin utility to mark suspicious messages using its text analysis features.

Although possible, it is not convenient to read the mailbox file directly. It is recommended to use a email client program instead (e.g. Thunderbird, Evolution, or KMail), which will parse the file and appropriately manage the messages. Such programs also offer extra features, like shortcuts to common actions, inbox sub-directories, etc.

It is possible to write an email message directly in its raw format, but it is much more practical to use a client application — also known as an MUA (Mail User Agent) — to speed up the process and to avoid mistakes. The MUA takes care of the work under the hood, that is, the email client presents and organizes the received messages and handles the proper communication with the MTA after the user composes an email.

There are many distinct types of Mail User Agents. Desktop applications like Mozilla Thunderbird and Gnome’s Evolution support both local and remote email accounts. Even Webmail interfaces can be seen as a type of MUA, as they intermediate the interaction between the user and the underlying MTA. Email clients are not restricted to graphical interfaces though: console email clients are widely used to access mailboxes not integrated with a graphical interface and to automate email related tasks within shell scripts.

Originally, the Unix mail command was only intended to share messages between local system users (the first mail command dates back to the first Unix edition, released in 1971). When network email exchanges became more prominent, other programs were created to deal with the new delivery system and gradually replaced the old mail program.

Nowadays, the most commonly used mail command is provided by the mailx package, which is compatible with all modern email features. In most Linux distributions, the command mail is just a symbolic link to the mailx command. Other implementations, like the GNU Mailutils package, basically provide the same features as mailx. There are, however, slight differences between them, especially with respect to command-line options.

Regardless of their implementation, all modern variations of the mail command operate in two modes: normal mode and send mode. If an email address is provided as an argument to the command mail, it will enter the send mode, otherwise it will enter the normal (read) mode. In normal mode, the received messages are listed with a numerical index for each one, so the user can refer to them individually when typing commands in the interactive prompt. The command print 1 can be used to display the contents of message number 1, for example. Interactive commands can be abbreviated, so commands like print, delete or reply can be replaced by p, d or r, respectively. The mail command will always consider the last received or the last viewed message when the message index number is omitted. The command quit or q will exit the program.

The send mode is especially useful for sending automated email messages. It can be used, for example, to send an email to the system administrator if a scheduled maintenance script fails to perform its task. In send mode, mail will use the contents from the standard input as the message body:

In this example, the option -s was added to include a subject field to the message. The message body was provided by the Hereline redirection to the standard input, but the contents of a file or the output of a command could also be piped to the program’s stdin. If no content is provided by a redirection to the standard input, then the program will wait for the user to enter the message body. In this case, keystroke Ctrl+D will end the message. The mail command will immediately exit after the message is added to the outbox queue.

By default, the email accounts on a Linux system are associated with the standard system accounts. For example, If user Carol has the login name carol on the host lab2.campus then her email address will be carol@lab2.campus. This one-to-one association between system accounts and mailboxes can be extended by standard methods provided by most Linux distributions, in particular the email routing mechanism provided by the /etc/aliases file.

An email alias is a “virtual” email recipient whose receiving messages are redirected to existing local mailboxes or to other types of message storage or processing destinations. Aliases are useful, for example, to place messages sent to postmaster@lab2.campus in Carol’s mailbox, which is an ordinary local mailbox in the lab2.campus system. To do so, the line postmaster: carol should be added to the /etc/aliases file in lab2.campus. After modifying the /etc/aliases file, the command newaliases should be executed to update the MTA’s aliases database and make the changes effective. Commands sendmail -bi or sendmail -I can also be used to update the aliases database.

Aliases are defined one per line, in the format <alias>: <destination>. In addition to the ordinary local mailboxes, indicated by the corresponding username, other destination types are available:

An unprivileged local user can define aliases for their own email by editing the file .forward in their home directory. As the aliases can only affect their own mailbox, only the <destination> part is necessary. To forward all incoming emails to an external address, for example, user dave in lab2.campus could create the following ~/.forward file:

It will forward all email messages sent to dave@lab2.campus to emma@lab1.campus. As with the /etc/aliases file, other redirection rules can be added to .forward, one per line. Nevertheless, the .forward file must be writable by its owner only and it is not necessary to execute the newaliases command after modifying it. Files starting with a dot do not appear in regular file listings, which could make the user unaware of active aliases. Therefore, it is important to verify if the file exists when diagnosing email delivery issues.