108.3 Mail Transfer Agent (MTA) basics¶

Weight: 3

Candidates should be aware of the commonly available MTA programs and be able to perform basic forward and alias configuration on a client host. Other configuration files are not covered.

Objectives

Create e-mail aliases.
Configure e-mail forwarding.
Knowledge of commonly available MTA programs (postfix, sendmail, exim) (no configuration)

Terms

~/.forward, sendmail emulation layer commands, newaliases, mail, mailq, postfix, sendmail, exim

What an MTA does¶

On Linux, every user has an inbox: a file only they (and root) can read, which holds their mail. Mail is moved around by a Mail Transfer Agent (MTA), a program that runs as a system service. It:

receives mail from local users and from other machines on the network, and puts it in the right inbox,
sends mail to other machines: it takes the part of the address after the @, finds the machine that handles mail for it, and passes the message on with SMTP (Simple Mail Transfer Protocol), normally on TCP port 25,
keeps an outbox queue of messages still waiting to be delivered, and retries when the other side is down.

The program you read and write mail with is a different thing, a Mail User Agent (MUA): Thunderbird, Evolution, KMail, webmail, or the mail command.

  emma (MUA: mail, Thunderbird)                                       dave reads the inbox
        |                                                                      ^
        v                                                                      |
  MTA on lab1.campus  ---- SMTP, TCP port 25 ---->  MTA on lab2.campus  ---> /var/spool/mail/dave
  (outbox queue)                                    (finds the inbox)

Finding the destination. For an address like info@lpi.org, the sending MTA asks DNS for the domain's MX record, which names the mail server for that domain. If there are several MX records, they are tried in order of priority. If there is no domain or no MX record, the part after the @ is used as the host name directly.

Today most people use a remote mailbox instead, like a company mail server or Gmail: the email client logs in and reads mail with IMAP or POP3, and no local MTA is needed. But on Linux servers, local mail is still how cron jobs, scripts and services report to root.

Security. An MTA that can be reached from the internet must not accept mail from just anyone for any destination. An MTA that forwards everyone's mail is an open relay, which spammers abuse to hide who sent a message. Accept mail only from authorized domains and use authentication. This is also why most distributions do not install an MTA by default.

The common MTAs¶

You need to know these by name, not how to configure them:

MTA	Notes
sendmail	the oldest and traditional Unix MTA. Very flexible, but huge, hard to configure and not designed with security first, so few distributions use it as the default today
postfix	a newer alternative to sendmail with easy-to-read configuration files. Supports multiple domains and encryption. The default on many distributions, the most common default MTA
exim	a general, flexible MTA with strong checking of incoming mail: ACLs, authentication and more. The default on Debian
qmail	another alternative MTA, security-focused, worth recognising by name

People choose an alternative to sendmail mainly because advanced setups are easier. On a desktop, install postfix and the mailx (or bsd-mailx) package if you want local mail.

The sendmail emulation layer¶

Because sendmail was there first, many programs and scripts call its commands directly. So every other MTA provides the same commands, as a sendmail emulation layer: sendmail, mailq and newaliases work whichever MTA is installed. Any program written for sendmail works with postfix or exim too.

Command	Same as	Does
`sendmail address`		send a message
`mailq`	`sendmail -bp`	list the mail queue
`sendmail -q`		try to deliver the queued mail now
`newaliases`	`sendmail -bi`, `sendmail -I`	rebuild the aliases database from `/etc/aliases`

Talking SMTP by hand¶

To understand SMTP, or to test a mail server, you can type SMTP commands yourself with nc (netcat), connecting to port 25. Here emma on lab1 sends a message to dave on lab2:

$ nc lab2.campus 25
220 lab2.campus ESMTP Sendmail 8.15.2/8.15.2; Sat, 16 Nov 2019 00:16:07 GMT
HELO lab1.campus
250 lab2.campus Hello lab1.campus [10.0.3.134], pleased to meet you
MAIL FROM: emma@lab1.campus
250 2.1.0 emma@lab1.campus... Sender ok
RCPT TO: dave@lab2.campus
250 2.1.5 dave@lab2.campus... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Subject: Recipient MTA Test

Hi Dave, this is a test for your MTA.
.
250 2.0.0 xAG0G7Y0000595 Message accepted for delivery
QUIT
221 2.0.0 lab2.campus closing connection

Command	Means
`HELO lab1.campus`	"I am lab1.campus"
`MAIL FROM:`	the sender
`RCPT TO:`	the recipient
`DATA`	the message follows. A `Subject:` line goes first, then an empty line, then the text. A `.` alone on a line ends the message
`QUIT`	close the connection

The numbers are the server's replies: 2xx means OK, 354 means "go on", and 5xx means an error. By default, an MTA accepts mail only for its own local users. Asking lab2 to deliver mail for lab3 is refused, because lab2 is not an open relay:

RCPT TO: henry@lab3.campus
550 5.7.1 henry@lab3.campus... Relaying denied

When dave next opens a shell, the message You have new mail in /var/spool/mail/dave appears. The inbox is one text file holding all messages one after another (the mbox format), with headers the MTAs added:

$ cat /var/spool/mail/dave
From emma@lab1.campus    Sat Nov 16 00:19:13 2019
Return-Path: <emma@lab1.campus>
Received: from lab1.campus (lab1.campus [10.0.3.134])
    by lab2.campus (8.15.2/8.15.2) with SMTP id xAG0G7Y0000595
    for dave@lab2.campus; Sat, 16 Nov 2019 00:17:06 GMT
Date: Sat, 16 Nov 2019 00:16:07 GMT
From: emma@lab1.campus
Message-Id: <201911160017.xAG0G7Y0000595@lab2.campus>
Subject: Recipient MTA Test

Hi Dave, this is a test for your MTA.

The Received: headers record each server the message passed through. Read them from the bottom up to follow its route.

sendmail and mailq¶

In practice you let your local MTA find the right server. The sendmail command composes a message for it; you type the headers, and again a . alone ends the message:

$ sendmail dave@lab2.campus
From: emma@lab1.campus
To: dave@lab2.campus
Subject: Sender MTA Test

Hi Dave, this is a test for my MTA.
.

If the remote server cannot be reached, the message waits in the queue. mailq (run as root) lists everything not yet delivered, with the reason:

# mailq
                /var/spool/mqueue (1 request)
-----Q-ID----- --Size-- -----Q-Time----- ------------Sender/Recipient-----------
xAIK3D9S000453       36 Mon Nov 18 20:03 <emma@lab1.campus>
                 (Deferred: Connection refused by lab2.campus.)
                                         <dave@lab2.campus>
                Total requests: 1

This is how an admin checks stuck mail. mailq is one of the emulation-layer commands, so it works under any MTA.

Each entry shows the queue ID, size, time, sender, the recipients still waiting, and why the last attempt failed. The MTA retries by itself, and sendmail -q forces a new attempt now.

A postfix-style queue looks like this:

$ mailq
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
AA52C228E6B      468 Thu Jan  7 19:59:41  nagato@funlife
(connect to alt2.gmail-smtp-in.l.google.com:25: Network is unreachable)
                                         nagato@gmail.com
-- 0 Kbytes in 1 Request.

Where the files live depends on the MTA:

	sendmail	postfix
outbox queue	`/var/spool/mqueue/`	`/var/spool/postfix/`
inboxes	`/var/spool/mail/<user>`	`/var/mail/<user>`

The mail command¶

mail is the classic command line MUA. Today it is usually provided by the mailx package (mail is a link to mailx), or by GNU Mailutils; the versions differ slightly in their options. It works in two modes:

send mode, when you give an address,
normal (read) mode, when you do not.

Sending¶

With an address, mail sends. Interactively, it asks for the subject and you type the body, ending with Ctrl+D:

$ mail news
Subject: Email to news user
This will go to root.
Cc:

The body can also end with a line holding only a .:

$ mail news
Subject: Email to news user
this will go to root, and by alias on to nagato
.
Cc:

In scripts, give the subject with -s and send the body on standard input, from a pipe, a file or a here string. Here a maintenance script reports a failure:

$ mail -s "Maintenance fail" henry@lab3.campus <<<"The maintenance script failed at `date`"
$ echo -e "email content" | mail -s "email subject" root

mail hands the message to the MTA's queue and exits at once.

Reading¶

Without an address, mail lists your messages with numbers and gives you a & prompt:

$ mail
Mail version 8.1.2 01/15/2001.  Type ? for help.
"/var/mail/nagato": 12 messages 12 new
>N  1 root@debian        Sat Jan 02 08:50   39/1373  apt-listchanges: news for f
 N  2 root@debian        Sat Jan 02 09:01  165/7438  apt-listchanges: news for f
(...)
 N 12 nagato@debian      Thu Jan  7 19:53   17/478   Email to news user
& 12

Command	Short	Does
`print 12`	`p 12`	show message 12 (or just type its number)
`delete 12`	`d 12`	delete it
`reply`	`r`	reply
`quit`	`q`	exit, keeping the messages you did not delete
`?`		help

Without a number, a command acts on the current message (the last one received or viewed). Your mailbox is /var/mail/<user> (also seen as /var/spool/mail/<user>), in the classic mbox format. A short session:

$ mail
"/var/mail/nagato": 12 messages 12 new
>N  1 root@funlife    apt-listchanges: news
 N 12 nagato@funlife    Email to news user
& 12          # read message 12
& d           # delete the current message
& q           # quit

Aliases: /etc/aliases¶

Normally each mailbox matches a system account: user carol on lab2.campus receives mail as carol@lab2.campus. Aliases add extra names that deliver somewhere else. They are defined system-wide, by root, in /etc/aliases, one per line, as alias: destination:

$ cat /etc/aliases
#
#  Aliases in this file will NOT be expanded in the header from
#  Mail, but WILL be visible over networks or from /bin/mail.
#
#       >>>>>>>>>>      The program "newaliases" must be run after
#       >> NOTE >>      this file is updated for any changes to
#       >>>>>>>>>>      show through to sendmail.
#

# Basic system aliases -- these MUST be present.
mailer-daemon:  postmaster
postmaster:     root

# General redirections for pseudo accounts.
bin:            root
daemon:         root
news:           root
www:            webmaster
webmaster:      root
nobody:         /dev/null

So mail for news goes to root, a destination of /dev/null throws the mail away (as for nobody), and mail for www goes to webmaster, which itself goes to root: an alias can point to another alias. To have mail for postmaster land in carol's inbox, change its line to:

postmaster: carol

After every change to /etc/aliases, run newaliases (or sendmail -bi, or sendmail -I). The MTA reads a compiled database, not the text file, and newaliases rebuilds it, otherwise the change is ignored:

# newaliases

A destination can be:

Destination	Example	Delivers to
a local user	`postmaster: carol`	that user's inbox
several destinations	`team: carol, dave`	all of them (comma separated)
a file	`archive: /var/mail/archive.txt`	the end of that file (a full path starting with `/`)
a command	`subscribe: \|subscribe.sh`	the standard input of the command (starts with `\|`, quote it if it has spaces)
an include file	`list: :include:/var/local/destinations`	every destination listed in that file
an external address	`boss: boss@example.com`	another mail server
another alias	`mailer-daemon: postmaster`	wherever that alias points

When sendmail runs in restricted shell mode, the commands allowed as destinations must be in (or linked from) /etc/smrsh/.

Forwarding: ~/.forward¶

A normal user cannot edit /etc/aliases. Instead, each user can create a ~/.forward file in their home directory to redirect their own mail. Since it only affects one mailbox, it holds just the destinations, one per line:

$ cat ~/.forward
emma@lab1.campus

All mail for dave@lab2.campus now goes to emma@lab1.campus. It can hold a local username or a full external address:

$ cat ~/.forward
nagato@gmail.com

Rules for .forward:

It can hold the same kinds of destinations as /etc/aliases, including full external addresses.
It must be writable only by its owner.
No newaliases is needed: changes work at once.
It starts with a dot, so ls does not show it. When mail goes somewhere unexpected, check whether a .forward file exists.

Summary¶

I only need the basics here. An MTA is the service that moves mail: it accepts messages from local users and the network, stores them in each user's inbox, and sends outgoing mail to other servers with SMTP on TCP port 25, finding them through DNS MX records and keeping a queue of messages it could not deliver yet. An MUA, like the mail command or Thunderbird, is what a person uses to read and write. The MTAs to know are sendmail (the old, complex original, hard to configure), postfix (modern and the usual default), exim (flexible, with strong checks, Debian's default) and qmail. Whatever is installed, the sendmail emulation layer keeps the old commands working the same everywhere: sendmail to send, mailq (or sendmail -bp) to show the queue of undelivered messages, sendmail -q to retry it, and newaliases to rebuild the aliases database. An MTA must not be an open relay, which is why servers refuse to relay with a 550 error.

I send mail with mail -s "subject" address, piping the body in from a script, and read mine by running mail alone; inboxes live in /var/spool/mail/ or /var/mail/, and the queue in /var/spool/mqueue/ or /var/spool/postfix/. As root I create system aliases in /etc/aliases to redirect mail for one name to another, with lines like postmaster: carol, pointing at users, other aliases, files, commands, include files or external addresses, and I run newaliases after each change. As a user, I forward my own mail by listing the destinations, local or external, in ~/.forward, which needs no newaliases and must be writable only by me.