SMTP is the Simple Mail Transfer Protocol (defined in RFC 821). Among other tasks, its job is to receive mail by accepting connections on TCP port 25 from remote mail servers. By default, UNIX comes with the sendmail program, an age-old program that implements the SMTP protocol (and more).

SMTP Risks

Sendmail is one of those programs every administrator seems to have heard of. Its history of security problems is well known. It could be the most maligned UNIX software ever written. With that reputation, it should be clear that something is fundamentally wrong with sendmail—and that something is its monolithic design.

However, the security of sendmail has improved significantly in recent years because of the efforts of its author, Eric Allman, in response to the many security problems it suffered. It's debatable though whether sendmail is totally "out of the woods," or ever will be, because of its design.

Rather than repeat a history of security flaws here (I don't think there's space), these are some generic problems that a default installation of sendmail presents:

· Sendmail is "Yet Another Daemon" that runs as root. Therefore, an exploitable vulnerability in sendmail can mean giving away root to an attacker. Even though a root-run program might temporarily drop privileges, an attacker who is able run shellcode (through a buffer overflow or string format exploit) can simply make a call to seteuid() to re-establish those privileges and have her shell code running as root.

· Sendmail is incredibly complex—its configuration file uses m4—the GNU implementa tion of the UNIX macro processor. Few people truly understand m4, and fewer still understand sendmail configuration. As a result, it's easy to make blunders and hard to lock it down without help from outside.

· Sendmail can be used to elicit usernames. By connecting to port 25 and issuing VRFY and EXPN commands, sendmail will confirm valid usernames. This is the first step in taking over an account. Attackers can then use remote login services and attempt to guess passwords. This guessing attack can be automated and use a large dictionary of common usernames to increase the chances of finding a valid username.

· Older versions of sendmail allow spammers to relay mail through your system. Apart from using your resources, this can make you very unpopular and result in your site being listed on RBL (Realtime Blackhole List at http://mail-abuse.org/rbl/). This is bad news for you as any mail servers your site attempts to connect to will drop the connection if they follow the RBL list.

· If incorrectly configured, sendmail leaks internal address information to the outside. Attackers can send probe e-mails to a company mail-server. By sending a malformed message, they can elicit a bounce message possibly including internal IP addresses. This assists an attacker in mapping the internal network.

· The sendmail daemon outputs its version number upon client connection. This information helps the attacker select a relevant exploit.

Securing SMTP

In my experience, few machines on an organization's network actually need to be listening for mail—they just happen to be because sendmail is active by default. To put it simply, don't run mail transfer unless you need it. Turning off your mail transfer agent does not affect your systems ability to actually send mails (such as for the output of cron jobs).

· Consider using Qmail instead of sendmail. It has been designed and coded following sound security principles and has an impressive security track record—zero security holes. Visit http://www.qmail.org/ for more details. Recent versions of Qmail go further in easing the migration from sendmail. Qmail is available on a wide range of platforms.

· Postfix (formally Vmailer), written by Wietse Venema, is a popular sendmail-compatible alternative written to be fast, easy to install and secure. Full details are at http://www.postfix.org/. If you can't face Qmail, check out Postfix.

· If you must run sendmail, don't run it as root—build a chroot environment and run it as a non-privileged user. Russell Coker has detailed how he does this at http://www.coker.com.au/~russell/sendmail.html.

· A common misconception amongst administrators is that sendmail needs to be listening to the network in order to send mail from the local machine. While this is the default on many systems, it's not required. Sendmail can be invoked via cron with the "-q" flag to service the queue of outgoing messages on a regular basis. If all you want is the ability to send mail, then disable the sendmail startup script—you don't need sendmail listening on port 25.

Note

The authors of Qmail and Postfix have publicly locked horns a number of times on security-related mailing lists. There is clearly no love lost between them as they try and find security bugs in each other's software. Although this might not be a pleasant sight to the uninitiated, it does give a valuable insight into the security issues facing designers of Mail Transfer Agents (MTA), such as where the weaknesses are and how to avoid them. The bottom line is, if you want secure mail servers, use dedicated, hardened systems with shell access given to trusted users only