E-mail server section

E-mail server setup

What we want to accomplish here is the setup of a mail server with the following properties:

• can serve multiple mail domains
• can relay mail for other domains to other mail servers
• can have one or more mailboxes per domain
• users of these mailboxes can be virtual (do not need to have a Linux user account)
• can have multiple aliases per mailbox
• can forward mail for certain aliases to multiple mailboxes

For this type of mail server setup, we owe a great thankyou to Christoph Haas, whose advise has helped us create flexible and reliable mail servers since 2003.

Preparation

We'll assume that the server currently has no mailserver installed, at least no other than the default exim mailserver. Furthermore, the server is already fitted with MySQL, and this database is running without problems.

The hostname of the server must be set correctly, so that hostname -f returns a valid DNS name, like lighthouse.saruman.biz.. It might also be an internal name like lighthouse.saruman.lan. but that will require us to give extra attention to the name under which Postfix will contact its collegues on the Internet. Also, the server can correctly [Networking_section#DNS_resolution_under_Debian | resolve DNS names] like www.debian.org, preferably by running it's own caching DNS server.

The server is kept on the correct date and time using NTP, TCP port 25 is open on the server, the ISP will allow connections from Internet to this port, and if there's a firewall running on this server, then it has port 25 open so as to not block incoming e-mail.

Software installation

As a first step, we use apt or aptitude to make sure that our server is up-to-date. Then we can install the necessary software packages. Under Debian 5.0 "Lenny", the (single) packages is:

• postfix, the mail server itself - this includes the "virtual package" postfix-tls, that we want to use to secure connections to Postfix with the TLS protocol

At the same time we can - and must - purge the following packages:

• exim4
• exim4-base
• exim4-daemon-light
• exim4-config

In aptitude, only press "go" when you've marked all four of these packages "purge", or you cannot install postfix.

When installing the postfix package, the Debian installer script will ask you several questions, which you can answer like this:

• General type of mail configuration: Internet site
• System mail name: the FQDN of the mail server that you've verified in the previous section. Note that the script will try to guess the DNS name, but that might yield a DNS name with a trailing dot. That is technically correct, but the installation script will barf. Remove the trailing dot before hitting <enter>!
• Postmaster mail address: the address that all mail should go to that is addressed to "root" and "postmaster".
• Domain list: give the list of all domains that the machine can consider itself the FINAL destination for. This should at a minimum include an empty value, "localhost" and the FQDN of the machine itself (no trailing dots!); however, if you're running your own mail domain, you can also add that (e.g. "saruman.biz"). Thus, the list could look like this:

saruman.biz, lighthouse.saruman.lan, localhost.saruman.lan, , localhost

• Force synchronous updates? We think that's not necessary, but please read the question and decide for yourself
• Local networks: something like 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.67.0/24 (the default, augmented with your local IP range)
• Mailbox size limits: you can give postfix a limit in bytes, but we're going to use one single big mailbox for all users, so we cannot let Postfix guard it. Leave it at 0 (zero) so we don't have a size limit.
• Character for local address extension: we leave it at +
• Internet protocols to use: currently we don't have IPv6 support, so there's no sense in letting Postfix try to serve IPv6. We choose ipv4 only.

With the above data, the Debian install script for Postfix can do its job and configure Postfix with some basic settings.

Now that Postfix is installed, we can install some dependent packages (we could install them in the same run, but if anything goes amiss with the postfix installation, then these packages are going to remain unconfigured as well):

• postfix-doc, the accompanying documentation;
• postfix-mysql, necessary to have Postfix talk to our MySQL server;
• postfix-pcre to be able to parse regular expressions, which which we can combat spam;
• dovecot-imapd is a daemon that will provide your users with IMAP access to their mail;
• dovecot-pop3d is another daemon, but for the POP3 protocol.

Virtual Mailman creation

When we're done, we'll need a "system user", a sort of virtual mailman that is the owner of all mailboxes that we're serving. We suggest the name "vmail" for this user. Note: this user does not get his own mailbox (i.e. there's no mailbox at vmail@saruman.biz).

To create this user, and his home directory, we can run the following commands:

groupadd -g 120 vmail
useradd -g vmail -u 120 vmail -d /data/vmail -m -s /bin/false

As you see, we've chosen a group ID and user ID of 120 (after confirming that this ID was not taken by another group or user, by checking /etc/passwd and /etc/group. Furthermore, we've decided to keep the vmail user's home directory not under /home/vmail, but in a special place where we're going to expect server data to reside - in our server, the /data directory (which is a RAID-5 array mounted under root). By adding -m, we've actually created the home directory, and adding -s /bin/false makes totally sure that the user vmail can never ever log in - even if we've not created a password for this user, so vmail shouldn't be able to log in anyway. Better safe than sorry.

To tell Postfix that this vmail user is someone special, we run

postconf -e virtual_uid_maps=static:120
postconf -e virtual_gid_maps=static:120

That makes postfix understand that all mail for the virtual mail domains must be written to disk with these specified user and group ID's.

MySQL configuration

Database preparation

We will use the MySQL database to record data on our mail system, and then give our Postfix access to this database so that it can read its configuration from there. For starters, we'll create a MySQL database named "vmail", and a MySQL user named "vmail_admin" that can read all necessary data from that "vmail" database. Then, we create the necessary tables, and a view that links these tables. We do this with the MySQL client mysql. However, we're quite lazy, so we don't create this database by hand (that's error-prone), but by use of a script create.vmail.sql. To this end, feed the create.vmail.sql script into the mysql client like this:

mysql -u root -p < create.vmail.sql

(This of course assumes you have create.vmail.sql in your current working directory; if not you can include the path to the file.) Simply give the MySQL root user password, and the script creates the database, the user, the necessary tables, and the view.
A note of caution: it is never a good idea to just run scripts without a proper understanding of what it does. Especially with MySQL, it will be advantageous if you understand the SQL commands. Open the script in a text editor, open the MySQL command reference, and trace back what the script does exactly.

Inserting data

Next up, we fill the database with the first sets of values (either test data or the first of our production data). We'll start off with the virtual domains that we're hosting, by running the mysql client and feeding it information like this:

mysql -u root -p vmail
mysql> INSERT INTO virtual_domains (id, vdomain) VALUES (1, 'saruman.biz');
mysql> INSERT INTO virtual_domains (vdomain) VALUES ('wiki.saruman.biz');
mysql> INSERT INTO virtual_domains (vdomain) VALUES ('shop.saruman.biz');

This has the effect of creating three entries. You can check that everything worked as planned by executing

mysql> select * from virtual_domains;

Note: only the first entry needs an id value, because in MySQL we've defined that field as AUTO_INCREMENT. After creating your first virtual domain in the table, you never have to use a statement like the first INSERT again, only statements like the other two.

Now the MySQL database has the information needed by Postfix to recognise that you have three virtual mail domains (namely the three domains in the VALUES section) for which it hosts virtual mailboxes. Postfix cannot read this information yet, but that'll be taken care of in the next section.

Whle still within the mysql client, we can now create users:

mysql> INSERT INTO virtual_users (id, domain_id, user, passwd) VALUES (1, 1, 'jan', MD5('JanSecret'));
mysql> INSERT INTO virtual_users (domain_id, user, passwd) VALUES (1, 'mike', MD5('MikeSecret'));
mysql> INSERT INTO virtual_users (domain_id, user, passwd) VALUES (3, 'shopkeeper', MD5('ShopSecret'));

This has the effect of creating three users "jan" (in domain saruman.biz), "mike" (in domain saruman.biz) and "shopkeeper" (in domain shop.saruman.biz). Again, the id value is only ever needed in the first statement, because from now on every user addition will auto-increment id.

The passwords shown are encrypted with MD5, and put in the passwd field. Later on, the users will be able to access their mailboxes using this password; we won't tell Postfix anything about them, because it doesn't need the passwords. You can check the content of the virtual_users table with the right "select" statement, and you can now also see that the view view-users works:

mysql> select * from virtual_users;
mysql> select * from view_users;

Next up, we're going to add some aliases, which are alternative e-mail addresses for the users; their primary e-mail address can already be seen in the view_users view, but perhaps you also want mail to "webmaster@shop.saruman.biz" to arrive at one or more mailboxes, e.g. in the mailbox for user "shopkeeper" (within domain "shop.saruman.biz") and this guy's home address ("j.doe@example.com"). Furthermore, we'll define catchall-addresses for all domains, that'll send all mail for which no mailbox can be found to the mailbox of user Mike (for the first two domains) and Webmaster (for shop.saruman.biz; Webmaster himself is an alias yet again, that points to shopkeeper@shop.saruman.biz). To create the catchall, leave out a value for the source address. This creates the empty value for source that will be expanded (using source@domain) to "@domain".

mysql> INSERT INTO virtual_aliases (id, domain_id, source, destination)
       VALUES (1, 2, 'webmaster', 'shopkeeper@shop.saruman.biz');
mysql> INSERT INTO virtual_aliases (domain_id, source, destination)
       VALUES (2, 'webmaster', 'j.doe@example.com');
mysql> INSERT INTO virtual_aliases (domain_id, destination)
       VALUES (1, 'mike@saruman.biz');
mysql> INSERT INTO virtual_aliases (domain_id, destination)
       VALUES (2, 'mike@saruman.biz');
mysql> INSERT INTO virtual_aliases (domain_id, destination)
       VALUES (3, 'webmaster@saruman.biz');

Again, we don't need to add the id value any more after the first ever insertion into this table.

We can check the result by running a select statement against the virtual_aliases table and view_aliases view:

mysql> select * from virtual_aliases;
mysql> select * from view_aliases;

Postfix configuration for MySQL lookups

Next, we're going to tell Postfix to use the vmail database, and also how to read the database (Postfix never writes the MySQL database). To this end, we're going to create three configuration files in directory /etc/postfix. We'll start off with one configuration file, with which Postfix can determine if a domain name is among the domain(s) that it's actually hosting mailboxes for. Then we'll create the config file that checks the table that contains all the users that have a virtual mailbox, and finally we create the lookup for the table with all the aliases.

Virtual mail domains lookup

Create file /etc/postfix/mysql-virtual-domains.cf with the following content:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT 1 FROM virtual_domains WHERE vdomain='%s'

Next, we tell postfix to check this configuration file when it needs to check "virtual_mailbox_domains":

postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-domains.cf

Use of this configuration file in postfix has the effect of returning "yes" when checking our database for the domain part of an email address. Naturally, this configuration file has to be fitted with the actual vmail_admin password rather than our example "SuperSecret".

Virtual mail user lookup

Create the file /etc/postfix/mysql-virtual-mailbox-maps.cf with the following content:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT 1 FROM view_users WHERE email='%s'

Next, we tell Postfix that this mapping file is supposed to be used for the virtual_mailbox_maps mapping:

postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

The lookup of virtual mailboxes should work with the data we've put into the database previously. i.e.

postmap -q jan@saruman.biz mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

should return a 1 to acknowledge that "jan@saruman.biz" is indeed a virtual mailbox in our Postfix configuration.

Virtual alias lookup

Create another cf file at /etc/postfix/mysql-virtual-alias-maps.cf:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT destination FROM view_aliases WHERE email='%s'

And create yet another cf file at /etc/postfix/mysql-email2email.cf:

user = vmail_admin
password = SuperSecret
hosts = 127.0.0.1
dbname = vmail
query = SELECT email FROM view_users WHERE email='%s'

Again, we tell Postfix that this mapping file is supposed to be used for the virtual_alias_maps mapping:

postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf

The lookup of virtual aliases should now work as designed:

postmap -q shopkeeper@shop.saruman.biz mysql:/etc/postfix/mysql-virtual-alias-maps.cf
postmap -q webmaster@shop.saruman.biz mysql:/etc/postfix/mysql-virtual-alias-maps.cf

If you've put in the data from the previous section, then both commands should return the same result: shopkeeper@saruman.biz. This shows that Postfix will deliver mail to shopkeeper or to webmaster to the mailbox of shopkeeper.

Finishing up configuration files

When everything works as planned, then we secure the configuration files against prying eyes. Remember, the configuration files contain the user/password combination which which to access the vmail SQL database. The vmail_admin user has only read rights, and passwords in there are encrypted, but nevertheless, this is information we need to protect. Thus:

chown root:postfix /etc/postfix/mysql-*.cf
chmod u=rw,g=r,o= /etc/postfix/mysql-*.cf

This protects all four configuration files since only root can write them, members of group postfix can read them, and the rest of the world cannot access them at all.

Configuring mail delivery through Dovecot LDA

E-mails get delivered to the virtual mailboxes by means of a Mail Delivery Agent (MDA). With Postfix, the standard MDA for delivery to virtual mailboxes is called virtual, but we're not going to use that; we're going to use Dovecot LDA, which is a program called deliver. By the way, the abbreviation LDA stands for Local Delivery Agent, which of course means more or less the same as MDA.

To make Postfix use Dovecot LDA, we add the following line to /etc/postfix/master.cf:

dovecot   unix  -       n       n       -       -       pipe
    flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient}

This declares the service "dovecot" to mean using the program /usr/lib/dovecot/deliver with the right parameters. If we now let Postfix reload its files, it'll learn about this new service. Subsequently we can instruct Postfix to start using it, by declaring service "dovecot" to be the virtual transport of choice, and specifying one Dovecot-specific variable:

postfix reload
postconf -e virtual_transport=dovecot
postconf -e dovecot_destination_recipient_limit=1

Now to configure Dovecot itself: open /etc/dovecot/dovecot.conf with your favourite editor vim; first make sure the protocols = line reads

protocols = imap imaps pop3 pop3s

(which it should by default). The second setting is quite crucial. By default, the location of the mail directories is found automatically by Dovecot, but here that won't work. Find the line that reads #maildir_location =, and change it to

mail_location = maildir:/data/vmail/%d/%n/Maildir

This will look for mail in directory /data/vmail/DOMAIN/USER/Maildir, and expect this directory to be in "maildir" format. Next look for a section called "auth default". First define the allowed authentication mechanisms:

mechanisms = plain login

Then inside that same section you need to uncomment:

passdb sql {
    args = /etc/dovecot/dovecot-sql.conf
}

which tells Dovecot that the passwords are stored in an SQL database (it's obvious that we'll now have to create this conf-file) and:

userdb static {
    args = uid=120 gid=120 home=/data/vmail/%d/%n allow_all_users=yes
}

to tell Dovecot where the mailboxes are located. This is similar to the mail_location setting. Next task: comment out the section passdb pam, so that Dovecot doesn't accidentally try to service local users as well.

Next up: tell Dovecot to handle IMAP the same as previous versions of this virtual mailserver have:

namespace private {
    separator = .
    prefix = INBOX.
    inbox = yes
}

(this section is not necessary if you're building this server from scratch).

You'll also want to comment out the passdb pam sections, since we're not hosting mail for local users.

Next up, we'll edit the section socket listen so that Dovecot can access the user database (master section), and our Postfix mailserver can access Dovecot (client section):

socket listen {
  master {
    path = /var/run/dovecot/auth-master
    mode = 0600
    user = vmail
  }
  client {
    path = /var/spool/postfix/private/auth
    mode = 0660
    user = postfix
    group = postfix
  }
}

Finally, the protocol section, where we specify log files, an address to put in mails when we send out nondelivery reports, and a place to store scripts:

protocol lda {
    log_path = /var/log/appslog/mail/dovecot-deliver.log
    auth_socket_path = /var/run/dovecot/auth-master
    postmaster_address = postmaster@saruman.biz
    mail_plugins = cmusieve
    global_script_path = /data/vmail/globalsieverc
}

Now that we've finished the Dovecot configuration file, we need to create a fitting MySQL configuration file for Dovecot. The file is /etc/dovecot/dovecot-sql.conf, and its contents are

driver = mysql
connect = host=127.0.0.1 dbname=vmail user=vmail_admin password=SuperSecret
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM view_users WHERE email='%u';

When you've created this file (as root), change the attributes for safety, and then restart Dovecot to start using the new configuration:

chmod u=rw,g=,o= /etc/dovecot/dovecot-sql.conf
/etc/init.d/dovecot restart