CiviMail Architecture

CiviMail, described in general previously, is our component for mass-mailing the CiviCRM contacts. In this entry, we’d like to get a bit more into the details on how CiviMail exactly works ‘under the hood’. Recipients List Building Getting the final list of recipients is not as easy as it may seem. If you look into the getRecipients() method of the CRM_Mailing_BAO_Mailing class, you’ll see that we’re building some temporary exclude (X_*) and include (I_*) tables, based on

• the user’s selection – excluded/included CiviCRM groups,
• the user’s selection – excluded/included previous mailings’ recipients,
• whether the contacts already received this mailing in a previous run,
• whether the contacts have the DO NOT EMAIL preference set,
• whether the contacts have explicitely opted-out from mass-mailings in the past, and
• whether the contacts’ email addresses are not on hold due to excessive bouncing.

The Outgoing Channel For every cron-started run, CiviMail selects which mailings are ‘due’, creates the list of recipients for the first of the mailings, creates the emails from the provided templates and then starts sending the emails up to the number specified in the Mailer Batch Limit setting. The sending is actually straightforward; CiviMail dumps the mails to any SMTP ‘sink’ specified in the general CiviCRM config. Most probably, it’s either port 25 or 10025 (see ‘Changing the Outgoing SMTP Port’ in the CiviMail Installation docs) of the localhost machine. The Incoming Channel The tricky part of the CiviMail setup/operations is how to make sure the incoming emails reach the CiviMail instance and get acted-upon properly (the contact is (un)subscribed, the bounce is counted, etc.). For this, we have created our own, ‘forked’ AMaViS filter, which, once properly hooked-up to your email server, makes sure any emails targetted at CiviMail reach CiviMail’s SOAP interface. So, to racap the incoming channel, the icoming email passes through the following chain: the mail server → the (modified) AMaViS filter → the target CiviCRM’s SOAP interface → CiviMail. The Email Address Stucture Except for bounces, CiviMail doesn’t really care about the body (or contents) of the incoming emails; all of the needed information is encoded in the email address. For example, a proper email address to subscribe a contact to a group might look like this: subscribe.1.5@civimail.example.com; this means that the CiviMail instance working on domain_id of 1 should reply (to confirm whether the originating email address in not forged), and if confirmed, should subscribe the mailing person to the group with group_id of 5. To present another example, an unsubscribe email address might look like this: unsubscribe.1.2.3.39fdba0d747c1dc06eaac0e41d8bf236c322385f@civimail.example.com. This, in turn, means that CiviMail working on domain_id of 1 sould unsubscribe the mailing person from the group that they were mailed during the mailing of job 2 and queue 3; this action does not require a confirmation, as the last part of the email (the 32-character-long part starting with ‘39fd…’) is a confirmation hash, virtually impossible to guess by a third party. Every CiviMail email has a different Return-Path: header (for example, Return-Path: bounce.1.2.3.39fdba0d747c1dc06eaac0e41d8bf236c322385f-user=example.net@civimail.example.com) – this way, if the target server is bouncing mail for any reason, the bounces can be traced back to an idividual contact (in this case, the contact with the email address of user@example.net; see the last part before the @ sign). More on CiviMail in future installments of this blog; feel free to comment and request which parts should be described in more detail. :)