Leveraging CiviCRM Email-to-Activity Processing in Gmail

CiviCRM has a great feature that allows it to check email boxes and add new contacts and activities based on the emails in that box. Gmail has an awesome system of tags and filters. Emails in Gmail can be very tagged and sorted using a vast number of criteria. Together, Gmail can filter and sort large volumes of email with excellent accuracy, then CiviCRM can import the relevant, and only the relevant emails into the contact records.

This processor checks for emails in a specified folder, downloads them, processes the relavant content to an activity on a new or existing contact, then moves them to a new folder. In the days where everyone ran a standard email server accessible with POP3/IMAP, this was perfection.

The most difficult part I found of this entire setup of the Gmail setup, was getting authenticated with Gmail. Google employs multiple layers of security for their accounts and letting in some random login attempt from and unknown ip at an unknown location is a big red flag. Also, recently, Google has been restricting the use of non-Google made applications over security concerns. This includes the IMAP protocol CiviCRM will use to interface with Gmail.

Emails in Gmail are behave somewhat differently. Gmail emulates POP3/IMAP functionality, but, ultimately, does not work in the exact same way. Gmal uses tags instead of folders, but for our purposes they work the same. All tags are folders. There is a set of special [Gmail] folders. These all look like standard IMAP folders. Inside the [Gmail] folder resides other folders: All Mail. Sent Mail, etc. These are the same as in the online version of Gmail. There is also the special folder INBOX. This not located inside the [Gmail] but along side it. Items deleted from the INBOX are not actually deleted, they are just no longer listed there. It looks something like this:

• INBOX
• Tag One
  • Tag Under Tag One
• Tag2
• CiviCRM
  • CivicMail
    • processed
    • ignored
• [Gmail]
  • All Mail
• • Sent Mail
  • ...

Here is where using Gmail becomes very useful. CiviCRM, by default, imports email from INBOX, moving it to the INBOX/CiviMail/processed or INBOX/CiviMail/ignored folder after processing. We can change this by change this behaviour in CiviCRM. By having CiviCRM import mails from another folder (tag), we can use Gmails filters to only tag specific emails that we want imported into CiviCRM. For obvious reasons, I chose the tag CiviCRM (see above). In Gmail, I created a number of filters to apply the tag CiviCRM to all the emails that I was sure I wanted in CiviCRM. If you are unfamiliar with Gmail filters, you can learn more here.

With this in mind, first, we need to setup CiviCRM. Log into CiviCRM and go to the Administrator Console, then to the Mail Accounts (civicrm/admin/mailSettings). Create a new account and put int your credentials.

• Name: (what you will call this account)
• Server: imap.gmail.com
• Username: (part before @gmail.com or full email address if not at gmail.com)
• Password: (Gmail password or application password if you have two factor authentication)
• Local Part: (blank)
• Email Domain: gmail.com (or part after @ if not at Gmail)
• Return Path: (blank)
• Protocol: IMAP
• Source:  CiviCRM (or whatever tag you chose for emails CiviCRM should import)
• Use SSL: (checked)
• Used for: Email-to-Activity Processing

Enable the Fetch Activities scheduled job in the Administrator Console -> Scheduled Jobs (civicrm/admin/job).

Setting up Gmail is a little more detailed. First enable IMAP access in Gmail. Go to the settings and click on the "Forwarding and POP/IMAP" tab. Click "Enable IMAP".

Next we need to enable Access for Less Secure Apps. Its easier to just click on the link while logged into Gmail. To find it manually: go to Gmail, click on the picture in the top right, click "My Account", then click "Connected apps & sites". It is currently the last box.

Finally, Google with still probably block CiviCRM's login attempt. Open the Unlock Captcha page and run the below command in the command line of the server (ssh probably required).

php5 [CiviCRM Path]/civicrm/bin/cli.php -s site -u [CMS User] -p [CMS Path] -e Job -a fetch_activities

Likely you will get an error similar to: "Died during callApi". Error codes are pretty undescriptive. If you get and error, fill out the Captcha and try again; tt is probably a Gmail authentication error. Try some of the other procedures here.

The possibilities of the setup are great. I have only begun to explore the multitude of filters I can use. Please comment if you have found any good ones to use.

Cross post from: Leverging CiviCRM Email-to-Activity Processing in Gmail