CiviCRM has a great feature that allows it to check email boxes and add new contacts and activities based on the emails in that box. Gmail has an awesome system of tags and filters. Emails in Gmail can be very tagged and sorted using a vast number of criteria. Together, Gmail can filter and sort large volumes of email with excellent accuracy, then CiviCRM can import the relevant, and only the relevant emails into the contact records.
This processor checks for emails in a specified folder, downloads them, processes the relavant content to an activity on a new or existing contact, then moves them to a new folder. In the days where everyone ran a standard email server accessible with POP3/IMAP, this was perfection.
The most difficult part I found of this entire setup of the Gmail setup, was getting authenticated with Gmail. Google employs multiple layers of security for their accounts and letting in some random login attempt from and unknown ip at an unknown location is a big red flag. Also, recently, Google has been restricting the use of non-Google made applications over security concerns. This includes the IMAP protocol CiviCRM will use to interface with Gmail.
Emails in Gmail are behave somewhat differently. Gmail emulates POP3/IMAP functionality, but, ultimately, does not work in the exact same way. Gmal uses tags instead of folders, but for our purposes they work the same. All tags are folders. There is a set of special [Gmail] folders. These all look like standard IMAP folders. Inside the [Gmail] folder resides other folders: All Mail. Sent Mail, etc. These are the same as in the online version of Gmail. There is also the special folder INBOX. This not located inside the [Gmail] but along side it. Items deleted from the INBOX are not actually deleted, they are just no longer listed there. It looks something like this:
- INBOX
-
Tag One
- Tag Under Tag One
- Tag2
-
CiviCRM
-
CivicMail
- processed
- ignored
-
CivicMail
-
[Gmail]
- All Mail
-
- Sent Mail
- ...
Here is where using Gmail becomes very useful. CiviCRM, by default, imports email from INBOX, moving it to the INBOX/CiviMail/processed or INBOX/CiviMail/ignored folder after processing. We can change this by change this behaviour in CiviCRM. By having CiviCRM import mails from another folder (tag), we can use Gmails filters to only tag specific emails that we want imported into CiviCRM. For obvious reasons, I chose the tag CiviCRM (see above). In Gmail, I created a number of filters to apply the tag CiviCRM to all the emails that I was sure I wanted in CiviCRM. If you are unfamiliar with Gmail filters, you can learn more here.
With this in mind, first, we need to setup CiviCRM. Log into CiviCRM and go to the Administrator Console, then to the Mail Accounts (civicrm/admin/mailSettings). Create a new account and put int your credentials.
- Name: (what you will call this account)
- Server: imap.gmail.com
- Username: (part before @gmail.com or full email address if not at gmail.com)
- Password: (Gmail password or application password if you have two factor authentication)
- Local Part: (blank)
- Email Domain: gmail.com (or part after @ if not at Gmail)
- Return Path: (blank)
- Protocol: IMAP
- Source: CiviCRM (or whatever tag you chose for emails CiviCRM should import)
- Use SSL: (checked)
- Used for: Email-to-Activity Processing
Enable the Fetch Activities scheduled job in the Administrator Console -> Scheduled Jobs (civicrm/admin/job).
Setting up Gmail is a little more detailed. First enable IMAP access in Gmail. Go to the settings 
and click on the "Forwarding and POP/IMAP" tab. Click "Enable IMAP".
Next we need to enable Access for Less Secure Apps. Its easier to just click on the link while logged into Gmail. To find it manually: go to Gmail, click on the picture in the top right, click "My Account", then click "Connected apps & sites". It is currently the last box.
Finally, Google with still probably block CiviCRM's login attempt. Open the Unlock Captcha page and run the below command in the command line of the server (ssh probably required).
php5 [CiviCRM Path]/civicrm/bin/cli.php -s site -u [CMS User] -p [CMS Path] -e Job -a fetch_activities
Likely you will get an error similar to: "Died during callApi". Error codes are pretty undescriptive. If you get and error, fill out the Captcha and try again; tt is probably a Gmail authentication error. Try some of the other procedures here.
The possibilities of the setup are great. I have only begun to explore the multitude of filters I can use. Please comment if you have found any good ones to use.
Cross post from: Leverging CiviCRM Email-to-Activity Processing in Gmail
Comments
Thank you for providing a detailed explanation of how this approach might be useful to CiviCRM users with Gmail accounts. I look forward to having an excuse to try it out
thanks for sharing, i've used it on an almost similar configuration, but adding an extra step. our aim was to avoid sharing the password of gmail in civicrm, so we used getmail (we found it working better than fetchmail) to read the folder and download it to a local maildir, and then have the processor you describe feeding from that files instead of gmail over imap
I tried to keep it simple :) It got pretty long by the end.
The best way is to use Two Factor Auth with an App Specific Password: https://support.google.com/accounts/answer/185833?hl=en
This is a great summary! For reference, this works well with Google Apps, too, and many of our clients do exactly this. The only difference is that your "email domain" is the actual domain of the email. The connection to Google itself is the same, however.
We actually set this up originally on Google Apps too!
Thank you for these instructions. For our purposes we can not turn off the 2-Step Verification, which then blocks access to allowing "Less Secure Apps." The alternative was generating an App Password for each Gmail account and using that in the password field in Civi Mail Accounts settings, as shown in your example. All the other fields and settings were as shown.
Everything worked great with that change, and did not did not require the additional command line step. Thanks again.