Upcoming Events
Configuring, Customizing and Extending CiviCRM - New York
September 16th, 2010
This comprehensive two-day hands on training course is targeted at (more...)
CiviCRM User and Administrator Training - New York
September 16th, 2010
A comprehensive two day hands on training course covering the configuration, (more...)
CiviCRM Forum Dresden, Germany
September 17th, 2010
Für alle Interessierten, die eine freie Fundraisingsoftware suchen oder sich zu (more...)
CiviCRM Code and Test Sprint - New York
September 18th, 2010
This code and test sprint is targeted at experienced developers who want to (more...)
CiviCRM Toronto Meetup
September 21st, 2010
Come meet others from the Toronto Area who are interested in, using or (more...)
CiviCRM Philly Meetup – September 2010
September 23rd, 2010
Come meet others from the Philadelphia Area who are interested in, using or (more...)
CiviCRM Seminar - Dublin
September 28th, 2010
NfP Services are hosting a free seminar at The IBOA, Stephen St Upper, Dublin 8 (more...)
London developer and implementer training
September 30th, 2010
This comprehensive two-day hands on training course is targeted at implementers, (more...)
London user and administrator training
September 30th, 2010
A comprehensive two day hands on training course covering the configuration, (more...)
CiviCRM Code and Test Sprint - Bristol, United Kingdom
October 2nd, 2010
This code and test sprint is targeted at experienced developers who want to (more...)
Berlin user and administrator training
October 6th, 2010
A comprehensive one day hands on training course covering the configuration, (more...)
Berlin developer and implementer training
October 7th, 2010
This comprehensive one-day hands on training course is targeted at implementers, (more...)
Benelux meetup in Brussels: Connect, communicate and activate your supporters and constituents
October 11th, 2010
Come meet others who are interested in, using or developing for CiviCRM. For (more...)
CiviCRM Toronto Meetup
October 19th, 2010
Come meet others from the Toronto Area who are interested in, using or (more...)
CiviCRM Toronto Meetup
November 16th, 2010
Come meet others from the Toronto Area who are interested in, using or (more...)
Integrating Sendgrid into CiviMail
- Not Just a Contact Database
-
These optional components give you more power to connect and engage your supporters.

civiCASE
Case management for clients and constituents.

civiCONTRIBUTE
Online fundraising and donor management.

civiEVENT
Online event registration and participant tracking.

civiMEMBER
Online signup and membership management.

civiMAIL
Personalized email blasts and newsletters.

civiREPORT
Report generation and template management.
Sendgrid is a relatively new cloud-based service that takes care of mailing list SMTP delivery and attempts to get the highest possible deliverability rate. We didn't want to have to manage our own SMTP server here at the Participatory Politics Foundation, so when we migrated to CiviCRM recently, we decided to take the sendgrid route for SMTP.
Integration with CiviMail is pretty straightforward: all you really have to do is change your SMTP settings in CIviMail to match your sengrid credentials. But sendgrid is different from a typical SMTP server in the way it handles bounces and spam reports. First of all, sendgrid uses a reputation score to keep spammers away -- the gist of it is that 80% or more of your messages must deliver without failures. If you go below 80%, your sendgrid account can be suspended. Furthermore, they don't send any bounce or spam report messages back to you, the sender. And they don't attempt redelivery. Instead, they record the e-mail addresses that bounced and prevent further messages to those addresses from being delivered -- so your score will never drop if you try to send further messages to bounced addresses.
So, we could actually stop once we change the SMTP settings, but then we'd keep trying to mail to all of those bounced addresses, and that's kind of annoying even if it doesn't impact our sendgrid reputation. It would be better to keep the bounce and spam report lists on sendgrid in sync with CiviCRM, and mark CiviCRM contacts as "on hold" when they bounce. Sendgrid has a simple Event API that will push notifications of bounces and spam reports to a URL on your server.
We wrote a PHP endpoint to handle these events. You can grab our code here. You will want to change the civicrm path and unique key, which you'll provide to sendgrid in your post notification URL.
Make this script publicly accessible and test it out like this:
$ wget -O - -S --post-data 'email=bouncing-contact@hotmail.com&event=bounce' 'http://yoursite.org/sendgrid-notifier.php?key=CHANGE_THIS_TO_YOUR_UNIQUE_KEY_FROM_ABOVE'
Once it's working to your satisfaction, you can point sendgrid to your URL, and away you go!
Some changes we'd love to see in the future:
- Turn this into an official module/plug-in?
- Sendgrid also sends a bounce reason message and a bounce status code in the push notification -- it would be nice to record these somehow.
- ... and, for some temporary failures, we could tell sendgrid to remove the address from their bounce list, via their web api.
I welcome your feedback and suggestions for other improvements.
PS. The sendgrid Event API also notifies you of opens, clicks, and unsubscribes, but we turned all of those off in our sendgrid filter settings, since CiviMail already provides all of these for us.







Comments
Cool stuff!
This is great work, guys! I'm about to start using it with a client.
I made some changes to your code to use a CIVICRM_SENDGRID_KEY constant defined in the civicrm.settings.php file rather than hard-coding the shared secret. I have it up on github here: http://github.com/cap10morgan/civicrm_sendgrid
I'm also discussing ways we can make this more secure with the SendGrid folks. I'd rather not have to have the shared secret go across the wire in cleartext, right there in the URL. They say they're discussing my suggestions and will get back to me. I'll update this code if/when they provide a more secure way to verify that the post events are really coming from them.
Once that's done, it will probably be easiest to just use the CIVICRM_SITE_KEY as the shared secret. But since the shared secret gets exposed in the URL currently, I didn't want to use site key just yet.
Looks great. Would be great
Looks great. Would be great to have an option beyond CiviSMTP for email deliverability.
Have you compared the open rate ?
HI, seems that sendgrid tracks as well the open rate. Have you compared it with civimail internal tracking ? Do you get the same % and could you share what it is with us?
nope
We've disabled all of that tracking on their end and are just going with CiviCRM's stats -- but if we ever do turn on open tracking, I'll let you know how it goes.
What are they ?
Hi,
It'd be nice you try to enable the opening rate tracking once or twice to confirm if that's the same as civi (it should, obviously).
X+