Newsletters and other mailings are an important way to keep in touch with your supporters and mailing statistics like click and open rates can help you learn what your supporters respond best to. CiviCRM provides these features, similarly to many/all other email marketing solutions. But what do those numbers actually mean?
Spoiler alert: not very much in themselves, but they're still useful.
Measuring delivery performance is at least fairly simple: Bounces, bounce rate, Unsubscribes are all evidently useful, so I'm not focusing on those here.
The most important metrics for measuring content performance
Numbers abound in CiviCRM's mailing reports, so which are the useful ones? The report at Reports » Mail Summary Report lets you choose various columns:
- 🟢 Unique Open rate: ✔ useful
- 🟢 Click through rate ✔ useful
- 🟡 Click to open rate: possibly useful
- 🟡 Unique Opens: possibly useful
- 🟡 Unique Clicks: possibly useful
- 🟠 Total Opens: not that useful!
- 🔴 Total Open Rate: downright misleading
The unique open rate (I'm just going to call this "open rate" from now on) tells you what percentage of the contacts opened the email, and the click through rate is the same for those who click a link. Multiple clicks/opens by the same contact will not affect the figures, so these are fairly stable.
These are by far the most useful statistics and you can fairly reliably compare two given mailings against each other using these. Below I look at what they actually mean, and conclude that for a single mailing they are not that meaningful. But that doesn't take away from their value in comparing different mailings.
The click-to-open rate is complex, and potentially too hard to understand the meaning of. In theory, it sounds good: a 100% here would mean that everyone who opened your mailing clicked at least one link, so that would be great. Again, it might be useful to compare one mailing with the next.
The unique opens/clicks are an absolute count of contacts who did that. Percentages are usually more readily understandable as they don't depend on variance in mailing size.
Total opens really means nothing (I cover why below), and the total open rate is a percentage that can exceed 100%, so ...
Introducing Olivia Open Rate
Me: Hi, Olivia, thanks for joining us for this candid discussion of your role.
Olivia: Please see my remote images.
Me: Sorry, I didn't get that, erm, (bit awkward), maybe later?. Anyway, could you explain what you do?
Olivia: It's very simple, I let you know how many people opened your email, of course!
Me: Impressive, how do you do that?
Olivia: A link to an invisible remote image gets placed in every email and when you open your email this image is fetched from the internet and because the link is unique to you, CiviCRM knows you have requested the image, and therefore that you have opened the email.
Me: I see, that's clever, so when the email gets opened, the image gets loaded, and we know whose opened it. Perfect!
Olivia: Yep. Oh, but I forgot to say that it sometimes happens before you've opened the email.
Me: Huh? What?
Olivia: Yeah, well it depends on your email client and your email service provider.
Me: Email client?
Olivia: you know, the app you use, like Gmail, K9-Mail, Outlook, Mac Mail, Thunderbird and many, many others. And service provider might be Gmail/Microsoft/your homespun email server/your company's email server etc.
Me: Right, so why might it be "opened" before it's actually opened?
Olivia: well, some apps offer a little preview of the email - that might cause the tracker image to be fetched; some providers, especially corporate ones, might perform some security scans which involve loading images and following links to see if they seem safe. And Mac Mail now offers a privacy option that means the image will always be fetched by Apple whether you open it or not.
Me: Oh right...that's a shame. So you might give us numbers that are a bit on the high side, with the real open rate being a bit less. That's ok, I can work with that.
Olivia: Yeah, possibly. Although I also forgot to say that sometimes it never happens even when someone reads the email several times.
Me: Really? Why would that be?
Olivia: Well, some people are all fussy about their privacy. And what's worse, some email clients - particularly those goodie-goodie open source ones - tend to default to being privacy aware. So these apps might not show images, or not show them until the user consents, e.g. by clicking a Show Remote Images button or similar.
Me: Hey, that's me! I never click that button unless I think I'm missing something, and often it's just logos and stuff I can't be bothered with anyway. I don't want to be spied on by my own software.
Olivia: I can go off people, you know.
Me: Oh sorry, it's not that I don't trust you, it's more about who you work for. Hmmm, so you can't really tell us if someone has opened their email or not then?
Olivia: Sorry, not really, no. Not on an individual basis. But if you regularly email a big list of people then those people are probably all still using the same email services as they did last time you sent an email, so those that report opens will report opens, and those who don't still won't. So for a given list, you can still use my numbers to understand if your mailings were successful.
Me: Ok, thanks so much for your time.
Olivia: See remote images?
Me: Sure, now I know we're friends. {click} (sigh, just some logos again..)
Olivia: (note to Civi: Rich got in touch, they've opened the email!)
Introducing Clive Click Rate
Me: Clive, welcome, I've just been chatting with your friend Olivia Open Rate - wow, she's pretty deep. I'm hoping this chat will be easier! Could you start by saying what you do?
Clive: Hi Rich, thanks for the click. I can tell you what percentage of people clicked a link in your email.
Me: That is what I suspected, good so far, so dare I ask: How do you do it?
Clive: Well, it's simple really
Me: I've heard that before...
Clive: When an email gets sent through CiviMail with click tracking enabled, all the links that the author put in get swapped out for special links that go to your CiviCRM instance. So when someone clicks a link their browser asks CiviCRM for the page, and it counts their click and redirects them to the real URL/link.
Me: Oh right, like a link shortener.
Clive: Yes, exactly.
Me: Phew, that's simple. So if someone wants to click a link in my email, then my CiviCRM will definitely register the click.
Clive: Yes!
Me: So much simpler. So nice to know that it's exactly right and not as woolly as Olivia's open rate!
Clive: I didn't say it was exactly right.
Me: I had a sinking feeling you'd say that.
Clive: Well, sometimes, before your email gets to you, some other systems might have had a peek at the link as they passed your email along the way.
Me: But! But! That's rude! That's my private email!
Clive: Well the good ones are doing it for your protection; they just want to make sure the link doesn't go to some phishing site or something like that. Of course the bad ones just want to trawl data about you, so want to know what links you might see, so they can deliver adverts to you.
Me: How much does that happen?
Clive: Surprisingly little at the moment, actually. But these things are always changing.
Me: That's a relief. I suppose you're going to tell me that sometimes there's an exception to the rule about people clicking are definitely registered, too?
Clive: Actually not so much. There's nothing to stop it happening, e.g. maybe an email provider might choose to look up tracking links and replace them with direct links, thereby meaning the recipient can click in peace without being registered - although CiviCRM will show that they clicked every link once. But I don't know of any that do that ... yet. The other thing to watch for is that if your link text (the bit people click/tap/activate) is itself a URL, then the actual URL, the tracked one, is different and some email clients might warn users "This link looks like it goes to x.com but it actually goes to y.com" or such, which might put people off clicking. For this reason it's best to avoid putting URLs in the link text.
Me: Good tip, thanks Clive. So your click rate numbers are generally pretty accurate then?
Clive: I do my best.
Me: Thanks very much for your time.
In Summary
You can't really know whether a given person has actually opened a given email. But you can use ("unique") open rate to compare mailings sent to the same cohort or list over time.
Watch out for big shifts that could mean a big provider has changed its rules or algorithms (example: Mac Mail). Typically, the open rate is a bit conservative; it's likely that your mailings are a bit more read than it says.
Don't be That Organisation which sends the "We're sad that you've not been opening our emails..." type mailings - because you'll make people cross, either because they don't like the idea of you watching them, or because they always read your emails and possibly don't even realise why you might think otherwise.
Click rates are generally more reliable/accurate, down to the individual. Also, the mailing report you get by clicking Report by the mailing listed under Scheduled and Sent Mailings has a useful list of which links were clicked the most, which can help you understand what people want from your mailings.
Errata
Clive and Olivia - who both make me feel a lot less geeky than I thought I was - got in touch after publication of this blog post to say that just before anyone relaxes, they may have over simplified things a little too much, and that another thing to consider is what happens when someone forwards their email containing all those links and images that uniquely identify the original recipient to their friends, or share a link they copied from the email on social media...
Comments
Brilliant! Thanks for the down-to-earth explanations Rich. I look forward to meeting Olivia and Clive at a future CiviCRM event!