Chained SMS

For the past few months we have been working with Future First, who have an ambitious mission of 'creating an alumni network for every state school in the UK'. A key part of this work is the ability to track and engage with recent school leavers and we've spent a fair amount of time building tools that allow them to do this. One of these is an extension that allows them to hold scripted SMS conversations with their contacts in CIviCRM.  We've just piloted this functionality with them and the results look quite promising.

The background

Over the summer Future First signed up over 25,000 school leavers to their alumni networks. They did this in two ways. Approx. 50% filled in their details just before they left school using an online form that we built with webform_civicrm. The form collected their name, email, mobile number and optionally facebook details. The other half filled in paper forms that were later electronicified and imported via CiviCRM's import. Now it is autumn: these former students are taking their first post-school steps, and Future First is keen to find out what they are up to.

As you may or may not know, email is considered a bit 'passe' with the younger population, so we had to explore other ways to collect information on what they are doing now. Thanks for funding from Zing and PTP (amongst others) CiviCRM 4.2 shipped with CiviSMS - a tool that allows you to send/receive SMS to/from contacts in your database. Out of the box, CiviSMS is great for sending out information blasts and for having unstructured conversations with individual contacts, but it doesn't provide an easy way to automate two way conversations or facilitate the collection of structured data - something that is key to this project.

In this instance, for example, Future First wanted to know if former students were: a) working, b) at university, c) doing an apprenticeship or d) something else, and they wanted to about each of these options in more detail (e.g. what university they were at and what course were they doing, etc.) Of course, if they were using a webform, collecting this data would be easy - they'd just create a whole set of questions and have the later questions depend on answers to the earlier questions.  But how do you do that with SMS? Thankfully it was not to difficult to russle up an CiviCRM extension to help out and allow Future First to hold conversations like the following with thousands of students.

FutureFirst: Hi Michael! Rodborough School would like to know what you are up to now you have left. Text back A for uni, B for working, C for apprenticeship, D or other [text A, B, C or D - texts cost your normal network rate]

Michael: B

FutureFirst:Thanks for letting us know. Can you let us know your job title and employer seperated by a comma? (e.g. 'Doctor, NHS'). Last question, we promise!

Michael: Community Manager, CiviCRM

FutureFirst: Thanks for your help Michael! :)

The results

We've just finished a pilot where we compared sending an SMS conversation to sending a email with a link to webform civicrm.  It is still early stages, but the results look promising.  42% of former students that we texted replied to us compared with 14% of people that we emailed - that is a significant difference for this demographic at least.

Behind the scenes - the SMS extension

There are a few different parts to the chained SMS extension:

• An SMS API which allows you to send text or template based SMS to individual contacts or groups of contacts
• Inbound SMS processor that reads incoming SMS and replies to them when necessary
• A set of questions and answers that extend message templates to form the script of the conversation

SMS API

We implemented SMS as an Contact action.  It takes either a contact_id or a group_id parameter, and either a msg_template_id or a text parameter to send SMS to contacts.  I'm interested to know what people think about this way round of doing it. The alternatives would have been (and still could be) phone.sms, sms.send, sms.create, etc.  And it would be nice to standardise this with the new mailing.create api.

Inbound SMS processor

The inbound SMS processor is fired each time an SMS is received. It decides whether it needs to reply.  This is one of those things that is trickier than it initially sounds.  Us humans are quite good at it but we needed to teach CiviCRM some text message ettiquite.   Each time an SMS is received Civi asks itself "Did I just ask this person a question? If I did, have they replied yet. If they haven't did they say something that I should reply to?". As soon as it works out what it needs to say, it uses the API to send another SMS.

SMS questions and answers

We decided to store all SMS questions in message templates.  We then created another table for SMS answers.  The answers table lists all possible answers for all questions and also stores another message template id for a further outbound SMS.  So we can construct logic like "if this person replies C to message 75, then ask them what they are doing for their appreniceship".  or "When someone replies to this text, send them a thankyou text".

Current status of the code

You can download the latest version of the Chain SMS extension can from github. It was developed to work with CiviCRM 4.2 and requires a few changes to core SMS functionality and minor changes in other files and the DB.  You can see the code changes by diffing this version of CiviCRM against v4.2.6.  I'm happy to walk people through the other changes if they want to get this working for themselves.

From out point of view, the next steps in this project are to find an automated and reliable way to clean and validate the information we've collected, what do I do if I was expecting 'A' and someone replies 'A :-)' or 'A - who is this, by the way?'.  And how do we move replies it into the right place in CiviCRM (something that is much harder to automate in SMS than it is webform, because the SMS is a string with no validation.

If you want to test out, improve or extend this functionality, then please feel free to dive in (and also feel free to let me know what you are doing / what you think of it).

If you can wait a bit, this functionality should find its way into a supported extension and Core CiviCRM by 4.3 at the latest.