The Progressive Technology Project (PTP) works with groups who organize community members around social justice issues. These groups recruit members and help to develop their leadership abilities. In order to measure the impact of the organization's work to develop leaders and more effectively reach out to more people in the community, these organizations want to be able to track how often and how much members interact with them, and in what ways (door-knocking, volunteering at events, attending rallies, etc.). These interactions can be summed up as "member engagement".
The typical mechanism for tracking member engagement is the memory of organizers. Not only is this inefficient - and potentially inaccurate or incomplete - but when the organizer leaves the organization, that information usually leaves with her. Unfortunately when the information remains with that individual rather than becoming part of shared organizational knowledge, this knowledge is lost.
PTP's proposal is for CiviCRM to be able to track an organization's interactions with an individual as an "engagement score". It would be possible to measure member engagement at least minimally by adding a numerical field to Activities so that each group could decide what level of engagement a particular activity is worth to them, and aggregate an individual's interactions as a total engagement score.
For example, PTP would use a scale of 1 to 5 to rank Activities, where an activity that requires the most engagement would be ranked 1 and one that required the least would be ranked 5. The score for attending an event could be set to a 4, while participating in a GOTV effort might be scored a 3. An individual's score would be the average of all their accumulated scores (i.e. a 1.5 would show high engagement). We may need to query some Activities to better capture engagement. For example, someone who speaks at a meeting should get a better engagement score than someone who attends that meeting. Both activities are tracked in event attendance but the different scores would be associated with different Roles (i.e. Speaker vs. Attendee).
Tracking member engagement will give groups a clearer picture of their constituents, and will let them reach out to members with different engagement levels in more targeted ways. It will also create new groupings of members with the same score who may have done different activities (someone who makes a contribution could have the same score as someone who completes a survey and signs up for the newsletter). This gives organizing groups ways of deciding explicitly how to value different kinds of engagement.
Using an engagement score to track member interactions with an organization is also useful because it allows the organization to develop a strategy for moving members towards greater engagement. This concept is known as an "Engagement Ladder". The goal is to understand where your members are on the ladder, and devise strategies to move them towards higher-ranked activities, or greater engagement.
PTP will be commissioning a custom search and report that will allow you to see the most active people in a period of time. Has your organization created a Ladder of Engagement? How do you move people up the Ladder to more engagement? Do you have an idea for how you would track your member engagement score?
We welcome all thoughts, stories, suggestions, and discussion about how to make this more useful for community organizing groups, and for the CiviCRM community at large.
Written by Robyn Perry and Josue Guillen, PTP Staff.
Comments
The average doesn't seem like it captures everything that is relevant. In fundraising, recency, frequency and monetary amount are commonly combined to provide basic targeting (see http://en.wikipedia.org/wiki/RFM). Your metric seems to focus on the equivalent of monetary amount while ignoring frequency and recency.
Still, this is a nice start.
On another matter, I tried install CiviEngage last August or September and it was difficult to turn off some features. You seem to be developing what is like a menu of 'best practices' or at least PTP suggested practices for how to configure CiviCRM for phone and foot canvassing. In many cases clients don't want the whole model you are using for engagement, just a bit of functionality you have added like a walk-list report.
In Drupal Commons, there's a User Points function, and in Joomla, Dioscouri has one in their Ambra User Manager, and AlphaPlug.com has AlphaUserPoints. Wouldn't it be nice to use one of these existing frameworks? It would seem that maybe the two CMS communities would be more able to pitch in. Then, the project could focus on the scoring algorithms, and hand off implementation to the existing modules...theoretically, at least.
Hi,
Is the idea of putting a score for each activity type (meeting is 4, each action on the civicampaign is 3...) and so the score can be done automatically or to do it everytime one enters an activity (this meeting was a solid 2...) to have more work/flexibility?
As Joe pointed out, time matters and 20 "engagments points" 3 years ago don't have the same value as those now.
One simple thing might on the report to filter by date, and be able to compare (manually) who was active last year and isn't this year.
On mepwatch, we score the "engagement" (number of speeches or documents) of meps per keyword/topic and display them as "nameclouds". The most active a contact, the bigger the font. To visualise the freshness of the engagement, we use the color, so someone that has been very active but isn't a lot now is big font with washed out color, and someone that has just started being active is in small font but dark color. Works ok I think.
eg
http://www.mepwatch.eu/mepwatch/cloud/topic/6037
(btw, I'd expect the other way around, 5 is higher engagment than 1, but detail)
This is a good idea. We have to be clear, though, in our thinking about what statistics are offered for making an analysis, so that the results aren't misleading.
What I suggest is that the online help for such a feature should guide them to use statistics that are appropriate to their "measure of engagement".
@Josue writes ...
"For example, PTP would use a scale of 1 to 5 to rank Activities, where an activity that requires the most engagement would be ranked 1 and one that required the least would be ranked 5. The score for attending an event could be set to a 4, while participating in a GOTV effort might be scored a 3. An individual's score would be the average of all their accumulated scores (i.e. a 1.5 would show high engagement)."
@Josue's example measure is a "category" of engagement.
Unfortunately, categorised information can't be summed or averaged. Adding an instance of Category 1 and and instance of Category 2 doesn't give a Category 3 instance. The only statistic that makes sense for Categories is the "count" in each Category.
If the Categories are ordered, then "minimum" and "maximum" become valid statistics.
If the measure of engagement counts something (ie, integers) (eg, Frequent Flyer points) then the "sum" becomes a valid statistic.
If the measure of engagement "measures" something (ie, real number) (eg, Hours of engagement) then the "average" becomes a valid statistic.
let me explain some more and provide an example.
with this proposal each activity in my database can have a rating. the purpose of this rating is to show how much engagement a person is showing when they do this particular activity.
a rating on 1 is bestowed upon people who are really active and providing leadership. the reason, xavier, for doing it this way is because people aspire to be number 1, not number 5. :) organizing a phonebank would have a rating of 1. writing a letter to an elected official would be rated a 4. not that hard but still some engagement. attending a meeting would be a 3.
when person1 attends a meeting (on 8/7/10) they get a 3. they wrote a letter (11/2/10), they get a 4. another letter (1/2/11), another 4. then they attended a meeting (1/16/11), for another 3.
because we are tracking this in the database, here is the kind of reporting i am talking about.
we can show the first time someone took a scored action: 8/7/10
we can show the last time they took one: 1/16/11
we can show the total number of actions they took: 4
we can show the average leadership score they have: 14 / 4 = 3.5
we can show the number of the kinds of leadership actions they took:
1 - 0
2 - 0
3 - 2
4 - 2
5 - 0
this actually tells me quite a bit. 3.5 on an engagement ladder that goes from 5 up to 1 means i have someone who is willing to do some work. i can send her more, see if she responds to greater engagement.
it would be nice to be able to set the default value for a particular activity when i create the activity, but it is essential that the individual activity be able to be rated independently. attending the meeting at the state capitol could be rated higher than the meeting at the local school, but it would be nice to not have to create a new kind of activity.
hope this helps...
Hi,
We have tentatively decided to use Events, rather than Activities, to track member actions such as attending hearings, providing testimony, signing petitions, contacting elected officials, and participating in rallies. Events seems more robust than Activities, especially the Manage Events part of it. We are not intending to be highly quantitative about this, in the sense of developing an engagement score. We want to have a count of the events a person has participated in, and also want to track their reliability -- what percentage of events that a person commits to do they actually show up at? It will be a bit confusing since some of our "events" are really actions. I don't think we can change the name, though. All events will have a date range, and will be linked to a specific project -- which will also give us some ability to track effort, such as how many phone calls, emails, pre-meetings, and so on went into a particular policy win.
The blog is a great idea, by the way.
Joy Williams