Canvass and Phonebank

Introduction: The Need

Progressive Technology Project (PTP) supports grassroots community organizing groups nationwide that engage in social, economic and environmental justice efforts to empower low-income people and communities of color. These groups conduct phone banking and field canvassing as an integral part of their base-building and voter engagement strategies to engage existing and new constituents. PTP hired DharmaTech to help them build PowerBase: a customized CiviCRM for use by community organizing groups.

PowerBase helps groups manage the following data related to phonebanking and field canvassing work:

• voter info, history, and demographics
• reason why the phonebank or canvass is taking place
• responses to questions asked during the phonebank or canvass
• phonebank or canvass status
• primary language spoken

CiviCRM falls short of generating phonebank and canvass lists, because these features were missing:

• phonebank and canvass list report templates based on voter info, reason for canvass, responses, etc.
• a report filter to search on specific criteria such as language spoken and smart groups
• ability to display non-European alphabet (e.g. Chinese characters) in the PDF format of the report
• address parsing (a critical feature for creating canvass lists, a.k.a. "walk lists")
• ability to sort lists without affecting householded contacts

DharmaTech's Challenge

We used CiviReports to implement phonebank (call list) and canvass lists (walk list) for PTP's groups.  The goal was to generate a PDF file of information about selected contacts that could be used by people phonebanking and canvassing.

To do this we had to overcome a number of challenges. First, we drafted workflows for phonebanking and canvassing and reviewed them with PTP.  Second, we tackled the reporting mechanism.  The screen that the user uses to generate a call or walk list exposes data fields and provides filters to select data values from tables.  These fields and filters are defined by an array whose structure is not documented.  Therefore, we had to read the code to understand how to produce the needed screens.

Some of the logic for field definitions depends on the table Data Access Object (DAO).  The DAO for a table is generated when a CiviCRM distribution is created, and so no DAO exists for a custom table.  After lengthy study of the code, it became apparent that the need could be met by using the DAO of that table which was extended by the custom table.

The available filter operators are very limited, particularly for select and multiselect operations.  In particular, the operator for a single select is configurable, which is only occasionally useful, and there is no way to replace the operator with the static word "is".  The operator for the multiselect, on the other hand, is not configurable.  We needed to allow the user to choose among interpretations of the multiple selections.  For this reason, we provided alternative implementations of the select and multiselect operators.  The CRM_Report_Form class defines filter operators with class constants and a case statement, which provides no method of adding or overriding operators in a subclass.  Therefore it was necessary to modify the code of the distributed CRM_Report_Form class to define the needed filters.

The walk list required pages separated by street name and odd/even house numbers, ordered by house number.  The columns in the civicrm_address table for street_number, street_number_suffix, street_number_predirectional, street_name, street_type, street_number_postdirectional and street_unit are not populated in the database.  It was necessary to parse this information from the street_address and supplemental_address_* columns.  Then a temporary DB table was built to permit selection of the needed information.

A requirement was that the PDF show names in the contact's native language.  The CiviCRM database is internally UTF-8 Unicode, so storing and retrieving characters in any language is not a problem.  The challenge was to output characters outside the western European alphabet to the PDF file.  The basic problem is that PDF is not designed to support an alphabet with more than 255 characters, so direct support for Chinese, Japanese, Korean and Vietnamese character sets is not possible by the usual PDF mechanisms.  dompdf, the PDF generation software distributed with CiviCRM, in particular doesn't support Unicode.  However, there are various work-around methods used by certain software packages.  Firefox can display Unicode characters and output them to a PDF file, with certain restrictions.  This is a work-around until a Unicode PDF facility is implemented.

To see a demonstration of what was discussed:

http://ptp.dharmatech.org

username: demo
password: demo

You can view the work-around described by logging in - click on canvass list - select filter "group",  "is equal to", "Mandarin" - print preview. You'll see Firefox display Unicode characters and can save that html to pdf format.

Want to contribute to this project?

1. A related feature needed in conjunction with producing phonebank and canvass lists, is the ability to input responses gathered from the phonebank or canvass, through a "batch" data entry screen that looks similar to the reports. A user can:
• pull up the list of contacts for the phonebank or canvass, view a batch data entry screen that resembles the report, and begin entering the responses, or
• have a search view that resembles 1 record of the report, where the user can either enter the contact id number or the barcoded representation of the contact id (barcodes are a type of font read by a barcode reader device) to find the contact, and then record the responses manually or use barcoded representations of the responses.
2. Implement a PDF output that supports Unicode characters. Note: After researching, one approach to achieve this is using a free Java library called iText and a package called xhtmlrenderer which uses iText as an exact drop-in replacement for dompdf.