CiviCRM in India...

It's been almost 2 wonderful years that I have been working on CiviCRM project, since we started development in India. From then things have changed. We started with a single file, (I think it was CRUD.php) which no longer exists :). What excites me the most about CiviCRM is the challenges that we face in implementing new features and the opportunity to work on advanced technologies. The scope is simply unlimited...

CiviCRM is not just another software, it is driven by the needs of the community, which makes this project very special to me. Since this software has also simultaneously been developed in US and Poland, makes it global in nature. This project has been crafted meticulously, so that there is a lot of scope for extensibility and it can be tailor-made to suit the requirements of host of organizations and non-profits. A lot of credit needs be given to the pioneers of this project, who are harbingers of the today's IT Industry.

Experiences with ajax and dojo ...

So this week, our focus has been on getting some cool new features into CiviCRM v1.6. We figured that we'd take some baby steps into the wide world of web 2.0 and ajax and use Dojo as our toolkit of choice.

Yes we do know that Drupal chose JQuery and Joomla v1.5 has a combination of a few different ajax utilities. However, we did like the direction Dojo was moving in with a fairly nice large community along with some of the big names in the industry (IBM, Sun etc). We also did meet Alex Russell, one of the lead developers/architects and its always a good practice to use tools from folks you know :) (since u can then pester them if you need help at some stage)

Team Goals for the Week of October 30

We made decent progress on our 1.6 queue last week. Some of the issues we targeted for resolution are still in progress, but the queue is now down to 23 items (from 39 last week).

For the coming week we'll try to get the queue down to around 10 issues and resolve the following larger issues:

  • Add support for free membership signup. This should also allow for contribution pages which solicit in-kind (non-monetary) contributions.
  • Finish work on Access Control (ACL) functionality for group permissioning (CRM-1308).

PayPal Web Standards and IPN added to CiviContribute

Another feature that we worked on this week was integrating the "standard" version of PayPal as an additional payment option for CiviContribute. We waited for quite a few months for PayPal (or Google) to release a nice web 2.0 api for subscriptions / recurring transactions, but that did not seem to be happening. So we took the plunge and decided to implement PayPal Web Standards as the first step towards implementing PayPal Subscriptions (basically recurring donations)

PayPal has done an incredible job of documenting their API's and making the integration fairly easy. It helped that the Drupal e-commerce package has this feature, so we could peek into their code base and figure out how to do a few things. Getting a system up and working using the sandbox makes development so much easier, since the sandbox behaves identically to the live system. I was pleasantly surprised to see that we did not have to write a lot of code for this case, but primarily had to rearrange and cleanup the existing code to handle the synchronous (SOAP api) vs the asynchronous (IPN) implementation.

Access Control List (ACL) implementation

We are just wrapping up our first implementation of access control lists (ACL) in CiviCRM (for v1.6). This gives us a framework for building fairly sophisticated access permissions into the data model and allowing fine granularity of who can view/edit/delete various sections of your contact database

Our first implementation replicates what we currently achieve with drupal's permissioning system. A CiviCRM admin can partition their contact database into multiple sections, and give view/edit access to different "roles". Integrating this into the core allows us to expose this functionality to our Joomla! users also. We do not use a matrix of checkboxes to present all potential combination, but rather present one "ACL" at a time. Obviously each method has its advantages/disadvantages, but the former is inherently non-scalable (as some of our friends in canada discovered when trying to partition their db into 300+ ridings), while the latter would probably need an external script to populate the database tables for significantly large sets of ACLs

Team Goals for the Week of October 23

We have been thinking it would be useful to share our team goals with the community on a regular basis - so folks have a clearer idea of what we're working on (without wading through the details in the Issue Tracker).

We currently have 39 open issues on the 1.6 queue - and we need to knock-off at least 10 next week in order to be on-track for 1.6 beta. Here's the issues we'd definitely like to resolve:

  • Link custom fields to specific types of activities, relationships, etc. (CRM-1177).

Exporting CiviCRM data for deeper analysis ..

We've been working on the Branner Project and writing code to help analyze the applications and process them in a suitable manner. The application collects a fair amount of data across multiple pages and as such is not very conducive to reading on the web.

We wanted to analyze the data in Excel. The database is relational and hence hierarchical. CSV is not a great format for this purpose. So we wrote a CiviCRM -> XML converter and got all the data we needed in a nice easy to understand / parse xml. We then used a PEAR package (Spreadsheet_Excel_Writer) on windows to convert the xml to a set of excel worksheets. As part of this package we also included the contact_id, which allowed us to import results back into CiviCRM using the contact_id

Moving into CiviCRM v1.6

Last week was a pretty intense week for the team. The quest application process had the application deadline and as expected the load on the servers was really high for an extended period of time. As always we learnt quite a few lessons and a few things to do differently the next time around. Some of the lessons learnt include:

  • Balance is not a very good load balancer. It croaked under heavy load and did not distribute the traffic evenly. An not aware of any other open source "web server balancing" solution. If someone knows other solutions, please let us know.

Batch Update Feature in v1.6

We've been polishing up a cool new 1.6 feature over the past few days - and I got pretty excited seeing it working with a real set of data. The feature is Batch Update Via Profile. It allows you to define a profile with (almost) any standard and custom fields and use that to display a table for updating many contacts at once.

For example, if you want input hard-copy responses from a mailer for a group of contacts...You could define a profile with the fields you want to update. Then use Find Contacts to retrieve contacts in the group and select "Batch Update via Profile" from the -actions- drop-down menu. You then get a table with a row for each contact in the group and the fields you've selected for updated. Update some or all of them and click "Update" - and your changes are saved.