Blogs

Over the next few weeks, I will blog about the architecture of CiviCRM and some basic principles that we follow in the design/coding phase. In this series I’ll also talk about how we integrate with Drupal/Joomla and the scalability/performance/memory bottlenecks in CiviCRM.

CiviCRM is built on top of some incredibly powerful open source toolkits. They include:

It's been almost 2 wonderful years that I have been working on CiviCRM project, since we started development in India. From then things have changed. We started with a single file, (I think it was CRUD.php) which no longer exists :). What excites me the most about CiviCRM is the challenges that we face in implementing new features and the opportunity to work on advanced technologies. The scope is simply unlimited...

CiviCRM is not just another software, it is driven by the needs of the community, which makes this project very special to me. Since this software has also simultaneously been developed in US and Poland, makes it global in nature. This project has been crafted meticulously, so that there is a lot of scope for extensibility and it can be tailor-made to suit the requirements of host of organizations and non-profits. A lot of credit needs be given to the pioneers of this project, who are harbingers of the today's IT Industry.

So this week, our focus has been on getting some cool new features into CiviCRM v1.6. We figured that we'd take some baby steps into the wide world of web 2.0 and ajax and use Dojo as our toolkit of choice.

Yes we do know that Drupal chose JQuery and Joomla v1.5 has a combination of a few different ajax utilities. However, we did like the direction Dojo was moving in with a fairly nice large community along with some of the big names in the industry (IBM, Sun etc). We also did meet Alex Russell, one of the lead developers/architects and its always a good practice to use tools from folks you know :) (since u can then pester them if you need help at some stage)

We made decent progress on our 1.6 queue last week. Some of the issues we targeted for resolution are still in progress, but the queue is now down to 23 items (from 39 last week).

For the coming week we'll try to get the queue down to around 10 issues and resolve the following larger issues:

Another feature that we worked on this week was integrating the "standard" version of PayPal as an additional payment option for CiviContribute. We waited for quite a few months for PayPal (or Google) to release a nice web 2.0 api for subscriptions / recurring transactions, but that did not seem to be happening. So we took the plunge and decided to implement PayPal Web Standards as the first step towards implementing PayPal Subscriptions (basically recurring donations)

PayPal has done an incredible job of documenting their API's and making the integration fairly easy. It helped that the Drupal e-commerce package has this feature, so we could peek into their code base and figure out how to do a few things. Getting a system up and working using the sandbox makes development so much easier, since the sandbox behaves identically to the live system. I was pleasantly surprised to see that we did not have to write a lot of code for this case, but primarily had to rearrange and cleanup the existing code to handle the synchronous (SOAP api) vs the asynchronous (IPN) implementation.

We are just wrapping up our first implementation of access control lists (ACL) in CiviCRM (for v1.6). This gives us a framework for building fairly sophisticated access permissions into the data model and allowing fine granularity of who can view/edit/delete various sections of your contact database

Our first implementation replicates what we currently achieve with drupal's permissioning system. A CiviCRM admin can partition their contact database into multiple sections, and give view/edit access to different "roles". Integrating this into the core allows us to expose this functionality to our Joomla! users also. We do not use a matrix of checkboxes to present all potential combination, but rather present one "ACL" at a time. Obviously each method has its advantages/disadvantages, but the former is inherently non-scalable (as some of our friends in canada discovered when trying to partition their db into 300+ ridings), while the latter would probably need an external script to populate the database tables for significantly large sets of ACLs

We have been thinking it would be useful to share our team goals with the community on a regular basis - so folks have a clearer idea of what we're working on (without wading through the details in the Issue Tracker).

We currently have 39 open issues on the 1.6 queue - and we need to knock-off at least 10 next week in order to be on-track for 1.6 beta. Here's the issues we'd definitely like to resolve:

We've been working on the Branner Project and writing code to help analyze the applications and process them in a suitable manner. The application collects a fair amount of data across multiple pages and as such is not very conducive to reading on the web.

We wanted to analyze the data in Excel. The database is relational and hence hierarchical. CSV is not a great format for this purpose. So we wrote a CiviCRM -> XML converter and got all the data we needed in a nice easy to understand / parse xml. We then used a PEAR package (Spreadsheet_Excel_Writer) on windows to convert the xml to a set of excel worksheets. As part of this package we also included the contact_id, which allowed us to import results back into CiviCRM using the contact_id

CiviMail is a component that allows the users to create, send, track and manage CiviCRM-based mailings. In CiviCRM 1.5, CiviMail underwent speed and memory optimisations; in CiviCRM 1.6 we’re concentrating on usability improvements.

Last week was a pretty intense week for the team. The quest application process had the application deadline and as expected the load on the servers was really high for an extended period of time. As always we learnt quite a few lessons and a few things to do differently the next time around. Some of the lessons learnt include: