CiviCRM Architecture - Object Structure and SQL/DAO Generation

CiviCRM uses the PEAR package DB_DataObject to access the mysql database. Doing so reduces the amount of sql we need to write to fetch / store values for a single table. For queries involving more than one table, we write them manually since since it was fairly painful to try to force complicated queries using DB_DataObject (we did use DB_DO for multiple table queries when we wrote EmailNow, in the end we felt the hoops we had to jump through was not worth the effort and hence the current scheme)

The basic PHP classes that represent a table are called DAO's (Data Access Objects). These are generated automatically by (xml/GenCode.php) which also generates the sql structure for that specific table. The xml files representing the schema is stored under CIVCRM_ROOT/xml/schema. The sql file generated (civicrm_41.mysql and civicrm_40.mysql) are stored under CIVICRM_ROOT/sql (note that the generated files are not present in the svn repository). The DAO files are stored under CIVICRM_ROOT/CRM/Core/DAO (and under appropriate directories based on what table it is, this is configured in the xml representation.

CiviCRM v1.6 release schedule

here is a tentative schedule for CiviCRM v1.6. The feature list for v1.6 is here. As all good open source software projects, the below schedule is subject to change :)

  • Nov 10: Code Freeze, all major features implemented
  • Nov 10 - Nov 17: Internal QA
  • Nov 17: v1.6 alpha released. Sandbox site up and running for community testing
  • Nov 17 - Nov 29: Community testing on CiviCRM sandbox
  • Nov 29: v1.6 beta released
  • Mid December: v1.6 released

CiviCRM Architecture - CiviCRM directory structure

Make sure you read the Introduction Chapter of this series for a better understanding of the below. Developers might be interested in installing CiviCRM from our svn repository

Some of the important directories under CiviCRM svn root are:

  • CRM: Most of the CiviCRM specific source code is stored in the directory. This directory is further sub-divided into other directories based on functionality (Core, Utils, Contact, Contribute, Mailing etc).
  • xml: CiviCRM schema and the base Data access objects (DAO) are automatically generated from a simplified xml scheme which contains a fair amount of sql and type information. This makes it relatively easy for us to change the sql code, figure out what changes were made in a version and add meta information to a table within PHP.

Team Goals for the Week of November 6

We made excellent progress on our 1.6 queue last week. All but one of the issues targeted have been moved into resolved status. The queue is now down to 12 items (from 23 last week). Some team members will be moving into active QA mode this week - doing regression testing on the 1.6 resolved queue to verify that things haven't gotten broken since they were resolved earlier in the release cycle.

For the coming week we'll try to get the queue down to 5 issues or less... and finish the following larger issues:

CiviCRM Architecture - Introduction

Over the next few weeks, I will blog about the architecture of CiviCRM and some basic principles that we follow in the design/coding phase. In this series I’ll also talk about how we integrate with Drupal/Joomla and the scalability/performance/memory bottlenecks in CiviCRM.

CiviCRM is built on top of some incredibly powerful open source toolkits. They include:

  • PHP, we use PHP 5.1 for development and will probably switch to PHP 5.2 in the next few weeks as our primary development platform. We restrict ourselves with the features that we use in PHP 5 to enable an auto-translation of the code to PHP 4. I suspect we will drop support for PHP4 around the time we release CiviCRM 2.0 :)

CiviCRM in India...

It's been almost 2 wonderful years that I have been working on CiviCRM project, since we started development in India. From then things have changed. We started with a single file, (I think it was CRUD.php) which no longer exists :). What excites me the most about CiviCRM is the challenges that we face in implementing new features and the opportunity to work on advanced technologies. The scope is simply unlimited...

CiviCRM is not just another software, it is driven by the needs of the community, which makes this project very special to me. Since this software has also simultaneously been developed in US and Poland, makes it global in nature. This project has been crafted meticulously, so that there is a lot of scope for extensibility and it can be tailor-made to suit the requirements of host of organizations and non-profits. A lot of credit needs be given to the pioneers of this project, who are harbingers of the today's IT Industry.

Experiences with ajax and dojo ...

So this week, our focus has been on getting some cool new features into CiviCRM v1.6. We figured that we'd take some baby steps into the wide world of web 2.0 and ajax and use Dojo as our toolkit of choice.

Yes we do know that Drupal chose JQuery and Joomla v1.5 has a combination of a few different ajax utilities. However, we did like the direction Dojo was moving in with a fairly nice large community along with some of the big names in the industry (IBM, Sun etc). We also did meet Alex Russell, one of the lead developers/architects and its always a good practice to use tools from folks you know :) (since u can then pester them if you need help at some stage)

Team Goals for the Week of October 30

We made decent progress on our 1.6 queue last week. Some of the issues we targeted for resolution are still in progress, but the queue is now down to 23 items (from 39 last week).

For the coming week we'll try to get the queue down to around 10 issues and resolve the following larger issues:

  • Add support for free membership signup. This should also allow for contribution pages which solicit in-kind (non-monetary) contributions.
  • Finish work on Access Control (ACL) functionality for group permissioning (CRM-1308).

PayPal Web Standards and IPN added to CiviContribute

Another feature that we worked on this week was integrating the "standard" version of PayPal as an additional payment option for CiviContribute. We waited for quite a few months for PayPal (or Google) to release a nice web 2.0 api for subscriptions / recurring transactions, but that did not seem to be happening. So we took the plunge and decided to implement PayPal Web Standards as the first step towards implementing PayPal Subscriptions (basically recurring donations)

PayPal has done an incredible job of documenting their API's and making the integration fairly easy. It helped that the Drupal e-commerce package has this feature, so we could peek into their code base and figure out how to do a few things. Getting a system up and working using the sandbox makes development so much easier, since the sandbox behaves identically to the live system. I was pleasantly surprised to see that we did not have to write a lot of code for this case, but primarily had to rearrange and cleanup the existing code to handle the synchronous (SOAP api) vs the asynchronous (IPN) implementation.

Access Control List (ACL) implementation

We are just wrapping up our first implementation of access control lists (ACL) in CiviCRM (for v1.6). This gives us a framework for building fairly sophisticated access permissions into the data model and allowing fine granularity of who can view/edit/delete various sections of your contact database

Our first implementation replicates what we currently achieve with drupal's permissioning system. A CiviCRM admin can partition their contact database into multiple sections, and give view/edit access to different "roles". Integrating this into the core allows us to expose this functionality to our Joomla! users also. We do not use a matrix of checkboxes to present all potential combination, but rather present one "ACL" at a time. Obviously each method has its advantages/disadvantages, but the former is inherently non-scalable (as some of our friends in canada discovered when trying to partition their db into 300+ ridings), while the latter would probably need an external script to populate the database tables for significantly large sets of ACLs