CiviCRM Architecture - CiviCRM directory structure

Make sure you read the Introduction Chapter of this series for a better understanding of the below. Developers might be interested in installing CiviCRM from our svn repository

Some of the important directories under CiviCRM svn root are:

  • CRM: Most of the CiviCRM specific source code is stored in the directory. This directory is further sub-divided into other directories based on functionality (Core, Utils, Contact, Contribute, Mailing etc).
  • xml: CiviCRM schema and the base Data access objects (DAO) are automatically generated from a simplified xml scheme which contains a fair amount of sql and type information. This makes it relatively easy for us to change the sql code, figure out what changes were made in a version and add meta information to a table within PHP.

Team Goals for the Week of November 6

We made excellent progress on our 1.6 queue last week. All but one of the issues targeted have been moved into resolved status. The queue is now down to 12 items (from 23 last week). Some team members will be moving into active QA mode this week - doing regression testing on the 1.6 resolved queue to verify that things haven't gotten broken since they were resolved earlier in the release cycle.

For the coming week we'll try to get the queue down to 5 issues or less... and finish the following larger issues:

CiviCRM Architecture - Introduction

Over the next few weeks, I will blog about the architecture of CiviCRM and some basic principles that we follow in the design/coding phase. In this series I’ll also talk about how we integrate with Drupal/Joomla and the scalability/performance/memory bottlenecks in CiviCRM.

CiviCRM is built on top of some incredibly powerful open source toolkits. They include:

  • PHP, we use PHP 5.1 for development and will probably switch to PHP 5.2 in the next few weeks as our primary development platform. We restrict ourselves with the features that we use in PHP 5 to enable an auto-translation of the code to PHP 4. I suspect we will drop support for PHP4 around the time we release CiviCRM 2.0 :)

CiviCRM in India...

It's been almost 2 wonderful years that I have been working on CiviCRM project, since we started development in India. From then things have changed. We started with a single file, (I think it was CRUD.php) which no longer exists :). What excites me the most about CiviCRM is the challenges that we face in implementing new features and the opportunity to work on advanced technologies. The scope is simply unlimited...

CiviCRM is not just another software, it is driven by the needs of the community, which makes this project very special to me. Since this software has also simultaneously been developed in US and Poland, makes it global in nature. This project has been crafted meticulously, so that there is a lot of scope for extensibility and it can be tailor-made to suit the requirements of host of organizations and non-profits. A lot of credit needs be given to the pioneers of this project, who are harbingers of the today's IT Industry.

Experiences with ajax and dojo ...

So this week, our focus has been on getting some cool new features into CiviCRM v1.6. We figured that we'd take some baby steps into the wide world of web 2.0 and ajax and use Dojo as our toolkit of choice.

Yes we do know that Drupal chose JQuery and Joomla v1.5 has a combination of a few different ajax utilities. However, we did like the direction Dojo was moving in with a fairly nice large community along with some of the big names in the industry (IBM, Sun etc). We also did meet Alex Russell, one of the lead developers/architects and its always a good practice to use tools from folks you know :) (since u can then pester them if you need help at some stage)

Team Goals for the Week of October 30

We made decent progress on our 1.6 queue last week. Some of the issues we targeted for resolution are still in progress, but the queue is now down to 23 items (from 39 last week).

For the coming week we'll try to get the queue down to around 10 issues and resolve the following larger issues:

  • Add support for free membership signup. This should also allow for contribution pages which solicit in-kind (non-monetary) contributions.
  • Finish work on Access Control (ACL) functionality for group permissioning (CRM-1308).

PayPal Web Standards and IPN added to CiviContribute

Another feature that we worked on this week was integrating the "standard" version of PayPal as an additional payment option for CiviContribute. We waited for quite a few months for PayPal (or Google) to release a nice web 2.0 api for subscriptions / recurring transactions, but that did not seem to be happening. So we took the plunge and decided to implement PayPal Web Standards as the first step towards implementing PayPal Subscriptions (basically recurring donations)

PayPal has done an incredible job of documenting their API's and making the integration fairly easy. It helped that the Drupal e-commerce package has this feature, so we could peek into their code base and figure out how to do a few things. Getting a system up and working using the sandbox makes development so much easier, since the sandbox behaves identically to the live system. I was pleasantly surprised to see that we did not have to write a lot of code for this case, but primarily had to rearrange and cleanup the existing code to handle the synchronous (SOAP api) vs the asynchronous (IPN) implementation.

Access Control List (ACL) implementation

We are just wrapping up our first implementation of access control lists (ACL) in CiviCRM (for v1.6). This gives us a framework for building fairly sophisticated access permissions into the data model and allowing fine granularity of who can view/edit/delete various sections of your contact database

Our first implementation replicates what we currently achieve with drupal's permissioning system. A CiviCRM admin can partition their contact database into multiple sections, and give view/edit access to different "roles". Integrating this into the core allows us to expose this functionality to our Joomla! users also. We do not use a matrix of checkboxes to present all potential combination, but rather present one "ACL" at a time. Obviously each method has its advantages/disadvantages, but the former is inherently non-scalable (as some of our friends in canada discovered when trying to partition their db into 300+ ridings), while the latter would probably need an external script to populate the database tables for significantly large sets of ACLs

Team Goals for the Week of October 23

We have been thinking it would be useful to share our team goals with the community on a regular basis - so folks have a clearer idea of what we're working on (without wading through the details in the Issue Tracker).

We currently have 39 open issues on the 1.6 queue - and we need to knock-off at least 10 next week in order to be on-track for 1.6 beta. Here's the issues we'd definitely like to resolve:

  • Link custom fields to specific types of activities, relationships, etc. (CRM-1177).

Exporting CiviCRM data for deeper analysis ..

We've been working on the Branner Project and writing code to help analyze the applications and process them in a suitable manner. The application collects a fair amount of data across multiple pages and as such is not very conducive to reading on the web.

We wanted to analyze the data in Excel. The database is relational and hence hierarchical. CSV is not a great format for this purpose. So we wrote a CiviCRM -> XML converter and got all the data we needed in a nice easy to understand / parse xml. We then used a PEAR package (Spreadsheet_Excel_Writer) on windows to convert the xml to a set of excel worksheets. As part of this package we also included the contact_id, which allowed us to import results back into CiviCRM using the contact_id