24 May, 2013
By totten
Filed under API, Architecture, Extensions

CiviCRM configuration is largely driven through the web interface and the database: if an administrator wants to add a new "report" or new "relationship type", he can accomplish this with a few clicks of the web interface. The new item is inserted into the database and immediately becomes live. This is great for web-based administration, but it's inconvenient for developers: if a developer writes a module or extension that registers something in the database, then he needs to write an installation routine to insert the item (and an uninstallation routine to delete the item). CiviCRM 4.2+ includes a better way: use the API and hook_civicrm_managed. This technique is already used in "civix" based extensions, but it also works with Drupal modules, Joomla plugins, etc.

Example Use-Case

As an example, suppose we are creating a module/extension called "fancyreports" which defines three new report classes. Each of these classes...

Read more
16 March, 2013
Filed under Architecture, CiviCRM

After my previous blog post, i have been working on making progress on working model w.r.t NoSQL and config. Starting with civicrm cache was a good idea. Keeping in mind NoSQL, new config system and what Eileen has already done with settings, here is what i planned and accomplished  :

Placed cache and cache-type (mongodb) settings file on disk with default values
Sticking to plans in previous post, and eileen's work on richer metadata, created two files in settings directory of civicrm. Richer metadata does help in creating web configuration forms without any extra coding for the config-items, and i was able to use it for building the new simple config UI.

deepak@bfc:civicrm$ ls settings

deepak@bfc:settings$ cat system.cache.php
return array(
  'class' => array(...
Read more
01 March, 2013
By totten
Filed under Architecture, CiviCRM

As of March 1st, the official source-code repository of CiviCRM has switched from Subversion to Github. Git and Github provide a number of advantages:

  • Popular among FOSS projects and web developers.
  • Free as in beer and (mostly) free as in speech.
  • Supports off-line development.
  • Supports lightweight branching, merging, and code-review.
  • Supports open teams – anyone can jump-in, make changes, and share changes.

For instructions on converting an existing installation (based on SVN or tarball) to a git checkout, see:


As you get to developing and submitting patches, please try out Github's "pull-request" process. This process allows you all the benefits of git and github (publishing, offline development, lightweight branching, online code...

Read more
10 February, 2013
Filed under Architecture, CiviCRM

As part of my course i have been doing research on what would it require plug an external storage engine into CiviCRM, and how other open source systems doing it. Answer lies in a better config system which allows doing it in a scalable pluggable manner. As i make progress i'll be showing more reasons to get excited and curious about building a better config system. Drupal 8 has spent a fair bit of time on configuration management to make things easier. And we shouldn't shy learning from them and others.     

Standardizing config with hierarchical storage

The second layer of drupal's proposed configuration system is to store config data from file system to database for fast access. That is storing all config data in one place. CiviCRM has already been doing this decently for quite some time with civicrm_settings table. Whats new things to learn here are -

1. Standardising access to config

So we...

Read more
20 November, 2012

Just created a quick ERD for CiviCase, and shared it on this page http://wiki.civicrm.org/confluence/display/CRMDOC42/CiviCRM+ERD+3.3.

It is version 3.3, so not the latest and greatest. But I am sure I will have to check the same ERD for version 4 at a near point in the future and update the ERD too. And I do not think there are major differences in the data model......

I have also attached the ERD to this blog post.

26 September, 2012

Extensions are a growing part of the CiviCRM way of doing things. We need to develop a process and toolset to facilitate getting them translated and making those translations easily installable. This post is intended to lay out some issues and a potential approach in order to generate discussion.

Here are some assumptions and suggestions I have:

  • Extension developers should be responsible for coding their extensions with ts(), and can be expected to do things like extract strings from their extension and upload somewhere for translation.
  • Transifex is the tool used for translating CiviCRM strings. It has a team for each language translation, who do the translations. This approach is working fairly well. Having extension strings presented for translation in the same space, and displaying percent translated for them, etc. would be ideal.
  • Translations are done in spurts and at somewhat unpredictable times, depending on the interest and...
Read more
22 July, 2012
By totten

Brian Shaughnessy (lcdweb), who has been working with the New York State Senate's CiviCRM project, recently raised the issue of simultaneous editing: What happens when two users simultaneously make changes to same contact record?  We've held a few discussions on IRC to examine the issue and draft some solutions.  We would appreciate further feedback and ideas on how to address the issue.

The Problem

As described by Brian:

The issue is that if you and I get into the same record around the same time, and you save the record first, when I save, it will overwrite your changes.  So let’s say you add a middle name, and I get in and add a birthdate.  The data is not in conflict, and both changes should be preserved, but they aren’t -- when you save the birthdate (where the edit form was loaded before I saved the middle name), the empty middle name overwrites the value I had saved. 


Read more
27 March, 2012
By totten
Filed under API, Architecture, Extensions

Many CiviCRM customizations have been packaged and distributed as Drupal modules. This can be desirable when a customization delves into both the CMS and CRM functionality, but -- when a customization focuses only on CiviCRM -- Drupal modules are a drag: they need to be patched for CMS upgrades (D6/D7) as well as CRM upgrades (Civi 2.x/Civi 3.x), and they don't work with CiviCRM's other CMS's (Joomla and WordPress).

Fortunately, dlobo has been making progress on support for native modules (built around the "CiviCRM Extension" system) in 4.1 and 4.2. An example module is here:

Of course, this still poses a challenge: a native module needs to use native tools for packaging code, adding new web pages, developing templates, etc. -- and all those tools come with a learning curve. To improve the learning curve, I've taken a...

Read more
26 March, 2012
By Eileen
Filed under Architecture, Sprints

Last week I wrote a blog about technical debt (comparing it to keeping a kitchen in order). I got a lot of feedback - most of it constructive. I'm going to resist belabouring the whole metaphor & limit this blog to a quick summary of some of the discussion that came out of it.


It's about making it better

There is such a thing as code without technical debt. It is found in textbooks & outputs the words 'hello world'. Code that is actually used has technical debt. Technical debt is a natural by-product of writing code without unlimited resources & time. So, while addressing technical debt is a good thing - saying that CiviCRM has technical debt is like saying CiviCRM is software.


On the flipside, it's important to note that CiviCRM is a remarkable piece of software & it represents a remarkable commitment by...

Read more
19 March, 2012
By Eileen
Filed under Architecture

Do you like to whinge about CiviCRM code? Have you sat through others doing having a rant? I've certainly done both. Being in the drupal world people often like to compare CiviCRM code with drupal & CiviCRM usually comes up a bit short. I think that's like comparing my kitchen with Bill Gate's kitchen. There are a few good reasons why my kitchen is not as nice as his. However, should I look at his kitchen (in a magazine) then I might glean a few good ideas that I could use in designing my own. (Copying the colour scheme would be in my budget :-))


To continue the metaphor, Bill Gates gets to rip his kitchen out & put a new one in whenever it's getting a bit tatty whereas I have to patch mine up & incrementally improve it. Unfortunately, each time I get a new whiz-bang gadget I find it harder & harder to get the cupboard door to shut. And then I can't find anything or open the door without everything falling out.


Read more