A common question on the mailing list has been about CiviCRM search, scalability and speed. Currently basic/advanced search has been designed to try to get as many matches as possible from your contact data. However trying to get as many matches as possible results in some major inefficiencies which does not work for data sets of reasonable size
Currently the default search does the following:
1. For any value typed in the name field, it searches the civicrm_contact table and the civicrm_email table. It connects the two tables using two LEFT JOIN's via the civicrm_location table. LEFT JOIN's are not very efficient sql operators
CiviCRM Forms and Wizards (multi-page forms) are based on PEAR's HTML_QuickForm_Controller. (QFC). QFC in turn is based on HTML_QuickForm (QF). It was easier for us to model a single form as a one page wizard, and hence all CiviCRM forms are instances of QFC
The basic Form object is CRM_Core_Form. All forms are derived from this class. Each derived class is expected to implement the following functions
- function preProcess( ): This function is called before a form is built. All objects needed to build the form should be built in this function.
- function buildQuickForm( ): This function builds the form. There are some helper functions in CRM_Core_Form to build some elements (Radio, Select, Yes/No etc). Classes typically call a mixture of these helper functions and QF functions directly
This is what worked for me using Control Panel, FTP, File Manager and PHPMyAdmin.
Before you begin make sure you have a working Drupal 4.7 already installed. Have the name of the database, database user and password handy as well as the name of your mysql host. If you can’t find the name of your mysql host anywhere on your host’s site then your mysql host is probably localhost. Save yourself the aggravation and get all these things written down beside you before you even start installing CiviCRM.
1. Use FTP to upload the zip/tar CiviCRM directly into the drupal/modules folder.
2. Go to your Control Panel or whatever method you use to create new databases and create a new database called civicrm. You will already have a database user name and password from your Drupal installation. Go to your list of database users and add a user to the new civicrm database you just created. Make sure to give the user full privileges.
We now move onto the more interesting stuff of what really happens when a request is made and the objects that are responsible for building the response. The top level CiviCRM objects are:
- Form (CRM_Core_Form): CiviCRM forms are based on HTML_QuickForm. For drupal folks, this is more of an object representation of the much loved form api
- Wizard (CRM_Core_Controller): A set of CiviCRM Forms that collectively make up an action (Import Wizard, Mailing Wizard etc). For simplicty sake a one page Form is represented as a Wizard (CRM_Core_Controller_Simple)
CiviCRM uses the PEAR package DB_DataObject to access the mysql database. Doing so reduces the amount of sql we need to write to fetch / store values for a single table. For queries involving more than one table, we write them manually since since it was fairly painful to try to force complicated queries using DB_DataObject (we did use DB_DO for multiple table queries when we wrote EmailNow, in the end we felt the hoops we had to jump through was not worth the effort and hence the current scheme)
The basic PHP classes that represent a table are called DAO's (Data Access Objects). These are generated automatically by (xml/GenCode.php) which also generates the sql structure for that specific table. The xml files representing the schema is stored under CIVCRM_ROOT/xml/schema. The sql file generated (civicrm_41.mysql and civicrm_40.mysql) are stored under CIVICRM_ROOT/sql (note that the generated files are not present in the svn repository). The DAO files are stored under CIVICRM_ROOT/CRM/Core/DAO (and under appropriate directories based on what table it is, this is configured in the xml representation.
here is a tentative schedule for CiviCRM v1.6. The feature list for v1.6 is here. As all good open source software projects, the below schedule is subject to change :)
- Nov 10: Code Freeze, all major features implemented
- Nov 10 - Nov 17: Internal QA
- Nov 17: v1.6 alpha released. Sandbox site up and running for community testing
- Nov 17 - Nov 29: Community testing on CiviCRM sandbox
- Nov 29: v1.6 beta released
- Mid December: v1.6 released
Some of the important directories under CiviCRM svn root are:
- CRM: Most of the CiviCRM specific source code is stored in the directory. This directory is further sub-divided into other directories based on functionality (Core, Utils, Contact, Contribute, Mailing etc).
- xml: CiviCRM schema and the base Data access objects (DAO) are automatically generated from a simplified xml scheme which contains a fair amount of sql and type information. This makes it relatively easy for us to change the sql code, figure out what changes were made in a version and add meta information to a table within PHP.
We made excellent progress on our 1.6 queue last week. All but one of the issues targeted have been moved into resolved status. The queue is now down to 12 items (from 23 last week). Some team members will be moving into active QA mode this week - doing regression testing on the 1.6 resolved queue to verify that things haven't gotten broken since they were resolved earlier in the release cycle.
For the coming week we'll try to get the queue down to 5 issues or less... and finish the following larger issues:
Over the next few weeks, I will blog about the architecture of CiviCRM and some basic principles that we follow in the design/coding phase. In this series I’ll also talk about how we integrate with Drupal/Joomla and the scalability/performance/memory bottlenecks in CiviCRM.
CiviCRM is built on top of some incredibly powerful open source toolkits. They include:
- PHP, we use PHP 5.1 for development and will probably switch to PHP 5.2 in the next few weeks as our primary development platform. We restrict ourselves with the features that we use in PHP 5 to enable an auto-translation of the code to PHP 4. I suspect we will drop support for PHP4 around the time we release CiviCRM 2.0 :)