API in CiviCRM 4.1 - under-the hood

Although the API changes in 4.1 are not as obvious as took place in 3.4/4.0 there have been quite a few changes under the hood. This is a fairly detailed explanation of what's going on inside the API wrapper.

The way in which required fields & default values are set has been changed to be more consistent and to be self documenting.

For example if you look at the auto-generated (PHPDOC) documentation for Note API you will see that entity_id & note are marked as 'required' and that the default value for 'entity_table' is civicrm_contact. The default value for modified_date is 'now'  (the api accepts 'now' along with other relative dates for date fields).

This information, about required fields defaults set at the API level is available by querying the API - e.g.

civicrm_api('note','getfields',array('version' =>3, 'action' => 'create')); 

The api explorer also uses the getfields function to provide information about the API. The metadata is defined in the API using _spec functions which are called by the getfields function- e.g.

/*
 * Adjust Metadata for Create action
 *
 * The metadata is used for setting defaults, documentation & validation
 * @param array $params array or parameters determined by getfields
 */
function _civicrm_api3_note_create_spec(&$params){
  $params['entity_table']['api.default'] = "civicrm_contact";
  $params['modified_date']['api.default'] = "now";
  $params['note']['api.required'] =1;
  $params['entity_id']['api.required'] =1;
}

The Metadata functions can also be used to document additional fields - for example 'country' has been added to the bottom of address_create.

When you call an API (e.g. note_create) it follows this process:

1. It converts any aliases defined in the getfields function

   - e.g. it understands from getfields that pledge_contact_id is the same as contact_id. pledge_contact_id is the unique name defined in the schema whereas contact_id is the field name. After quite a lot of discussion we settled on the standard being the field name for create functions & the unique name for get functions. This was due to the requirement of unique names by the query object. However, in the past the two have been used somewhat randomly so the alias conversion should mean that both work. In the delete function '$entity_id' is converted to 'id'. The alias mechanism can also be used to state that 'if label is not set then use the value in name'.

2. If 'id' is not set then the defaults are retrieved (using getfields) and merged into the array being passed in.

   (in the past this was done ad hoc in the functions & didn't always take account of whether or not id was set (e.g. CRM-9763). It also interacted badly at times with the required checking).

3.  The resulting array is checked against the mandatory fields.

   If id is set then only version is mandatory.

4. The validate fields function is called.

   This performs transformation on any date fields using string-to-time. If the date is not valid it will throw an error.

5. At the end If the function fails due to a foreign constraint error (e.g. invalid contact_id) then the validate_fields function is called

   by the create_error function in error mode and it will check all the foreign constraints & provide more detailed information. This is intended to replace the numerous checks ad hoc in the various API for validity of fields. Doing it only on failure is more efficient. We intend in future to allow a 'strict' mode where you can invoke this extra error checking up-front while developing.

The validate_fields mechanism has a way to go but we hope to completely replace functions like _civicrm_api3_contact_check_params with api layer wrapping. The movement of so much of the api functionality to the API wrapper is why we do not support any access to any api functions from outside the API except via the wrapper. At the moment about 40% of the API crud functions are only 1 line of code. Our goal is that should reach about 80%. See survey api for an example 

Going forwards on our API simplification / refactoring we want to try to better resolve how the API interacts with the BAO

. In particular we want to standardise the BAO functions for create and delete so that we can always call the same function from the api (not sometimes add, sometimes create or on the other side sometimes del sometimes delete) and so that they always expect the same variable input.

The other area where there are often problems is that the various bits of processing & setting of defaults in Core is sometimes done in the form and sometimes in the BAO - this leads to the API not acting as people expect. It also limits the extent to which we can document the defaults. In some cases functionality that people expect the api to leverage is not there because the forms implement it. Or, in the case of sending contribution receipts, it is attached to the payment processor IPN rather than the contribution BAO. Unravelling these issues will take time.