API and the Art of Installation

CiviCRM configuration is largely driven through the web interface and the database: if an administrator wants to add a new "report" or new "relationship type", he can accomplish this with a few clicks of the web interface. The new item is inserted into the database and immediately becomes live. This is great for web-based administration, but it's inconvenient for developers: if a developer writes a module or extension that registers something in the database, then he needs to write an installation routine to insert the item (and an uninstallation routine to delete the item). CiviCRM 4.2+ includes a better way: use the API and hook_civicrm_managed. This technique is already used in "civix" based extensions, but it also works with Drupal modules, Joomla plugins, etc.

Example Use-Case

As an example, suppose we are creating a module/extension called "fancyreports" which defines three new report classes. Each of these classes must be registered in CiviCRM's database. (Specifically, new rows must be inserted into table "civicrm_option_value".)

Cumbersome Approach: hook_civicrm_install, etc

One's first impulse is to write an installation routine which inserts the new reports in the database. For example:

function fancyreports_civicrm_install() {
  $result = civicrm_api('ReportTemplate', 'create', array(
    'version' => 3,
    'label' => 'Fancy membership report',
    'description' => 'Membership report with some cool doodads',
    'class_name' => 'CRM_Fancyreport_Form_Report_Membership',
    'report_url' => 'fancy/member',
    'component' => 'CiviMember',
  ));
  if ($result['is_error']) {
    CRM_Core_Session:setStatus(ts('Failed to register report'));
  }
  $result = civicrm_api('ReportTemplate', 'create', array(
    'version' => 3,
    'label' => 'Fancy contribution report',
    'description' => 'Contribution report with some cool doodads',
    'class_name' => 'CRM_Fancyreport_Form_Report_Contribution',
    'report_url' => 'fancy/contribute',
    'component' => 'CiviContribute',
  ));
  if ($result['is_error']) {
    CRM_Core_Session:setStatus(ts('Failed to register report'));
  }
  $result = civicrm_api('ReportTemplate', 'create', array(
    'version' => 3,
    'label' => 'Fancy event report',
    'description' => 'Event report with some cool doodads',
    'class_name' => 'CRM_Fancyreport_Form_Report_Event',
    'report_url' => 'fancy/event',
    'component' => 'CiviEvent',
  ));
  if ($result['is_error']) {
    CRM_Core_Session:setStatus(ts('Failed to register report'));
  }
}

This is serviceable, but (by itself) leaves some bugs. When an administrator uninstalls the extension, the reports are still listed in the database. So we need to define an uninstallation process, e.g.:

function fancyreports_civicrm_uninstall() {
  $getResult = civicrm_api('ReportTemplate', 'getsingle', array(
    'version' => 3,
    'name' => 'CRM_Fancyreport_Form_Report_Membership',
  ));
  if ($getResult['id']) {
    $delResult = civicrm_api('ReportTemplate', 'delete', array(
      'version' => 3,
      'id' => $getResult['id'],
    ));;
    if ($delResult['is_error']) {
      CRM_Core_Session:setStatus(ts('Failed to register report'));
    }
  }
}

That gets us closer, but still there are issues:

• The uninstall needs to run for all three reports.
• We've handled "hook_civicrm_install" and "hook_civicrm_uninstall" but not "hook_civicrm_enable" or "hook_civicrm_disable". We should implement those hooks, too -- when the extension is disabled, we should flag the reports as inactive; when re-enabled, we should flag the reports as active again.
• These four hooks -- "hook_civicrm_install", "hook_civicrm_uninstall", "hook_civicrm_enable", and "hook_civicrm_disable" -- only work with CiviCRM native extensions. With Drupal modules or Joomla plugins, one must identify a different (but comparable) hook for that platform.
• If we release an upgraded version of the module/extension, the upgrade might define additional reports, might remove defunct reports, or might rename existing reports. All of these require extra upgrade logic.
• As we write the install/upgrade logic and fine-tune the report, we'll need to test the install/upgrade logic -- usually, that means repeatedly and manually installing/uninstalling the extension.
• If an error arises during installation/uninstallation, then the administrator is left with a database in an inconsistent state and no simple way to recover.

Joomla and Drupal Approaches

The difficulty of using hook_civicrm_install arises because it's a procedural approach to managing the full lifecycle. Many platforms adopt a declarative approach to registering items -- an author declares what should be in the system, and the system takes care of any needed installation (or uninstallation or deactivation re-activation)  steps. For example, in Joomla, an extension author declares permissions by creating an XML file called "access.xml". In Drupal, a module author declares permissions by implementing hook_permission and setting it to return a certain list of permissions.

Unfortunately, adopting a Joomla or Drupal approach within the CiviCRM 4.x series would present a challenge -- inertia. We already have a long list of items which are managed as database resources -- report-templates, report-instances, payment-processor-types, relationship-types, activity-types, ad nauseum. Adapting each would require a lot of incremental work (coding, documentation, tests, etc) because our code assumes that these records are in the database -- aside from "stored in the database", there's no single standard, format, or entry-point that applies to all these interesting resources.

Managed Entity Approach: hook_civicrm_managed

Correction: there is a single standard that we've adopted, documented, and tested for most interesting resources -- APIv3! But APIv3 is clearly procedural. Can we use APIv3 declaratively? As of CiviCRM 4.2+, yes.

The solution is hook_civicrm_managed. Whenever the CiviCRM cache is cleared, CiviCRM will invoke the hook and perform reconciliation:

• Invoke this hook (to build a list of entities which should be in the database)
• Insert new entities in the database (using the API)
• Update existing entities in the database (using the API)
• Delete stale/unnecessary entities from the database (using the API)
• Flag entities as active or inactive depending on their module's status (using the API and is_active property)

For our "fancyreports" example, one would declare:

function fancyreports_civicrm_managed(&$entities) {
  $entities[] = array(
    'module' => 'com.example.fancyreports',
    'name' => 'fancymember',
    'entity' => 'ReportTemplate',
    'params' => array(
      'version' => 3,
      'label' => 'Fancy membership report',
      'description' => 'Membership report with some cool doodads',
      'class_name' => 'CRM_Fancyreport_Form_Report_Membership',
      'report_url' => 'fancy/member',
      'component' => 'CiviMember',
    ),
  );
  $entities[] = array(
    'module' => 'com.example.fancyreports',
    'name' => 'fancycontribute',
    'entity' => 'ReportTemplate',
    'params' => array(
      'version' => 3,
      'label' => 'Fancy contribution report',
      'description' => 'Contribution report with some cool doodads',
      'class_name' => 'CRM_Fancyreport_Form_Report_Contribution',
      'report_url' => 'fancy/contribute',
      'component' => 'CiviContribute',
    ),
  );
  $entities[] = array(
    'module' => 'com.example.fancyreports',
    'name' => 'fancyevent',
    'entity' => 'ReportTemplate',
    'params' => array(
      'version' => 3,
      'label' => 'Fancy event report',
      'description' => 'Event report with some cool doodads',
      'class_name' => 'CRM_Fancyreport_Form_Report_Event',
      'report_url' => 'fancy/event',
      'component' => 'CiviEvent',
    ),
  );
}

The entity and params are exactly the same as normal APIv3. There are only two additions (which are needed for reconciliation):

• module: The fully-qualified name of the module which declares the entity. (If this is a CiviCRM module, then the name looks like "org.example.fancyreports". If this is a Drupal module, then the name looks like "drupal.fancyreports".)
• name: A locally-unique name for the entity.

Managed Entity Approach: *.mgd.php

When the civix code-generator creates a new module, it provides this glue code as part of hook_civicrm_managed:

function _fancyreports_civix_civicrm_managed(&$entities) {
  $mgdFiles = _fancyreports_civix_find_files(__DIR__, '*.mgd.php');
  foreach ($mgdFiles as $file) {
    $es = include $file;
    foreach ($es as $e) {
      if (empty($e['module'])) {
        $e['module'] = 'org.example.fancyreports';
      }
      $entities[] = $e;
    }
  }
}

Any files with the extension "*.mgd.php" will be automatically added to hook_civicrm_managed. Consequently, you don't need to put all declarations in one file -- you can find a more convenient place to put them. For example, when creating a report class, it's handy to put the *.php and the *.mgd.php next to each other:

• CRM/Extendedreport/Form/Report/Price/Lineitem.php - The source code of the report class
• CRM/Extendedreport/Form/Report/Price/Lineitem.mgd.php - The metadata for the report class

Limitations

The "Managed Entity" feature is convenient for several use-cases but can't do everything -- sometimes it may be better to use the more flexible (but more cumbersome) install/uninstall hooks. A few key limitations with managed entities:

• The reconciliation process is not designed for high-performance -- if there were a couple hundred entities, then reconciliation could be slow. However, reconciliation only runs rarely -- when CiviCRM is upgraded, when new modules are installed, when an administrator explicitly flushes the cache, etc. It shouldn't happen on a day-to-day basis.
• It's only been used with independent entities. If you need to create a sequence of related entities (e.g. create a UFGroup and then create a UFField inside that group), then it may be hard to link the related entities. (This is generally untried, though. API chaining might help, but again --I haven't tried.)
• For upgrades, it handles simple replacements but not complex migrations. For example, suppose v1.0 of a module defines two reports (Reports A and B), but v2.0 uses a generalized report to replace both of them (Report C). If the module author simply replaces reports A+B with C, there may be some stale references to A+B (e.g. old hyperlinks or report-instances) which should be transitioned.

[EDITED 27-May-2013: Add more hyperlinks and more limitations]