What the Xml? Where did the DAO go?

If you're a developer of any CiviCRM extensions or customizations, read on... (otherwise, hi, CiviCRM just got a bit more efficient! Now you can move on with your day. Thanks for stopping by).

Hey developers. Have you ever thought to yourself, "Gee I love editing xml files and then running a weird code generator every time I want to make any schema changes!" No? Neither has anyone else! Which is why I just deleted the entire xml/schema directory. Now before this causes a panic, take a moment to wipe up the coffee you just spewed all over your laptop, and I'll explain:

The CiviCRM schema (aka tables and fields) has always (since version 1) been declared in the form of xml files. They looked a bit like this:

<table>
  <class>Contact</class>
  <name>civicrm_contact</name>
  <title>Contact</title>
  <description>Individuals, organizations, households, etc.</description>
  <icon>fa-address-book-o</icon>
  <field>
    <name>id</name>
    <type>int unsigned</type>
    <title>Contact ID</title>

... except muuuch longer, but you get the idea. These files don't do anything at runtime. Instead they serve as a source for autogenerating several other files, most notably:

• Sql files used by the installer
• DAO files which contain all the data from the xml + some other boilerplate

So if you'd like to make any changes, even something simple like updating a description, you'd have to do these steps:

1. Make your changes to the xml file
2. Run GenCode.php
3. Commit your changes and make a PR
4. Groan in frustration when the updated GenCodeChecksum causes merge-conflicts and someone asks you to rebase your PR
5. Try to change another small thing like a field title, and now it's going to conflict with your first PR

As developers, we're always on the lookout for ways to streamline systems by making them less stupid, so the above workflow seemed extra-ripe for optimization. Combine that with the fact that the generated sql files were static and could not adapt to e.g. your chosen database collation, and we've got a system badly in need of rewriting. And now... we've done just that!

Today, the schema is defined by .entityType.php files, which look a bit like this:

return [
  'name' => 'Contact',
  'table' => 'civicrm_contact',
  'getInfo' => fn() => [
    'title' => ts('Contact'),
    'description' => ts('Individuals, organizations, households, etc.'),
    'icon' => 'fa-address-book-o',
  ],
  'getFields' => fn() => [
    'id' => [
      'title' => ts('Contact ID'),
      'sql_type' => 'int unsigned',

These files serve as both the canonical schema definition and are used at runtime. To make simple changes to e.g. icons or descriptions, just edit the file and that's it! If making changes to the sql table you'll still need to add an upgrader, but we've got some new helpers to streamline that process as well.

But what about GenCode.php? We don't need it anymore. DAO files? We don't really need those either, but for backward compatibility with extensions or custom code that relies on them, we've kept them around as empty stubs that extend a common base-class. Think about DAOs this way: historically, DAOs and BAOs were essentially a single class split into two files, the DAO was the autogenerated portion, and the BAO was handwritten. We've now eliminated the autogenerated stuff by moving it all to a common base-class that reads from the entityType.php file, so there's no longer a practical reason for the split. Someday in the future we might eliminate DAO files altogether (but not anytime soon).

How does this impact your extensions? Well, it doesn't have to. Extensions that use traditional autogenerated DAO files will be supported indefinitely. But if you're interested in modernizing your code, we've got a handy new civix convert-entity command which will transform your xml files into entityType.php. There's still some work being done to make this work on extensions that need to support older versions of CiviCRM. But we're nearly there. Ping us on https://chat.civicrm.org/civicrm/channels/extensions with any questions.