As part of the ongoing website upgrade, the CiviCRM extension directory has received a few upgrades. More precisely, we changed a few configurations and workflows on the page listing CiviCRM extensions and how new releases (extension updates) are regularly added to that directory.
Starting with the interface changes on the directory, we adjusted the filters to remove some redundancies:
Previously it looked like the image below. Note: not a criticism! It's just to highlight the changes, described more in detail below.
By default, extensions are sorted by popularity (the number of installations according to pingback stats).
The "CMS Compatibility" filter has been removed since CiviCRM extensions and CMS-specific plugins/modules tend to follow very different workflows (how to install, where to get support, etc). However, CMS-specific plugins/modules still have their own separate listings (and are clearly advertised in the introduction of the extension directory).
As for the "Installable through CiviCRM's user interface?" filter, it now only proposes "Yes" or "No", instead of "Needs work" or "Closed". When extensions were first introduced, the civicrm.org website was used to track the review progress for extensions, but this is now done on Gitlab. We are still looking for a better label for the filter, or perhaps a way to explain it with a help popup.
The "status" filter was added so that we can more easily find, for example, "Work In Progress" extensions (most of which could previously only be found on Gitlab, more on that later). We also simplified the list of statuses (removed "Incomplete", "In Planning"), which were rarely used. A new status "Unsupported" was added, since previously we would completely remove unsupported extensions.
Finally, when viewing an extension, the download link for the latest releases is displayed directly on the same page, to avoid a further click:
Every click counts, right?
Power users may also have noticed that the "Git URL" field now links to the project development space associated with that project, which is where bug reports and feature requests are usually handled.
The next sections are more oriented towards developers. As a CiviCRM administrator or user, we would love to hear more from you about your experiences installing extensions. Feel free to skip to the comment section below!
Changes for extension developers
This is where we have done some rather big changes.
To add a new extension, go to "My account", then click on the "Participate" link, then click "Add an Extension":
If the option is not displayed, ask in the extensions channel of the CiviCRM chat ("Mattermost") for the "blog and extensions" role.
The "New Extension" form was greatly simplified. We wanted it to be as easy as creating a new project in Gitlab. When adding a new extension, there are now only 4 mandatory fields to fill in, and two optional fields:
There are also two fields with default values: the development status (usually Work-in-Progress at first) and the CMS compatibility (almost always "CiviCRM Native"). Many other fields were hidden because they are distracting during the initial extension creation. For example, extensions can have a summary and a description field. At first they are set to the same thing, and we can edit the content later on to update the description (and screenshot). The maintainers will be automatically set to the user that is creating the extension. Lots of small tweaks so that adding an extension is very quick, but we still kept most of the features in place.
Automatic Gitlab project creation
Creating a new extension will automatically create a project on CiviCRM's Gitlab, under the extensions group. More precisely, it will:
- Create the project, ex: lab.civicrm.org/extensions/myextension
- Add the logged-in user (e.g. you) as a maintainer of that extension on Gitlab
- Set the description of the extension on Gitlab using the description entered on civicrm.org
- Set the "Git URL" of the extension on civicrm.org to point to Gitlab. We can change this afterwards by editing the node (the field is initially hidden, displayed on further edits).
If you haven't browsed Gitlab, you should! We have been trying to encourage people to use this space as a way to share work in progress, encourage collaboration between organizations, and make it easier to end-users to participate. Most developers may have a Github.com account, but most of our users do not. CiviCRM's Gitlab is also where most of the community organization happens, bug reporting, etc. Eventually, we hope to rollout some better messaging around "how to get support for an extension".
Faster release publication
A long time ago, a nightly job was setup so that when extension authors tag a release in their git repository, civicrm.org will automatically pickup the new tag and create a new release automatically. This was great, because it meant that we did not have to login to civicrm.org and tediously copy-paste information already in Git, but the job only ran once a day, which meant that sometimes we finished work on something, only to get an email 24h later that the release had failed validation, for example. If we wanted to speed up the process, we had to login to civicrm.org. If you're wondering, the scanner checks over a hundred Git repositories, so we could not run it, for example, on an hourly basis.
One of the benefits from Gitlab, is that when a new project is created, a webhook is automatically added to the project. So when a tag is added to the git repository (creating a new release), the webhook automatically calls the "release scanner" so that it runs only for that specific project. Therefore the success/fail notification is immediate and the release is available online after a few minutes.
The daily scanner still runs, so that new release on projects hosted on Github are still supported. We will also add the webhook to existing projects on Gitlab, created before this feature was added.
Automatic releases opt-out, PHP/JS bundled libraries
Extensions increasingly depend on third-party PHP or JavaScript libraries. Some may commit them to their git repository, some may use Composer or NPM to manage the installation of those libraries. Since admins usually do not have the tools to install those dependencies, we usually make sure that the "zip" files of extension releases include those dependencies.
We are looking at various options, such as comex, see dev/core#793, but meanwhile it is possible to opt-out from automatic release publication by editing the node associated to an extension in the directory (on civicrm.org) and set the value of the "Release management" field to "Create new releases but keep unpublished". With this option, the new release is picked up by the new release scanner, but it allows the extension authors to update the "Zip file URL" to point to the URL of the zip file that includes all dependencies.
Previously, extension authors had to create the release manually on civicrm.org, which was easy because the scanner ran only nightly (they only had to remember to do it before sunrise). However, we have completely removed the possibility to create new releases on civicrm.org. This was done to remove duplication in the validation code, which was fairly complex (hat tip to the original authors, it must have been a ton of work to write initially and it still works well).
Having statistics on how many extensions bundle libraries will also help guide us with data and use-cases about the requirements for those extensions.
Feedback?
The tools to handle everything around CiviCRM extensions can be, at times, both surprisingly simple and complex. We would love to hear from your experiences, whether as an user, admin or developer. What would you like to improve?
Comments
Thanks, these are all very positive developments.