Upcoming token changes and a request for help

Published
2021-10-15 19:22
Written by

In the upcoming 5.43 release significant changes have been made to the code that handles token rendering. The primary goal was to fix technical debt issues that were tying us in knots and holding us back. However some new usability features are also included in 5.43

  • Tokens for participant details now available from scheduled reminders and the send pdf letter and email tasks from search results. Event tokens are also available from pdf & email tasks (they were already available in scheduled reminders)
  • Tokens for membership details now available in the send email task
  • Domain tokens (e.g {domain.name}) are available in more places.
  • A new ‘Current time/date’ token is available. 
  • Date tokens can be rendered in a variety of formats - e.g to show the year of birth you could use {contact.birth_date|crmDate:"Year"} 
  • Tokens for money will have a greater degree of localisation

As mentioned the primary goal for the changes was to reduce technical debt and these changes will simplify the way in which extensions inject tokens into core. Prior to these changes extensions had to do hugely convoluted things to interact with the token system and the tokens that were available in CiviCRM varied depending on where you were using them from and in some cases (like {contact.gender_id}) gave different output from different places. 

However, in order to reconcile these competing code paths and practices we’ve had to make some changes to how some tokens render and some interactions with extensions. We’ve tried hard to use upgrade scripts and legacy handlers to minimise any impact. But now we are asking for YOUR HELP to test 5.43 while it is still in the pre-release ‘rc’ state to find any regressions or changes that won’t play well with your extensions and customisations. You can download the latest rc copy here.  Please try it out on your staging and development sites. 

We are particularly keen to have people do testing with varied language configuration to make sure nothing has regressed in terms of being parsed through translation.

The main code areas affected are

  • Rendering tokens as part of tasks from searches (email & pdf)
  • Rendering tokens in scheduled reminders
  • Rendering tokens from civimail with flexmailer installed
  • Rendering tokens in workflow templates - although these generally use a limited range of tokens at the moment

The following tokens are still being resolved through legacy and / or mishmash records

  • Greeting tokens (e.g email greeting) - we are still hoping to fix this in time for 5.44
  • Tokens rendered through the ‘print label’ task
  • CiviMail tokens when flexmailer is not being used. We are deprecating not using flexmailer although we have not determined what (if anything) we will do about this difference in behaviour between the two

I want to make particular mention of 2 people who have done a tonne of work to test and validate the work that has been done - Dave D (demeritcowboy) and Richard van Oosterhout (magnolia61).