The quest for performance improvements - 4th sprint

Last week we had a fourth sprint to improve CiviCRM performance at the socialist party. 

During this sprint we started with looking at why the screen for adding and editing memberships loaded slow. The issue reported was that it took some time before the end date field jumped from the right side of the screen to the middle of the screen. It turned out that as long as the field was displayed at the right side the screen was still loading.  Timing this gave a time of about 18 seconds before the screen was fully loaded.

We discovered a few causes:

• The javascript files which the browser had to download are over 2MB in size
• Every request in CiviCRM was also logged in Google Analytics and in Piwik.
• The PHP function getGroupsHierarchy in CRM/Contact/BAO/Group.php took around 900ms to execute (see the issue: https://issues.civicrm.org/jira/browse/CRM-19831)
• On that same screen the custom fields are retrieved with an AJAX request and in each request all contributions linked to the membership are also retrieved.

The getGroupsHierarchy function is performing slow because it needed to loop three times through all groups in the database in this case there are around 2.687 groups. In addition one of the loops uses the php function array_merge to merge the result of the loop. The array_merge function is known for performing slow when an array contains more than 40 elements.

After rewriting the function we reduced the execution time from 900ms to 20ms.  At https://github.com/civicrm/civicrm-core/pull/9633/files you can find the patch of the rewritten code.  In that patch we use for loops and not foreach as we discovered that a for loop is performing a lot faster than a foreach. If we replace the for to a foreach the total time of execution of that function will drop to 2.360 ms a factor of 100 slower.

We have also turned off Google Analytics and Piwik for CiviCRM pages as there is no need for this. Nobody really knew why it was turned on in the first place.

After fixing the getGroupsHierarchy function we reduced the total loading time in the browser from 18  seconds to 10 seconds. Which is still slow!

Read the previous blog posts:

• The quest for performance improvements
• The quest for performance improvements 2nd sprint
• The quest for performance improvements 3rd sprint