Optimizing CiviCRM ...

I've spent a fair amount of time in the past two weeks figuring out how we could optimize and improve CiviCRM. Its been an interesting few days and I suspect will become more interesting over the next few days as we start implementing a few things. All this is in preparation for doing a pretty major load test for the Branner project. I did read a few articles on the web about optimizing LAMP. The article was fairly helpful, but I disagree quite a bit with some of the conclusions. I think there is also a big difference between building something for one site vs building an open source product. I do agree that PEAR DB, QuickForm and Smarty add a lot of overhead to CiviCRM. On the other hand, I also think we get an incredible increase in productivity because of these components. If I was building stuff for Yahoo or Mozilla, i would definitely skip those components and make them more lightweight. There is also work started on HTML_QuickForm2, with the goal being to make it more lightweight and faster. I suspect we will participate and help with it, since it is such an important component for CiviCRM. I installed XDebug to profile various CiviCRM pages. To make sense of the output, i needed to install KCacheGrind. My labtop is a MacBook Pro, so I had to install Fink. To install KCacheGrind via Fink, I followed the instructions from this article. It took a long long time (8-12 hours or so), but in the end I had a working KCacheGrind and could visualize the profiles generated by XDebug. I'm analyzing various CiviCRM pages with this currently and changes are being incorporated into CiviCRM v1.7. Found a couple of places in HTML_QuickForm that we could optimize quite nicely. Also in v1.7, we'll be splitting more forms into smaller sections and load them in dynamically, which should make things significantly faster, similar to the Advanced Search form in CiviCRM v1.6. Hopefully Dojo will fix the TitlePane widget and allow lazy loading of content in Dojo v0.5 My current focus is on installing and using Memcached using the PHP Memcache Pecl extension. For folks using MacOSX, I installed memcached using Fink and not MacPorts. I could not get the MacPorts version to work properly. You should also be aware of this issues with MacOSX and memcached. There are quite a few things that can be stored in memcache. Specifically, I'm thinking of

• The config object which is mainly static but used on all page loads
• All the pseudo constants in CiviCRM. Things like states, countries and various other option/value pairs, which rarely change
• Definitions of Custom Groups, Custom Fields, Contribution Pages

I suspect doing the first two will increase performance by a decent percentage and reduce the number of db queries / page quite a bit. Detailed results and graphs as I make more progress