Testing CiviCRM in the real world ....

We've had 1.5 deployed and running for our partner Questbridge and their scholarship application program. This week is the final week for submitting applications, and as expected the servers and systems are under a fair amount of load. Lucky for us, we did go through a similar experience with the college prep application in april and have some idea of what to expect etc, but we are still learning a fair amount from this. For more details on this project check our Balance to distribute http requests among our application servers. Its a pretty decent load balancer and allows us to survive machine reboots etc, but was not impressed by its performance during heavy load. The load balancer seemed to be the bottleneck

From our past experience, we decided to use 2 database machines. One for the drupal database (which contains the session table, which we use quite heavily) and one for civicrm. This worked out very well for the College match application and the db servers were definitely not the bottleneck. At some point in the future, i'd like to use memcached to hold the sessions in memory rather than a db table. We also switched to using an innodb table for the sessions, since in the past the myisam session table frequently got corrupted. We clean the session table every day and get rid of old entries.

The architecture includes a front end load balancer spreading the requests among the application servers all of which access the common 2 backend database servers. In addition to CiviCRM v1.5, we also use the logintoboggan module and a custom locker module to display summary information to the students and recommenders.

Currently the application has created more than 75K contacts, 65K relationships, 128K locations. I'll probably come back and add more stuff to this post as i think about stuff. While all this was happening in production, we were busy hacking away on 1.6 features (acls, bulk update etc) that we need to roll into production for them early next week. lobo