Report from the CiviCRM testing sprint in San Francisco

The past two days a group of us gathered at the Mitchell Kapor Foundation offices in downtown San Francisco for the first CiviCRM Test Sprint. Some of the highlights of the event were:

• Introducing the concept of testing and our current framework for unit testing. CiviCRM uses PHPUnit for unit testing. We also use XDebug for code coverage. You can see the latest results of our testing here
• Improved the test coverage for our upcoming CiviCRM 3.1. Our goal before the final release of 3.1 is to have 80% coverage of the CiviCRM API and 50% coverage of the CRM classes. We are well on our way to meeting this goal.
• Improved the framework so we can make things easier and more efficient for future testers. Sasha worked on automating creation of CiviCRM objects via a generic method. We extended this to create all required Foreign Key Objects dynamically also along with creating multiple objects at the same time if needed. We complemented it with a deleteTestObject function that goes in and does the required cleanup. This makes testing a whole lot easier and more stable since we can create objects fairly easily without writing new code for every object.
• Experiment with adding Selenium based testing to the framework. So far our work has been primarily focused on unit testing the API and BAO's. We wanted to expand the test suite to include integration tests and work flow tests. PHPUnit has built in support for Selenium. We can now write Selenium tests using PHP (which is significantly better than Selenese). We hope to add quite a few tests using this feature for the 3.1 release.
• The speed of running the tests is becoming a bigger issue. Sasha is going to experiment with hosting the mysql database in memory. We are also thinking that it might be a useful exercise to port CiviCRM to SQLite for testing purposes
• We also had some great discussion on how to improve and use the power of the community. We do hope to get more folks involved at various other locations where CiviCRM has a good presence (New York, London, Boston, DC).

Thanx to Cedric Brown and MKF for being such awesome hosts. We truly do appreciate their support for CiviCRM and Open Source. Thanx to all the participants (Jim, Sasha, Walt, Jason, Mark, Andrew, Sarmeesha and Alice) for being so generous with their time and taking part in the sprint. Kudos to DharmaTech for organizing the sprint (and being such a HUGE driver in the test initiative)