Testing achievements in 3.3 cycle and going ahead

When the dust settled after releasing 3.3 stable, we started planning for next steps in testing area. As looking a little bit back is always good for planning, we've took a look at what we managed to achieve during 3.3 release cycle. In short, we increased the number of tests from 792 to 920 (so nice, round number of 128 new tests) and added quite a few new test suites. Most of the work was put towards webtests this time (as opposed to unit tests), and those check the funct ionality quite extensively. While our unit tests are quite isolated and check for relatively small pieces of functionality, webtests are usually extended scenarios checking multiple pieces of user interface.

Among the things that we added to our webtests are (just the highlights): CiviPledge tests, Advanced Search tests, a lot of tests for adding different objects from contact record perspective, started tests for Administer CiviCRM area, also started testing "anonymous" processes, like contribution page or membership sign-up. During 3.3 cycle, we welcomed new component, CiviCampaign, and started a good practice of providing tests from the very beginning after new code appeared. From outside of webtesting area, some BAO tests were added (e.g. Discount and PaymentProcessor tests) and some new API tests. Detailed differences can be seen on 3.3 test results and 3.2 test results websites.

So now we're starting to plan for next release and setting up goals to achieve during next release. There is definitely increasing the coverage further, setting up some additional testing infrastructure and introducing testing related requirements for code contributions, but all that is under discussion at the moment. Want to join the discussion? You can see first issues already filed for 3.4 testing run, and you can also tell us about your ideas and join the discussion on Testing board on our forums. Let us know what you think!