CiviCRM Developer Hack-a-thon: Days 7-9

Time flies - it's been already a week since we've been together in New Zealand, enjoying face to face conversations and working together. My personal take on communication is that there is nothing like real time conversation involving two persons located in the same physical spot, so I must say I'm really enjoying this opportunity to hang out with Yashi, Dave, KJ & Lobo. Apart from work, we decided to have some fun too - which turned into two days full of adventure. First day, we went on a hike, with careful plan to get on top of Mt. Arthur. Weather wasn't really nice and people that we met on the way were mentioning something about snow and cold on higher parts of the trail, so we decided just to give it a try, go as far as we can and come back if weather gets worse. However, the trail turned out to be a "one way pipe" - the higher we were getting, the less interested we were in going back down the very steep, slippery path. Even the snow blizzard (doh! didn't someone mention this before?) didn't really force us to turn back - and finally, we arrived on our last legs to a little hut near the mountain top. We had a bit of rest there and took off on our way back using a different trail, which actually was very rewarding: a very well maintained forest path. The farther we went, the nicer the weather was. Wet, freezing (but happy!), we finally arrived at the parking lot and took off for home. Since this post wasn't supposed to be describing fun activities, but work stuff, second day report is going to be short: we went kayaking on the sea in Abel Tasman National Park, where we played with baby seals, had lunch on a beautiful island and made our hands sore from paddling. Definitely worth trying out. Coming back to original subject: after some UI "ajaxification", next thing on our plates was straightening out the situation with unit and functional tests in CiviCRM. We've tried a few approaches to this in the past, but it hadn't yet worked out quite like we wanted it to, and was never a very high priority to be sincere. However, before every release, we were putting quite a lot of manual effort into regression testing, so we decided to reduce this recurring investment (definitely not give it up!), and put some more attention into writing test cases that will be executed on regular basis and ensuring that we're not moving backwards with stability. First decision was choosing the testing framework. Since we had quite good experiences with SimpleTest, and one of platforms we work with is Drupal, we decided to start off with using the Drupal-SimpleTest module to begin with. After short initial training and brainstorming, we kicked off writing our first tests. The initial priorities are unit tests for BAOs and functional/browser tests for the most important parts of user interface - we managed to assemble some of that just to see what does it take to build them. Next thing will be putting behaviour tests on some bug reports from our issue tracker to facilitate work on fixing them and make sure they do not come back in new releases. Finally, we also started work on porting some of our previous tests (mainly API) into new framework. To keep the work running, we also came up with some rules that all the team members have to follow while writing the code, which will ensure that test coverage will be growing after this hack-a-thon ends, and also some plans for trying out new things to make our testing practices even more useful. Every team member working on tests will be also encouraged to take a closer look at the code that is being tested and refactor it if needed. This always works well - looking at your own code (or even better, at code written by somebody else!) after some time gives every programmer some ideas about what could have been done better or what became obsolete as the whole framework moved forward. To me personally (having some background in Java project management and programming) it is an extremely awesome feeling to see us moving with little steps towards the world where test coverage is 100% and code refactoring is flawless and painless. Not that this world really exists anywhere in the reality of software development. But it's nice to have dreams. :-D

Comments

About the existing tests (as on the svn), would it be possible to clarify how they are structured ? They seems to contain valuable examples on how to hack stuff on php.

About the browser testing, have you considered doing "real" browser testing (eg. selenium http://selenium.openqa.org/ ) ? it would also allow to test js code.

Swimming with baby seals ? Looks like civi got a new mascott ;)
X+

Hey Xavier,

Regarding tests clarifications - if you think this knowledge would be useful, would be worth to move this conversation to forums. Could you repost this question on developer discussion board (and also say a bit more about what you would like to know - all the tests are in tests/ directory in the repo, divided into API, BAO, browser/functional and issue specific groups.

As for browser testing - yes, we made an approach some time ago to use Selenium, but it didn't work out too well for us (for different reasons). We decided to focus on SimpleTest, since we already had some in-house experience with it, and also thought it will be more flexible for us to be able to run functional tests within the same environment that the system is running - it is easier to check database or runtime objects to verify if clicks are doing what they are supposed to. :-)

As for seals - yeah, that was pretty cool. :-)

Thx,
m