After what seems to be an incredibly long time, we released CiviCRM v3.2 a few weeks ago to good reviews and not too many critical bug reports :). Now that is out of the way, we are starting to plan the feature set for v3.3

We've been fortunate to get a grant from Open Society Institute to help add some features important for Human Right Organizations, specifically for Front Line. We've been working with Front Line for a few years and this grant allows us to target their needs more effectively. Some of the features that are included in this grant are:

• Extending Address inheritance from Organization to an Individual (or from any contact to any other contact - TBD).
• Building Advanced Reports and Dashlets for Human Rights Organizations
• First version of Logging

While doing a deployment for a large organization, our good friends at Rayogram hit upon a pretty severe export scalability issue. The primary export was failing for approx 40K rows. They contacted us to see if we can figure out whats happening and why. They first assumed that the big issue was using a custom feature (merge same address / merge household address) and wanted us to look at it and potentially optimize and fix the issue.

In general dealing with memory issues in PHP are not something i look forward to (it comes a close second to debugging core dumps in PHP). Basically you cannot afford to leak a lot of memory in every iteration. For the export, if you leak 10K for every iteration, u will need 400M for a 40K export not including any of the memory used before export was called. On my initial tracing of the code we were leaking between 15K of memory for every iteration. On my local labtop, i could process approx 5K contacts before running out of memory :(

It took me some time to get my head around the code and figure out what the potential problems were. I first eliminated our good friend DB_DataObject (which has a tendency to store things in a static global variables). I also ensured that we were free'ing various objects as soon as possible. Spent a few hours on this and realized that i was not making any significant progress.

We've been hard at work implementing an online set of forms to collect family information for the school module. This is one of our final projects for the year and eliminates the tedious summer ritual for the admin staff of sending paper forms to the 200 school families and for the parent to fill out the same information every year. This also saves the admin staff from entering that same information into the SIS

We've built this work on the parent portal that we launched late last year. Parents can update information on themselves and their children anytime via their drupal account. The form is composed of 5 sub-forms:

• Household Information: Name, Email, Phone and Address of the household. We currently support 2 household and 4 contacts. These are stored as CiviCRM contacts with a relationship link of type Parent / Child to the student. We created a custom group to store which household a parent belonged to. We did not use CiviCRM's household functionality.
  

At CiviCon SF 2010, we met the folks from Emotive LLC (chang and matthew b) who expressed an interest in helping scale CiviCRM and in specific help with building an import system that can handle large data sets.

I started discussing this with chang on the forums and IRC. You can read our forum thread here: http://forum.civicrm.org/index.php/topic,13630.0.html

We decided on implementing an Import API and started work on this on an svn branch. We also decided to incorporate unit tests with this feature at a very early stage. The API is composed of five functions:

The past few weeks have been just an amazing ride through the CiviCRM universe. We've had some excellent training sessions, an awesome CiviCon and then the sprints which produced a nice new book and some major improvements to our translation process. We've also been spending a lot of time thinking about what we want to accomplish with CiviCRM 4.0. So here are some of our thoughts, note that many of them are speculative and if something is really important to you or your org, consider making that part of YOUR OWN personal battleplan and contribute a design and code to make it happen.

My top 4 guidelines with regard to 4.0 are:

• No new features. 4.0 is a restructuring and refactoring of part of the code base. Our objective is to make the code base more solid and not focus on adding functionality to the application.
• Improved extensibility. We've heard the message loud and clear. We need to continue making it easier for developers to extend and customize CiviCRM. More documentation, More examples, More API's
• Improved testability. We need to continue increasing the test coverage of the code base. We are continuing with a strong push on this with 3.2 and have introduced Selenium into the mix. We hope to get the community involved in improving our test coverage in a big way
• Switch to a modern framework. The current CiviCRM code base is based on PEAR which is old and no longer maintained. We will upgrade the core libraries to a more modern object oriented framework, with a strong userbase / documentation / testability. To ease the transition we will stay with our current templating system but we will upgrade Smarty to v3.0.

We've received a small amount of funding to help start our work on Canvassing and GOTV. We are still looking for sponsors, if this work is important to you, please consider supporting the project by making a contribution. We are also coordinating with Xavier on his work with CiviPetition.

We revised the implementation details a few days ago to allow the canvassing module to use more of the civicrm core objects rather than creating new objects/tables. We also generalized the structure to enable CiviPetition to share the same objects. The key idea was to treat canvassing and GOTV as Survey(s). A petition should also be treated as a survey and hence reuse the same code structure. Fields specific to Canvassing or Petition can be added via CiviCRM's custom group mechanism.

The past 8 days have been an amazing period for the CiviCRM community and core team members. Its been incredibly intense, extremely fulfilling and mind-blowing. A huge thank you and tip of the hat to the members of the community who participated in the event and came together from various parts of the world (asia, europe, north america) to push the project to greater heights, from a usability, documentation and localization viewpoint.

Thank you to Jimmy H, Erik B, Goran G, Matheiu L, Mathieu P for working on improving CiviCRM's localization and internationalization features. Thank you to Michael M, Xavier D, Adam H, Sarah G, Mari T, Alice G, Jack A, Josue G, Kyle J for burning the midnight oil to update, improve and extend the CiviCRM: A comprehensive guide. Thank you to OSI and our program officer: Janet Haven, Chintu Gudiya Foundation, Yellow Dog Foundation and Mitch Kapor Foundation for supporting our work and this sprint.

So a brief recap on some of the amazing events and highlights of the past 8 days:

We had another super awesome day at DrupalCon SF 2010 today. Was great to watch and hear Cricket from Jane Goodall Institude proclaim: "I love CiviCRM!" to the audience :) To keep the momentum going, we've scheduled two BoF sessions for today. Please check the BoF notice board for the rooms.

We had a great day at DrupalCon SF 2010 today, especially in our BoF sessions :) To keep the momentum going, we are planning to have a few more BoF sessions:

• Canvassing and GOTV (Get Out The Vote) functionality in CiviCRM. We'd like to extend the current CiviEngage module and incorporate the work done by Will B on Voter Canvassing. We'd also like to incorporate some of the ideas floating around with regard to CiviPetition and build campaign functionality in CiviCRM. This BoF will be scheduled for Tuesday afternoon, 3:00 pm. I'll update the blog post with room details soon.
• CiviSchool module for K-12. We've built and deployed a school module to manage a K-8 school interaction with staff / parents / students. We'd like to extend this module, deploy it in more schools and integrate it with open source and commercial SIS packages (Student Information Systems). We'd like to use this BoF to explore how more folks can be involved in this project. This BoF will be scheduled for Wednesday morning, 9:30 am.

The FSF just put out a press release recommending CiviCRM for non-profits. We've been working with the FSF very closely over the past 4 months to ensure that CiviCRM is licensing compliant (i.e. all the packages that we use and ship within CiviCRM are all compatible). This is an important milestone and a great endorsement for the project.

Some snippets from the FSF press release