12 January, 2007
By lobo
Filed under v1.7, CiviCRM

NTEN (Non-profit Technology Enterprise Network) recently posed a series of questions to software vendors serving the non-profit community. Here's a draft of our response. We will be revising this entry over the next few days. Would be great for the community to comment and/or contribute to our response. The questions are in bold type.

Does your product have APIs that allow other applications to access data from your application?

Yes. CiviCRM includes a substantial set of API's which are being used to exchange data and integrate CiviCRM with other applications. We also use SOAP API's internally to integrate the CiviMail high-capacity broadcast email component with the CiviCRM Core.

Do you have features that call APIs of other applications?

Yes. We use this fairly extensively to add more functionality to CiviCRM. A few ones that come to mind are...

Read more
07 January, 2007
By lobo
Filed under v1.7, CiviCRM

Over the past few releases we've been having quite a few people use and exercise the api. We are getting to the stage where people want to use the api to integrate with other applications (e.g. integration with Thunderbird).

Our current API has a few issues. Firstly it was designed and implemented very early on (CiviCRM v1.0) and we've learned quite a bit since then. We decided to use PHP class objects in the API, which in retrospect was a mistake. Secondly, some of the api calls (crm_get_contact) are very heavy and return fairly complex objects. Other api calls expect the same object (crm_add_group_contact) to be passed back in. Finally, the api is exclusively SOAP which is a barrier for a few people.

I think it might be a good idea to come out with a simplified api that addresses the above issues in the next release of CiviCRM (i.e. v1.7). For this version our goals are:

  • Keep the api simple and lightweight.
  • All parameters are basic types:number...
Read more
06 January, 2007
Filed under v1.7, CiviCRM

The team is making excellent progress on the 1.7 release. We've got 19 issues resolved of the approximately 50 issues posted for this release - and the new CiviEvent component is starting to take shape.

We are targeting code-freeze / alpha release for the end of February, and there's still quite a lot to do. Hence, our plates are getting pretty full in terms of additions for 1.7. That said, if you have critical requirements or fixes that aren't already on the list, speak up! If you're able to pitch in with some code, and engineering resource and/or detailed specifications - that's a big plus.

You can see the current to-do list here - and the completed items here.

29 December, 2006
Filed under v1.7, Architecture

I just checked in a few changes that allows a CiviCRM install to use a memcached server if available. We use the php memcached integration to make this possible.

To use this feature you'll need to do the following

  • Compile php with the --enable-memcache flag
  • Download and install memcached. Get the memcached daemon running on localhost and the default port (11211)
  • Define the CIVICRM_USE_MEMCACHE constant in your civicrm.settings.php file

Thats all there is to it. Currently we are storing the Config object and all the PseudoConstants in memcached and are seeing some performance improvements. As we figure out more bottlenecks, we'll migrate a few more core objects to the memcached partition.

We'll need to tweak things a bit to move this to production, but this looks like a pretty good first step. Overall, I'm quite happy with the integration...

Read more
27 December, 2006
Filed under v1.7, Architecture

I've spent a fair amount of time in the past two weeks figuring out how we could optimize and improve CiviCRM. Its been an interesting few days and I suspect will become more interesting over the next few days as we start implementing a few things. All this is in preparation for doing a pretty major load test for the Branner project.

I did read a few articles on the web about optimizing LAMP. The article was fairly helpful, but I disagree quite a bit with some of the conclusions. I think there is also a big difference between building something for one site vs building an open source product. I do agree that PEAR DB, QuickForm and Smarty add a lot of overhead to CiviCRM. On the other hand, I also think we get an incredible increase in productivity because of these components. If I was building stuff...

Read more
19 December, 2006
Filed under v1.7, CiviCRM

Trying to get our arms around how to load test CiviCRM and it is turning out to be a major issue with quite a few steps. This post is primarily for us to sort things out and make sure we are on the right path. So here are the specific steps and various programs we'll use to test things out:

  • Build and deploy the CPS application using CiviCRM. To keep things a bit simple, we'll eliminate any file uploads.
    Language: PHP, Status: DONE
  • Run CiviCRM using xdebug profiler. Isolate and optimize the code here. Could potentially use selenium to automate this a bit.
    Language: PHP, Status: Not Started
  • Take our existing data set of 3000 xml files and randomize the information to generate a new data set of 3000 xml files.
    Language: PHP, Status: DONE
  • Take this set, and convert it into an easy to use pickle format for Grinder. Basically into name/value sets for Grinder to use.
    Language: Python2.5, Status: In progress
  • ...
Read more
14 December, 2006
Filed under v1.7, Architecture, CiviCRM

One of our major goals for this year is to optimize CiviCRM to handle load in a graceful manner. This is extremely important for us with the Branner Project. One of our main goals for the project this year is to optimize CiviCRM to be able to handle the last weekend load nicely (similar to the slashdot effect, the application process has most of the applications being filed and completed the weekend it is due)

The application is a multi-step form, made up of approx 25-30 forms. We have data from last year that we can scramble and potentially get a good random data set that is fairly accurate. Writing a functional test for this is potentially quite painful, but there is the wonderful testing tool Selenium. We've been using it for testing CiviCRM with pretty decent results. We hope to make it part of our...

Read more
02 December, 2006
Filed under v1.7, CiviCRM

Now that 1.6 beta has been released - it's time to look ahead and prioritize the NEXT release (1.7).

There are a few "big features" we're targeting:

... and a number of smaller but significant feature candidates.

Check out the proposed release features here.

We strongly encourage you to take some time to review the list, and add your feedback. This is one of the best ways you can ensure that CivCRM meets your current and future CRM needs.

AND... if there's a "candidate feature" that's important to your organization - help "make it happen" by contributing financial or engineering resources.

Read more
27 November, 2006
Filed under CiviReport, v1.7, CiviCRM

It was a pretty productive thanksgiving weekend :) Learnt way more about reporting, how complex an issue it is and the number of companies that are built on reporting (or to use a more trendy phrase, Business Intelligence). So here are some conclusions:

  • PHP does not have a decent open (or closed) source reporting tool.
  • Reporting is too complex an application and fairly well addressed by other open source projects. We should use one of those applications rather than doing it ourselves
  • All the reporting open source projects use Java / Tomcat. CiviCRM users will need both java and php tools if we adopt this route
  • I dont think we really have a choice with regard to this.

I continued my adventures with BIRT and read the IBM tutorial (registration required) and learnt a fair bit more. The BIRT ...

Read more
27 November, 2006
Filed under CiviEvent, v1.7, CiviCRM

We've posted a draft specification for CiviEvent (phase 1) on the wiki. We are actively soliciting community feedback prior to finalizing the specifications on or around December 7. If you are interested in an integrated Event Management component for CiviCRM - please review the spec carefully and post your feedback and questions as comments on the wiki. We are targeting this for the 1.7 release.

This specification reflects feedback and suggestions from many folks in the CiviCRM and Drupal communities. We'd like to especially thank and acknowledge Jeff Porter of The Foundation for Prader-Willi Research, and Dan Robinson of CivicActions for their extensive contributions.