- What data is stored in what tables?
- What type and size of data is valid for a given field?
- How are the various tables connected to each other?
We made a few major changes to the v1.7 search interface for a big improvement in performance. The first change was to ot use a wildcard for the prefix. Thus when a user searches on NAME, we only search for 'NAME%', in older version we would search for '%NAME%'. This allows mysql to use the index on sort_name and is significantly faster than a full table scan. The second change involved not searching the 'email' table when doing a search on 'name'. This allows us to avoid two very expensive 'LEFT JOIN' sql statement and speeds up search significantly.
You can get around the above limitations by manually specifying the prefix. There is also a new text search box for email in advanced search. This is probably not a very good user experience and the community has started bouncing around ideas on potential ways to improve the experience without sacrificing speed.
We also want a full text search engine for CiviCRM and some of my earlier thoughts are...Read more
The nice thing about developing software is that the work is never done. There are so many cool things that we can do to make CiviCRM a better application. I've been having some pretty good conversations with David Strauss of Four Kitchen Studios on our IRC channel (#civicrm at irc.freenode.net). Four Kitchen Studios has been a great resource for CiviCRM and were instrumental in deploying CiviCRM within Wikipedia. In the recent past, David has initiated the de-dupe and contact merge algorithm which we hope to incorporate in CiviCRM v1.8.
Some of the various things that we hope to be part of v2.0. We will use a fair amount of the design and code from the v1.x series, but at the same time make significant changes when needed.
- Support for php5 / php6 only. This will enable us to use some of the more advanced functions specifically interface classes, iterators, PDO's etc. At this point, we'll probably also consider...
lighttpdpackage. This will get the server installed and running; you can verify it by opening http://... Read more
I've spent a fair amount of the weekend attempting to install FishEye from Cenqua. Its awesome that companies like Atlassian (wiki and issue tracking software) and Cenqua (coincidentally both these firms are Australian!) give away free licenses to open source projects like CiviCRM.
FishEye helps you analyze, search, share and monitor your source code repository (in our case svn). We've always wanted something a bit more fancy than what subversion offers out of the box (a vanilla http interface to the code). We also wanted better integration with JIRA and link issues to the appropriate revision of the code. FishEye promised to deliver on both these cases. We had also heard pretty good reviews of the product.
Getting a license was incredibly easy. I filled out a form and gave them all the information and in 24 hours received the license. Installation was quite easy and I had...Read more
An interesting discussion spawned on the civicrm-dev list recently regarding our implementation of custom groups and fields. We have been super cautious about this and have advised people not to create more than 20 custom fields per object (contact, activity, group, relationship etc). However creating a custom field is relatively easy and people have been doing it at a fast and furious pace. We've heard of implementations with more than 100 custom fields. This has worked for some people, but a large number of custom fields has the following problems:
- Custom values are stored in a thin table, one per value. This table will grow significantly for large number of custom values
- Primary Export is not possible since it will exceed the mysql JOIN limit of 61 tables
- You cannot make a large number of these...
To use this feature you'll need to do the following
- Compile php with the --enable-memcache flag
- Download and install memcached. Get the memcached daemon running on localhost and the default port (11211)
- Define the CIVICRM_USE_MEMCACHE constant in your civicrm.settings.php file
Thats all there is to it. Currently we are storing the Config object and all the PseudoConstants in memcached and are seeing some performance improvements. As we figure out more bottlenecks, we'll migrate a few more core objects to the memcached partition.
We'll need to tweak things a bit to move this to production, but this looks like a pretty good first step. Overall, I'm quite happy with the integration...Read more
I've spent a fair amount of time in the past two weeks figuring out how we could optimize and improve CiviCRM. Its been an interesting few days and I suspect will become more interesting over the next few days as we start implementing a few things. All this is in preparation for doing a pretty major load test for the Branner project.
I did read a few articles on the web about optimizing LAMP. The article was fairly helpful, but I disagree quite a bit with some of the conclusions. I think there is also a big difference between building something for one site vs building an open source product. I do agree that PEAR DB, QuickForm and Smarty add a lot of overhead to CiviCRM. On the other hand, I also think we get an incredible increase in productivity because of these components. If I was building stuff...Read more
One of our major goals for this year is to optimize CiviCRM to handle load in a graceful manner. This is extremely important for us with the Branner Project. One of our main goals for the project this year is to optimize CiviCRM to be able to handle the last weekend load nicely (similar to the slashdot effect, the application process has most of the applications being filed and completed the weekend it is due)
The application is a multi-step form, made up of approx 25-30 forms. We have data from last year that we can scramble and potentially get a good random data set that is fairly accurate. Writing a functional test for this is potentially quite painful, but there is the wonderful testing tool Selenium. We've been using it for testing CiviCRM with pretty decent results. We hope to make it part of our...Read more