courtly
courtly
courtly
courtly

Upcoming Events

San Francisco CiviCRM Meetup - February 8th, 2012
February 8th, 2012
Come meet others from the Bay Area who are interested in, using or developing (more...)

UK usergroup - London meetup
February 8th, 2012
Come and meet others from the UK that are using CiviCRM or are interested in (more...)

London user and administrator training
February 23rd, 2012
A comprehensive two day hands on training course covering the configuration, (more...)

CiviCRM London sprint Feb 2012
February 27th, 2012
Following the CiviCRM training here in London, we will have a CiviCRM code (more...)

UK South West - CiviCRM Meetup
March 20th, 2012
Come meet others from the Area who are interested in, using or developing for (more...)

[Bristol, UK] user and administrator training
March 21st, 2012
A comprehensive hands on training course covering the configuration, (more...)

San Francisco user and administrator training
March 29th, 2012
A comprehensive two day hands on training course covering the configuration, (more...)

CiviCRM Usability, Test and Code Sprint - San Francisco (March 2012)
March 29th, 2012
This usability, code and test sprint is targeted at CiviCRM users and (more...)

CiviCon 2012 San Francisco Bay Area - April 2nd 2012
April 2nd, 2012
CiviCon is THE annual event bringing together the people who use, develop, (more...)

CiviCRM Documentation, Test and Code Sprint - after CiviCon San Francisco (April 2012)
April 4th, 2012
This sprint is targeted at CiviCRM users and developers who want to work on (more...)

CiviCRM Components

Tools for engaging your supporters...

CiviContribute

CiviEvent

CiviMail

CiviMember

CiviReport

Expanding CiviCRM search ...

Not Just a Contact Database

These optional components give you more power to connect and engage your supporters.

  • civiCASE

  • Case management for clients and constituents.

  • civiEVENT

  • Online event registration and participant tracking.

  • civiMEMBER

  • Online signup and membership management.

  • civiMAIL

  • Personalized email blasts and newsletters.

  • civiREPORT

  • Report generation and template management.

April 1, 2007 - 15:28 — lobo

We made a few major changes to the v1.7 search interface for a big improvement in performance. The first change was to ot use a wildcard for the prefix. Thus when a user searches on NAME, we only search for 'NAME%', in older version we would search for '%NAME%'. This allows mysql to use the index on sort_name and is significantly faster than a full table scan. The second change involved not searching the 'email' table when doing a search on 'name'. This allows us to avoid two very expensive 'LEFT JOIN' sql statement and speeds up search significantly.

You can get around the above limitations by manually specifying the prefix. There is also a new text search box for email in advanced search. This is probably not a very good user experience and the community has started bouncing around ideas on potential ways to improve the experience without sacrificing speed.

We also want a full text search engine for CiviCRM and some of my earlier thoughts are here. Based on a discussion on the mailing list, I decided to check out what are some technologies we could adopt in the near future into CiviCRM.

After a few minutes of searching, I found Solr. Solr is an XML based frontend sitting on top of Lucene. Solr communicates with the rest of the world using HTTP and XML. Thanx to some prior work, we pretty much have a most of the code for spitting out contacts as XML (we use this route to interface with PDFlib). I did go through the Solr tutorial and it seemed fairly easy to add / delete XML documents. Just need to do a bit more research on how to represent hierarchical tables and see the features it offers us.

I'm quite excited about this, and is with any new technology will probably spend some time over the next few days figuring out how best to integrate it with CiviCRM. I suspect this will give us a pretty good search solution for minimal effort. Both Solr and Lucene are java based projects so folks will need a J2EE server to get this functionality when we do introduce it. However this fits in nicely with the need for a J2EE server for the forthcoming release of CiviReport.

Some useful Lucene and Solr links:

( categories: )

Comments

July 7, 2010 - 11:34 — Hans Rossel (not verified)

Apache Solr Search for Civicrm 3

Hi,

I found the code for this for Civicrm 2.2.9 and before: http://svn.civicrm.org/civicrm/tags/tarballs/2.2.9/tools/solr/.

Is there also code that can be used for Civicrm 3 and any more documentation on how to use it?

Thanks

April 5, 2007 - 16:02 — mixed

Dependencies

Hi Lobo

This sounds great. One of the great aspects of CiviCRM is the speed at which you guys are building in new features.

From your blog I note that this often involves implementing other projects as part of CiviCRM. The downside of this is that it creates a bunch of dependencies to manage in order to ensure stability/security.

I would be interested in your thoughts on this issue and what the developers do to deal with the risk/maintenance issues created by adding these dependencies?

Thanks for the great project!

April 17, 2007 - 01:28 — lobo

Dependencies ...

I think its a tradeoff between build vs reuse. At our current stage, it does help to reuse as much as possible especially with high quality open source pacakages. We spend a fair amount of time figuring out what packages we can use etc. Yes, this does increase the security burden on folks implementing and deploying CiviCRM and its dependencies.