CiviCRM 3.3 introduced a new hook that allows you to interact with and alter dedupe queries. Unfortunately, it was something of a "hidden hook" as it lacked documentation for quite a while. But it can be quite useful and powerful, and thus deserving of a review.
In version 3.3 and 3.4/4.0, the dedupe tools in CiviCRM received several notable improvements, including a restructuring of how the dedupe queries are built to improve performance, and the introduction of a paginated display with table caching that significantly improves the dedupe review and merging process. However, running a rule can still take quite some time on a large database (250k+ contacts), and the matching mechanism is quite basic -- simply comparing field data with field data.
That's where the dedupe hook can come into play.
The dedupe hook allows you to rebuild the queries to improve performance, and add additional logic and comparison intelligence to increase successful matching. Working...Read more
Imagine for a moment that CiviCRM is a garden. In all its object oriented complexity lie bugs and weeds that need to be effectively discovered and managed. Managing a garden the size of CiviCRM is a daunting task for one individual, and even a team of developers along with a community of end-users and testers still need help. There is indeed help to be found!
Most gardens have a gnome or two. CiviCRM has dozens. Our little army of workers were deployed from the xUnit, a top secret, multi-lingual consortium who's primary concerns are the sanity of software developers and confidence of end-users. These assertive xUnit workers diligently patrol the garden for problems. When a problem is found, the local worker will switch xe's hat from green to red in order that...Read more
Some improvements on permissions and the API are landing on 3.4.2 and 4.0.2.
On the previous versions, the permissions where not enforced (on php and smarty) and checked on the same permission "access civicrm" for the rest and ajax no matter the operation and entity.
As API v3 is exposing more and more entities, it became necessary to control better what could be accessed. We re-used the permissions has defined on drupal for the menues, but instead of applying them to a path (in order to access civicrm/admin/event the user need the priviledge "access CiviEvent"), we applied them to the entities and action (in order to create an event using the api, the user need the priviledge "access CiviEvent").
We realised there isn't a perfect match between these two logics "protect what pages and form you can access" vs. "protect what type of content and action...Read more
Many modern web applications have a lot of spam deterrent such as Captcha, Bayesian filters, URL, ip detections etc. One example is trying to do 2 consecutive search on the CiviCRM.org forum and you will get a an error that look like
"Your last search was less than 5 seconds ago. Please try again later."
The concept behind this is flood control is to prevent a webbot (automated script) that is trying to spam and flood the server.
Sometimes this technique is useful in place of something such as a Captcha system because when someone performs a search on the forum, it would be annoying to have to play the "guess game" with a captcha everytime. Therefore discourages the usage of the searching functionality.
We are applying the same concept to CiviContribute contribution page in attempt to stop spammers from using the contribution form as a gateway to test fake or stolen credit cards. See the code in the...Read more
I’m happy to announce the rebuilding of CiviCRM translation resources. If you’ve been a visitor to our Transifex page previously, you know that CiviCRM always ran two concurrent sets of translation resources – one for the stable version and one for the upcoming version (since its first beta release). This was cumbersome – and is the past now.
From now on there’s only one set of resources (POT templates, translated to per-language PO files), cunningly generated from a set of recent versions (the current set is built to cover CiviCRM 3.2, 3.3, 3.4 and 4.0). This is a welcome change for both our beloved translators (no more version confusion!) and your l10n maintainer, who no longer needs to synchronise translations across versions. Our full translation...Read more
This new API is the recommended method to Create, Get, Delete or Update any CiviCRM data from your code or an external system. It is much more coherent and easier to use than the api v2.
As of the civicrm 3.4/4 and as already available for your viewing pleasure on sandbox.civicrm.org (login/password demo/demo), we have developed an API explorer and code generator that is shipped with civicrm and will let you use interactively the api directly on your own data.
It seems like time to give people a quick update as to what api v3 is about. API v3 will be shipping with CiviCRM 3.4 and basically it's like v2 but more so. Well, it's like bits of v2 api and all the other bits have been changed to be like those bits... make sense?
The key to v3 is standardisation. All the api have files now represent their entity (which is generally their table name) - membershipContributionLink is now called MembershipPayment. The mismash of things in the UFGroup (or whatever it was) have been split out into UFField, UFGroup, UFJoin, UFMatch (who knew we had an api call that could return a contact's user_id & vice versa?). We're not 100% there on everything but we've made a big step forwards. Some api that I couldn't make sense of haven't been brought across at this stage (volunteers welcome) - the civicrm_uf_profile_groups_ functions spring to mind.
The functions in each api are now all called
This is a summary of ideas from this forum topic, http://forum.civicrm.org/index.php?topic=15983 , and discussion should continue here.
I'm working on a CiviCRM/Drupal installation for an organization that puts on workshops and houses and feeds people for the duration of the workshops. CiviCRM's built-in way of handling price sets as flat lists of options and prices, doesn't quite do what we want.
We need events to have many signup options, including length, since we sometimes give the option to attend partial workshops, and fee level, since we have different pricings for students and the unemployed. Many other options' prices, in turn, need to depend on the length field and the fee level field - for example, housing option prices need to be adjusted based on length of attendance selected, since a private room for a weekend would not cost...Read more
I have been working on dedupe optimization, part of 3.3 release and a make it happen project, and we are quite happy with the results. A fuzzy rule (first+last+email) which would take 4.3 mins on a 65K contact database, now takes 1.02 sec (tested on a iCore5, 4Gig machine). On a 1.45 million database same rule which used to take forever (i had to quit after 1 hr), now takes 13 sec. Below are some more stats.
|65 K Contacts||1.45 M Contacts|
|Old code||New code||Old code||New code|
|Fuzzy Rule||Strict Rule||Params Strict Search||Params Fuzzy Search||Fuzzy Rule...|
For the training in london, we wanted a simple example that illustrates how to customise and improve civicrm for specific usages using the ajax interface. I'm sharing the result with you, hoping you will find it useful.
One common workflow we have is to change the status of an activity from "scheduled" to "complete". The default way is to click on edit, go to the full form, change the status, save, and go back to the list of activities
We are going to improve it with a "one click click complete": on the list of activities, we transform the status column into an action (when "scheduled"), and when I click on it, it changes it to Completed, without changing screen. For that, we are using the ajax interface and the activity api....Read more