Enhancing Greetings Handling

English has been my favourite language for a long time now.

There is more than one reason to this. I feel I should write a long and beautiful essay (in English!) about it... But alas, that's not what I'm going to do now. (Sorry -- I hope you can bear the deprivation ;-) ) Rather, I need to help making the world a better place, by saving some poor German NGOs from starving. Which I mean to achieve by -- can you guess it? -- of course: by improving CiviCRM! Easy one :-)

And that's why I'm going to talk about one particular reason for liking the English language: the beautiful simplicity of (certain aspects of) English grammar. Or more to the point, about the fact that those not having the wonderful luck of living in regions doing all communication in English, often have to deal with more complicated grammatical rules -- including, among others, pretty much the entire continental part of Europe.

(Actually, I will mostly talk about Germany only -- after all, I don't want to offend our dear fellow Continental Europeans by implying that they are just as complicated :-) Though I must say we *did* hear some anecdotal evidence -- i.e. rumours -- that some others might happen to be kinda somewhat in the some boat with this...)

So what does grammar got to do with greetings in CiviCRM? Don't worry -- getting there :-)

We all know there is this nifty greeting template system in CiviCRM. We just need to set up a default greeting template in line with the communication style of our organisation, and CiviCRM will generate according greetings for each contact automatically. We can further set up extra greeting templates, so we can easily select some other standard form for certain contacts who we want to address with a different level of familiarity for example. Only for special cases we need to enter individual greetings by hand.

Now the bad news: while implementing CiviCRM at digitalcourage and other German NGOs, we (that is, us folks from Software für Engagierte e.V.) soon realised that this approach doesn't really scale to German greetings grammar. Drat.

What makes English greetings easier to tackle? First and foremost, there is little gender distinction in English. The only difference between "Dear Mary" and "Dear John" is the name, which we just insert literally as a token -- the "Dear" part is always the same. When going more formal, it's "Dear Mr. Smith" vs. "Dear Mrs. Smith"; so we need an additional token for the gender-specific prefix ("Mr." or "Mrs.") -- which we can also take directly from the contact entry form.

Not so in Germany. We have to distinguish between "Sehr geehrte Frau Müller-Lüdenscheidt" and "Sehr geehrter Herr Müller-Lüdenscheidt" (spot the difference!); or (informal) "Liebe Brunhilde" vs. "Lieber Gottlieb". So it's not just the gender-specific prefix ("Herr" vs. "Frau") that differs, but also the actual greeting word. ("[Sehr] geehrter" vs. "[Sehr] geehrte"; "Lieber" vs. "Liebe".) We can't just insert some contact field as a token to make this distinction -- we need to have different templates for male and female forms, and select the right one manually for every single contact... Ouch.

Not convinced of the superiority of the English language yet?... ;-) Well, there's more. In modern-day English, while the greeting can range anywhere from "Hey slacker" to "Your Majesty", the pronoun used throughout the actual letter is (almost) always "you" -- formal and informal alike. It's not so easy where other languages come into the picture. (Sweden being the only exception I'm aware of...) In Germany for example, there is the familiar "Du" and the formal "Sie". (And also "Ihr", if you feel an urge to cover the majestic plural as well... ;-) )

Now how does that affect greetings? Well, indirectly it does: to properly handle formal and informal modes of communication, we need to vary not only the greeting, but also the message body. We can handle this with CiviCRM of course -- but to do that, we need a separate, explicit data field telling us the mode for each contact. Which means we have to select the appropriate greeting template, *and* the matching pronoun (in a custom field) for each contact. Did I say "ouch" before?...

There is yet another issue with the greeting handling in Germany -- though this one is not really more complicated for a change, just different: the way titles are included in names.

(Well, it's not more complicated, unless we wanted to do it *really* correctly... But the finer rules are so intricate and obscure, that nobody really bothers to have them implemented in software :-) )

Titles are very important to Germans. So much so, that instead of appending them after the name like English titles ("MBA" etc.), in Germany virtually all titles are prepended, between the gender prefix and the actual name. Thus, all contact entry forms in Germany begin with the gender prefix, followed by a title field, and only then the first and last names. (There is actually a national standard for that...)

What's more, the title field in Germany is generally free-form: as titles are so important, there are way too many obscure titles and combinations of titles to make a select box with a number of fixed choices feasible. (How about a "Prof. Dr. Dipl.-Ing." for example? ;-) )

Note that not having a set of predefined choices here -- while it may sound horrible to us database nerds -- is not really a problem in this case: the title is usually not processed, but rather just included as a literal token in addresses and greetings -- so there is no real need for normalisation! (I'm surprised at myself for saying this... ;-) )

In the face of all these almost horrible shortcomings, I must wonder why (to the best of our knowledge) nobody has attacked this issue before? I subtly hinted above that these problems are not all exclusively German -- yet nobody else seems nearly as bothered as we are? (In lack of a better explanation, all I can do is resorting to stereotypes: attributing it to the proverbial German "Gründlichkeit" ;-) )

Whatever the reasons, the fact of the matter is that these waters remain uncharted thus far -- so we were left with no other choice but to set out on a heroic mission to solve this properly once and for all. In the course of this adventure, we first went through the deathly perils of intense brooding. Then through more brooding. Then some empirical research. (Read: failed early attempts at a working solution ;-) ) But most importantly, more brooding. And last but not least -- no doubt the hardest part -- convincing ourselves that we are indeed finally on the right track!

So what have we come up with? At the centre of our elegant and finely crafted solution, resides a simple yet powerful concept: let's process the greeting templates with Smarty, so we can include conditionals! (We already discussed that bit on IRC a while back...) This way, we can create a complex default greeting template, which will automatically adapt the wording to different genders and formal vs. informal addressing modes.

While technically this is not a big change, it has major consequences -- greeting handling will never be the same again... ;-) Which is to say, using this will somewhat change the modus operandi for the greetings mechanism: rather than explicitly choosing between different greeting forms by selecting distinct templates, most variants can now be covered using only a single (complex) default template -- deriving the appropriate form from other data, such as a separate formality level field. Explicitly choosing different greetings will now only be necessary in special cases.

(But fear not, traveller: if this new world order intimidates you, it will still be possible of course to use the original approach!)

Allowing conditionals is only part of the story: we also need appropriate database fields to feed these conditionals. In our cautious adventurers' wisdom, we have already mapped out these parts of the expedition as well :-)

For the gender, let's keep using the conveniently already existing gender-specific name prefix field. Although it was originally conceived by its creators as a token to be directly included in the greeting as text, we can use it as a switch for driving conditionals just as well -- I'm sure the field won't mind ;-) (That's indeed how it is generally handled in German contact input forms... Remember that standard I mentioned?)

For recording the formality level, our grand plan stipulates planting a brand new field in the communication preferences block. We are not sure yet what talents to endow this field with exactly: for digitalcourage -- and probably most other organisations too -- a simple binary choice is fine; but it seem quite fathomable that some organisations -- perhaps in other countries? -- might feel more ambitious about this. Thus it could be preferable to implement this as a mighty configurable option group from the start?

Finally, we need some place to stick our fancy prefix titles. For this we shall introduce another new field, right in the main contact block with the other name components. We will make this one optional, so those lucky organisations only needing to deal with English naming conventions won't be bothered by it; while us others can tune up our happiness by ticking a simple configuration option :-)

(By the way, we contemplated extending our mission by adding Smarty processing to the address format templates as well, while already navigating neighbouring waters... However, there are some other deep perilous considerations lurking there, suggesting a whole new adventure on its own -- we shall follow up in a forum discussion.)

Now having established the best course, it's time to fetch the oars! In fact, we have already plunged ahead with an implementation of the Smarty processing in greeting templates -- you can marvel at our achievements so far in our greetings handling branch :-)

(To shelter the faint of heart, it might be desirable to add a configuration setting enabling this only optionally, the same way as Smarty processing in message templates needs to be enabled explicitly? This doesn't seem nearly as crucial here though, as the greeting templates are generally only set up once during the implementation phase, and not modified in normal usage...)

With none of the other stages of our planned route appearing too challenging either, we are confident we can reach the destination harbour within few weeks, so as to safely land this in time for CiviCRM 4.4. And then we shall see plentiful rejoicing, with one of the major frontiers for broader CiviCRM adaptation in Germany finally taken care of! :-)

But let's not forget our dear fellow passengers. Having previously established hard anecdotal evidence that we are not exactly alone in this boat, it's time now to pass the word to the other travellers. So, fellow Europeans -- or anyone else with a boarding pass for this cruise -- speak up now: do you think our plan can keep your side of the boat afloat as well?