Drupal 8 is just around the next (53) corners*

This weekend has seen 2014 Drupal South conference in Wellington with an influx of a couple of hundred of mostly Kiwis and Aussies - but a few others - to talk Drupal. Coming away from the conference I'm excited about Drupal 8 developments and even more excited about convergence between CiviCRM and Drupal. With both Drupal and CiviCRM moving towards Symfony and Drupal having selected a bunch of other 'best of breed frameworks' that we in CiviCRM are also looking at (including Backbone), one of the big wins is transferability of developer skills. This is a win in both directions - it will be easier to get people to develop for CiviCRM and Drupal as we adapt to frameworks that are becoming industry standards. Conversely developers will be better able to apply their skills elsewhere.   One 'best of breed' framework Drupal adopted some time ago was jQuery and jQuery has pretty much become the industry standard since then. One of the speakers pointed out that at the time Drupal adopted jQuery there were several competing frameworks (CiviCRM was then using Dojo) and by a combination of good selection and community leadership, Drupal selected the 'winning horse' in that race. Thinking about that gave the Civi-developers at the conference reason to feel pretty good about taking strong leadership in frameworks from Drupal.   At the Twig presentation (a templating engine) many of the advantages Morten was excited about in Twig seemed to also to apply to our much maligned Smarty - being able to wrap chunks of text in 'translate', being able to pipe output from a variable into a 'replace' or 'formatter' function and not having to put in php around text outputs.  However, I came away with a couple of thoughts there too. Twig would be a fairly easy replacement for Smarty as it didn't feel that different and we can expect it to be maintained well into the future. However, simply replacing Smarty with Twig won't remove people's annoyances with Smarty, because most of the annoyances are about the implementation - the amount of logic in the form & templates layer and the difficulty in over-riding them in a maintainable way. So, when one of the speakers pushed the importance of 'loose coupling' I realised that was the jargon for our calls to 'get the logic out of the form layer'.   Loose coupling provides opportunities. We were shown a phone app that rendered a Drupal 'form' - without passing html back and forth to the website. Drupal now sees HTML as merely one of many possible outputs for a form. I think that would be a great challenge for CiviCRM  -  to reach the point where we could render and submit Contribution / Membership forms with all their clever logic in another application, without the expectation of HTML being part of the exchange. But, that required a rethink of how we use Smarty and whether we can move the 'doing' part of the form into its own function which can be accessed in multiple ways.   The real lightbulb moment, for me, however was in the talk about Drupal's Migrate in Core by (chx) (slides are here). As you may be aware I have written a CiviCRM extension to make CiviCRM a destination for migrations. The speaker advised we shouldn't need to do that in Drupal 8 "if you need to write a destination plugin you are doing something wrong". If your destination is an entity then Drupal provides Migrate destinations as part of the entity interface. In fact over the last year Fuzion has been investigating declare CiviCRM entities as Drupal entities as a solution to multiple problems - in particular we have looked at Rules and Views Bulk Operations. It has long been my feeling that eventually declaring CiviCRM entities should replace most of our views code (as you get Views 'for free' when you declare entities) but I now think we should do that INSTEAD of migrating our drupal 7 views code to Drupal 8.   Looking at the list of drupal 7 modules we have  
  • CiviCRM Contact Reference - we already use Drupal EntityReference fields instead of that, and I think we can avoid migrating it to drupal 8
  • CiviCRM engage - we should turn that into an extension and not migrate it to drupal 8
  • CiviCRM group roles - this can ship as a set of Rules
  • CiviCRM Member Sync - this can ship as Rules
  • CiviCRM OG Sync - this can ship as Rules
  • CiviCRM Rules - this is Rules - but would need to be reviewed against the entity approach to Rules
  • CiviCRM Theme - this would probably get migrated
  • CiviCRM Views - hopefully we can replace this with entity code.
One question would be whether we need any form of upgrade path for any of the above modules. I would note that in Drupal 8 there is no upgrade path - you create a new database and use Migrate module to put your data in there.   Another message from Lullabot keynote speaker Emma Jane was 'Start on Odd, Stay on Odd'. In other words, if you are currently on Drupal 7 you should wait for Drupal 9. The implications of this are of course that they are not expecting people to move off Drupal 7 'anytime soon'. When Drupal 7 was first integrated with CiviCRM it was expected that CiviCRM 6 would be removed more or less straight away. However, a few people, including myself, were concerned about Drupal 6 users and 2 years later we still have an important but decreasing set of sites using Drupal 6. The good news and bad news about the D6 experience - good: after the initial restructure keeping Drupal 6 integration has not required much work, bad: people have been confused about whether Drupal 6 was still supported. We've had several Drupal 6 organisations tell us that they would like to use CiviCRM but they are on D6 and we have had to explain to them that there is a Drupal 6 release. Having been the 'lead supporter of Drupal 6' for the past 2 years I have probably spent more time explaining that D6 is supported than supporting it. I think going into D8 we should be anticipating that Drupal 7 will need to be supported for some time and be consistent on that message.   I know other people are far more 'up' on D8 than I am but I think we should plan to migrate to it without copying and pasting our d7 code & trying to make it work. I've started a bit of tinkering and will push it somewhere to Github soon. Who wants to get involved in Drupal 8?   *53 is the number of critical bugs left


I think moving forwards the civicrm as drupal entity path, would be the greatest win possible for CiviCRM. So i really would welcome this integration. Not only rules and views would come for free. But also displaying civicrm data in drupal with display suite for example. Getting features export code for civicrm entities would also be a huge win. 

Supporting migrate in D7 (and D8) even further this way could also be a great catalyst for an 'easier' switch of getting complex data in to civicrm. And would also be easier to get drupal integration D8 ready. 

You have started that work already, haven't you? Is this usable/in prod already?

As for the view integration, who took the lead recently? your suggestion makes sense I think, would be great if view could follow the path you describe.

We have been using CiviMigrate for our CiviCRM migrations for quite a while now. WRT views - I think the problem with the change to using entities is migration - but D8 is such a big change anyway I think it's OK if views need to be rebuilt when people switch to D8.


Chris Ward is our views maintainer at the moment

Civicrm is sometimes rejected by Drupal devs becuase it does not conform to entities (like redhen or crm core) and I expect this could really stimulate adoption.  As I've mentioned in the past, using entities will also allow System Builders a greater set of tools without requiring API module development.

Views is definitely the most utilized of the integration modules - could we have them coexist such that civicrmentity would use one approach and the existing module another?  Perhaps this could work the same way with rules.  Using either method would allow a gradual testing & migration from one approach to another.

Thanks Eileen for the post and pushing this into the mainstream dialog.

I'm not quite sure at this stage whether switching to entities will necessarily break views - at least the tables & field names are likely to still be the same as presented to views. In Drupal 7 the civicrm_entity module adds a few fields without breaking views.

It would certainly need more testing.


Note that in discussion about GSOC (since Torrance has shown interest in this as a GSOC project) we thought it might be the case that we have a backdrop integration that follows the d7 path & a d8 integration that uses entities.

How does this discussion relate to the options available to sites running Drupal 6, as referenced in this post announcing extended support for Drupal 6: https://www.drupal.org/node/2288521 ? Is it realistic to expect that a direct migration from 6 to 8 might be possible in the three months that they will have overlapping support, or is it unrealistic to think that a Drupal 8 ready version of Civi will be ready by then?