Native Module Development

Submitted by totten on March 27, 2012 - 08:59

Many CiviCRM customizations have been packaged and distributed as Drupal modules. This can be desirable when a customization delves into both the CMS and CRM functionality, but -- when a customization focuses only on CiviCRM -- Drupal modules are a drag: they need to be patched for CMS upgrades (D6/D7) as well as CRM upgrades (Civi 2.x/Civi 3.x), and they don't work with CiviCRM's other CMS's (Joomla and WordPress).

Fortunately, dlobo has been making progress on support for native modules (built around the "CiviCRM Extension" system) in 4.1 and 4.2. An example module is here:

https://github.com/dlobo/org.civicrm.module.cividiscount

Of course, this still poses a challenge: a native module needs to use native tools for packaging code, adding new web pages, developing templates, etc. -- and all those tools come with a learning curve. To improve the learning curve, I've taken a first stab at implementing a code-generator:

https://github.com/totten/civix

For example, to make a new module with a new web-page, one would go to the command-line and enter:

civix init com.example.mymodule
cd com.example.mymodule
civix add-page HelloWorld civicrm/hello

That works for development on one CiviCRM site; to create a redistributable .zip file, just do one more step:

civix build

At this point, civix is a proof-of-concept -- it works for me, but it hasn't been stress-tested, and it lacks some important features (such as creating web-forms, web-services, or DB tables). Before doing another development session to tackle these features, I wanted to circulate the PoC for a few reactions.

So: what do you think?

totten's blog
Log in or register to post comments

Comments

Tim - sounds like a great

Permalink Submitted by Eileen on March 28, 2012 - 02:10

Tim - sounds like a great idea - although I'm miles behind you (as usual) on this & haven't even got my head around the new packaging!

We might be in the same spot

Permalink Submitted by totten on March 28, 2012 - 03:26

FWIW, we might be in the same spot -- we know a bit about the Drupal approach to modules, and a bit about the CiviCRM approach to core code, but this approach is somewhere in between -- it's a new mix of mostly old pieces. It's useful to figure out how common use-cases (eg "declaring a new page" or "adding a new table") play out with this new mix. That requires some reading/thinking/testing. As I started down that path, I wanted to record the results/lessons.

My thinking with "civix" is that the results should be recorded as little metaprograms rather than wordy explanations. The little metaprograms ("civix add-page ControllerName web/path") should be simpler to understand than the explanations ("Update this XML, and create these 2 files, and make sure 5 things match up.")

The approach isn't unique -- I think a lot of frameworks (Rails, Symfony, etc) have new developers get started with similar code-generators. I don't think this feature would ever be as robust with civix as with those those frameworks, but (for the near term) it could be useful in bridging some gaps in our development resources.

Is there a way to port

Permalink Submitted by xavier on March 28, 2012 - 03:18

Is there a way to port existing "drupal but civi only" modules?

It depends on how strictly it

Permalink Submitted by totten on March 28, 2012 - 03:43

It depends on how strictly it's "only civi". Some expectations:

You can probably copy hook_civicrm_* implementations as long as they don't call Drupal helper functions ("url()", "drupal_get_path()", etc). Just copy the code from your old "modules/foo/foo.module" to the new "extensions/com.example.foo/foo.php".
If there are calls to Drupal helper functions, they should be replaced with CiviCRM equivalents. The cheatsheet might help: http://arms.dl.locker10.com/devel-doc/cheatsheet.html
Other hooks (hook_menu, hook_schema) and related codes would have to be ported over more carefully. I don't think there's a good resource describing the new alternatives -- but I'd like to see civix give some kind of kickstart for those hooks.

[Edit] Lobo can probably give some better comments on that... he's been porting "cividiscount" which was a Drupal module.

What if 90% similar?

Permalink Submitted by xavier on March 28, 2012 - 23:25

Hi,

That looks awesome. I was wondering if it's possible to put some CMS specific code too and still benefit from civi extensions?

if it's a function, I can always wrap if into a if function_exists ('drupal_specific') {} ...

that would allow to add elsif function_exists ('joomla_equivalent') by the author of the one wanting to run it on J!

(or implementing a common parent class being overrided by the CMS specific class in the init, like civi does in the core)

For the hooks, is there a way to easily expose some D6/D7/J!/WP specific code containing the hooks to the CMS?

"For the hooks, is there a

Permalink Submitted by totten on March 29, 2012 - 22:11

"For the hooks, is there a way to easily expose some D6/D7/J!/WP specific code containing the hooks to the CMS?"

That is an interesting challenge. I don't think there's an "easy" way. Some ideas (of varying quality):

Without changing any infrastructure code, you could obviously write separate but tightly coupled modules which need to be deployed together (e.g. one portable CiviCRM module, one non-portable D6 module, and one non-portable D7 module). But the developer and the deployers need to be conscientious about building and installation.
In D7, there's something called "hook_module_implements_alter"; I've never used it, but it might allow one to hijack D7's hook processing and direct calls to extensions. However, that seems pretty intricate, and I don't think it would work with D6/J!/WP.
The portable bits and the non-portable bits really should be separate modules (complying with the constraints of each environment's module system) within a shared super-project. One could write a code-management tool (eg civix) which manages that super-project. The challenge is the file-structure -- I don't believe any one file structure is likely to work well with all environments. We need some layer of (developer-friendly) indirection -- such as symlinks or CMS-specific build files (composer/drush make).

I think that's solvable, but it feels like a lower-priority issue -- though I'd be happy if someone proved me wrong.

Let's keep it simple

Permalink Submitted by xavier on March 30, 2012 - 04:44

Agree, way easier to say that it's separate modules.

Would be nice to put in place a convention already. ie. generate a folder CMS (whatever the name) with a README

"Put inside this folder your folders containing Drupal6 Drupal7 Joomla & WP modules that are needed in conjonction of this civicrm extension if needed.

The installation of these modules into their CMS will not be automated but is likely to need symlinks. Please add here the specific instalation instructions"

It is not solving the problem automatically, but at least the specific is going to be stored in the same way by different modules and might make easier the manual steps.

Is it possible to define dependencies (either a version of civi, or module(s) installed on the CMS?

hooks

Permalink Submitted by colemanw on March 28, 2012 - 10:38

Hooks are normally prefixed with the module name. How does that work in civi extensions?

Almost the same. Hooks are

Permalink Submitted by totten on March 28, 2012 - 12:33

Almost the same. Hooks are still implemented as stand-alone functions. The only difference is that you can choose the prefix. Specifically, in org.civicrm.module.cividiscount/info.xml, there's a configurable element:
 
	  <file>cividiscount</file>

	which correlates to:

	  https://github.com/dlobo/org.civicrm.module.cividiscount/blob/master/cividiscount.php

	I'm a little torn about the merits of this. On the plus side, it's familiar and makes it easier to copy-paste code from a Drupal module (if you choose the same name). On the down side, it's less consistent with the other Civi extensions and has the same ambiguity as Drupal's approach (there's a small chance of accidental name collisions). My own aesthetic would be pleased more  by ( http://pastebin.com/GTjrP2du ), but I could go either way.

Did think about it ...

Permalink Submitted by lobo on March 28, 2012 - 13:22

But decided that making it as close to a drupal module (since thats a large part of the developer community) seemed a better decision. A bit easier for those folks to write the drupal extension :)

lobo

extensions have a name also

Permalink Submitted by lobo on March 28, 2012 - 12:49

So the hooks are prefixed with the extension name. so for the cividiscount extension, the

name is cividiscount and its packages under org.civicrm.module.cividiscount (though u could also package it as cividiscount.zip also)

lobo

Support the MIH needed for this!

Permalink Submitted by Joe Murray on March 31, 2012 - 13:35

totten, this is great work - thanks very much.

For others reading this, you might want to help out with the Make It Happen initiative that enables totten's approach to be really useful - http://civicrm.org/participate/mih#cmsagnostic. My company put up $2k as the lead sponsor for this MIH, and only $660 more is needed for it to be fully funded. Developers and consultants in particular should consider contributing since it will make it much easier to get a larger takeup on custom modules and thus share the support needed for them. As well, I see this as a good way to start making more custom functionality available for WordPress and Joomla! installations without having to learn their programming models.

Url:

http://jmaconsulting.biz

Followup: Wiki documentation

Permalink Submitted by totten on October 19, 2012 - 16:57

Just a quick note: The material covered here has been extended on the wiki:

http://wiki.civicrm.org/confluence/display/CRMDOC42/Create+a+Module+Extension

GROWING AND SUSTAINING RELATIONSHIPS

You are here

Blog

Native Module Development

Comments

Tim - sounds like a great

We might be in the same spot

Is there a way to port

It depends on how strictly it

What if 90% similar?

"For the hooks, is there a

Let's keep it simple

hooks

Almost the same. Hooks are

Did think about it ...

extensions have a name also

Support the MIH needed for this!

Url:

Followup: Wiki documentation

CIVICRM

GROWING AND SUSTAINING RELATIONSHIPS

GROWING AND SUSTAINING RELATIONSHIPS

Search form

You are here

Native Module Development

Comments

Url:

CIVICRM

GROWING AND SUSTAINING RELATIONSHIPS