Fair warning, this post is intended to the technical part of our community, if you don't care about the architecture of civi, please skip this one, I'll come back to you soon with awesome datavisualisation and an interview of Micah about security and privacy (you'll like it).
And if you read anyway, I'm a bit of a drama queen and some of the mountains I describes are probably hills, at best.
Replacing the database abstraction layer
was discussed recently with very promising results, and doctrine seems to be the obvious choice to get rid of DataObject.
However, despite all the issues we have dealing with the existing libraries, I'm not sure replacing them with more modern ones will really solve my (and I suspect most) problems: boilerplate and duplication.
To add a simple "whatever that is stored in a new table" in civi you need to (at least):
describe the schema (xml)
have a sql upgrade to create the table
write a minimal bao
write a minimal crud api
write a php form
write a page to list the whatever
write a template for the form
write a template for the list
Some steps can be skipped (eg. by not providing an api) or combined (eg. the sample template for the list and the form) but it's not best practice nor does it save much time.
Adding a new field in this whatever implies at least
update the schema (xml)
have a sql upgrade to alter the table
alter the php form
alter the template for the form
Quite a few steps, isn't it? and plenty of options to miss one and create issues for some users, or (ab)use one size fits all OptionValue/Group even if it means no more DB enforced coherency. Or skip unit tests. Or inject business logic in the template....
I'm sure that replacing quickform and pear classes are going to help, but will it be really worthwhile if, at the end, we still have to touch half a dozen files spread all around to implement simple changes?
I know of a framework that makes it easier: with django, you can write a single file and it manages to provide the rest and have a decent looking interface that stores and retrieves from the db. Moreover, does a pretty good job letting you override any of these parts if you need something more advanced than what is provided out of the box.
Can we have in civi something like that that makes the trivial automatically done and the complex easier to customise and maintain too?
I've seen some examples of symfony where the db+form layers are one step, others with form+templates, but I haven't found my holly grail of "change a single file and get everything updated automatically".
Are you aware of such an example? ideally, integrated with backbone and that handle sql schema upgrade automatically too.
As for the way forward, we are probably not going to change everything at once so we won't benefit directly from that integration, but wouldn't it be good to know it will be possible before we continue this journey?
Xavier, for the committee of the better tomorrows ;)