GROWING AND SUSTAINING RELATIONSHIPS

GROWING AND SUSTAINING RELATIONSHIPS
Close
Jamie McClelland

DEVELOPER AND IMPLEMENTER

PROGRESSIVE TECHNOLOGY PROJECT

http://progressivetech.org
GROWING AND SUSTAINING RELATIONSHIPS
Close
John Derry

Implementor, Administrator, Trainer, Architect

John Derry

http://www.johnderry.com.au

Magazine subscriptions management can be done by CiviCRM.
GROWING AND SUSTAINING RELATIONSHIPS
Close
Guillermo de los Santos

Administrator

Medecins Sans Frontieres Argentina

http://msf.org.ar

with the translation Spanish-English of the module and with the up-to-date upgrade of the modules e.g. peer to peer and campaigning
GROWING AND SUSTAINING RELATIONSHIPS
Close
Michal Mach

Core Team Member, Developer, Implementor

CiviCRM, Caltha

http://civicrm.org

I've always been passionate about what non-profits and advocacy groups can achieve using technology. For me, CiviCRM shows an essential example of how non-profit and technology worlds can come together to provide real change - working as community, creating value for yourself, but also for others in non-profit sector.
GROWING AND SUSTAINING RELATIONSHIPS
Close
Chandra Sekhar Putchakayala

End-User

Organization using CiviCRM

http://vidyahelpline.org

1. To maintain a track of all the workshops conducted till date, who attended the program, who funded the program etc.,
2. To regularly keep in touch with all key stakeholders
GROWING AND SUSTAINING RELATIONSHIPS
Close
Sushant Paste

Developer

Implementors and Developers

http://osseed.com

Worked with CiviCRM as core team developer for more than 2 years. Now we are working as a team and providing service with CiviCRM installation, customization and training. One thing about CiviCRM community is that it's very healthy and really helpful. It's really great that i am part of this community and we want to grow this more and more . Also whatever the problems we are facing there is a solution on forums, or we will get the proper guidelines to solve the issues. Big salute to the CiviCRMcommunity :)
GROWING AND SUSTAINING RELATIONSHIPS
Close
Young-Jin Kim

Developer, Implementor, Trainer

Emphanos

http://emphanos.com

The community around CiviCRM is both welcoming and vibrant. CiviCRM as a software solution is a powerful and flexible data management solution for a vast array of nonprofit organizations ranging from the startup NGO to the established multi-million dollar foundation. In our daily work we are seeing more and more NPOs moving away from proprietary systems and single vendor SaaS solutions and embracing the open source community around CiviCRM. Organizations using CiviCRM love the extensibility and the freedoms that come with open source, freedom to choose hosting, freedom to choose project partners, and the freedom to re-use, re-purpose and re-deploy without paying extra.
GROWING AND SUSTAINING RELATIONSHIPS
Close
Rachel Daniell

end-user, implementor

consulting/multi

CiviCRM provides a vital tool whereby nonprofits and other social projects can implement strong contact-relationship management capabilities without high monthly fees. It also provides the integration and customization capabilities necessary to make such software useful in the complex, lived reality of doing social engagement work. Plus it continues to build the open source toolset made available to the Commons and grow the common good.
GROWING AND SUSTAINING RELATIONSHIPS
Close
Jake Martin White

Implementor, Developer

PeaceWorks Technology Solutions

http://www.peaceworks.ca

PeaceWorks provides technology solutions for not-for-profit organizations. CiviCRM fills an important niche among our clients who need a flexible, comprehensive, user-friendly, web-integrated CRM solution.
GROWING AND SUSTAINING RELATIONSHIPS
Close
Oliver Gibson

Consultant, Implementor, Trainer

Northbridge Digital

http://www.northbridgedigital.co.uk/

The community provides excellent forum support, new ideas and feedback on suggestions. The CiviCRM software suits many use cases and allows us to support a large number of diverse UK voluntary sector organisations.
GROWING AND SUSTAINING RELATIONSHIPS
Close
Allen Gunn

Ally, FanBoy

Aspiration

http://aspirationtech.org/

By giving the nonprofit sector a values-driven, free/open source solution for CRM needs!
GROWING AND SUSTAINING RELATIONSHIPS
Close
Michael McAndrew

Implementor, Trainer, Documentator and Developer.

Third Sector Design

http://www.thirdsectordesign.org

CiviCRM helps us help non profits to do fantastic things with their data.
Being closely involved with the developers and documentation team on a daily basis ensures that we can give our clients the best and most up to date advice on how they can use CiviCRM to meet their needs.

You are here

Home » Blogs » rasantiago's blog

Proposing a New Architecture for CiviCRM

Submitted by rasantiago on June 22, 2009 - 17:56

Here at raSANTIAGO we are entering our third year with CiviCRM and still find ourselves struggling to make desired changes to the codebase. Too often we have expressed desired to re-architect and re-factor the CiviCRM. Recently we have completed two major projects that had us deeper in the codebase then before and realizing that we had to stop complaining. So we set ourselves to figuring out a good target architecture and a migration path that would not interfere with current development but give us the opportunity to engineer in patterns which have produced stable, extensible and maintainable systems for us. In the process, we wanted to deal with serious issues like advancing the API, advance our abilities to test and radically increase code reuse.


After a lot of work and prototyping we think we may have found an architecture that will work and a path to that architecture. From this work we offer a proposal for a new CiviCRM architecture. The proposal covers our analysis of the current architecture, some criticisms and details on a proposed architecture. At the end it lists out some activities for getting to the new architecture, many of which we have already begun and will blog about.


In short, we propose using PHP Doctrine for the ORM layer in place of DAO/BAO. Fortunately, both of these layers can exist in parallel using the same database without stepping on one another. PHP Doctrine provides a good implementation of the ActiveRecord pattern. Instead of QuickForms and Smarty we propose using thin controllers which speak XML and JSON. All UI is subsequently pushed directly to HTML, JS and jQuery. The controllers and the UI are connected through authenticating RESTful interface.


We will be posting and blogging about some of our prototypes and working code to show the pieces of the architecture in action. The most exciting news right now is that we have a 90%+ working RESTful API as a natural byproduct of our efforts so far.


This is a work in progress and appreciate any feedback, criticism and contributions. We hope this effort will serve the community well.


Peace


Roberto

Comments

Submitted by lobo on June 22, 2009 - 23:48

while the document does give us a fair amount of detail, seeing a working example would be even better. So looking forward to seeing the release of the code under an OSI license in the near future

lobo

Submitted by rasantiago on June 23, 2009 - 07:02

No problem. The code will come next. I always prefer to let analysis and documentation lead code.

Submitted by kurund on June 23, 2009 - 16:17

Is your code available in public subversion?

Submitted by pasamio (not verified) on June 24, 2009 - 04:53

It sounds like you're almost going to require JS support on the part of the client to run CiviCRM, seems like a lot of effort. Personally I'd suggest porting it to something like Joomla! which would make stand alone versions easy (you ship a stripped down Joomla! with the CiviCRM in it) and you can relatively easily then embed it into Drupal (unlike the other way which requires you to hack apart a tonne of Drupal's files just to get something to work because you put it in a different directory). But thats just me and I'm slightly biased towards Joomla!.

Submitted by rasantiago on June 24, 2009 - 13:04

Not sure embedding Joomla in CiviCRM in Drupal makes a lot of sense. The new architecture is meant to reduce the number of places where you can place business logic and application logic. Javascript seems like a minimal and very reasonable requirement.

Submitted by mchapman2000 on July 1, 2009 - 18:07

haha. :-)

Drupal Core is lean enough, no stripping needed.

DAO/BAO -> nodeapi / FieldAPI (drupal7) (+ DB:TNG)
QuickForm -> FormAPI
Smarty -> PHPTemplate

It's worked for Ubercart.

(I'm curious how Joomla would map like this. I don't really think of Joomla as a Framework at all like Drupal is, but maybe I can be persuaded?)

Anyway, I like a lot of what's been proposed, although I'm not especially impressed by my first glance at PHP Doctrine.

Even apart from this idea of using a CMS Framework as the framework, the proposal does seem to miss a lot in regards to CMS integration. It seems you would need a View that produced HTML to, for example, put profile data collection into CMS registration forms.

Submitted by JoeMurray on June 25, 2009 - 10:16

I'd like it if more data integrity and business logic could be pushed down into the dbms. As MySQL 5.1 is now out in stable GA release, we could be looking at expanding the use of stored procedures, functions and triggers. This independent of whether the main PHP code is using the existing libraries or this proposal's.

My concern is that I'm looking at a couple of very large CiviCRM implementations (500k - 25M contacts) which will require regular synchronizations with external data sources that do not have shared contact identifiers or consistent name or addressing for the same person. As a result bulk operations will need to be performed at a scale that is not really practicable with a record by record PHP approach.

Submitted by rasantiago on July 10, 2009 - 00:19

So there are a number of huge architectural disasters which come from business logic in the DBMS. So I would avoid this like the plague. Been down that road a few times and while in the short term you gain huge performance you lose all long term maintainability and reuse. Bleh!

But, this is not to say that you cannot take advantage of the DBMS for optimizations of simpler CRUD operations especially in situations of Bulk updates involving related objects. That is a good thing but for the type of problem you are facing I think you are choosing the wrong tool.

Data Synchronization can be handled in number of ways. I would not recommend using the API to do very large bulk operations record by record. We have used Talend and other open source toold to do synchronization on large data sets distributed across numerous heterogeneous databases. This type of issue is not relevant to CiviCRM, though. So it should not be a consideration in the architecture.

Submitted by dharmatech on June 25, 2009 - 15:14

Roberto,

Have there been any updates regarding your proposed changes to the CiviCRM architecture? Did you prototype this for standalone or is it riding on Drupal/Joomla? Do you have a blog/wiki for developers w/instructions on where we can download the prototype code, dev environment requirements, etc.?

Submitted by rasantiago on July 10, 2009 - 00:09

Take a look at http://civicrm.org/node/597. The blog entry posts to the repository at GitHub. Sorry for the delay on it.

We are using it for Standalone, Drupal and Joomla.

Submitted by dalin on June 25, 2009 - 18:45

+1 for this. I think you've got some great ideas here that are well thought out. I think the current architecture has been great at helping the application move as quick as it has, but is now holding back a few key areas.

After working with Drupal's elegant FormsAPI, QuickForms is anything but quick and infuriatingly difficult to use due to zero documentation. Furthermore it doesn't play well with Smarty in some very important areas. I'll be glad to see it go.

But that begs the question, what will create forms? And how will we programmatically alter them (i.e. hook_form_alter)?

And will there be some sort of equivalent to Smarty where we could override the HTML of a page?

I have similar feelings about Pear DB. As you point out it is a bit limited. My cursory review of Doctrine is that it is very well documented and has an active development community. Combined with your proposed model there's the potential to eliminate a load of duplication while making things generally much more consistent. +1

My one concern about the JavaScript dependency is that the application not loose any accessibility (or if possible that we become more accessible). And by that I mean things like screen readers, mobile devices etc. Dojo had some usability issues and moving to jQuery has helped, but I think if we don't keep bringing up the topic, we tend to forget about it.

Another thing that we need to keep in mind is self-documenting code. Currently you can open any file and you'll see a whole lot of
foreach ( $dpml as $p => $n ) {
Which means that it takes an order of magnitude more time to figure out how something works. Your proposal incorporates some really elegant tools that could markedly increase the maintainability of the code. However this can easily be made naught if the code is not understandable on a micro level.

Lastly I think you've made a good move with the idea of incremental migration. Release-early-release-often has served CiviCRM well and I wouldn't want to see a 2yr re-architecting plan kill off the community.

Submitted by xavier on June 26, 2009 - 04:12

Hi,

Waiting on the code to be sure, but for what I understood, you plan to throw out any regular html form and replace the usual post by ajax to a rest api ?

That's a bit of a usability concern here. It should work by default (or as fallback) with the good ol' PUT ;)

Couple of comments:
I'd suggest JSON over xml at anytime, especially to talk to the browser
and I'd use a templating jquery solution (eg pure)

X+

Submitted by dalin on June 29, 2009 - 21:16

I generally prefer JSON as well. PHP's native JSON functions work much better than PHP's native XML functions. I would think that things would be more performant on the JavaScript side too.

I'm not too concerned about requiring a fallback for no JavaScript. The iPhone was the first mobile device to be somewhat bearable to surf the net and it has JS support. Future phones will follow suit. Screen readers do have limited support for JS, they generally do better with the "renderthe JS when the page is loaded, clicking a link sends you to a new URL" approach rather than the "load a huge JS library upfront, clicks make AJAX calls and you always remain at the same URL". As I understand it this new architecture uses the more compatible method.

Submitted by rasantiago on July 10, 2009 - 00:06

I am not sure I agree that REST + JSON/XML is really extreme. Its a great dev pattern in use on a lot of big sites and one that has served us well. I am not sure what the complaint is here. Can you describe a scenario where this would pose a problem? Perhaps that would help clarify your concern.

The suggestion for JSON is a good one and we are planning to support it since Doctrine already natively supports XML and JSON (and a couple more serialization patterns)

Submitted by lobo on July 14, 2009 - 18:41

a few BIG sites that use a pure REST + JSON/XML model. Also links to other articles/websites that implement it would be appreciated

lobo