Notice to non-developers: This post is about how some functionality in 4.2 will be implemented in code and in the database, with very minor changes to anything visible through a browser. If you're not a developer, it probably won't interest you.
Simplifying the Codebase
As part of the CiviAccounts project we are looking to redo some of the implementation of the configuration and processing of payments for contributions, memberships, and events. Currently the processing for each of these three types of objects has two paths: one for a simple configuration of the objects, and one using price sets. This means there is more code, more complexity, more possibility of errors, more work when making changes, and more need for testing.
As we refactor the existing code we're looking at keeping the simplified UI for configuration and administration, but implementing everything under the hood using price sets. Before going ahead with that, we wanted some feedback from developers in the community.
Price sets are now supported for contributions, events, and memberships. You can use them to sell individual or multiple tickets at a time, items along with memberships and subscriptions, or allow contributors to determine how much to give / buy using text/numeric fields, select widgets, radio buttons, or checkboxes. Support for contributions on the same form the same as ticket purchases, and for contributions along with membership purchases is facilitated with price sets. They will be a good way to implement a page that allows people to get a discount on their annual membership if they buy a ticket to an event, though I haven't seen that yet.
One change that CiviAccounts will be making to price sets is allowing each option to be associated with a different general ledger account. This would allow, for example, revenue for one type of ticket to go into one account while revenue for a different type of ticket would go into a second account in your bookkeeping system.
The downside of price sets is that they are time consuming to create, and can be confusing for non-techies or new admins. First the price set needs to be created; then each of the fields within it; and within some file types, options can be specified. Lots of steps, clicks, page refreshes, and not intuitive how it will turn out for many CiviCRM admins.
By contrast, it's possible to enter the Fixed Contribution Options directly on the Contribution Amounts tab when configuring a Contribution Page, or select the Membership Types to sell on a page in the Membership Settings tab. 
The Regular Fees section on the Fees tab when configuring an Event is just as simple. These simple configuration pages are equivalent to specifying a radio widget option field in a price set, without the complexity or flexibility provided by certain options.
We think it would be good to allow admins to continue to use the simplified interfaces and have the option of the power and flexibility that comes with the complexity of the full price set UI.
Proposed UI Change
In addition, we think it _might_ make sense to allow admins to switch from the simple configuration of a page to editting that information via the more complex price set configuration forms. But not the other way - going from complex to simple.
What's this mean in practical terms? Once any change has been saved to a configuration as a price set, even one that includes only one radio button field, the simple configuration approach for the contribution page or event would not be available for editting it. This would prevent confusion in various ways, e.g. from not seeing in the simplified form a change on the complex forms that was changing the behaviour of the field behaviour. Of course, you could turn off the price set you just created, and go back to the simplified UI and start configuring it from scratch again.
Proposed API
We would expect the price set api to follow our existing structure with an API to get, delete, create each of the Price set entities – Price sets, fields, values and for hooks to be called PRE & POST the main CRUD functions on these entities. The buildForm hook & validate hook are already available for editing price sets. We would probably discontinue the buildAmount.
For API users the change is beneficial with regards to interacting with events. At the moment it is extremely difficult to create events through the api as you also have to create an option group & option values to go with it. More problematically you can’t re-use the same event prices to create a new event via the api so this makes api creation of events much easier (and the eventual implementation of a front end event import option.) It is also currently hard to GET event prices through the API. This is likely to still require the use of API chaining but the chain will be a little more intuitive. Contribution Pages do not yet have an API & the issues are likely to be similar but most sites seem to create many events but only a few contribution page so the relevance is more for events.
Feedback?
Comments or concerns about these change would be appreciated.
Comments
Does this change mean that if an admin creates an event/contribution page with "simple" pricing, it would be stored in the dababase layer as a priceset with one field?
If so, would the admin be able to change the price, such as changing the amount from 10 to 15 in the same way as in CiviCRM 3.4.7? ie they just edit the contribution/event page directly?
Yes, it will be stored as a priceset, and yes, people will be able to make changes to prices right in the contribution and event pages as they do right now.
Pretty much the same user experience.
Hi,
Seems to clarify and simplify indeed, great job. About the UI, I think having one simplier UI for the complex cases that works as well for the simple one is easier to understand and implement than having two separate UIs (and the usual head scratching of the user wondering which one they should use).
I think the problem is that the UI is implemented in a way that doesn't fit the natural workflow. It expects the priceset to be created before where it's used, but in practice, you start with the event/contribution page/... and add the price or priceset. (That's the same issue with profile, with an extra step of dealing with custom fields).
By allowing to create or update the priceset where it's used instead of simply selecting one, it would likely makes it easier enough so we don't have to provide a "dumbed down" alternate UI.
X+
I agree that it would be better if CiviCRM supported natural workflows better through its interface, and this is another instance where that is not the case. Unfortunately that improvement is a bit out of scope given our funding for this initiative.
I pushed for this task orientation when we redid the menuing back in 2.1 or so, but it wasn't accepted then - perhaps too much work? I think when creating a contribution page, for example, a user may need to create a profile, and while creating a profile, may need to create one or more fields. Keeping the context so the user popped back to the right place from this deep dive may have been an issue in terms of usability. For example, how to save the 'workspace' to continue where one left off the day before. Anyway, I think this question deserves to be discussed again more broadly for the whole interface.
We are going to have more priceset, and I'm wondering if we risk having the same problems than for the profiles, namely:
1) getting pricesets that aren't relevant to the specific page (eg. priceset about memberships when creating an event).
eg. when creating an event, I have a long list of contact profiles, most don't make sense for events
2) Not knowing where the priceset/profile is used
Can I delete it? Is this used still? where?
Doing cleanup on the priceset will be needed, and if we don't implement it right, will be same issue as with the profile. It should be possible to "obsolete/disable", and it would be useful to have more data exposed about their context:
- where it's used
- if it's made for specific components/usage (eg. for events and donations but not memberships)
- creation date
- author
Make sense?
Yes, you're onto something here. I think we could try to start along this path by providing a link from the Manage Price Set page to a page showing the places where a price set is used.
In order to ensure data integrity over time, we will need to keep price sets around once there is data for them, and prevent destructive changes if there is data. Users will need to be told to delete the financial transactions from the system before the changes can be made. So this will result in cruft in terms of the profiles displayed in the interface. I like the idea of 'obsolete' status. Another idea was versioning of objects. I'm reluctant to commit to either of them for phase 1, however.
While you're at it: A sophisticated capacity system is needed!
Currently it is not possible to restrict the number of Tickets per Person!
Example:
Right now we have one Event with an incredible high demand. We want to limit the total amount tickets purchased per Person to 3. There are two kinds of Tickets - Student and Normal - both we just have a certian number of. If student tickets are sold out, people can only buy normal tickets and other way around.
I'm able to restrict the capacity of Student-Tickets, and the amount of Normal Tickets - but it is not possible to handle the overall Capacity of the Priceset itself.
I would love following solution where you can group the Priceoptions and give them a overall capacity and order limit. This way Capacity is ensured and options are not independent (which they are often aren't in real life).
Then it would look like this:
Priceset: Tickets for Crazy-Event
- Price-Option-Group -
Overall Capacity Limit: 200
Maximum Capacity per Person: 3
Price-Option: Student-Ticket
Type: Select-List [max. 3 or less depending on other selection in P-Group]
Maximum Capacity: 100
Participant Count: 1
Price: 18
Price-Option: Normal-Ticket
Type: Select-List [max. 3 or less depending on other selection in P-Group]
Maximum Capacity: 50
Participant Count: 1
Price: 25
Price-Option: Just Pre-Event-Party
Type: Select-List [max. 3 or less depending on other selection in P-Group]
Maximum Capacity: none
Participant Count: 1
Price: 5
Price-Option: Just Outside Bar
Type: Select-List [max. 3 or less depending on other selection in P-Group]
Maximum Capacity: none
Participant Count: 1
Price: 0
- - - -
Whats important as well, is to check if capacity is reserved when multiple people are registering. Seems like right now a capacity check is just done when entering the registration process - not on submit (500 people registering at the same time = trouble).
This comment is longer then assumed. Maybe I'm heard.
Hi,
Interesting use case, would be great you write a case study about it.
I think you should use hook to implement it, as it might be too hard to find something generic enough in the core.
Have you tried?
X+
Hi xavier,
I will make a Case about the whole System/CRM/Views as soon I have the time; especially because CiviCrm was such a pain at some spots (not a fault; just the maturity of the system I guess). We are one of the biggest Program oriented Student-Alumni Clubs in Germany (pimandcems.de); located at the University of Cologne. The System is not just a tool to manage our members, but mostly to track them what events they are attending and so on.
However, the price-set-case above really gave me headaches. I finally decided to force people in registering additional Participants for additional tickets; implemented a dynamic "additional drop down-list" which lets you only select so many slots as specified in a costume field of the event. Made a bunch of template changes so the number of participants_registered_by_id and the slot per person restriction blocked a user registering additional participants. Then set the overall capacity restriction in the event and of the two different kinds of tickets.
It works - BUT - people do not accept that they need to register people for a ticket; this leads to confusion and to a lot of communication because people buy tickets before they know who is going with them.
Hooks:
Honestly - the only time I really used hooks is when it was merely possible to relate a hook example. Nowhere I could find a easy to understand work flow of creating a hook from start to finish. There are a bunch of hook-types - I read all the development stuff (at least I think so) - but I couldn't find a step by step introduction from realizing a problem, finding the code, figuring out what/how variables are responsible and how to make changes with a hook.
Right now we are using template overrides - I know it will be trouble some to update - but it is easier to understand, to debug and simpler then maybe having a whole set of hooks just to perform a easy task.
Development:
Right now it is really hard to speed the customization up, because I can't find volunteers - just for most organizations - php-professionals are to expensive, so everything relies on one person who made his way through try and error. I think the CiviCRM community forgets that - that the people on the other end need to be able to grow not only on the CiviCRM but also the PHP-Skill side. Sometimes I think the development section really assumes that a CiviCrm developer/customizer is a php professional. That is maybe 1 % of the time the case. The Forums are full of proof.
This system would really evolve if interested folks/volunteers who like to make changes would be provided with a less crazy learning curve. Finally, it wouldn't be much. Know how to work with arrays, know how if-else statements work, how to get stuff in and out the system and how to easily debug everything from finding variables and template files - all related to CiviCRM. Then maybe hooks.
Greetings and Thanks
sondy
Hi,
Complex use cases and workflow are better dealt with hooks IMO, otherwise the admin insterface will even more look like the cockpit of a jet. It is usually cheaper/easier to do it that way (unless your need is one that is the same than someone else sponsoring the dev that is).
Have you tried our second book? http://book.civicrm.org/developer/ Should make the learning curve faster (as you seem to know already how to find variables...)
Gook luck
Hi,
the community - and for most - the developers do a great job with CiviCrm. It is a open source extensible system and I didn't want to complain about the possibilities to realize those extensions.
Right now I'm just not yet in the position to gather fellow students (volunteers), teach them "a apart" of the system and tell them what we need - you can use hooks, api, templates - here you have example work flows and here a great resource how php is used.
As soon I figured out how to turn work into "grunt work" - I'll share it with the community.
sondy
I agree that there is a very high learning curve for new developers getting to a level where they can modify core CiviCRM behaviour for their own uses or especially contributing new features back to the community.
Overriding files was an early way to customize behaviour that was designed to have a low barrier to entry - copy, make small or large and edits, and run.
ErikHommel, Eileen, Xavier (and I'm forgetting someone else, oh dear!!) made a lot of effort to standardize and extend the API in a new v.3 in order to ease the on-ramp for developers.
Hooks are another way that the core team has been attempting to make it easier for developers that was inspired by the success of the Drupal hook system.
Improving the documentation with more elaborate developer use cases is a good idea. The Drupal documentation team has developed some example modules showing best practices in developing modules, and CiviCRM could likely do better. Lobo has put together a bit of one with his CiviSchools module, but it isn't focussed on providing explanations and typical usages of lots of hooks. Some of the CiviCRM modules on drupal.org provide some good examples, such as the one for webform integration.
The developer training sessions are reasonable ways to ease people into customizing and developing for CiviCRM. There are also usually some sessions at CiviCon's that go through a bit of code that does interesting things. Perhaps I'll try to present something along those lines at CiviCon 2012 in San Francisco.
Thanks again for you feedback.
Hi Sondy,
Thanks for taking the time to explain your use cases. I have a client who would like to have capacity limits for breakout sessions within an event, which are currently subscribed to via a price set. This would allow the room capacities for parallel sessions to be different than the capacity for the plenary session in the biggest room. I've posted a link to this comment at http://forum.civicrm.org/index.php/topic,23116.0.html.
Note that my blog post is not about enhancing functionality for price sets, but about changing so that all financial information is recorded and processed via price sets. Sorry but there is no budget in this project for this sort of functional improvement. Thankfully, many similar initiatives have succeeded when someone puts in the effort to work with the core team to develop a spec and then raises money from the community via a Make It Happen initiative.