I agree that it would be quite complex to define great locking behavior for every single form -- which is why that's left out of scope for the proposed project. :) If I could restate the proposal, it's this: "The Edit Contact screen [and View Contact inline edit blocks] will claim and respect preventive locks. All other existing ('legacy') UIs will treat concurrency as they do today -- with the extra side-effect of interrupting anyone using the Edit Contact screen." To show this, I'll try to answer your points individually.

With edit in place (including at the single field level) and the new setvalue action on all the entities and civimobile that comes on the top of the not so edge case problems you already described, not sure the lock solution in the right answer.

There are a few options for those screens: they can do nothing and use the default lock-stealing behavior which will basically allow the edits to go through (as they do today) but which will kick-out anyone concurrently using the "Edit Contact" screen. Alternatively, the client could request a lock while rendering the screen. (For CiviMobile, this might mean chaining 'contact.get' with 'contact.lock'.) The subsequent AJAX calls ('contact.setvalue', 'email.create', etc) would not need any significant changes. Why? Because the contact-lock would be held on behalf of the user's session -- there's no need for callers to pass around or keep track of lock IDs, revision numbers, last-modified-times, etc.

The problem becomes even more complex with profiles that are used by the end user directly. What happens when a end user register to an event/make a donation/renew their membership/register to the newsletter...?

The lock has to take into account profiles (and weforms) as well and it can become explosively more complex (an event registration with a profile containing a name,+phone number+email+country needs to lock all these entities?

If the registration/renewal only deals with the participant/membership record, then locking is irrelevant because it's not within the scope of the lock system (which only covers core contact+address information). If there is a profile form which touches the contact, then the default policy (lock-stealing) is probably best -- it lets the constituent's changes go through and breaks any concurrent edits on the staff's "Edit Contact" screen.

And then what about creations? if I add a primary phone using a profile, what happens in the backoffice when editing existing phones? should the creation be locked, or should changing the is_primary be uncovered by the locks?

If the phone is part of a contact record that is locked, then the choices are the same: use the default lock-stealing behavior or block the creation/edit.

It is safe to assume you can't update all the edit forms/chanels, both within the core civicrm and even more with alternate interfaces (eg. the mobile interface, the xmpp...), so "legacy code" will exist and ignore locks.

Yes, definitely. Perhaps "legacy code" was too pejorative -- "default behavior" or "lock-stealing behavior" would be better descriptions.

Suggestion:

Wouldn't a monitoring framework answer better the need?

When I'm starting to edit/view a contact, it flags that I'm monitoring it. When someone else modify it, I get a notification (new phone number xxx for the contact yyy done by user zzz).

From a UI perspective, real-time monitoring is much better than locking. A good example might be Google Docs -- multiple users can edit the same document, and they don't have to do locks because everyone sees the same changes in real time. The problem is more with implementation: PHP kind'a sucks at letting servers push data to clients. That's not saying it's impossible, but the best techniques (eg "Comet") break the traditional HTTP request cycle, and many tools common to CiviCRM installations (i.e. PHP interpreters, firewalls, reverse-proxies) cater to the traditional HTTP request cycle. The technique is more commonly used with different stacks (like node.js).

I'm personally down with a "monitoring" approach if other folks find these consequences acceptable:

• The "Edit Contact" (and "View Contact-Inplace Edit") will make more AJAX requests and run more concurrent PHP processes -- possibly affecting performance with large userbases or slow network links. (Alternatively: This could be mitigated somewhat by introducing a major new dependency like node.js -- but that won't be available in many commodity hosting environments.)
• Firewalls and reverse-proxies need to be tested and tuned for compatibility. (In commodity hosting environments, the firewall policy is usually "take-it-or-leave-it".)
• If the hosting environment cannot support monitoring, then the feature must be disabled. (I'm not sure if it should be turned on or off by default, but it needs to be toggleable.)
• For legacy code (screens which don't incorporate monitoring -- ie most screens), there are no options for tweaking behavior. (By contrast, with a pessimistic lock, the default behavior of "ignore/steal/fail" could be a *policy* question which could change on a per-site/per-page/per-user basis using config-options or hooks.)
• The solution will mitigate problems when users are working at roughly the same time (ie an overlapping time frame of a minute); but if two users actually click "Save" within a second of each other, then the changes may unknowingly overwrite each other. (Unless, of course, we implemented optimistic locking on top of monitoring -- but then our project's total-cost goes up.)
• We need to draft a new plan/estimate.

My expectations: When the staffer submits the edited contact screen, he gets a warning. The warning is generated as part of the form validation, so nothing is saved to the database (even if the staffer focuses on different, non-conflicting fields). The warning would be coarse and would not highlight individual fields.

It's not a coincidence that the proposal involved neither automatic merging nor highlighting of conflicts. Either of those features would increase complexity by roughly the same degree -- i.e. they require retaining a copies of the old revision of the contact (and closely related entities), then performing field-by-field comparison across the entities, then communicating those details back to the user. If we did provide such fine-grained feedback, then we could reasonably say that "We have the best damned concurrency support on the market!" (Have you seen another product in this space with this functionality?) OTOH, the increased complexity would increase technical risks and probably increase costs. Is this a feature that justifies the risk/cost?

FWIW, in a "monitoring" approach, there's a similar trade-off between granularity and complexity/risk -- there, the choice is whether the monitoring system reports a coarse message ("This contact is being edited by another user") or granular message ("Field X has been changed by another user").

Last thought: If our goal is "to provide the best damned concurrency mechanism for web CRMs", then a fine-grained monitoring approach is a good design. If our goal is "to mitigate a rare issue with minimal cost/risk", then a coarse-grained lock is a good design.