Adding Privacy and Comments to CiviCRM Notes

At Two Mice and a Strawberry, a client has asked for some customization of the Notes features for Contacts. It happens that these features may also be useful for other objects that use Notes, but at the moment we are primarily interested in getting this working for Contacts. In the CiviCRM forum, I had a short exchange with Lobo about the two primary improvements we're looking at:

1. Adding a "private" indicator to allow notes to be visible only to the note author (or otherwise limited based on logic within a hook).
2. Allowing comments to be attached to a note.

With Lobo's helpful input, we've come up with a target that lays the foundation for some improvements in all uses of notes, but starting out by only exposing these features for Contact notes. Here's a summary of what we're looking at, targeting inclusion in CiviCRM 3.3. We'd love to get some feedback from other CiviCRM users and developers, so please comment below if you have any input.

---

Note Privacy

This idea is fairly straightforward: mark a note as "Private" and it will, by default, be only visible to you, the note author. For the note author, this adds a "Private?" check-box to the note editing form; only the note author can mark the note as private. Module developers are able to override the default privacy rules using hook_civicrm_notePrivacy(): /**  * Custom logic to hide private notes  *   * @param array $note (reference) Associative array of values from   * civicrm_note for the given note.  */ function civitest_civicrm_notePrivacy (&$note) {     /* CiviCRM will check for existence of $note['notePrivacy_hidden'].      * If this value is not set, CiviCRM will show or display the note      * based on the default, which is to display private notes only to      * the note author.      * If this value is set, CiviCRM will hide the note if the value is      * TRUE, and display the note if the value is FALSE.      */     if ($note['is_private']) {         if ($my_business_rules_say_so) {             $note['notePrivacy_hidden'] = TRUE;         } else {             $note['notePrivacy_hidden'] = FALSE;     } }

---

Note comments

Note comments are simply notes having a parent_id of another note. Technically, this would allow for comment threads many layers deep, but at this point we're only planning to deal with the first layer, supporting only comments directly attached to a note, not comments attached to comments. To support this, we add a couple of new actions to each row in the "browse notes" table, as seen for example in the Notes tab under civicrm/contact/view. The new items are "Comment" and "View Comments", as seen here: The "Comment" action opens up the Note form in add/update mode, assigning the correct parent_id to tie the comment to the parent note. Since a comment will inherit the parent note's subject, this note add/update form does not expose the "Subject" field. In its place it displays the static text: "Re: $subject", where $subject is the parent note's subject. The "View Comments" action uses AJAX to call the new API function civicrm_note_getTree(), which fetches all descendants of the given note, in tree form, up to a given maximum depth. In this case we set a maximum depth of 1 because we're not supporting multi-level threaded comments here. Comments fetched via this action are then inserted into the table in rows directly below the note row. This action also replaces itself with the "Hide Comments" action, which can be used to remove those comment rows from the table and once again expose the "View Comments" action. You'll see in the above image that, although notes are sorted in descending order by date, comments are always sorted ascending by date. The idea here is that, while top-level notes are not necessarily related chronologically to each other, comments are directly related to their parent note and logically flow in a chronological order. Adding these rows does raise some question about the proper behavior of the sorting controls at the top of each column. Ideally, the comments would be sorted as a block with their parent note. We're currently playing with the jQuery DataTables plugin (which powers the sorting functionality) to see how close we can get to that. Presently the comment rows are simply removed when the table is re-sorted. Finally, note comments are displayed below the note on the "View Note" page. That's it. This adds note privacy and non-threaded note comments to Contact notes, and leaves room for adding similar features in places where notes are used. Please let us hear your comments below.

Comments

Hi,

If you want to avoid modifying the schema

1) Use entity_table and entity id ?
Beside creating a new parent id field, you might want to use the existing entity_table (civicrm_note) and id (the id of the parent node)

Con: you can't rely on the db for integrity, but you can't already with entity_table+entity_id

2) Use a tag "private" for privacy
This one is less convincing, but it would allow easily extend
- (eg. easy to extend different privacy level, eg. tag "private", "staff only", "advisors"...
- and use the tags in general to qualify your notes

if you don't go for tags, can you implement it as priority level (int) instead of a private (boolean) ?
public =0 & private = max(int)

Will make it easier to extend for custom implementations

I do like the idea of leveraging existing columns entity_table and entity_id. For one thing, it allows existing code to continue fetching only top-level notes, since no existing code is looking for notes with entity_table = 'civicrm_note'. This was one of my concerns, adding the comments to the notes table without throwing surprises at existing code.

This comment and others have valid points about handling privacy. We're trying to find a solution that works well, is forward-thinking, and doesn't eat up our development budget (considering both deadlines and hours). Will have to give this some thought.

I think this is a great addition, and would second Brian's suggestion to keep the "drilldown" UI for showing / hiding comments related to a note in the Notes listing consistent with other screens - e.g. Case and Pledge dashboards and search results. This approach gives you both consistency, and the fact that the "pointer" (right / down) model immediately allows the user to see which notes have comments.

Looks like a v useful improvement. Do you expect that the 'count' in the Notes tab will be just for Notes or for Comments too? ie if 2 Notes with 5 Comments will the tab show
Notes (2) or
Notes (7) or
Notes (2|7)

Seems like the focus of the Notes tab is Notes. In your example, showing "(7)" seems a little confusing, and "(2|7)" or "(2|5)" seems like too much information.

Can you think of some cases where users would need to know that level of detail at this level of view?

I guess in my mind each 'comment' is effectively another 'note' - ie it is part of a 'conversation' that is being recorded - and seeing a contact that says (1|3) v one that says (1) provides an immediate visual cue that there has been more activity in that area.

This looks great -- could be very useful. Couple suggestions:

Instead of using more > view comments, I think would be easier to add the right-directed triangle at the beginning of a note that contains comments, and allow the user to toggle to view/hide comments -- similar to how a case can be expanded to display activities in the case dashboard tools. This would reduce the number of clicks (from 2 to 1) and provide a visual cue to the presence of comments. In the examples above, perhaps only foo note would have the triangle, as it is the only one of the three with children comments.

Re: permissioning -- I agree with X+ that you should anticipate something other than just boolean logic for the privacy. One feature that would be great to anticipate would be if the privacy could be attached to groups -- either through the interface or via hook. For example, you might have an administrators group that is permissioned to view private notes (in addition to note authors). That could be done with the existing boolean logic, but it might be worth anticipating into the structure so that it's easy to configure multiple groups. Ideally it might be cool to have the end user select one or more groups that are permissioned for the comment. Maybe the comment is about financial matters and the user flags it as private, but selects the Finance Team, Executive Team, and Administrators to view it.

This is really the right way to go. Thanks for pointing it out. Good use of space, intuitive pattern, and best of all, consistency with other similar behavior elsewhere in Civi (as Dave pointed out in another comment).

As for privacy, it does seem from comments here that a simple boolean is too limiting for many users. A good start might be to use a select list that's driven by an option group in civicrm_option_group/civicrm_option_value.

Default configuration would have two options for the "Privacy" field: "Author only"(1) and "None"(0), with default behavior being to hide notes with any privacy level > 0.

At this point I don't think we would build the UI to configure privacy level options, but that could easily be added later under Administer > Option Lists or similar. That would give users the ability to add as many privacy levels as needed. Combine this with hook_civicrm_notePrivacy, and it becomes pretty flexible; it's not total integration with groups, tags, etc., but that at least becomes possible, more than in the original proposal.

Thanks for the ideas.

This is a great idea! Thanks for the work.

I'd just like to discuss the ordering of the comments. I think the assumption being made is this is like a blog, where the comments are part of a discussion. I'd like to question that assumption.

The two scenarios I have in mind are fund-raising and relationship-building. I could deal with someone over a number of months either seeking their support, or influencing them. Currently, I can either track this in a single note (very long sometimes) or a set of notes (each with the same subject, sorted in date order).

In both those cases, the most recent data is the most relevant. (If someone decides whether to support us, or changes their mind on an issue, I want to know their current opinion first, and then secondly how they came to that point.)

I suggest that the default ordering of comments and notes should be the same: most recent first.

Thanks again,
Ken

Within my organization, I would prefer to see private note permissions assigned to a group vs. author. I see that the two are quite distinct, but I can't think of a use case where such specific control is necessary, and I can see some downsides as admins come & go. I suppose the author would have to migrate their private notes to another user?

If this scenario is specifically needed, would it might make sense to start the "connection" at a higher level? That is, a number of civi users OR roles could be assigned to a contact. After that connection has been made, privacy can be reflected in any number of fields. I know this is much more complex, but it seems relevant.

It's more complex. If you really want it, I'd suggest you to invest some money on it, it will likely help the developer to be convinced ;)

+

+1 for groups based permissioning

 

Also, please add number of comments to the "View Comments" link so "View Comments (3)"

 

Looking forward to seeing this!