Following #1549442: Use Message API for Activity stream functionality and #1551368: Use Message notify for email notifications, it's time for a more detailed plan on how to use these modules.

The proposed Commons notification system allows users to receive updates in their activity stream as well as by email. The currently proposed verb for creating a subscription is to "follow" but this could change to "subscribe" or something similar.

Users can configure their subscriptions in a single UI, and subscribe to taxonomy groups, comments and edits on individual nodes, as as users. Individual entities that the user can subscribe to have ajax toggle links .

Correction: The tabs in this screenshot actually correspond to Flags, not message types.
Commons notifications UI

Overall, this breaks down into 3 kinds of functionality:

- Generating messages for events & showing the right messages in the right context (e.g., site wide stream vs group-specific stream)
- Allowing users to subscribe (Flag) to nodes users, and taxonomy terms in a given vocabulary
- Sending emails to subscribed users

Proposed breakdown of functionality across modules:

Commons_follow_* - Underlying modules for flagging the subscription.

- Exported flags to flag nodes, groups, users, taxonomy terms
- Config to limit flagging of terms to specific vocabularies (Flag may support this already)
- Foundational code in place at http://drupal.org/project/commons_follow_users http://drupal.org/project/commons_follow_group http://drupal.org/project/commons_follow_node http://drupal.org/project/commons_follow_term

Commons follow

- Filter a view of content based on content, users, groups, terms you're following
- Filter a view of messages (activity stream) per:
-- Related group (likely determined by joining against the nid being acted upon, in part to simplify when a node changes groups and there are existing activity messages about that node)
-- Relevant node being acted upon (versus group?)
--Acting User (Store in message field)
-- Site-wide
-- Post mvp: per-topic

Commons_follow_UI (Depends on commons_follow):

(Likely one central commons_follow_UI module that the commons_follow_* modules can extend.
- Central UI to manage subscriptions (stream and email) that are defined by other modules.
- Other modules add controls to that UI via an API - Other modules likely define:
- - Message type(s) that should be visible in the UI. Generates a tab (likely QuickTabs) for each one
- - Embeds a View (the control UI) in each Quicktab
- A variable specifies the default view to use for these tabs (initially one View for all tabs. Could later support different views for different types of subscriptions)
- Other modules can extend the specific view via export alters. (Eg, radioactivity to add "Last Activity" column)
- Business logic to disable the unsubscribe link on an entity when an existing subscription overlaps with the current entity (e.g., user is subscribed to a taxonomy term assigned to the current node: that user can't follow or unfollow the node, because she is already subscribed via the term subscription)

Commons_streams (depends on commons_follow)

- Define stream message types (see below) - Store acting user and target entity.

- Batch generate messages for past events up to a certain period back (e.g., for upgrading sites)
- Support TimeAgo to update timestamps for messages (http://drupal.org/project/timeago) (Also simplifies showing an "Ago" format in cached content)
- Potentially: Index activity stream messages via ApacheSolr to speed up entity-specific streams (e.g. group)

Commons_notify (Depends on Commons_streams)

- Define email notification message types
- Relate each message type to the corresponding type defined by commons_streams (e.g., email notifications for the "comment_insert" message type)
- Store subscribed/unsubscribed for mail delivery option. Perhaps another (internal) Flag?
- Queue & send email notifications based on subscriptions defined in Commons_follow
- Integrate the unsubscribe from email control into the Commons_follow UI (likely by adding a field to the view)

Initial list of message types

For the Commons MVP, we'll generate a message for the following kinds of site activity:

- Node is created
- Wiki Node is updated
- New Comment
- Content is +1'd
- User follows another user
- User updates profile

Example notification event

Example notification event:

Bob subscribes to:
- NYC (group)
- Cakes (taxonomy term)
- Jane (user)

Jane posts "Checkout these great cakes in NYC":

* Commons_streams: Create the stream message
* Commons_notify: Queue this action for processing based on existing Message templates created.

* Process the queued event - Find all users subscribed via (all defined subscription types):

* Uids by group
* Uids by Topic
* Uids by Person

* Queue jobs to send message to all UIDs subscribed

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

ezra-g’s picture

Issue summary: View changes

Adding message types

amitaibu’s picture

Coming here from #1710696: Sanity Check on Commons/Message_Notify Plans, the issue is actually not much with Message notify, but on the custom subscription and query builder that is needed, and the above use case makes it an interesting one...

Some general suggestions:

  • Mark the subscription in a single way -- Flag or Field, but avoid both as it will only complicate the query
  • That's arguable, but maybe instead of multiple entityFieldQuery, maybe consider one big nasty, hardcore db_select() query with all the needed joins. Needs benchmarking...
  • Provide a Drush command that can process a range of users and send the email and maybe one to normalize the message stream. That Drush command can be invoked by crontab. This is because hook_cron() might not be able to process so many emails at once, better to try and process every 5 minutes

Other than that -- good luck :)

ezra-g’s picture

Title: Proposed Architecture for Streams, Email Notifications » Activity stream & Email notification system
Status: Needs review » Active

Thanks for the input.

This is because hook_cron() might not be able to process so many emails at once, better to try and process every 5 minutes

Yep, that's why I proposed using a queue . A drush command to process items in the queue would definitely be useful.

It sounds like the plan above is inline with your feedback - thanks for weighing in.

lightsurge’s picture

All sounds good to me as to what sort of stream messages / notifications to have and how to configure them.

Only thing I would say is that time spent wrangling out how to create a mechanism for dealing with overload in outgoing email might be better placed working out how to digest information into as few emails as possible and therefore avoid the overload.

I think everyone's been in the situation on gdo where we're subscribed to a thread, it becomes popular, and suddenly what was once useful occasional emails becomes dangerously close to feeling like spam in our inbox as it spirals off into quick-paced discussion. Not only that our mail exchanges also start to get suspicious it is spam. It's for that reason in particular that imho I would have avoided instant, per node / per comment notifications unless a user expressly identifies that they want this for the entity they're following. It really seems like in this day and age we need to clean up our email outgoings in a major sort of way. Plus I can only think of a few occasions where I'd want instant notification, other than those it's just nice to be occasionally kept updated.

So I really liked the idea of digests in Commons 6.x, and it's for this reason that I patched simplenews to do this the same thing that 'Activity Log Email Digests' module was built to do but didn't seem to work at the time. It actually worked out fairly painless to send out Views formatted, user-argumented queries of the streams to each user filtered to a specific i.e. daily/monthly etc period, and I believe the functionality for doing this in Simplenews has now been improved for 7.x. #700308: Allow separate theming of each individual outgoing email instead of caching the prepared newsletter

It wouldn't take much change to the screenshot here apart from to have, under 'Send email', a dropdown to select either 'Add to digest' or 'Send instant notification' or 'None', with perhaps 'Add to digest' as default. And then a user could define in their profile when they want to receive their digest.

lightsurge’s picture

In terms of queuing, would http://drupal.org/project/elysia_cron be a better option than drush to limit additional server config?

lightsurge’s picture

Issue summary: View changes

Example notification event.

ezra-g’s picture

Issue summary: View changes

Update with reference to commons_follow_*

gmclelland’s picture

The Flag module also has 7.x-3.x branch with fieldable flags that might could be used to indicate how you would like to subscribe using a field on the flag. -Just a thought

crimsondryad’s picture

I'm for using elysia_cron. That module looks like it has a lot of nifty features and I know using it would make site config more granular for me. And exportability via ctools is definitely a win.

lightsurge’s picture

This invision Commons prototype is a prime example of notifications gone wrong. Has anybody else who's commented on the prototype seen their inboxes full of needless notifications? I say needless because I don't need to be informed of comments posted just seconds/minutes apart on the same website.. what if I'm watching umpteen different websites, I shouldn't have to be reading so many emails just to read through what's been going on at each over the day. Crazy.

To be fair the invision website seems to default you to receiving notification on every comment thread that's opened on the prototype, but also in fairness that gives you a reasonable idea of the level of notifications on a Commons website, unless users did the reasonable thing and made the effort to switch them off completely.

Notifications can be either useful or irritating.. to me, g.d.o notifications are currently irritating, invision notifications are very irritating, and currently Commons notifications are planning on being about equally irritating as g.d.o notifications. Activity log digests (working) were actually useful - not irritating at all.

You can tell this is coming from somebody coming off from holiday and wading through the inbox ;-)

crimsondryad’s picture

@lightsurge I get that totally. And the thing is, it depends on the topic ( ie, not a global setting ). On one group, I might be intimately interested in every single thing that gets said. Another group I may just be lurking in and kinda keeping an eye on. The question is, how do you get notification configuration in front of people so they know and/or bother to change it? They're more likely to just set a rule on their mailbox to ignore it, which basically defeats the entire purpose of notifications.

There was a study done that says 95% of people don't change their MS Windows configuration because "Microsoft must have known what they were doing", even when the example was that a developer had put the setting on entirely by accident. Whatever the default is, most people will just blindly use it.

I would be more in favor of a daily digest by default with some way to tell people how they can adjust it ( actual users, not site admins...though I guess in some cases the site admin would want the ability to force a certain level to avoid loading up their server ).

Hope you had a great holiday!

ezra-g’s picture

@lightsurge, Thanks for weighing in here.

Did you notice that you could chose to receive updates only in your activity stream and not by email? It's true that with the current design you'd have to specify this on a per-subscription basis. As an improvement, would you be more satisfied if you could specify a default setting for your account so that updates always went to your activity stream when a new subscription was created? That seems like a reasonable addition to the 3.0 MVP.

I'd definitely like to see digests functionality added to Commons 3.x, but realistically plan to prioritize other aspects of the build above that feature as part of the emphasis on MVP. I would absolutely welcome patches to help make email digests happen sooner - Amitaibu has done some work around Message + Email digests that might be a good starting point.

I've filed #1743014: Support activity digest emails so we can be sure to track the digests feature for 3.x.

lightsurge’s picture

Did you notice that you could chose to receive updates only in your activity stream and not by email?

Actually no I didn't :-/ I suppose I assumed it would work the same as current Commons whereby if I'm a member of a group I get that group's updates in my stream. Will it be possible to force users to follow updates in a group? Aren't there a lot of use cases like mine where a site owner wants to force users to have updates in their feed, for example if group membership is mandatory/automatic, users are administratively added?

I was just about to ask why a user would join a group if they didn't want updates in their feeds and I realise I may have misunderstood the whole nature of this.. ouch, looking at it properly as I didn't before, there's no sign of group access in create group, or requesting to join in group directory etc, just 'follow'.... won't groups still be able to be moderated, invite-only or closed? Can't believe I didn't notice that this was absent in the MVP and the prototype!! Us even using it relies explicitly on that.

As an improvement, would you be more satisfied if you could specify a default setting for your account so that updates always went to your activity stream when a new subscription was created? That seems like a reasonable addition to the 3.0 MVP.

That sounds great and at least it would pave the way for digests in the future :-)

ezra-g’s picture

tere's no sign of group access in create group, or requesting to join in group directory etc, just 'follow'.... won't groups still be able to be moderated, invite-only or closed?

By default, groups are unpublished so that they can be moderated by site administrators. The proposed design optimizes for increased participation for groups that are public - Since they're public, there's no reason to provide the obstacle of first joining the group to post content into it.

However, there's no reason you couldn't have a group be private or use moderated membership.

lightsurge’s picture

That makes sense about public groups, that'll be much better without the barrier.

However, there's no reason you couldn't have a group be private or use moderated membership.

That's a relief, I'm glad that's still in the plan.. In fact, I see mariomaric's comment now in the prototype https://projects.invisionapp.com/share/FW3OD3SQ#/2402312 seems others were more zoned in on group access absence than I was initially ;-)

ezra-g’s picture

Note, the architecture conversation around Message module and *displaying* Messages continues at #1762558: Architecture for "Your activity stream".

ezra-g’s picture

Issue summary: View changes

Adding a node about commons_follow_ui

ezra-g’s picture

I updated the proposal slightly to move the views integration for "user is following" for messages into commons_follow.

Andrea C’s picture

Hi everybody

I would like to give a little contribute to the discussion. In my experience, a notification message is not only a notice of something new, but sometime can be also a reminder of something to do. Sometime a post deserve to be re-read more carefully later when you are not busy as when you receive the notification. Or sometimes the post contains a task you need to complete.

Currently, with Commons 2.x, I use the emails to keep track of the post that I need to take care again, deleting or archiving the notification emails of read posts and leaving in the inbox the notification email of the ones I need to work on.

Currently, with Commons 2.x, you cannot achieve the same result with the Stream page because (may be I’ve done some configuration mistake …) only the new content is reported there and not the related comments; looking only at the Stream page I’m not able to know that a new comment has been posted on a subject I’m following.

New comments are reported under the page “Home”, but only for the last three post of each group. The new comments on older post are lost.

I like the way new comments are notified in the Drupal.org dashboard because it let float the related post on top of the list, and the new comments are marked in red. But still, there is no way to mark it to as something to work on later.

The page that better summarize this behavior is, in Commons 2.x, My Stuff -> My Content, in which is possible to mark a post as read, but not as unread (and then, if you read the post from another entry point, this page seams to not recognize the page as read) .

Conclusion: I think it would be nice to have a page, a stream, that summarize all the new post and also the related comments, may be like the “Your post” in Drupal.org Dashboard, with the possibility to mark a post or a comment as “To do” (and then “done”), with a second page (or a filter) that show only the page marked “To do”

Andrea C’s picture

Issue summary: View changes

Moving functionality to dedicated commons follow module.

thomjjames’s picture

Hi everyone,

Sorry I'm a bit late to this thread, only just found it.

I developed a module a few years back for my own Drupal community site (http://drupalsn.com) which could be a good fit for the user notifications part here.
It's called User Notifications: http://drupal.org/project/user_notifications.

Key points:

  • it uses Rules to trigger notifications so it's very flexible to all kinds of events & integrates with many modules right out the box (flag, flag friend, Facebook Style Statuses, and more)
  • It can handle sending notification via different channels (email, PM, SMS) as it can plug into the messaging API modules (http://drupal.org/project/messaging)
  • Has per user settings so users can disable notification/emailing per trigger event
  • Built with expansion in mind so has a number of useful API & alter hooks

It's currently only available for Drupal 6 but if there was some more interest in using it in Commons I could & would fast track (with hopefully some help :) ) a Drupal 7 version!!!

Were it needs work:

  • Code clean up & official release
  • Drupal 7 release
  • Full Views support
  • Full documentation
  • Document how provide multilingual notifications

Maybe I'm too late to this one or it's not quite what's wanted but thought I write this anyways
Cheers
Tom

ezra-g’s picture

Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Adding a correction.