I've been tinkering around with Statuses now (both D6 and D7) and I'm very impressed with the work you've put into the module. I read on the project page that you have plans for a new version for D7, and I was wondering what those plans are. I would personally love seeing an entity based system that utilizes rules and views for as much as possible. Perhaps core comments as well. That would make Statuses really become the ultimate social communications framework for drupal because it would be extremely flexible. Sort of like what Commerce guys did when they created Commerce.

What are your thoughts on this?

Files: 
CommentFileSizeAuthor
#7 statuses-entity-1311468.patch1.52 KBIceCreamYou

Comments

IceCreamYou’s picture

Component:Code (Features)» Documentation
Category:feature» support
Status:Active» Fixed

I haven't done the research yet so I don't know exactly what will be involved; the most I can say for sure is that probably most of the currently postponed issues will get in. It might turn out that no structural changes are needed at all. Entities are obviously interesting because of their ability to be fieldable, but I am unconvinced that there is any reason for status updates to have fields, and it adds a lot of overhead. I do want to look at ways to make the Micropublisher (sub)module more natural and fields are the obvious way to do that, but the most important quality is for the UI to be easy and intuitive for end-users, and adding additional fields complicates things.

Other than that I'm not sure what additional flexibility you think you're going to get -- Statuses already integrates heavily with Rules and Views (and 25+ other modules) and has a very robust API. And it's very lightweight. (Also the core comments module is a mess and totally incompatible with anything that isn't a node. Believe me, you don't want to go there.)

AdamGerthel’s picture

I understand your points, and I didn't think about comments on entities not being possible (#731724: Convert comment settings into a field to make them work with CMI and non-node entities). The reason why I stumbled across Statuses is because I had the following use case:

A micro publishing feed for a company intranet. Users should be able to tag (#hashtag), reference other users (@user) and post images along with their statuses/tweets. A "Status" should also be able to become a task for another user. So writing

"!ticket @user do foo, and also check bar"

would produce a "status" that has a checkbox for done/not done. It could also easily be filtered out using views to produce task lists for each user. This would be very usable for most company intranets, as it would become a social communications tool along with a GTD tool. The "!ticket" could be used in rules to alter an entity field value on the status, send an email, or similar.

My use case above is specific, yes. But having field-able statuses opens up to a LOT of possibilites.

If "private" statuses was an integer field on status entities, it would be easy to control through views and core permissions. It wouldn't actually even have to be a part of the statuses module, but it could easily be added by anyone with views/fields/permissions knowledge.

Entities already have core functionality in terms of field max length, input filter, field row size, default value, so all of that added functionality is already there.

Another (quite different) suggestion - which just crossed my mind is separating some modules from Statuses core. Mentions could perhaps be user references (provided by a hidden field on the entity, and perhaps handled inline via an input filter). The same could go for taxonomy terms / hashtags (term reference field). There are two modules (http://drupal.org/project/hashtags and http://drupal.org/project/mentions) doing this currently. However, those modules are in poor condition. However, there is a good point in having that type of functionality in separate modules, because they could then be used in other situations. URL's could be handled the same way. If Facebook style url's are desirable (import image and data from the actual URL, and show beneath post) this could also be handled by a contrib filter. Much like the way Media module has separated it's contrib modules for other media (youtube, vimeo etc.).

The main reason that I see regarding entities is for the fieldability, and the added versatility that comes with it. Thats also why I mentioned Commerce. Its predecessor Übercart was plug and play. It had similar integration possibilities to FBSS regarding rules and views. The difference between Commerce and Übercart is that Commerce is built using views, entities and rules. It's not plug and play the same way übercart is, but it's easy to integrate other modules with it, and also very easy to customize it, since it all uses core functionality (+ views, ctools, rules, entity API) that a lot of Drupal developers already know and use.

Another obvious advantage is easier integration with Features.

Like I said. Statuses is really a great module - and you've done an amazing job with it. I just wish I was able to use it on more projects, and I'm sure others do as well :)

IceCreamYou’s picture

Status:Fixed» Active

Regarding the !ticket concept -- there was a similar issue about that while ago, but I can't find it now -- it's not something I'm interested in adding to the module. This is a status update module focused on social networks -- it isn't and wasn't designed to be a ticket system, no matter how similar the workflow for some visions of ticket systems might be. The more the module moves toward something like a ticket system, the more it moves away from a status update system, and I don't want that.

If "private" statuses was an integer field on status entities, it would be easy to control through views and core permissions. It wouldn't actually even have to be a part of the statuses module, but it could easily be added by anyone with views/fields/permissions knowledge.

Statuses already has a privacy submodule that supports Views and core permissions. One of the main arguments against exposing that kind of functionality through fields, in my mind, is that I don't want people configuring it in incorrect ways and creating bad user experiences, broken security models, etc. (not to mention the support requests!). This is somewhat against the usual Drupal model which tries to expose as much functionality as possible with the caveat that it's the developer's fault if they do something stupid. Statuses is explicitly designed with something of an opposite idea in mind: it should always require as little configuration as possible, while remaining as powerful and flexible as possible for its central purpose. That doesn't mean I don't want to expose powerful functionality to site administrators -- it mainly just means sensible defaults -- but anything that lets site administrators fundamentally change the user experience to make it do something other than what it was designed to do requires an extra level of scrutiny. It's a source of pain for everyone: site administrators don't get quite the functionality they want because the module wasn't designed for it, site users don't get quite the experience they want for the same reason, and I get a significant support burden from people trying to push the limits.

There are several frontiers where I do see Statuses innovating that are very important to me, but these innovations are things that make for a better end-user experience of status updates for a social network, and generally not things that turn statuses into nodes. Those frontiers include user-controlled privacy settings, friend-list building, and audience control; aggregating all site activity into a single stream; automatically "bubbling up" more important activity in the streams; adding a "previously loaded" marker in the streams and making streams scroll infinitely; adding more integrations; automatically suggesting #hashtags and @mentions as you type; and pulling status updates from other sites like Facebook and Twitter into the Drupal status stream. These are the kinds of things that are good candidates for the 7.x-2.x branch of Statuses.

Another (quite different) suggestion - which just crossed my mind is separating some modules from Statuses core.

Statuses core is already very modular; you can essentially enable only the features you want. Each module is specifically designed for the optimal experience with Statuses specifically. When people download Statuses, they are looking for a complete package. I have heard time and time again how confusing it is to require separate modules for status updates and site-wide activity streams. Believe me when I say that separating out more components into separate modules -- especially ones that aren't widely applicable in other contexts -- wouldn't help anyone.

Mentions could perhaps be user references (provided by a hidden field on the entity, and perhaps handled inline via an input filter). The same could go for taxonomy terms / hashtags (term reference field).

There is a lot of unnecessary overhead associated with a solution like that; on top of all the extra completely irrelevant code, you have to worry about hiding the field from view, what happens if administrators add/remove/change the field you're using, etc. #hashtags and @mentions are already their own lightweight form of taxonomy term and user references.

However, there is a good point in having that type of functionality in separate modules, because they could then be used in other situations.

I generally agree with the philosophy that a modular architecture is a good idea, on the premise that it distributes the maintenance burden and allows tools to be used flexibly for other purposes. However, there's a balance between building something that actually works and building something that relies on a lot of other tools. One of the fundamental design goals of Statuses is to provide very comprehensive, smooth, and most of all working integration with every other layer of Drupal and contrib (that makes sense). If core Statuses functionality depended on a lot of other projects with a lot of moving parts, it would be very hard to keep them all in sync and working together flawlessly.

If Facebook style url's are desirable (import image and data from the actual URL, and show beneath post) this could also be handled by a contrib filter. Much like the way Media module has separated it's contrib modules for other media (youtube, vimeo etc.).

Attaching various kinds of media to status updates is a big challenge to Statuses and to the general philosophy I've explained in this post. That's currently handled by an external module for the D6 branch and the code to do it is almost as large as Statuses core. It's an obvious area where relying on other modules would be a good strategy, both for the power and flexibility other modules provide and because it could allow combining or moving attachment data in more natural ways. However, other modules tend to be written explicitly for node integration (or increasingly, in D7, for entity integration). If I can figure out how to manage media integration well, that would be a very powerful argument for making statuses entities. However there are a lot of factors to consider, not least of which is that entity fields are designed to appear on one very long form, which is a completely different approach than Statuses takes. I'm not sure how feasible it would be to format arbitrary fields in a friendly way. This is a big yet-to-be-determined question -- probably the central one for 7.x-2.x.

Thats also why I mentioned Commerce. Its predecessor Übercart was plug and play. It had similar integration possibilities to FBSS regarding rules and views. The difference between Commerce and Übercart is that Commerce is built using views, entities and rules. It's not plug and play the same way übercart is, but it's easy to integrate other modules with it, and also very easy to customize it, since it all uses core functionality (+ views, ctools, rules, entity API) that a lot of Drupal developers already know and use.

Commerce is quite a different beast. There is a huge variety of needs when it comes to building any sort of store or checkout process online, so it made sense to make it more abstracted. On the other hand, there is basically only one status update experience. There are things you can add to it, but it all comes down to the same basic workflow.

Now, there is a higher-level question here, which is that the experience of submitting an AJAX form and seeing your content appear in a stream immediately is much nicer than the usual node workflow in many cases. Would it be nice to provide an infrastructure for AJAX-y forms of any type, even those that are not necessarily based on status updates at all? Yes, possibly. That would be a very radically different route than Statuses has taken to date. It's not a route that I'm ruling out, but it might turn out that this would turn into a separate module that encompasses Statuses, i.e. would allow showing the status update form inside of a separate infrastructure. This would change the purpose of the module from a focus on statuses to a focus on creating content in general, and I think address a lot of the things you're getting at in terms of flexibility to push the limits of the module. On the other hand, it seems to me there are a lot of very status-focused features that really haven't been built yet, and maybe it makes more sense to build those first than to dramatically expand the scope of the project.

Another obvious advantage is easier integration with Features.

There is not anything in Statuses to bundle in Features ATM; it doesn't provide any interface for site administrators to create "components" of any sort. Its functionality is currently almost entirely end-user-facing.

I'm going to mark this issue active for awhile. I'm curious what other people think about the philosophy I've laid out and the features I've suggested.

vinoth.3v’s picture

+1 for making statuses as entity. so site builders can attach any fields to statuses. It would be great feature.

please have a look at "privatemsg" module, which doing the same.

Michelle’s picture

I think making them entities is a good idea. Entities do not have to be fieldable so you could go that route. Or make them fieldable but not use fields on them from within the Statuses module itself. Basically, I think you should make them entities because Drupal is moving towards having that be the "atom" of content and gradually getting to where there's somewhat of a universal API when dealing with entities whether in core or contrib. Sure, it plays nice with other modules now but, if they were entities, you'd get a lot of that for free. (In theory; obviously entities are massively in flux right now and the kinks are being worked out.)

Michelle

IceCreamYou’s picture

Yes, one option is to make statuses entities, and then make a submodule (as a replacement for FBSMP) that makes statuses fieldable. The challenge comes mostly from the visual presentation of fields on the status update form, and Michelle is right that there are some benefits that come from entities even without fields (most notably Rules integration). This is most likely the route I'll take. It might not even require any significant changes to the module as it is.

IceCreamYou’s picture

StatusFileSize
new1.52 KB

In fact, attached is an (untested) patch which adds the beginnings of Entity and Entity API support. This really belongs in a new issue, but leaving it here for now since I haven't decided this is definitely what I want to do.

zilla’s picture

very exciting stuff...any plans to roll out a new beta/alpha of this entity implementation in spring 2012?

IceCreamYou’s picture

I have several large projects going on right now so it's hard for me to project a timeline for when this might be done, but obviously the more testers and contributors the faster it will go.

Right now the roadmap is:

  • Fix the integrations in 7.x-1.x and get a release out.
  • Create a 7.x-2.x branch where statuses are entities and a submodule adds field support.
  • Create a 7.x-3.x branch (possibly even at a different namespace) where the status-update workflow can be used with any kind of entity, including nodes.

However how soon and even whether this gets done depends a lot on the other projects I'm working on and whether other people step up to help.

fenda’s picture

I have a project in sandbox that suggests a possible solution to entity walls.
http://drupal.org/sandbox/fenda/1471568

pribeh’s picture

If I can figure out how to manage media integration well, that would be a very powerful argument for making statuses entities. However there are a lot of factors to consider, not least of which is that entity fields are designed to appear on one very long form, which is a completely different approach than Statuses takes. I'm not sure how feasible it would be to format arbitrary fields in a friendly way.

Would it be nice to provide an infrastructure for AJAX-y forms of any type, even those that are not necessarily based on status updates at all?

I have to say that although FBSS for D6 worked quite well for the projects I used it for during the past few years, progressively I find myself looking for another level to the status paradigm. I've been studying user interfaces for awhile now and I think your statement above hits at what Drupal (along with various web platforms) lacks. To me the big social networks deliver an incredible user experience for their members largely because of the refined "interface" and content structure their users are exposed to. The effortless form submission process along with the streamlined structure of content are both crucial elements to the social network. For D6, the FBSS module provided me the first glimpse at what a refined (or simplified) interface and content structure could look like within Drupal. That said, I've been trying to theorize a flexible ajaxy (form) interface for:

  • More than just statuses. Think of event submission "like" on Facebook. Users are often permitted in different places to input a status, then attach a date, hit submit and voila (you have an event). Just as users can then add other elements (such as images, locations, etc.) think of statuses that could eventually turn into articles or perhaps building blocks for other pieces of content (this could be provided via an embeddable status of course).
  • Posting content to different back ends. This could be another Drupal site (via services and json) or perhaps just an external service such as Facebook, Twitter or another framework (think nodejs' express coupled with mongodb). This could be done depending on what type of content (what attachments or fields) users are submitting or even whether the user wants the content to be submitted to one place or another ("check this box to post to Facebook").

Just as Views 3 finally allows for querying of different services (Solr or Flickr), Drupal 7 & the Services module (included in D8 from what I hear) allow you to build new types of web/mobile apps with backbone.js (or other such front end frameworks), the future of Drupal development (let alone all web development) seems to be interoperability. Building sites or apps off of Facebook, Twitter or within mobile devices or across several websites is becoming key for the survival of so many web projects these days. This is due to project complexity and server loads as well as competition (from other projects) and ADD (of our users).

So, I hope my rant helped to explain how by making statuses work as/with entities/fields, and creating an ajaxy form with an API to post various types of content to wherever (perhaps a couple different places to start), we could push (albeit subtly) a transformation in Drupal content submission workflow and interoperability. This could be the case for various types of projects whether they be social networks or simply any web app with a social or refined UX requirement.

Of course, I don't have much insight yet as to where to begin with an undertaking such as this. Does this resonate with anything you were discussing/thinking Ice?

IceCreamYou’s picture

I don't think the "Posting content to different back ends" point applies here -- that would be a lower-level paradigm than what Statuses deals with -- but you're preaching to the choir when it comes to streamlined user interfaces. I know conceptually how I want it to look and feel, but it's technically difficult to develop a modular system that integrates with the rest of Drupal and meets these usability goals. In D8 the Node module will be optional, so maybe we'll see more supportive core abstractions at that stage. In the mean time entity-based fields are a place to start from the perspective of richer, AJAX-driven content. However you hit on something even more central, which is that content is content and users don't care about content type, and there are deeper complications to making a site (appear to?) use a single "content type" so that users can focus on the content itself and not the method of input.

AdamGerthel’s picture

The effortless form submission process along with the streamlined structure of content are both crucial elements to the social network. For D6, the FBSS module provided me the first glimpse at what a refined (or simplified) interface and content structure could look like within Drupal

Most of this can, and should, be completed in the theme layer in my opinion. A lot of the "smoothness" and UI elegance can be achieved in Drupal using for example jquery plugins. Some things, however, are nested too deep into Drupal to be able to handle on the theme layer. Ajax forms is one of those things. Drupal has support for that though, and there are contrib modules that handle a lot of those things as well (such as Ajax Comments).

The problem that I see (and the reason why I started this issue) is that the D6 version of FBSS has a lot of that stuff built into the module. That's great if you're looking for that exact functionality, but not so great if you need to do things in a different way.

We're actually building a small social "wall" right now for a small intranet. It will consist of a "wall entry" content type, Ajax Comments, a view to display the wall entries + comments (with Load more), a node form above the view (using Form block) and some custom code to handle ajax submit of the node form to display the entry in the view without reload. Since the "wall entry" is a node, it's easy to add the ability to tag content or add media fields. All of this is actually quite simple to make, and it has a lot of flexibility.

My example is nowhere near the complete functionality of FBSS, but it's an example of the strength in Drupal, and the reason why I don't think modules should handle too much UI "goodness", and especially not design.

mathankumarc’s picture

How about automated tests.

I think tests will be a mandatory one(May be not)

IceCreamYou’s picture

I would like to have tests. Not mandatory for release, but certainly helpful for stability.

IceCreamYou’s picture

Category:support» task
Status:Active» Postponed

Let's revisit this after 1.0.

adamtong’s picture

Right now the roadmap is:

Fix the integrations in 7.x-1.x and get a release out.
Create a 7.x-2.x branch where statuses are entities and a submodule adds field support.
Create a 7.x-3.x branch (possibly even at a different namespace) where the status-update workflow can be used with any kind of entity, including nodes.
However how soon and even whether this gets done depends a lot on the other projects I'm working on and whether other people step up to help.

When will the 7.x-2.x branch comes out? I really want to have the "add filed" functionality as the FBSMP is not doing the exact thing i want.

IceCreamYou’s picture

Since it came up in #1787930: Support Flag 7.x-3.x, here's my status:

I am overcommitted and putting all my time into other (new) projects, which means I am not going to be able to contribute significant new development energy towards this project in the foreseeable future. If development energy doesn't come from someone else, there probably won't be a 7.x-2.x branch. It's possible that in a year or so things could change when we get to work porting Statuses to D8, but the porting effort alone is going to be a huge task given all the API changes from D7 to D8.

I'm going to be around in the issue queues and will fix smaller bugs and of course security issues if only out of habit, so I am by no means suggesting that I am abandoning Statuses, but I simply do not have the bandwidth to work on another big new re-structuring (nor even to dive into most of the thornier issues in the queue at the moment).

I don't have any great solutions. Maybe mathankumarc is more available than I am. I would like to see more independent contributions, and maybe even a new co-maintainer depending on what mathankumarc thinks.

In the short term, I'd like to push a new beta out this weekend just to get the majority of people using the most stable version since there haven't been any commits in 3 months.

adamtong’s picture

Hi IceCreamYou,

Thank you so much for your effort and contribution to this module. Though it is very pitiful that there may not be 2.x branch in near future.

BTW, will the entity patch in #7 included in the new beta (stable version you mentioned)?

I have tested this patch and work fine for status. However, it is not for status comment (fbss_comment).

If I applied #7 patch to turn status to be entity, Flag 3.x working well with status and actually no need the sub-module of "Statuses Flags". If the "Statuses Comments" can be entity, I think we don't need the "Statuses Comments Flag Integration" sub-module as well. (am I right?)

I really hope the new version of statuses can fully utilize the D7 entity feature and make it more flexible and powerful.

Thank you very much.

eidoscom’s picture

I'm with adamtong that if is not going to be a new 2.x branch it is a must to convert statuses to entities for take direct solutions based on entity API itself. I think that a lot of problems will solve with this.

IceCreamYou’s picture

BTW, will the entity patch in #7 included in the new beta (stable version you mentioned)?

Pending agreement from mathankumarc, the new beta will probably not contain any new commits. It's actually pretty stable and I want to keep it that way.

As #7 notes, that patch is an incomplete solution that really belongs in a new issue. I'm glad to hear it solves some problems but it may also introduce new ones. I think it's reasonable to get entity support in the 7.x-1.x branch as long as they're not fieldable.

mathankumarc’s picture

@adamtong, eidoscom

The patch in #7 is like a proof of concept, it wont take more time to write a patch for fbss_comment(Maybe we can use the comment entity which is available in core now). The problem is we need to change lot of things in the module and that will take time. I'm accepting the fact that If the Statuses are converted to entity we need not spend more time on the integration with other modules(Flag and Rules submodule not needed anymore, views integration will become very simple) and as Isaac mentioned it would be very easy to port the module to D8.

Give us some time to get the stable release of 1.x.

@Isaac
Will get back to you on the availability.

eidoscom’s picture

You have all the time you need, as I said other times I offer mine time to test but I can't help on hardcoding :S. I have skills to code in PHP but my understanding of the module at code level is a limitation at the moment. I worked with statuses, configuring, testing and using it for almost a year if not more.

If Isaac or mathan tell me what I must do, I can code what you need letting you check and revise what I code. I can spend time on this as I need the module working with a big project that I'm involved.

Thanks!

adamtong’s picture

Hi all,

I am glad to know that you all agree to make status to be entity! I am not a programmer so i cannot help in coding. But I am much willing to spend my time on testing and give you feedback at no time.

Thank you very much.

adamtong’s picture

#22 @ mathankumarc

Using core comment module for status comment is a good idea. However, I don't know whether the ajax update is available for it. I am using another module for light weight comment (http://drupal.org/project/reply).

This module is an "reply bundle" entity and so can add fields to the "reply bundle". The reply bundle then can be a field to attach to any content type.

I am not sure whether this module can help.

Thank you.

IceCreamYou’s picture

@eidoscom: Thank you for your offer to help. I just updated #1551542: [META] Release Plan and the issues we need to work on are listed there, so that would be a good place to start. If you need help with coding the solution to a specific issue, you can ask in that thread.

@adamtong: The one issue that is really ready for solid testing right now is #1342628: Allow showing statuses in the current user's stream and in the current user's groups' streams - it's not on the release path but testing is always appreciated.

@mathankumarc: I think using core comments as entities would require completely rewriting the comments submodule and would create more problems than it's worth, in addition to creating an unnecessary dependency.

IceCreamYou’s picture

Also, I created a new issue for Statuses-as-entities: #1912970: Make Statuses into non-fieldable entities

andypost’s picture

@IceCreamYou Suppose you need to look at #731724: Convert comment settings into a field to make them work with CMI and non-node entities where we trying to introduce a comment field

IceCreamYou’s picture

@andypost: Worth looking into for D8, but it would still require completely rewriting the Statuses Comments submodule. (And we would still need a Statuses Comments submodule.)

adamtong’s picture

HI all,

I just found a very nice open source script for Facebook Wall. I found it is lightweight and with good JQuery effect. Will it be possible to easily rewrite the statuses module if the script are provided?

Please see the following links:

Script download and documentation:
http://www.9lessons.info/2011/05/facebook-wall-script-with-php-and.html

Live Demo:
http://wall.9lessons.info/index1.php

IceCreamYou’s picture

Will it be possible to easily rewrite the statuses module

It is taking all of my willpower not to make a joke about rewriting the module for WordPress! :)

The Statuses suite is ~18,000 lines of code, including comments/whitespace. That script is about 100 lines of code and contains serious security errors. Statuses' complexity is due to integrating with Drupal and other modules, providing a lot more functionality, and providing a comprehensive API (and related documentation). The few remaining problems that exist are due to certain complexities of Drupal, which that script certainly doesn't address.

Anyway, the module doesn't need rewriting. It is very nearly stable and works well as-is for almost all use cases.

vinoth.3v’s picture

Statues It is a great module and it has lot of drupal specific implementations and integration with other module. We should keep going with that, + we should use the new power of Drupal I mean Fieldable Entity.

So the first step will be,

add declarations for statuses entity info and define helper functions. Sill we can use the current table for status as the entity base table.

One big place where we should take care of is when saving and loading the status entity. Instead of direct update, we should call entity save or load functions.

With the help of Entity API module dependency we can easily get ride of custom integration code for views.

This sandbox project "Entity wall" has the basic modal for this.

I would like to see the entity statuses in action, Of course, I can contribute to it too.

Statuses Comment will be next step.

To make these all process simple, we should remove other module integration related codes from core module and keep the core module simple and beautiful. all other integrations or modifications should go to appropriate sub modules, so user can enable or disable extra features only if they needed.

adamtong’s picture

Hi IceCreamYou

Sorry for asking the #30 question. my intension is just want to give you more idea and want to help. As i am not programmer so i don't know how complicated and how difficult to write this module. Anyway, thank you so much for your effort in Statuses Module.