Nested editables: macro filter tags (first: token_filter)

Let's begin with token_filter. I pushed code to the Edit module that prepares it for this: http://drupalcode.org/project/edit.git/commitdiff/227b881. I've also prepared a patch for the token_filter module that allows us to easily experiment with Aloha Blocks for macro filter tags that are editable through Aloha Editor.

(I had created a new issue for this comment, which I shouldn't have. #1697594: Aloha Editor + Spark sprint: token_filter fork on GitHub is now closed.)

For prototyping purposes, I have included a modified check_markup() function in the Edit module, so that we could experiment with "nested editables" (i.e. things that are editable *inside* the WYSIWYG editor). I've then modified the token_filter module so that the custom AE plug-in for editing tokens within the AE WYSIWYG would have the necessary API endpoints to call.

My modifications:
- https://github.com/wimleers/token_filter/commit/90a4f33dc73b0b48eac7472c...
- https://github.com/wimleers/token_filter/commit/deb17de7b167990b4f7a60e1...

The initial AE plug-in that was implemented by Tobias Steiner of Gentics/Aloha Editor:
- https://github.com/wimleers/token_filter/commit/232923a2a6ba615f2d4719f2...

Also see #1605412-50: Brainstorm: provide WYSIWYG [filter compatibility] and only store the proper, "pre-transform" mark-up and that issue at large.

Finally, the generic issue for dealing with "true WYSIWYG" + format/filters compatibility is now #1699722: "True WYSIWYG" and compatibility with Drupal's text formats/filters.

@sun: OMG YES! That's such an awesome issue! Coincidentally, I took nearly the same route here, for the Edit module :)

Token vs. Inline API == plain text vs. rich text: agreed. I don't intend to use token_filter for rich text. It's just the most simple replacement system possible, and that's why I chose that as a starting point.

RE: Filter module's text format + filter system not being suitable.
Well, yes and no. #226963: Context-aware text filters (provide more meta information to the filter system) would allow me to not have to hack (or rather: provide an alternative to) the Filter module AFAICT, but at the same time, that issue attempts to do much more than I actually need for just the Edit module. For the Edit module, it's sufficient that we get filters to wrap the expanded/replaced values of macro tags in a standardized way, so that we can show the author the eventual value and still allow the author to change macro tags, while using a WYSIWYG editor. (See more below the horizontal rule.)

And now for the really interesting part…

I always agreed with that statement, given the Filter module's current design and involved filter_cache (which ultimately is the reason for why contexts cannot be supported). However, the intended usage/purpose of Edit module (as well as Inline API) is way beyond input filters in the first place. That behavior should not be tied to the actual text being rendered in the first place — it is primarily tied to the context and field the content is rendered in. In fact, the usage doesn't actually care that much that it is dealing with text. The primary relation to "text" is that something (another resource) is rendered within (embedded) in text. Fundamentally, this does not require a text format with filters to be involved at all. (!)

+9000

I wholeheartedly agree! To me, it seems people in those issues confused two types of context: internal and external. The internal (inherent) context is information that is really just part of the entity of which the text field is being filtered: the author, the language it is written in, etc. The external context is where context-aware filters become problematic: global/request information such as the current user, the current user's language, but also "page context" such as region in which it is rendered, whether it's rendered as part of a view, etc.

(Note that even in the case of external context, there are partial solutions to the problem that would limit the performance impact: render everything except these highly volatile replacement values, cache the result, upon viewing, get the result and only apply filters with highly volatile replacement values.)

So, IMO, internal context should be part of the filter system. Unless I'm missing something, I don't think it would cause a combinatorial explosion. External context could cause that, but there are ways around it. So unless we're willing to build the fairly sophisticated system I briefly described above, I don't think we should do that.

The changes I made:

• http://drupalcode.org/project/edit.git/commitdiff/227b8819f056f3acad3504... : My hacky customization of the Filter system: provide an alternative to check_markup(): check_markup2(), which accepts a "contextual" $wrap_macro_tags filter that defaults to FALSE (in this case, its behavior is identical to check_markup()).
  I also added a filter_wrap_macro() function that is used to apply standardized wrapping.
  In edit_field_attach_view_alter(), I then re-render each field using check_markup2(), to make sure all macro tags are wrapped.
• https://github.com/wimleers/token_filter/commit/90a4f33dc73b0b48eac7472c... : Overrides token_filter's filter callback; when the newly added "contextual" $wrap_macro_tags parameter is set, it will wrap the (replacement) value for each token using a standardized method.
• https://github.com/wimleers/token_filter/commit/deb17de7b167990b4f7a60e1... : Nothing interesting: just adding API endpoints so that some JS can 1) show a list of available macro tags, 2) pass a chosen macro tag to an endpoint and get the rendered result.

Note that for my own "contextual" $wrap_macro_tags parameter, it may not be really necessary to cache the results. Though that would assume only a limited number of users would be able to view the content in this special way, otherwise it may tax the server too much. So it may not be a safe assumption after all. If that's the case though, it's very likely a very advanced site, and developers would be able to override the lack of caching and simply make it cache after all.

I'm not sure what you mean.

Edit module is not in any way tied to the token_filter module. It's just the most simple example possible. I think it makes perfect sense for it to also support the Inline module. But really, it's not up to the Edit module to implement this. In Edit, there's no token_filter-specific code at all. The Edit module only provides a slightly extended version of the Filter system, that is all! :)

1) Those JSON responses are for the AE plug-in we wrote for token_filter: https://github.com/wimleers/token_filter/commit/232923a2a6ba615f2d4719f2.... We'd need to write a similar plug-in for Inline. But maybe (well, *hopefully*) the Edit module (actually, the edit_aloha module) can provide a generic implementation of that, which can accomodate most of the "macro tags" filter modules :)

2) See 1.

P.S.: heh … oops! I'll make sure to not do that again :) Thanks for pointing that out!

An interesting problem with the Inline module + having image captions:
- We wanted to be able to have an "image caption" filter disabled and simply have all the image captions disappear without having to do anything at all. We achieved this by deciding that we were going to have a "data-caption" attribute on the img tag (so: <img src="flower.png" data-caption="Such a beautiful flower!" />).
- This can never work well with an image that is rendered by the Inline module. The above assumes we have an actual HTML tag to work with, whereas the inline module uses a special syntax, in any case not HTML-tag based.
- The only way this could be resolved AFAICT is by having the inline module switch to a HTML-based (custom tag-based) approach, a la <inline type="image" node="345" field="field_main_image" data-caption="Such a beautiful flower!" />. I.e. let modules extend functionality provided by "filter modules" by adding new attributes.

What are your thoughts on this, sun?

Ontopic: token_filter has received significant changes on Friday (3 days ago):
- no more Aloha Block handles: https://github.com/wimleers/token_filter/commit/1f1e7442c3ed1ac489b6f5da...
- code clean-up
- only show the <select> when the token (the Aloha Block, really) has focus: https://github.com/wimleers/token_filter/commit/17e1b3ddda75f9d9233b09f5...
- this last change requires a patch to Aloha Editor: https://github.com/alohaeditor/Aloha-Editor/pull/622

My patch/pull request for Aloha Editor has been merged: https://github.com/alohaeditor/Aloha-Editor/pull/622. :)

@javier: the problem is that this is a filter and thus it is executed as part of the filter system, whereas the Inline module would run *after* the filter system, during hook_field_attach_view_alter()… :(

.