[meta] Refactor Render API

Problem/Motivation

The Twig initiative provoked about how the Drupal render system worked, and what could be accomplished or refactored with the move to Twig. The promise of Twig demands a drillable, accessible variable structure for themers.

Consider some examples from a node template. Here's directly rendering the first item in a multi-value image field (the variable names here aren't exactly what we have yet, this is just an example, and syntax of translatables/plurals may differ):

  <!-- First image, with manual tag creation -->
  {{ hide(content.field_image.0) }}
  <img class="banner" src="{{ content.field_image.0.attrs.src }}" alt="{{ content.field_image.0.attrs.alt }}" />

  <!-- Remaining content -->
  {{ content }}

  <!-- Links -->
  {{ links }}

  <!-- Comments -->
  {{ comments }}

We cannot do this currently. This is because things like the image src are prepared in the field theme callback preprocessor, which isn't available yet to the node template. Furthermore, the structure of render arrays tends to be chaotic as any #-prefixed key is generally used as a tool to communicate state downstream the array chain. This is bad design. See http://lb.cm/arrays-of-doom

Further examples of what we want to do in Twig:

Field UI provides for user-configurable formatters. Here's the first image for the field rendered using its formatter:

  <!-- First image, field UI formatter -->
  {{ content.field_image.0 }}

  <!-- Remaining content -->
  {{ content }}

  <!-- Links -->
  {{ links }}

  <!-- Comments -->
  {{ comments }}

Suppose we want all items in the field (and any field wrappers):

  <!-- Images first -->
  {{ content.field_image }}

  <!-- Remaining content -->
  {{ content }}

  <!-- Links -->
  {{ links }}

  <!-- Comments -->
  {{ comments }}

...or if we just want to see the node content itself with the field rendered according to its formatter and associated field UI weight:

  <!-- All content -->
  {{ content }}

  <!-- Links -->
  {{ links }}

  <!-- Comments -->
  {{ comments }}

Here's a more elaborate example:

  <!-- First image individually with some template class. -->
  <div class="banner">
    {{ content.field_image.0 }}
  </div>

  <h2>{{ title }}</h2>

  <a class="permalink" href="{{ url }}" title="{{ created|format_date('n/j/Y') }}">{% trans %}Permalink{% endtrans %}</a>

  <!-- Remaining images. Perhaps we have altered their presenters. -->
  {{ content.field_image }}

  <!-- Remaining content. -->
  {{ content }}

  <!-- Links separately as a list of tasks in sentence form. -->
  {% if links %}
    <aside>
      <h4>{% trans %}Tasks:{% endtrans %}</h4>
      {% for i, link in links %}
        {{ link }}{% if i != (links.count - 1) %},{% else %}{% trans %}or{% endtrans %}{% endif %}
      {% endfor %}
    </aside>
  {% endif %}

  <!-- Comments with comment count. -->
  <h4>{% trans -%}
    {{ comment|count }} recent comment
    {% plural comment|count %}
    {{ comment|count }} recent comments
  {%- endtrans %}</h4>
  {{ comments }}

These shortcoming threaten the success Twig in D8, and they are not the fault of Twig. Though Twig does give us some benifits natively, if we are not able to achieve the above examples, we feel that D8 is simply shipping with something different, not necessarily something better.

Proposed resolution

We believe in order to achieve what we want to accomplish, render arrays must move toward an Object Oriented design. We're referring to these objects as Renderables or Presenters.

At DrupalCon Portland, c4rl and msonnanaum discussed some potential design patterns with effulgensia and moshe weitzman. It was clear at the time that given the proximitiy to the code freeze date (7/1/2013), this is a potentially frighteningly large undertaking, nevertheless, that is no reason to delay further as Twig contributor momentum increased significantly.

Guiding principles (in-progress):

1. Renderables know how to display themselves. Each sub-component of a renderable knows how to display itself.
2. Renderables know if they are empty.
3. Renderables access their properties in an intuitive way.
4. Renderables talk to their parents to understand context.
5. Renderables have configured defaults.
6. Renderables' implementaton can be altered and extended.

Potential implementation

We believe renderables will take the form of a deferred factory. That is, a renderable is defined with some default class that can be altered (in the same way that alters could change `#theme` parameters).

    // D6 REALITY. We invoke the theme layer and get markup too early, that
    // can't be altered. Aside from preprocessors, we don't have a good way to
    // alter variables. However, there are still some calls to theme() in the D7
    // codebase that need to go away; these are artifacts from the fact that
    // D7's render API wasn't implemented fully. :(

    $markup = theme('item_list', $items);

    // D7 REALITY. Though we can delegate the generation of markup to later in
    // the execution stack, these arrays can be come a free-for-all of #keys (to
    // see a good example just add an image field to a node and look at the 
    // devel render krumo output http://lb.cm/arrays-of-doom).

    // Furthermore, if anything inside $items is a further render array,
    // any variables its preprocessor invokes are not easily accessible in
    // a parent template.

    $render_array = array(
      '#type' => 'item_list',
      '#items' => $items,
    );
    $markup = render($render_array);

    // POTENTIAL D8 SOLUTION. A deferred factory. `Theme::create` specifies a
    // class `ItemList` that eventually will be invoked (this is so the class
    //  can be altered) and some arguments for the resulting class.

    // PHP's magic methods offer potential gains. We envision __construct()
    // working similar to a preprocessor, and __call() as an accessor method
    // that could use helper methods to get properties. Much TBD.

    $renderable = Theme::create('ItemList', array('items' => $items));
    $markup = $renderable->render();

    // Ordinarly we'd simply cast the renderable to a string, but PHP's
    // __toString() method cannot catch thrown exceptions until (at least)
    // PHP 5.5.

Remaining tasks

• Remove all calls to theme() aside from the call inside drupal_render(). @todo Create issue
• Develop base interface and classes. @todo Create issue
• Refactor core theme callback implementations. @todo Create lots of issues

Wishlist (courtesy of Moshe Weitzman)

1. Make it faster Don't make it slower
2. Don't break caching

User interface changes

None, likely.

API changes

Sweeping, likely. Render arrays will be replaced with OO architecture. @todo More details

Related Issues

@todo Add related issues

• #1886448: Rewrite the theme registry into a proper service
• #1757550: [Meta] Convert core theme functions to Twig templates

Original report by @c4rl

See the revisions tab. This github project was my first take at this work and mostly is a proof-of-concept; we didn't want to distract from Twig conversion at the time, so I kept it outside of d.o. Some of the theories are similar, but the implementation will likely look different. It is by no means efficient, secure, well-designed or battle-tested. Peruse at your own risk.