Problem/Motivation

Since Drupal 8 we've done some work to remove 'magic array keys'. But we still have render arrays, which every time you use them cause you to need to look-up the allowed keys. This is really bad DX. It's not clear what the allowed keys are, or where to find them. For backend developers who don't have years of experience with the nuts and bolts of Drupal's rendering system, this is more of a learning cliff than a learning curve, and it's very discouraging.

Proposed resolution

Provide builder classes for each element type to make defining render arrays discoverable from IDEs.

before

$variables['table'] = [
  '#type' => 'table',
  '#header' => $headers,
  '#rows' => $rows,
  '#attributes' => [
    'id' => $table_id,
  ],
  '#tabledrag' => [
    [
      'action' => 'order',
      'relationship' => 'sibling',
      'group' => $weight_class,
    ],
  ],
];

after

$variables['table'] = TableElement::create()
  ->setHeader($headers)
  ->setRows($rows)
  ->setAttributes([
    'id' => $table_id,
  ])
  ->setTableDrag([
    [
      'action' => 'order',
      'relationship' => 'sibling',
      'group' => $weight_class,
    ],
  ])->toRenderArray();

This sort of interface is a lot clearer, and borrows heavily from both Url helper and FieldDefinition stuff.

In this issue, we should only introduce a base class for these builders, plus one or two concrete implementations that we use in core, to prove they work. Then, in follow-up child issues, we can introduce more builders for the various core elements, piecemeal.

Remaining tasks

Refactor the patch in #106 to modernize it -- i.e., adding type hints -- and remove most of the builders it implements (punt to follow-ups). Keep one or two useful builders, and change core to use them where possible. Add unit or kernel test coverage. Then review and commit.

Manual testing

  • To check DropbuttonBuilder: go to /admin/content, check that the dropbutton display correctly
  • To check ImageBuilder: go to http://drupal.local/en/admin/help/contextual, make sure the icon displays in the help text

User interface changes

None

API changes

Creates a new optional way to create render elements.

Issue fork drupal-2316941

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

larowlan’s picture

larowlan’s picture

Status: Active » Needs review
chx’s picture

Edit: nevermind.

larowlan’s picture

Title: Add a helper class for creating inline template render arrays » Add an interface for defining render array helpers
Issue summary: View changes
StatusFileSize
new11.12 KB
new15.71 KB

From irc I was asked 'but we don't have these for tables' etc

But we could.

This adds TableElement.

We don't need to do everything now, but this minor change means we could do it later.

xtfer’s picture

I've written some of these for my own purposes in the past. They are quite useful. A general +1 from me.

joelpittet’s picture

I agree with @chx, make a template if possible. Also though, #type=>table won't go away that easy and this would make them much easier to build! so +1 from me too

tim.plunkett’s picture

This has some interesting overlap with #2305839: Convert hook_element_info() to annotated classes

larowlan’s picture

StatusFileSize
new5.21 KB
new10.5 KB

Removed InlineTemplate helper, too contentious.
Started out hating on #type => inline_template, but that's not the issue, the issue is render arrays keys are magic.
Removing InlineTemplateHelper to focus on the real issue.

larowlan’s picture

Note this can totally go in 8.1 or later - just pointing that out.

jibran’s picture

I think we should add a setRow() method as well to add a single row to rows. Just a thought.

The last submitted patch, 4: inline-render-helper.2.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 8: inline-render-helper.3.patch, failed testing.

kim.pepper’s picture

+++ b/core/lib/Drupal/Core/Url.php
@@ -18,7 +19,7 @@
+class Url implements RenderElementHelperInterface {

What about just RenderableInterface?

dawehner’s picture

Nice idea in the first place!

  1. +++ b/core/includes/common.inc
    @@ -3085,6 +3086,9 @@ function drupal_render_page($page) {
    +  if ($elements instanceof RenderElementHelperInterface) {
    +    $elements = $elements->toRenderArray();
    +  }
    

    I think for now we cannot support this line, rather people which create the form have to explicit call it on the render element. The problem is that we lose consistency for example when we do alter the elements.

  2. +++ b/core/lib/Drupal/Core/Render/TableElement.php
    @@ -0,0 +1,320 @@
    +  public function toRenderArray() {
    +    return array(
    +      '#attributes' => $this->attributes,
    

    Can we use some reflection on the object to figure this out maybe? This would dramatically improve the DX of writing those helpers

  3. +++ b/core/lib/Drupal/Core/Url.php
    @@ -18,7 +19,7 @@
    +class Url implements RenderElementHelperInterface {
    

    This feels wrong given how I consider URL as a value object, not something you directly render but yeah it does have the method as well

fabianx’s picture

Issue tags: +Needs benchmarks

All for it, but this will need benchmarks to show how bad the performance hit of adding many many more function calls is ...

jhodgdon’s picture

So...

For the really big win here... We are already proposing defining the render elements themselves as plugin classes on #2305839: Convert hook_element_info() to annotated classes. Is it possible that we could, instead of having 1 class that defines the render element as a plugin, and another that lets you insert it into a render array with various properties, we could combine all of that into one class? Because they both need to know what the properties are...

larowlan’s picture

I agree, and have spoken to tim about combining forces

star-szr’s picture

Just want to check in here, I confess to not having read much here but #2305839: Convert hook_element_info() to annotated classes has been in for a while now. Is this issue still relevant?

larowlan’s picture

Title: Add an interface for defining render array helpers » Add builder methods to make it easier to create render arrays

New name because #14 rightly points out that this has to talk in arrays still, but no reason we can't have nicer DX.
Also this can go in contrib or later

larowlan’s picture

Title: Add builder methods to make it easier to create render arrays » Use the builder pattern to make it easier to create render arrays
larowlan’s picture

Issue summary: View changes
larowlan’s picture

Issue summary: View changes
jhodgdon’s picture

Version: 8.0.x-dev » 8.1.x-dev
Category: Task » Feature request

I believe that this issue is properly classified as a feature request, not a task. It is adding a DX feature, right?

Even if it isn't a feature request... If it's a task, it isn't Major or Critical or in one of the "Unfrozen" or "Prioritized" categories. So at this point, unfortunately, I believe it needs to be pushed out to 8.1.x or a later release according to #2144413: Config translation does not support text elements with a format. If you disagree with this conclusion, my suggestion would be to reclassify and add a comment and a piece in the issue summary explaining why it fits into one of the categories of tasks/bugs that need to be done now.

Meanwhile, looking at the patch... I have a few docs suggestions:

a) You definitely need to add something to the Render topic about this functionality, so that developers can discover that it exists:
https://api.drupal.org/api/drupal/core!modules!system!theme.api.php/grou...

b) RenderElementBase has no documentation header at all.

c) RenderElementInterface has barely any documentation. I think it should explain what the class is used for and how to use it. Then the hypothetical section in the theme_render topic can be short and link to this class for further information.

d) RenderElementInterface::toRenderArray() references drupal_render instead of drupal_render()... but actually that sentence is misleading, because actually drupal_render() calls this method itself... That aside, the phrase used for this all over the rest of our documentation is "A render array suitable for drupal_render()" I think?

e) TableElement first-line docs do not end in . and a couple of the member variable docs headers are missing the blank line before the @var.

f) So... At this point reading the patch for TableElement, I'm like , uh.... is this really better than just making a render array directly? And if so, why? I don't really see the benefit of this whole patch. You've added a bunch more classes, just so that you can do this in field.inc:

before:

 $variables['table'] = array(
    '#type' => 'table',
    '#header' => $headers,
    '#rows' => $rows,
    '#attributes' => array(
      'id' => $table_id,
    ),
    '#tabledrag' => array(
       array(
         'action' => 'order',
         'relationship' => 'sibling',
         'group' => $weight_class,
       ),
    ),
  );

after:

  $variables['table'] = TableElement::create()
    ->setHeader($headers)
    ->setRows($rows)
    ->setAttributes(array(
      'id' => $table_id,
    ))
    ->setTableDrag(array(
       array(
         'action' => 'order',
         'relationship' => 'sibling',
         'group' => $weight_class,
       ),
    ));

This is about the same number of lines of code and I don't see that it is all that more understandable. And there's overhead to take this simple object and convert it back into an array.... Why do we want to do this at all? I don't get it.

g) I also don't see why you need all the get() methods on the helper object. They are not ever used and I can't see a reason why they would be.

larowlan’s picture

Hi @jhodgdon agree totally about a) feature request b) 8.1.x (or even contrib) c) the patch is rough (proof of concept) but in terms of 'whats the point' if you try and use the array format if you don't know what the keys are - there is no discoverability - you have to lookup the reference.
With the builder pattern - after you type TableElement::create() all the methods will autocomplete with your IDE.

So its completely about DX

jhodgdon’s picture

Ah, I see -- for the convenience of coders, at the probable expense of performance (that needs to be checked, but probably can't really be until we have more of these builder/helpers in place so that you can try building/rendering a typical page with mostly builders and see how much it slows down).

jibran’s picture

@larowlan are you interested in working on it in a sandbox as a contrib module?

sam152’s picture

This is an awesome DX improvement. I had a go at what a contrib solution might look like for this problem: https://www.drupal.org/sandbox/sam/2485729

It's a tad rough at this stage, but a great POC. I must admit after spitting out a few images in a matter of seconds, I didn't feel it would be much fun to go back to blindly guessing keys and waiting for an error.

jibran’s picture

That looks awesome. It can be a handy drush extension in D8.
Perhaps we can define @param type for all the setters by parsing function doc or something.

sam152’s picture

Thanks for the feedback Jibran. We actually have access to the default values for each variable in the theme definition so we can kind of derive the type from there. Arrays and bools are fairly straightforward and string usually default to NULL.

benjy’s picture

Awesome, i'm glad someone has started this in contrib.

fabianx’s picture

Great sandbox, please see #2485995: Please do not render early, return render arrays instead for an issue with the current DX, early rendering is a little problem atm. (also for attachments)

sam152’s picture

Thanks for the feedback. I'll continue the conversation in the issue you created.

sam152’s picture

If there was a patch generated to provide these wrappers for core is the suffix 'Element' too overloaded given Drupal\Core\Render\Element\Table exists and we are proposing introducing Drupal\Core\Render\TableElement? If they are "builder classes" why not use 'Builder' as a suffix for clarity. IDE integration for class inclusion will also probably match less results with 'builder' as a keyword.

They could probably also do with their own folder, perhaps 'Builder' inside 'core/lib/Drupal/Core/Render' making the final namespace for the table builder class 'Drupal\Core\Render\Builder\TableBuilder'.

sam152’s picture

StatusFileSize
new158.25 KB

Here is a patch which adds functioning builders to core using a modified version of the contrib solution posted above. I think the script that builds the method names needs some work and things like the naming convention and namespace will need agreeing upon, but it's an example of what a patch for this issue could look like.

sam152’s picture

Status: Needs work » Needs review

Testbot

Status: Needs review » Needs work

The last submitted patch, 34: 2316941-34-core-theme-builders.patch, failed testing.

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 34: 2316941-34-core-theme-builders.patch, failed testing.

sam152’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 34: 2316941-34-core-theme-builders.patch, failed testing.

benjy’s picture

Version: 8.1.x-dev » 8.0.x-dev
Status: Needs work » Needs review

wrong version for a testbot run.

sam152’s picture

Ah, good catch. Thanks.

sam152’s picture

StatusFileSize
new172.81 KB
new41.83 KB

Added some extra builders, removed some useless keys.

fabianx’s picture

Version: 8.0.x-dev » 8.1.x-dev

Just to not give any illusions, but being a feature request this can only go into 8.1.x.

Testing and making patches is fine however - of course :).

dawehner’s picture

Yeah I agree with @fabianx that this is a 8.1.x feature, BUT a really great one.
I'd even call this one of the most influential one, if you are honest.

sam152’s picture

Assigned: Unassigned » sam152

Good to hear the efforts aren't going to waste. I've been looking over the current progress and methods on each of the builder classes aren't good enough. I can't find a reliable way of determining the keys that are available for each one of the builders. From what I can tell things are documented as a combination of the keys inside getInfo and the preRender method comment.

Docs from 'Range':

<?php
  public function getInfo() {
    $info = parent::getInfo();
    $class = get_class($this);
    return array(
      '#min' => 0,
      '#max' => 100,
      '#pre_render' => array(
        array($class, 'preRenderRange'),
      ),
      '#theme' => 'input__range',
    ) + $info;
  }
?>
<?php
  /**
   * Prepares a #type 'range' render element for input.html.twig.
   *
   * @param array $element
   *   An associative array containing the properties of the element.
   *   Properties used: #title, #value, #description, #min, #max, #attributes,
   *   #step.
   *
   * @return array
   *   The $element with prepared variables ready for input.html.twig.
   */
  public static function preRenderRange($element) {
?>

Compared to docs from HtmlTag:

<?php
  public function getInfo() {
    $class = get_class($this);
    return array(
      '#pre_render' => array(
        array($class, 'preRenderConditionalComments'),
        array($class, 'preRenderHtmlTag'),
      ),
      '#attributes' => array(),
      '#value' => NULL,
    );
  }
?>
<?php
  /**
   * Pre-render callback: Renders a generic HTML tag with attributes into #markup.
   *
   * Note: It is the caller's responsibility to sanitize any input parameters.
   * This callback does not perform sanitization. Despite the result of this
   * pre-render callback being a #markup element, it is not passed through
   * \Drupal\Component\Utility\Xss::filterAdmin(). This is because it is marked
   * safe here, which causes
   * \Drupal\Component\Utility\SafeMarkup::checkAdminXss() to regard it as safe
   * and bypass the call to \Drupal\Component\Utility\Xss::filterAdmin().
   *
   * @param array $element
   *   An associative array containing:
   *   - #tag: The tag name to output. Typical tags added to the HTML HEAD:
   *     - meta: To provide meta information, such as a page refresh.
   *     - link: To refer to stylesheets and other contextual information.
   *     - script: To load JavaScript.
   *     The value of #tag is not escaped or sanitized, so do not pass in user
   *     input.
   *   - #attributes: (optional) An array of HTML attributes to apply to the
   *     tag.
   *   - #value: (optional) A string containing tag content, such as inline
   *     CSS.
   *   - #value_prefix: (optional) A string to prepend to #value, e.g. a CDATA
   *     wrapper prefix.
   *   - #value_suffix: (optional) A string to append to #value, e.g. a CDATA
   *     wrapper suffix.
   *   - #noscript: (optional) If TRUE, the markup (including any prefix or
   *     suffix) will be wrapped in a <noscript> element. (Note that passing
   *     any non-empty value here will add the <noscript> tag.)
   *
   * @return array
   */
  public static function preRenderHtmlTag($element) {
?>

My feelings are that the list of elements and the appropriate methods will need to manually be built. I don't think this will be a big problem in the scheme of things and there is probably a fair bit of code that can be written to make the process of extracting meaningful information from the comments/info methods a lot quicker.

Unless anyone has a recommendation I haven't come across for deriving appropriate methods I'll proceed with getting this underway. The alternative that I can see is a method similar to 'variables' on hook_theme which documents the keys on the element. Not sure how feasible or useful this would be outside this use case.

sam152’s picture

Version: 8.1.x-dev » 8.0.x-dev
StatusFileSize
new149.65 KB

I have attached a new patch which address a number of issues:

  1. The keys which form the basis for the builder methods have been built manually based on the theme/element definitions, templates and documentation.
  2. There are now two base classes. A generic one and one for form elements. These encapsulate things standard across these two areas, making it really easy to do standard render API things like setting prefixes and suffixes.
  3. The builders generated are ones which developers will and should regularly call out to, instead of a catch-all approach which included a lot of elements which were internal to core modules and would rarely if ever be used outside the context of those core modules.

On a cursory play around with the result, the builders seem more accurate and are improved by the base class.

sam152’s picture

Version: 8.0.x-dev » 8.1.x-dev
sam152’s picture

StatusFileSize
new149.66 KB
new464 bytes

Status: Needs review » Needs work

The last submitted patch, 51: 2316941-51-core-builders.patch, failed testing.

sam152’s picture

Version: 8.1.x-dev » 8.0.x-dev
Status: Needs work » Needs review

May as well get it green.

Status: Needs review » Needs work

The last submitted patch, 51: 2316941-51-core-builders.patch, failed testing.

joelpittet’s picture

I do really like this, especially for the Table. This looks ripe for refactoring. Can tableselect extend table builder?

  1. +++ b/core/lib/Drupal/Core/Render/Builder/ActionsBuilder.php
    @@ -0,0 +1,69 @@
    +  protected $renderable = ['#type' => 'actions'];
    ...
    +  public function setType($value) {
    +    $this->set('type', $value);
    +    return $this;
    

    If each builder sets the renderable array type, do we need a setType? Maybe you can cast between builders and remove that method?

  2. +++ b/core/lib/Drupal/Core/Render/Builder/TableselectBuilder.php
    @@ -0,0 +1,121 @@
    +  public function setAttributes($value) {
    +    $this->set('attributes', $value);
    +    return $this;
    +  }
    

    Can these setter methods be added to the base class to avoid the duplication?

sam152’s picture

Status: Needs work » Needs review
StatusFileSize
new135.43 KB
new19.48 KB

Thanks for the feedback.

  • I have removed the 'type' from actions builder.
  • Attributes shouldn't live on the base because it's not available on every builder.
  • I have added inheritance to builders. I have only created relationships between builders which are pre-existing in the \Drupal\Core\Render\Element\ namespace.

I'm interested to hear thoughts on how we could cast between builders of different types. Right now, we could add a key that sets 'type' but then you could have an instance of a builder which represents entirely different output. Perhaps the create method could accept an array of keys so you could:

$table = TableBuilder::create()->setHeader(['header']);
$table_select = TableselectBuilder::create($table->render()); 
dave reid’s picture

Version: 8.0.x-dev » 8.1.x-dev

No beta evaluation means this needs to go to 8.1.x.

joelpittet’s picture

Status: Needs review » Postponed

Yeah this has to get postponed to 8.1.x unfortunately, luckily it's a good candidate to be included as a feature in 8.1.x but right now it would likely just block or distract from issues blocking release.

Love this issue though, wish I could favrouite issues:)

Also to respond to #58 certain and most types it would be very tricky to cast between because there may be very few keys. But if you could turn it into an array, change the type and pass the array into say the new builder's constructor, that may work.

sam152’s picture

Sorry, version was just for the test bot.

sam152’s picture

Going to bump this for reviews now that we almost have a release and core is slowing down.

Crell’s picture

Note that core recently added a RenderableInterface, which is for objects that can be dumped to a render array. It's already being used in core. Any additional builder utilities (which I support) should tie into the same system. Ideally, they object can just be stuck directly into the render array and the render system will extract it as needed. (We should verify that works for all objects in core, but I think it does?)

sam152’s picture

StatusFileSize
new135.38 KB
new849 bytes

I've implemented RenderableInterface. Not sure how the objects tie into rendering and if they can be used interchangeably with the array equivalent. From what I can see they are only supported as renderable twig variables but this could probably be tied into the Renderer easily enough.

dawehner’s picture

Note that core recently added a RenderableInterface, which is for objects that can be dumped to a render array. It's already being used in core. Any additional builder utilities (which I support) should tie into the same system. Ideally, they object can just be stuck directly into the render array and the render system will extract it as needed. (We should verify that works for all objects in core, but I think it does?)

Well, theoretically this should indeed just works, but to be clear, we need to not try to go too many steps at the same time. Having a 150k patch seems to be one of those.

fabianx’s picture

Can we move that into a contrib module for now so its usable for people that want / need that, then include the patch here step-by-step for 8.1.x?

sam152’s picture

I have opened #2602368 to look at allowing RenderableInterface to be treated as a renderable array.

I can stick these in a contrib module. The original idea of the contrib module was to provide a build process to automate the creation of the classes but I think it makes sense to keep those two concerns separate for those who just want to leverage the ones in core.

sam152’s picture

Status: Postponed » Needs review

Testbot ahoy.

joelpittet’s picture

Can this be factored out any further to reduce some of the duplicate methods and maybe slim the patch down?

This grid may help visualize where the grouping (base classes) could occur
https://api.drupal.org/api/drupal/developer!topics!forms_api_reference.h...

sam152’s picture

I don't think we should introduce any base classes unless there is actually a polymorphic relationship in play. It could get very messy, difficult to follow and I think the value might be limited. By it's nature I don't think this patch will be very DRY.

Perhaps I could get on board with a more concrete example?

Crell’s picture

If all you're doing is avoiding boilerplate code, traits are a much better solution than base classes.

joelpittet’s picture

@Crell hoping to get some logical groupings of types elements not at all a goal to avoid boilerplate.

sam152’s picture

We already have inheritance where it exists for the element definitions. Traits are an option for the grouping but not sure how much value they will bring given we already have a lot of methods on the base builder and the form element base. Happy to look into that in any case.

If the feedback at this stage is optimisation related, can it assumed the approach and structure are validated?

jibran’s picture

Should we add some tests to theses classes?

cilefen’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

My rule of thumb is to test anything new and anything that has broken in the past.

cilefen’s picture

Issue summary: View changes
Issue tags: +Needs change record

This needs a change record because it is a new API.

xano’s picture

The core responsibility of our current render element plugins is to provide render arrays (that's essentially what "element info" has always been in Drupal). As @jhodgdon suggested back in #16, can't we add the additional builder logic to these plugin classes? That will make them easier to find, and will allow us to build on top of the current plugin swappability.

xano’s picture

Status: Needs work » Needs review

This is an extremely bare example of what I mean. Element plugins' builds are part of their internal configuration (which can be imported and exported verbatim through ConfigurablePluginInterface), and the base class automatically renders this based on elements' default info and optional cacheability metadata. Any child class can choose to expose additional methods to modify the build. This approach lets us convert elements individually, so we can gradually move towards an OO API for render elements, while maintaining compatibility and integration with our current approach.

xano’s picture

StatusFileSize
new2.54 KB
xano’s picture

Since we cannot entirely replace render arrays with objects because of BC (see #2602368: Allow objects that implement RenderableInterface to used interchangeably with render arrays.), here's a method to help configure elements based on render arrays, so we can easily alter render arrays by creating and configuring elements using this method, then export them back to render arrays using ::toRenderable(). I think it would be useful to add ::createInstanceFromRenderable() to ElementInfoManager, which isn't really just an element info manager anymore... We cannot add this method to ElementInfoManagerInterface, because that would break BC.

xano’s picture

StatusFileSize
new3.31 KB
xano’s picture

This will even allow elements to implement PluginFormInterface to provide their own configuration UIs, which could be useful for form building tools.

sam152’s picture

This was discussed in detail over IRC. Logs are here: http://codepad.org/CToqn2yu

Would appreciate some outside perspectives.

benjy’s picture

+++ b/core/lib/Drupal/Core/Render/Element/RenderElement.php
@@ -126,7 +131,88 @@
+    $this->configuration['build'] = NestedArray::mergeDeep($this->getInfo(), $element);

Does $this->getInfo() give us parity with the builder patch in #64, it was my understanding that "build" array in the builders patch had been manually constructed from a number of places?

I think the IRC discussions touch on it, but it does feel like we'd be making a step towards an OO Render API without actually knowing if it will be a step in the right direction. However, adding the builder classes gives us the DX improvement now and I can see that in the future, regardless of what direction our OO Render API goes in, we'll be able to update those builder classes to return instances of something that make sense in the new Render API.

Also, I think the IDE completion has to be solved, for me, it's a primary part of this issue.

xano’s picture

Once we have introduced and released any new classes, we can only alter parameter and return value types as long as those do not violate the Liskov substitution principle. This is especially relevant to class dependencies, because the original builder classes do not use generic factory methods for their dependency injection.

Does $this->getInfo() give us parity with the builder patch in #64, it was my understanding that "build" array in the builders patch had been manually constructed from a number of places?

My patches use ::getInfo() to provide default build arrays, something which any builder solution should be able to do using whatever implementation we see fit. Adding these defaults in the builder gives us more power to build arrays (e.g. we can depend on default property values, check/compare them and act depending on their values).

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

sam152’s picture

Assigned: sam152 » Unassigned
eric_a’s picture

Why don't we add a render element type that uses #lazy_builder to build a render array out of a proper ViewModelInterface object?
This #lazy_builder callback would be focused on mapping ViewModelInterface to a Drupal render array. Then Drupal\Core\Render\Renderer(Interface) and Drupal\Core\Render\Element\Element(Interface) don't risk becoming too complicated. We can use the same old renderer for now.

(It remains to be seen if we can easily implement full mapping of something resembling Zend\View\ViewModelInterface, but still.)

xano’s picture

Because it's not about when we build the render array, but about having a typed API to work with for improved DX. #lazy_builder will not help with that.

Could you clarify how you expect the renderer to get too complicated? Also, how does Zend's ViewModelInterface fit into this issue? We're not all familiar with it.

eric_a’s picture

A #lazy_builder can help with last-minute converting typed data to a render array suitable for the Drupal renderer. Using a separate converter is a nice alternative to each object having knowledge how to convert itself to Drupal render arrays, like with RenderableInterface. To me the separate converter sounds like the simplest solution with regards to our current renderer.
I haven't delved too deep into the patches, I'm sorry if I'm misunderstanding the issue summary.

Zend's ViewModelInterface or similar would be your typed API alternative to 'magic array keys'.
http://framework.zend.com/apidoc/2.4/classes/Zend.View.Model.ModelInterf...
http://framework.zend.com/manual/current/en/modules/zend.view.quick-star...

Pseudo code to compare with the issue summary:

$table_view_model = TableElement::create();
...
$variables['table'] = [
  '#type' => 'view_model',
  '#model' => $table_view_model,
];
xano’s picture

That would work and wouldn't be a BC break. It would, however, mean that quite some code may be converted to this, and that dependents would have to increase their minimum requirements. This is in accordance with our policies, but I expect it is a large and disruptive change, especially since both the render array and the object would have to provide the ability to contain children, which begs the question which tree of children would get priority over the other. It sounds similar to #2602368: Allow objects that implement RenderableInterface to used interchangeably with render arrays., but without the BC break.

Status: Needs review » Needs work

The last submitted patch, 81: drupal_2316941_80.patch, failed testing.

xano’s picture

Status: Needs work » Needs review
fabianx’s picture

#79 and #81 should live in its own issue - please - as they are unrelated to this issue - which is about adding the builder pattern to create renderable arrays.

And I really like #90.

Traversing is definitely still a problem.

I also will comment on #2602368: Allow objects that implement RenderableInterface to used interchangeably with render arrays. as I forgot to do so.

fabianx’s picture

xano’s picture

@FabianX: Please include arguments supporting your claim that is #79 out of scope. The patch shows a builder pattern, but based on our existing element plugins for future compatibility, rather than by introducing another new concept.

fabianx’s picture

#96: First: Pardon if I came through to harsh. Not meant like that.

Second: Please Fabianx (with lowercase x) - that is important as d.org is not case-insensitive.

Third:

Your patch might enable contrib to do so, but it does not match the issue summary and it feels that it is an implementation detail of Sam's patch.

I have "hijacked" issues before myself (and likely that was not always the best idea - as I had to learn, too - sometimes one is just so enthusiastic :D) and I am not saying it is bad, but I feel that Sam's patch, IS and idea goes into a very specific direction and especially introduces examples, while your patch goes into another and is very generic and abstract.

I of course might be wrong here, but my feeling is still that a new issue would be appropriate as I feel the architectural approach is different, which usually happens when the IS does not match.

With a good IS and supporting arguments your patch might be long in, while this one is still debated. So there might be something for you to win, too.

If things end up similar again, it might be a good idea to merge again or one issue might deprecate the other.

I hope my arguments for the general feeling I got have been easier to understand now.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

larowlan’s picture

Issue tags: +DrupalSouth 2017

tagging

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

markhalliwell’s picture

Status: Needs review » Needs work
Related issues: +#2972143: Create \Drupal\Component\Utility\ArrayObject, +#2869859: [PP-1] Refactor theme hooks/registry into plugin managers

This concept is something that I think is desperately needed in core.

We've been doing something similar in the Drupal Bootstrap base theme for a few years now, but not as "type" specific as this:
http://cgit.drupalcode.org/bootstrap/tree/src/Utility/Element.php

I've been working on abstracting this into a separate module though, Drupal+:
https://cgit.drupalcode.org/plus/tree/src/Utility/Element.php?h=8.x-4.x

Ultimately, I think core needs better utility classes in general, but I think it's going to have to start from the very basics: #2972143: Create \Drupal\Component\Utility\ArrayObject.

---

Regarding this issue specifically, I think the patches from #79 onwards, while similar, are really a separate issue altogether. I think we should keep this issue focused on the helper classes themselves.

I am going to set to CNW though because I don't think extending from RenderElement is the most appropriate thing to do.

These objects are ultimately just glorified arrays and I think we need some better groundwork (#2972143: Create \Drupal\Component\Utility\ArrayObject) in place first before continuing.

Also, not a huge fan of using the "Builder" suffix nomenclature on all the classes here. Seems a bit overkill, e.g. Table::create() works just fine.

Another thing that this issue should think about is allowing these element classes to be subclassed. Themes (especially base themes) will need to participate in these classes somehow. #2869859: [PP-1] Refactor theme hooks/registry into plugin managers may be able to help in this area eventually.

markhalliwell’s picture

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

kim.pepper’s picture

Status: Needs work » Needs review
StatusFileSize
new29.29 KB
new130.54 KB

Re-uploading #64 with some code style fixes, and to see if the testbot still likes it.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

phenaproxima made their first commit to this issue’s fork.

phenaproxima’s picture

Issue summary: View changes

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

kristen pol’s picture

Issue tags: +DrupalSouth

Tried to get a patch via gitlab and then apply it manually but it's not applying cleanly. Maybe this is due to how I'm getting the patch?

https://git.drupalcode.org/project/drupal/-/merge_requests/610/diffs.patch

...
patching file core/lib/Drupal/Core/Render/Element/RenderElement.php
patching file core/lib/Drupal/Core/Render/Element/Textarea.php
patching file core/lib/Drupal/Core/Render/Element/AjaxableElementTrait.php
patching file core/lib/Drupal/Core/Render/Element/Textarea.php
can't find file to patch at input line 13130
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|-- 
|GitLab
|
|
|From 66a7d0a60a02141cc0826db63046dd06fca7b7a1 Mon Sep 17 00:00:00 2001
|From: =?UTF-8?q?Ph=C3=A9na=20Proxima?= <adam@phenaproxima.net>
|Date: Thu, 29 Apr 2021 07:39:56 -0400
|Subject: [PATCH 5/7] Fix API conflicts to get tests running
|
|---
| .../Core/Render/Element/ElementAjaxTrait.php  |  2 +-
| .../Render/Element/ElementAttributesTrait.php |  5 ++-
| .../Core/Render/Element/RenderElement.php     | 14 ++++++
| ...eholderFormElement.php => TextElement.php} |  6 ++-
| .../Drupal/Core/Render/Element/Textarea.php   | 45 +++++++++++++------
| 5 files changed, 55 insertions(+), 17 deletions(-)
| rename core/lib/Drupal/Core/Render/Element/{PlaceholderFormElement.php => TextElement.php} (50%)
|
|diff --git a/core/lib/Drupal/Core/Render/Element/ElementAjaxTrait.php b/core/lib/Drupal/Core/Render/Element/ElementAjaxTrait.php
|index aa87f6b7bcb..96b46a7150d 100644
|--- a/core/lib/Drupal/Core/Render/Element/ElementAjaxTrait.php
|+++ b/core/lib/Drupal/Core/Render/Element/ElementAjaxTrait.php
--------------------------
File to patch: 

tim.plunkett’s picture

Idk what the terminology is, but that's a patch formatted for git am. Each commit is a portion of the patch (see the Subject: [PATCH 1/7] Patch #106. in the header)

If you instead use https://git.drupalcode.org/project/drupal/-/merge_requests/610.diff (which is linked at the top of the issue as "plain diff") will have the patch as you usually expect it.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

nod_’s picture

Been through the issue and I see that one of the reason for this is DX but there hasn't been discussion about the actual DX beside "IDE autocomplete". I guess it was a bit too early then, so can we talk a bit about what will be the DX of actual developers? like why are we prefixing all methods with set? especially since there is no get methods. I think we can borrow from JS/jQuery/DOM as far as DX goes.

What is the current API in the patch:

Image::getBuilder()
      ->setPrefix($prefix)
      ->setUri($this->get('url'))
      ->setAlt($this->get('alt'))
      ->setSuffix('</p>')
      ->toRenderable();

 $variables['original']['rendered'] = Image::getBuilder()
    ->setUri($original_path)
    ->setAlt(t('Sample original image'))
    ->setTitle('')
    ->setAttribute('width', $variables['original']['width'])
    ->setAttribute('height', $variables['original']['height'])
    ->setAttribute('style', 'width: ' . $variables['preview']['original']['width'] . 'px; height: ' . $variables['preview']['original']['height'] . 'px;')
    ->toRenderable();

What I would expect:

Image::getBuilder()
      ->prefix($prefix)
      ->uri($this->get('url'))
      ->alt($this->get('alt'))
      ->suffix('</p>')
      ->toRenderable();

 $variables['original']['rendered'] = Image::getBuilder()
    ->uri($original_path)
    ->alt(t('Sample original image'))
    ->title('')
    // There is no autocompletion for random attributes so we could group them
    ->attributes([
      'width', $variables['original']['width'],
      'height', $variables['original']['height'],
      'style', 'width: ' . $variables['preview']['original']['width'] . 'px; height: ' . $variables['preview']['original']['height'] . 'px;',
    ])
    ->toRenderable();

It's starting to look a bit like the DOM, except it's not; so we might need to explain clearly why we didn't go with that (IDE autocomplete, returns an array, not a string, Drupal elements don't map 1:1 to HTML elements, etc.)

About having a list of what goes where, like said in #48 looks like it'll need to be a manual step. As far as human documentation goes once the list is done, since it looks like the DOM, the MDN pages for DOM elements and attributes are a pretty good examples IMO. I like using it at least.

This also makes render array somewhat similar to the DOM… so Drupal Object Model, or DRAMA (Drupal Render Api Model Abstraction) :þ?

nod_’s picture

StatusFileSize
new15.3 KB

Umm I guess there are some existing code that use the set* pattern, still better than render arrays :)

Here is a patch for D10

mallezie’s picture

Added a comment about the phpstan fails in #3259355: Always do a full phpstan analysis on DrupalCI
This will need to be tackled first probably.

mallezie’s picture

Never mind my above comment. Phpstan is right here. Classes mentioned do not exist (anymore).

nod_’s picture

Version: 9.4.x-dev » 10.0.x-dev
StatusFileSize
new13.99 KB

yeah forgot to add the new files to my patch…

As per the issue summary I remove all the extra code that is not related to the image builder. I do not understand why we add methods to the RenderElement class, it seems completely unrelated. In any case I rerolled the patch with the bare minimum to make the image builder work according to the merge request.

nod_’s picture

Ok read the logs in #83 which explains why stuff is in renderElement. I'm with Sam152 from 6 years ago personally.

The new MR against 10.x is using code from #64. Not trying to be fancy, just providing IDE auto-completion to create render arrays. It's been years since the work started and #64 would have been a net improvement compared to the still current situation. Solution proposed from #79 feels like too big a step to wrap this issue up in a reasonable timeframe.

I updated the method names to remove all the set prefixes because it's nicer to work with. It's missing almost all the render API stuff, there is just enough to replace #theme => image. We can talk about what should be a trait, in the base class etc. once we agree to go the "easy" way :)

nod_’s picture

Added type hints

nod_’s picture

Back to green

jibran’s picture

I think we should add set prefix to all the setters because it makes it easier to drill down all the setters. Should we add __set as well to BuilderBase?

Then

      $icon = ImageBuilder::create()
        ->uri($icon_path)
        ->width($width)
        ->height($height)
        ->alt($this->getLabel())
        ->toRenderable();

can also be

      $icon = ImageBuilder::create();
      $icon->uri = $icon_path;
      $icon->width = $width;
      $icon->height = $height;
      $icon->alt = $this->getLabel();
      $icon = $icon->toRenderable();
nod_’s picture

If we don't declare width/height/alt explicitly an rely on __set, we're loosing the IDE autocompletion no? Also how to document the various possibilities?

In the case of dropbutton it allows us to alias properties nicely (assign #dropbutton_type by calling ->type() for example)

In my mind there are nothing else than setters so it's redundant to prefix them. Those are helpers to generate render arrays, altering the render array is still done old school (where you need to know the magic keys names)

larowlan’s picture

Yeah I opened this (7.5 years ago) because I was tired of looking up random array keys in docs, using magic setters is no better in my opinion

kristen pol’s picture

It would be good to add testing notes in the issue summary so we know how to manually test this.

For example, the following changed so I assume that checking the pencil icon is showing up would be sufficient for this one.

@@ -84,11 +85,10 @@ function contextual_help($route_name, RouteMatchInterface $route_match) {
       $output .= '<dd>';
       $output .= t('Contextual links for an area on a page are displayed using a contextual links button. There are two ways to make the contextual links button visible:');
       $output .= '<ol>';
-      $sample_picture = [
-        '#theme' => 'image',
-        '#uri' => 'core/misc/icons/bebebe/pencil.svg',
-        '#alt' => t('contextual links button'),
-      ];
+      $sample_picture = ImageBuilder::create()
+        ->uri('core/misc/icons/bebebe/pencil.svg')
+        ->alt(t('contextual links button'))
+        ->toRenderable();
nod_’s picture

Version: 10.0.x-dev » 10.1.x-dev
Status: Needs review » Needs work
Issue tags: +Needs reroll

MR doesn't apply and #130

Bhanu951 made their first commit to this issue’s fork.

bhanu951’s picture

I have re-rolled patch for 10.1.x branch, but as the initially MR is raised against 10.0.x branch It is showing as not mergeable. Anyone with edit access can pls change the target branch to 10.1.x so we can check MR status. Thanks. #130 needs to be addressed.

nod_’s picture

thanks, updated he branch

bhanu951’s picture

Issue tags: -Needs reroll
nod_’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs issue summary update

reroll looks good

nod_’s picture

We still need to add typehints and all the fancy php 8.1 stuff that's available now. leaving as NR just to get more eyes on it before sinking too much time here :)

manuel garcia’s picture

Issue summary: View changes

Updated summary examples array() -> []

voleger’s picture

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative

Was tagged for tests and change record that I believe still need to happen.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

chi’s picture

PHP has evolved a lot since this issue was created. Type hints, property promotion, named arguments, etc. So instead of builders we can create value objects. It'll give same DX in modern IDEs.

Builder

$variables['table'] = TableElement::create()
  ->setHeader($headers)
  ->setRows($rows)
  ->setAttributes(['id' => $table_id])
  ->setTableDrag([
    [
      'action' => 'order',
      'relationship' => 'sibling',
      'group' => $weight_class,
    ],
  ],
)->toRenderArray();

Constructor

$variables['table'] = new TableElement(
  header: $headers,
  rows: $rows,
  attributes: ['id' => $table_id],
  tableDrag: [
    [
      'action' => 'order',
      'relationship' => 'sibling',
      'group' => $weight_class,
    ],
  ],
)->toRenderArray();
chi’s picture

Overall, this will only mitigate the problem. Dropping render arrays is the only way to fully resolve the issue. I’ve filed a ticket in the ideas queue.

andypost’s picture

oily’s picture

oily’s picture

Issue tags: -Needs change record

Added change record based on the current issue description.

oily’s picture

@chi Re: #143 Good point. But this seems like a major step/ leap forward. I hope this issue be completed as-is as a first step?

chi’s picture

@oily The are dozens different render elements in Drupal core. The MR currently covers just a few ones.

ghost of drupal past’s picture

if people want to go ahead with this -- and I am not saying they either should or they should not, I really have no advice on that -- then, if someone wanted the opinion of an old ghost then #142 is the way because then it's possible to add RenderElementBase::fromRenderable so that form alter does not need to deal with arrays either. Otherwise, people would need to learn both syntax to be able to debug code. Not to mention how much less code would this require in core. One could even save this gist, run drush scr buildergenerator.php; composer phpcbf $(git diff --name-only core) and then smooth the results. Not saying you should, but, you know, you could. https://i.imgur.com/0Xt8Puu.png https://i.imgur.com/F6e8c1h.png

larowlan’s picture

Yes, agree #142 is nicer and it also gels with some thinking I had in this experiment

It also means we might not have to add new classes for everything, we could possibly use the existing render element plugins but instantiate them.

andypost’s picture

catch’s picture

#2702061-106: Unify & simplify render & theme system: component-based rendering (enables pattern library, style guides, interface previews, client-side re-rendering) and downwards is relevant, but the most likely outcome would be that the number of different elements is reduced, and eventually #theme gets dropped in favour of #type component, but don't really see any reason that this couldn't all be worked on in parallel without explicit dependencies.

ghost of drupal past’s picture

I tried #142 but the amount of methods you'd need to add is staggering and at the end of the day since people use arbitrary properties in render arrays you would need to support some sort of set/get anyways so I leaned heavily on that and so #3525331: Reuse element plugins as object wrappers around render arrays is born. If that issue gets accepted this issue might be outdated.

nod_’s picture

I spent quite some time working on making just 1 conversion, I agree with #153, this is not the way to go about it. #3525331: Reuse element plugins as object wrappers around render arrays is a much nicer solution.

pdureau’s picture

Yes, extending existing Render Elements classes (as proposed in #3525331: Reuse element plugins as object wrappers around render arrays) looks better than adding a Builder class for each Render Element as proposed in the current MR.

Also, when we will replace most (but not all 😉) of Render & Form elements by SDC, all the logic to remove will be contained in a single file.

catch’s picture

Status: Needs work » Postponed

I think this is good background reading for the other issue, but let's explicitly postpone it on that one.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.