Background

There are several places in Drupal where elements can be arranged to control their placement.

  • Themes have regions, which contain blocks.
  • Content forms have form elements.
  • Content has fields.

Contrib modules such as Display Suite and Panels can be used to enhance or replace these systems.

Problem

  • Each of the above have their own mechanisms for laying out their elements. They are separate systems, with varying levels of flexibility.
  • In core, content view and content forms have no means of altering their layout in the UI, just rearranging the fields by weight.
  • Regions of a theme can be customized, but this is done in files (THEMENAME.info.yml and page.html.twig), and is global for that theme.

Some contributed modules aim to solve this problem, but they also provide many other features that can be overwhelming compared to the task of simply altering a layout.

Proposed resolution

  1. Bring the "Layout Plugin" API into core. This will be used by all core layouting systems, and can continue to be used by contrib.
  2. Enhance the Field UI to allow content and their forms to have customizable layouts. This mimics the basic functionality of Panelizer and Display Suite.
  3. Allow themes to provide layouts.
  4. Provide a means to have different theme layouts for different parts of the site.

There is currently discussion over whether the Layout API should live as an experimental module or as an experimental subsystem (or a combination of both).
See #2296423-120: Implement layout plugin type in core and later.

Documentation for transitioning from layout_plugin contrib module to layout_discovery: https://www.drupal.org/node/2619128

API documentation for layout_discovery now available: https://api.drupal.org/api/drupal/8.3.x/search/layout_discovery

Process and where to find it

The tag for these issues is Blocks-Layouts.

Proposal roadmap

Must have

Remaining
Completed
  1. The first step is to add the layout plugin API to core experimentally:
    #2296423: Implement layout plugin type in core
  2. To validate the implementation, an experimental core implementation should be added and ideally reach beta, one or both of:
  3. For the layout plugin API to be considered stable, it also needs to be validated by (ideally) beta contrib implementations, at a minimum:

    Additionally, we need to stabilize the layout plugin API in a way that allows these modules (and sites using them) to upgrade from the layout_plugin contrib module to the core module once it is ready to be stable:
    #2833976: How Panels, Display suite, Layout Plugin, Layout Discovery and core get to a stable layout plugin in core

Other must-haves:

Should have

Could have

Not in scope

  • Arbitrary page creation (like Page Manager)
  • Layout editing (flexible layouts builder)

Contributed projects

Panels, Panels Everywhere, Page Manager, Panelizer, Mini panels, Display Suite, Context

Team and resources

  • Developer/Architect: @tim.plunkett (funded by Acquia's Office of the CTO)
  • Developer/Designer: @ DyanneNova (funded by Acquia's Office of the CTO)

Signoffs needed

  • Framework manager
  • Usability topic maintainers
  • Accessibility topic maintainers

Signoffs given

  • Release manager: xjm in #22
  • Product manager: yoroy in #59

No other explicit signoffs have been given, but the initial discussion happened at DrupalCon New Orleans among the following people:
Gábor Hojtsy, xjm, japerry, tim.plunkett, Bojhan, DyanneNova, stevector, dsnopek, ted.bowman, alexpott, yoroy, dobe, hestenet, Crell, swentel, aspilicious

Comments

tim.plunkett created an issue. See original summary.

tim.plunkett’s picture

@todo, update this with the still relevant parts of #1696302: [meta] Blocks/Layouts everywhere and close that issue.

tim.plunkett’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes

(Just adding some markup for scannability.)

xjm’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes

Trying to clarify the plan for layout plugin per @tim.plunkett.

tim.plunkett’s picture

Issue summary: View changes

Moving #1787942: Allow assigning layouts to pages to could have, I mistakenly placed it in must have.

xjm’s picture

Status: Active » Needs review
xjm’s picture

@tim.plunkett explained that #2795833: [plan] Add layouts to entity displays (both form and view) is a separate idea, and the experimental module prototype for that may depend on and will validate this API.

tim.plunkett’s picture

Issue summary: View changes

Adding more issues to the summary.

tim.plunkett’s picture

Issue summary: View changes

Updated IS with info about the core/lib vs module debate, still needs general review.

alexpott’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes
tim.plunkett’s picture

effulgentsia’s picture

#2834019: Remove the "layout__" prefixing of theme hooks is currently in the "should have" list. If this is only with respect to committing the module to core as experimental, then I'm ok with that. But, per #2834019-3: Remove the "layout__" prefixing of theme hooks, I believe this is a "must have" for the module becoming non-experimental.

tim.plunkett’s picture

Issue summary: View changes

I was going by #2296423-187: Implement layout plugin type in core point 2.
I'm fine with it being a MUST, moved it.

I'm planning on doing all of the SHOULDs anyway, and I'd strongly bet they get done before all of the MUST section anyway :)

xjm’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes
Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs framework manager review, +Needs product manager review

The initial patch for this initiative has been reviewed and committed with ample community input, so based on that, I would consider this initiative plan RTBC.

I've discussed this plan in depth with @tim.plunkett. It reflects my feedback and addresses my earlier questions. Based on that, I'm also adding my signoff.

@catch, @alexpott, @effulgentsia, and I have also discussed the framework and release management especially for the first and third steps with their followups. The two parts of the second step probably need additional architectural review. Edit: Per @tim.plunkett, the page layout part will need a lot of ideation work still. OTOH the field (content) layouts already has an RTBC initial patch. :)

xjm’s picture

Component: Idea » Plan

Also, this is definitely at the plan stage.

xjm’s picture

For product managers and UX maintainers particularly: Note that the issue for field (content) layouts has its own sub-plan already: #2795833: [plan] Add layouts to entity displays (both form and view) So if there are questions about the specifics for that, UI prototype, etc. we should probably discuss on that plan. (This issue is the high-level roadmap for the core initiative overall.)

tim.plunkett’s picture

tim.plunkett’s picture

Issue summary: View changes

Per discussion with Bojhan, yoroy, Dries, Gabor, and webchick the additional layouts are now a Must Have.

tim.plunkett’s picture

Issue summary: View changes

Also adding #2660124: Dynamically generate layout icons based on well formed config as it will be greatly beneficial to visualize the layouts while selecting them.

mattlt’s picture

This is awesome.

All the talk about layouts seems very desktop-centric. Wondering how mobile-first responsive design fits in, if anywhere.

I see a lot of talk about left/right, one column/two column/three column, as well as other cool and complex layouts. For example, https://www.drupal.org/node/2660124 will show the icons as only one version of the layout, however that layout may look different on mobile, or have any number of variations.

In our practice, mostly using panels, we've shied away from using terms like two-column, left sidebar, etc, in favor of primary, secondary, etc, so site users are thinking more about importance than layout. I'll admit that this does nothing to help a person visualize where a piece of content will be placed and how it'll look depending on the size of the browser window.

Keep going! I'm just trying to wrap my head on how best to use this, and I'm sure you've all discussed these things in your internal discussions. I'd love to hear your thought on resolving some of these issues. If there's another issue somewhere that talks about this, please point me in the right direction.

Thanks,

•• matt

SKAUGHT’s picture

@mattlt
when it comes to desktop & mobile layouts it is/will be upto the layout plugin to work through that. ie: Bootstrap Layouts fills that use case (layouts built from responsive grid systems)

xjm’s picture

Title: Add layouts to Drupal » [plan] Add layouts to Drupal
Issue summary: View changes

Tim and I moved a couple issues to #2795833: [plan] Add layouts to entity displays (both form and view) that are in scope for that module, but not specifically blockers for Layout API itself being considered stable.

It also occurred to me that we might not want to mark Layout API as stable until the implementations validating it are beta, so that if we discover new additions or API changes that are necessary based on those implementations, Field Layout improvements can still happen in patch releases and are not pushed to the next minor. I see it going something like this:

  1. Layout API alpha (done)
  2. Field Layout alpha
  3. Contrib implementations alpha
  4. Layout API beta
  5. Contrib implementations beta
  6. Field Layout beta
  7. Layout API stable
  8. Contrib implementations stable
  9. Field Layout stable
  10. Updates to Standard, if any.

Any number of these steps could happen within a minor, though I'd imagine 6 on will happen after 8.3.0 so they can include feedback and testing from the experimental module process.

Adding a note about that too.

Dries’s picture

Component: Plan » Active Initiative
Status: Reviewed & tested by the community » Active

We discussed this on the core committer call today, and all agreed with the plan laid out here. I'm in favor of moving forward here, with Tim as the initiative lead as he’s been the one driving putting the plan together. Very excited to see this move forward (officially). Thanks everyone involved creating this plan and contributing to the layout initiative so far. As a next step, maybe Tim can create a patch making him the initiative lead?

tim.plunkett’s picture

Thanks @Dries and the other core committers!
Opened #2843407: Add layout initiative to MAINTAINERS.txt

xjm’s picture

Issue summary: View changes
tim.plunkett’s picture

xjm’s picture

Issue summary: View changes

Adding info on the contrib implementations we are tracking to validate the API.

xjm’s picture

Issue summary: View changes
webchick’s picture

Looked at this today with @japerry and @DyanneNova at PNWDS.

So the problem is we currently have distros such as Lightning or Thunder or whatever that are using Panels for layout functionality. At the moment, those distros are pulling in https://www.drupal.org/project/layout_plugin which, although the version number shows as alpha, has already been vetted by both Display Suite and Panels APIs, and is used in production on ~15K sites.

Now, 8.3.0 is poised to ship with Layout Discovery module as experimental, which is actively discouraged from real-world use. Display Suite and soon Panels have a -dev branch with compatibility with Layout Discovery, which in theory helps validate the API, but only if people actually use them. https://www.drupal.org/project/usage/ds as of today shows literally six users of the 8.x-3.x-dev release (which uses the core stuff), compared to 7600+ users of the latest 8.x-2.6 release (which uses Layout Plugin).

Which puts these distros, as well as core, in a pretty awful position. Do they rely on a deprecated/abandoned contrib module for the next 6 months until 8.4.0, when we will hopefully receive enough independent verification to ensure that we can elevate Layout Discovery to stable? And as the core development team, how can we in good consciousness elevate something from experimental to stable if the real-world usage is absolutely pitiful? This puts the entire ecosystem at risk, and actively harms overall Drupal 8 adoption. :\ And as a result, we're starting to see a fraction in the ecosystem around people who are using Paragraphs for this use (which is meant to be a field structure system, not a layout system), and one of the big points of putting this stuff in core in D8 was to avoid such fractions.

So what are our options here? Over the next month (the sooner the better, obviously), if Panels does a similar thing and starts a dev branch that utilizes the core stuff, and all tests are passing, can we elevate Layout Discovery to stable during RC? (This seems to be the only must-have in the list left, at least within core's purview.) Or, are we simply resigned to foisting 6 months of scary ambiguity among sites that require layout functionality?

yoroy’s picture

Where "Panels starts a dev branch that goes against the core stuff" means *using* the core stuff? Or moving away from it?

webchick’s picture

Oops, sorry. I meant using the core stuff. Edited to clarify.

yoroy’s picture

Thanks!

tim.plunkett’s picture

Layout Plugin is still alpha. There is the SAME risk for any module to use either one. Any module or distro depending on Layout Plugin that claims to be stable currently will have the same stability if they used the experimental Layout API.

Other than Panels making the branch with the patch I provided *four months ago*
#2821226: Update Panels for layout_plugin in core
there is only one remaining Must Have:
#2822758: Introduce a way to distinguish between different types of layouts

That I hope to focus on this coming week.

webchick’s picture

Well, I'm not sure "use this alpha that 15K other people, two major contributed projects, and several distributions are also using in production" vs. "use this alpha that has been refactored from the original and no one else has used in production yet" is the same level of risk, like, at all. :)

xjm’s picture

Which puts these distros, as well as core, in a pretty awful position. Do they rely on a deprecated/abandoned contrib module for the next 6 months until 8.4.0, when we will hopefully receive enough independent verification to ensure that we can elevate Layout Discovery to stable?

Actually, right now, the primary blocker to Layout Discovery being marked beta is that Panels has not opened a development branch. The primary blocker to Layout Discovery being RC will probably be Panels just confirming that it's good enough for them. Most of this module's roadmap is already done. Unlike Field Layout, it doesn't provide UI or anything. Panels is the independent verification we need.

xjm’s picture

Also, I already outlined this in #30 above.

tim.plunkett’s picture

#2821201: Update Display Suite for layout_plugin in core was opened the same day as the Panels issue, and has since been committed and released.

Opening both of those issues and providing patches was a part of #2296423: Implement layout plugin type in core

webchick’s picture

Ok, just told Jakob that. ;) Stay tuned.

swentel’s picture

Display Suite and soon Panels have a -dev branch with compatibility with Layout Discovery, which in theory helps validate the API, but only if people actually use them.

From a (semi) maintainer point of view of DS.

It might be important to notice that the 8.x-2.x branch of DS will /not/ work at all with core 8.3.x, which we already have put on the project page of DS itself. The main reason for this is not layout plugin in core itself, but due to subtle changes in entity displays and Field UI. Making 8.x-2.x of DS work with core 8.3.x BC wise and using layout plugin from contrib was an option, but in the end, it was kind of natural to start a new branch which uses core layout plugin and works with the Field UI/entity display changes in 8.3.x. This might be a little adventurous, otoh, given the little difference of layout plugin in core vs contrib, I'm confident that this won't be a problem at all for users when they have upgrade core. (core has a more interesting update anyway in core.extension for the profile for instance). And to be fair, imho, layout plugin is kind of beta ready anyway (at least for DS, can't tell for panels))

I don't remember having to make enormous changes in the code of DS (thanks to initial patches from Tim), so as soon as panels starts a branch on core layout plugin, we can deprecate the contrib version of it.

I know there's an issue to coordinate releases for several modules, I might ping Alex on that next week when I'm in london.

xjm’s picture

Put another way, this API is being added to core primarily to support DS, Panels, and future core implementations all at once. We're unlikely to majorly disrupt/break the contrib projects we're trying to support because that would defeat the purpose. :) We will break if they need us to for their stuff to be supported.

swentel’s picture

@xjm that's a good summary :) DS is using layout plugin since ages (I really don't remember anymore at which point we started using it). The last major major change for DS (#2612040: Allow DS to use Panels layouts and vice-versa (using the new layout_plugin API for settings)) happened in november 2015 after #2490680: Allow layouts to provide settings got committed in layout plugin, a week before we tagged a stable release of DS, two hours before core 8.0.0 ..

Now, I think I see the bigger picture/problem too: you can use Panels and DS in the same installation and then things are problematic since both can't be enabled right ? Which would potentially mean that we have to revert our branch of DS (or make 8.x-2.x work with 8.3.x) in case panels doesn't have a branch that depends on core ?

(bit late now, will think about it some more tomorrow)

DamienMcKenna’s picture

tl&dr: we're working on it.

Thanks for bringing this up, webchick.

We've been discussing this in the weekly Scotch meetings for a while. Our immediate goals were a) getting the time-sensitive improvements into core that were needed, b) stabilizing the five contrib modules focused around 8.2.x compatibility (CTools, Panels, Page Manager, Layout Plugin, Panelizer). We've suffered from a lack of contributors who understand the APIs, so the few people who do have been focused on the core improvements plus their day jobs. We've identified items in each project's queue that need to be done prior to them being marked "stable", now we just need time & people to work on them. Once all five are out with stable releases we can deprecate Layout Plugin and create new branches for the other four to focus around 8.3.x compatibility and the new layout APIs in core.

So, in summary, the plan is:

  • Get stable releases of the five modules with a strict line of compatibility with 8.2.x.
  • Once all five are out, deprecate Layout Plugin and create new branches of the other four.
  • Update the four modules to work with 8.3.x and core's layout API.
  • As always, more contributors who understand D8's APIs (and the Panels APIs) would be appreciated.

Thanks everyone!

Berdir’s picture

@DamienMcKenna:

To be honest, I'm not sure how realistic that plan is. 8.3.0 will be released in a bit more than a month and after that, 8.2.x no longer matters.

I've been involved a bit with Page Manger and progress there has been very slow, there were only a handful of commits since last April and a single new alpha release. And there are quite a few challenges left to resolve there IMHO before we can call that stable. And that's just one of 5 that you plan to get to stable in one month?

Why not just add a version dependency on 8.3.0 on panels and do a new release that uses the core layout discovery module? DS did a new major branch, but they also had other conflicting changes. If the layout discovery and the layouts themself are compatible, then I think it is fine to do that without a new major version and actually easier pre-stable.

That helps core and all users in two ways:

1) As mentioned above, the main blocker for layout_discovery in core getting (more) stable is panels using it and confirming that it works. If we do this now, then we can possibly make it beta in 8.3.x already which makes sure that there will be no API changes in 8.4.x and it hopefully can become stable there.

2) With DS already being updated and only compatible with 8.x-3.x, users will not be able to use DS and panels in 8.3.x.

Btw, I recently moved my custom widgets projects (https://www.drupal.org/project/widget) to drupal.org and ported it from using the very, very early layout.module (https://github.com/frega/layout). We have been using widget in NP8 since 2014 (http://www.md-systems.ch/en/blog/md-systems/2014/06/04/layout-for-drupal-8) in NP8 and it was kind-of-public on github. I actually have to change the templates in our case, but I hope it will not be too problematic for our projects. We don't actually use DS or panels right now in NP8, but I still plan on making this switch as part of the 8.3 update and will start to test this soon (I already tested 8.3.0-beta1 and it looks pretty good).

andypost’s picture

Looking from maintenance of few projects it's really not clear why lauout_plugin module can't implement service overrider to prevent collisions with core's discovery?
It makes total sense to have layout_plugin shim for migration to 8.3

andypost’s picture

Is there any real blockers in "ctools&co" to be compatible in-month with 8.3 core?

If yes it is separate blocker of 8.3 adoption because core's module experimental and the same alpha as layout plugin module

Also it's not clear now to contribute to panels&ctools because issues hangs for months (++ #51) and layouts are the least question...
So maybe better focus oo core layouts instead of solving compatibility of 8.3 & layout plugin

tim.plunkett’s picture

The 8.x-4.x branch of Panels is now open and is compatible with 8.3.x.
There is a bug in IPE that needs to be tracked down, but otherwise it should be fine

tim.plunkett’s picture

Issue summary: View changes

Updating the list of blockers. There are only 3 remaining:

  1. #2852608: Review layout CSS and markup
  2. #2824667: Update Layout Plugin handbook documentation
  3. #2834025: Mark Layout Discovery as a stable module

1 blocks 3.
2 does not need changes to the codebase.

webchick’s picture

For the next stage of the layouts in core initiative, targeting 8.5+, here are the notes from our discussion at the Layouts sprint last week!

https://docs.google.com/document/d/1rQukpyKFzeTIcVoRmCG6xTono9O2nhVxrzNS...

TL;DR: Work on the "I want to lay out node/5 differently" use case + a nice UI for content authors.

tonystar’s picture

@webchick regarding this:

Entity-based focus is still the most important (Panelizer in core). The ability to override node/5. (Only on content types for which layouts are enabled.)
...
Populate layout with fields, content, blocks, etc. [Broadest definition of "content"]
...
Should be able to add entire other entity, Powered by Drupal block, body field.

Wouldn't it be easier to create a single multi-value field using https://www.drupal.org/project/dynamic_entity_reference and then use Layout Plugin to place referenced entities in regions? Need to be able to set region per delta. Revisions and translation is out-of-the-box.

HongPong’s picture

Issue summary: View changes

I have updated the documentation page for layout plugin to help with the 8.3.x transition: https://www.drupal.org/node/2619128

Also note API documentation for layout_discovery now available: https://api.drupal.org/api/drupal/8.3.x/search/layout_discovery

yoroy’s picture

I think #57 has product manager review aplenty, so removing that tag.

tim.plunkett’s picture

Issue summary: View changes

Updating signoffs and team list

xjm’s picture

Responsive design support for layouts (e.g., breakpoint integration/support) has come up frequently. According Tim it's out of scope for the initiative (at least per https://events.drupal.org/baltimore2017/sessions/continuing-saga-layouts...).

Seems important and oft-requested. I personally wouldn't be too keen on using content layouts if they didn't respond the way the rest of Drupal does out of the box. Were the handful of default layouts added to core not responsive? If they weren't I was seriously not paying attention when I reviewed that patch. :P

At the least, I'd want a could-have, maybe a should-have. Maybe some user testing on Mobile and confirmation that the current state is okay from a product perspective?

xjm’s picture

Couple things from the Layouts session at DrupalCon:

This is a "mental note" on contextual filter stuff for Tim).

Also should stuff about the idea for the landing page builder on these roadmaps?

And revisioning stuff and data model for the stuff as discussed WRT Workflow. That's a big thing (when initiatives have an overlap, one of them has more technical debt).