Problem/Motivation

Thanks to #2469431: BigPipe for auth users: first send+render the cheap parts of the page, then the expensive parts a lot of brilliant work has gone into implementing the Big Pipe architectural concept into Drupal 8. What I understand of the work is that the critical path (the minimal set of markup that can be full cached and delivered quickly to user) can be determined thanks to the work done in annotating the cache contexts and cache tags of each render-able thing. Everything that is determined to be "expensive" is then placeholdered and replaced in a secondary process.

This allows the initial page to be delivered, extra data to be inserted into the page, and then the personalized, "expensive" components are triggered to get get their data.

Kind of like customizing a car. You buy a standard car and then get kits to replace the tires, stereo, and other things you want to replace. Getting the car can be easy, just get one from the lot. But getting the customized kits for accessories can be time-intensive since you need to get the ones that you like.

Anyway, as #2469431: BigPipe for auth users: first send+render the cheap parts of the page, then the expensive parts states in it's issue description, there are many ways that the replacement of placeholders can be handled. Web Components as defined by the emerging set of specifications (here: http://webcomponents.org/ ) can be compatible way. There are a few advantages and disadvantages of this approach and it would be worth our while to explore if the trade-off is in our favor.

Advantages:

1. Web components are built with HTML, JS, and CSS

Web components are built to define a set of inert markup that can be pushed into the DOM and later activated to drive a frontend component. Web Components are built to define their own behaviors and appearance. Portability is a key focus. The promise of building a component and then being able to reuse it over different projects is key here.

2. Web components are atomic and can be combined.

Many examples of component systems can be found on the interwebs. Using these comopnents together, developers can quickly build applications. While this is actually a pretty old concept the main difference here is how easy it is to make components and how developers can build their own component libraries instead of relying on slow-evolving vendor-driven ones.

3. Although incomplete browser vendor support exists today, the goal is to have these components supported natively.
Today polyfills such as found on http://webcomponents.org/ exist to ensure "evergreen" browser support (read: no oldIE) almost all browsers have annouced the intent to support or partial support for the standard. Chrome and Opera have full support, Firefox has partial, and MS Edge and Safari will eventually have partial support.

4. JS behaviors

The components themselves can take on duties such as observing when they need to update, reacting to other actions on the page, or on the server. Components can have 2-way data binding.

Disadvantages

1. Required to include another polyfill
Today the webcomponents.js polyfill is another 34 KB of javascript that would need to be loaded on any page a web component is needed. That's a 34 KB we wouldn't need if we didn't use Web components.

2. Speed
Today working with the polyfill is slow. Speed is improving but the sheer fact that you have to work with polyfills / secondary requests may be an issue. How much of an issue? We need to have numbers to put towards the performance question of Web Components and see if

3. Opinionated
Using web components is picking a path. While you can use web components with other frameworks (like Angular) there could be some scenrios where you don't want to work with them. I guess this is really a question about whether this should ever be brought into core or should be contrib-only.

See:
#2469431-108: BigPipe for auth users: first send+render the cheap parts of the page, then the expensive parts
#2469431-166: BigPipe for auth users: first send+render the cheap parts of the page, then the expensive parts

Proposed resolution

TBD

Remaining tasks

TBD

User interface changes

TBD

API changes

TBD

Data model changes

TBD

Comments

Wim Leers created an issue. See original summary.

cosmicdreams’s picture

Issue summary: View changes
skyredwang’s picture

Web Components are many things, but in the BigPipe case, we probably will only use two specific tools: Shadow Dom and HTML Imports.

By using these 2 tools in the implementation of BigPipe, we can completely eliminate Javascript in the process. However, we rely on Pollyfill (a JS library for browsers who don't support Shadow Dom natively). In other words, for modern browsers (ever green browsers), when they load a BigPipe page from Drupal, they will not use Javascript during this loading process.

For Drupal BigPipe maintainers, they don't need to think about JS anymore, another win.

Furthermore, Web Components are already production ready, but not mainstream yet. This list shows people who are building on Polymer (a framework based on Web Components): https://github.com/Polymer/polymer/wiki/Who's-using-Polymer%3F

Regarding tudo:

1. Change the markup of the placeholder to use "HTML Imports"
2. Change the markup of the "Expensive/Personalized HTML blocks" to be Shadow Dom compatible
3. Add Pollyfill library for backward compatibility support.

I am still learning more D8, maybe in a while, I will be able to write a patch or a new module for Web Components implementation of BigPipe.

Wim Leers’s picture

I am still learning more D8, maybe in a while, I will be able to write a patch or a new module for Web Components implementation of BigPipe.

That'd be great! :)

cosmicdreams’s picture

@skyredwang if you need code reviews / collaborators hit me up. I'm also keen on getting a patch that represents the advantages of Web Components.

Wim Leers’s picture

Version: » 8.x-1.x-dev

Looking forward to patches/prototypes :)

skyredwang’s picture

@Wim Leers, another thing related: HTTP/2 Push. When a request comes in, the BigPipe knows what parts are expensive. For HTTP 1.1, the client sends the requests for the expensive parts. But, with HTTP/2, it will be interesting that we take advantage of multiplexing and push to send the expensive parts automatically.

The performance gain will be the number of expensive parts on the page x the trip time it takes from client to server.

Wim Leers’s picture

Of course we'll use HTTP/2 Server Push for assets. We have the asset library dependency graph in D8 that allows us to automatically deduce which assets are worth pushing.

But that's 100% unrelated to BigPipe. It's only related in that BigPipe is possible thanks to finally knowing dependency metadata, and the same goes for HTTP/2 Server Push.

Note that I'm already working on this in collaboration with a university research project. More about that soon :)

skyredwang’s picture

I was talking about HTTP/2 push dynamic content. In the context of BigPipe. You send a static template first, then HTTP/2 push username, personalized flags, tokens, list of images, etc. I realize the current servers support for push is mostly for static assets.

It's always fun to read more of your R&D! Thanks for leading and pushing the boundary for our community.

Wim Leers’s picture

Project: BigPipe » Drupal core
Version: 8.x-1.x-dev » 8.2.x-dev
Component: Code » big_pipe.module
Category: Task » Feature request
Status: Postponed » Active

BigPipe has now landed in Drupal 8.1. It's now sufficiently stable/proven (~400 sites using it for the past few months) so unpostponing this. Feel free to post prototypes, research etc.!

cosmicdreams’s picture

Looking forward to it. In the meantime, browser support has grown. That should be in the research reporting as well. Let's shoot for March 21 as an arbitrary deadline for putting this all together.

@skyredwang does that work for you / can we get together to organize? Anyone else want to contribute?

Woohooo! Let's go!

skyredwang’s picture

I am spending full time on Commerce 2.x related stuff on D8 this month. So, I won't be able to do anything with Web Components soon. But, I am trying to get a GSoC student to work on that https://groups.drupal.org/node/508466#project4

tic2000’s picture

It seems the browser vendor reached some consensus and WebKit is starting implementing custom elements https://lists.webkit.org/pipermail/webkit-dev/2016-March/027995.html

Also you can test for custom elements support and load the polyfill only if needed with something like


var webComponentsSupported = (
  'registerElement' in document &&
  'import' in document.createElement('link') &&
  'content' in document.createElement('template')
);
if (!webComponentsSupported) {
  var script = document.createElement('script');
  script.async = true;
  script.src = '/bower_components/webcomponentsjs/webcomponents.js';
  script.onload = finishLazyLoading;
  document.head.appendChild(script);
}
else {
  finishLazyLoading();
}
function finishLazyLoading() {
  // do something;
}
Wim Leers’s picture

Yep, https://twitter.com/ryosukeniwa is tweeting about that quite often — he's working on the WebKit implementation.

cosmicdreams’s picture

Another excellent public signal https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/EDxhDZ-...

This community post points to the consensus that has been reached on custom elements (the key piece we're talking about here)

Wim Leers’s picture

Version: 8.2.x-dev » 9.x-dev

I think it's misleading at this point that this is marked for 8.2.x-dev.

It's clear this is still at least years away from being supported by a sufficient margin of browser share.

It's not remotely clear how BigPipe would ever use Web Components. Especially since it'd largely defeat the purpose of BigPipe: have a single response stream additional content as soon as it becomes available. Using Web Components implies using multiple requests.

So, moving to 9.x.

Wim Leers’s picture

Title: Look into using Web Components for BigPipe » Add a WebComponentStrategy
Component: big_pipe.module » render system

Furthermore, the IS says:

there are many ways that the replacement of placeholders can be handled

… this gets to the gist of what this issue is about: a new placeholder strategy, that doesn't render placeholders on the server side and blocks until they're all done (like \Drupal\Core\Render\Placeholder\SingleFlushStrategy does), nor renders placeholders on the server-side in a non-blocking way (like \Drupal\big_pipe\Render\Placeholder\BigPipeStrategy does). Instead, it'd map each placeholder to a web component … but then things become fuzzy: does it important fully rendered HTML? Or does it come with a HTML template + JS, with the JS taking care of filling in the variables in the template?

That just shows that this is not about BigPipe, this is about providing a new placeholder strategy that is independent of BigPipe, and is simply a different choice.

This is also basically what the GSoC project that skyredwang proposed and is mentoring (and that I'm co-mentoring) is about. See https://blog.radiumz.org/en/post/33/gsoc-2016-more-app-experience-drupal and https://drive.google.com/a/acquia.com/folderview?id=0B_ZsBQ4mpbKQN29peGV....

Retitling and moving accordingly.

catch’s picture

Project: Drupal core » Drupal core ideas
Version: 9.x-dev »
Component: render system » Idea

Moving to the ideas queue.