Problem/Motivation

Since Drupal 8.1, the following BigPipe changes were committed:

And we only have a single remaining open bug: #2738685: BigPipe leaves wrapper <div>s in place, because AJAX system leaves them — this can cause some CSS to break, which is actually a duplicate of #736066: ajax.js insert command sometimes wraps content in a div, potentially producing invalid HTML and other bugs, which has been open for >6 years, and is probably the oldest bug in the AJAX system.

I think this is a clear signal of stability.

Proposed resolution

  1. Mark BigPipe as having beta-level stability in 8.2's beta + RC phases.
  2. Consider marking BigPipe as non-experimental before 8.2.0.

There is only one significant feature that is IMO a must-have before making BigPipe stable: #2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits.

Remaining tasks

TBD

User interface changes

None.

API changes

None.

Data model changes

None.

Comments

Wim Leers created an issue. See original summary.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Issue summary: View changes

Manual HTML writing fail.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Issue summary: View changes
wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Issue summary: View changes

Also note that it'd be splendid if d.o DB admins could query https://www.drupal.org/project/usage/drupal's DB tables, because I understand that in there they can see which exact core modules are enabled. i.e. if only 100 D8 sites ever used BigPipe, then we shouldn't do this. But if 2,000 or 5,000 sites use BigPipe, then that's a strong signal of stability.

xjm’s picture

xjm
she/her
Primary language English
commented

+1 from me!

One question though. If Big Pipe is considered beta, RC, or stable, that would probably mean adding #2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits in 8.3.x rather than 8.2.1 or something. Do you think that makes sense?

xjm’s picture

xjm
she/her
Primary language English
commented

(Unless it's ready during 8.2.x beta i.e. August, in which case I'd be fine with marking the whole module beta at that point once that feature is added.)

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

I will work as hard as I can to have it RTBC by next Friday at the latest (August 5, 2016).

I think this feature is very important, because it makes BigPipe valuable for *every* site, rather than only those with sessions (authenticated users, or anonymous users that have sessions such as in the case of online shops).

fabianx’s picture

fabianx commented

+1 to beta stability during beta and potentially stable for 8.2.0
+1 to getting #2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits in before beta deadline (I can also help this week if needed, but road is pretty straightforward now)

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
Version: 8.3.x-dev » 8.2.x-dev

I may be mistaken, but I believe if all the parts were to come together prior to rc1 (Sep. 7, so really more like Sep. 5), this could still happen in 8.2. This isn't adding new functionality, it's simply stabilizing the existing functionality and asking for a stability change in response.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

Since August 1, not a single new bug report. Not against https://drupal.org/project/issues/drupal?component=big_pipe.module, nor against https://www.drupal.org/project/big_pipe. So that is further support for this gaining beta-level stability: as far as we know, it's bug free (famous last words, I know).

However, it'd be great to land #2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits before marking it beta-level stable. It's a pure feature addition, but it'd make BigPipe truly feature-complete. The patch has been ready for 26 days now.

catch’s picture

catch
he/him
Primary language English
commented

#2792817: BigPipe decorates html_response.attachments_processor, but uses an inner name based on the decorated service, which can cause problems just came in. It's an incompatibility with another module extending the same service.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

That's not a bug in BigPipe's functionality though; it's an implementation detail that causes problems because Symfony designed the mechanism for decorating services poorly. Anyway, it's an easy fix, and there's a patch there already.

If it's this kind of bug that gets reported, then YAY, let them come!

dawehner’s picture

dawehner
Primary language German
commented

That's not a bug in BigPipe's functionality though; it's an implementation detail that causes problems because Symfony designed the mechanism for decorating services poorly. Anyway, it's an easy fix, and there's a patch there already.

Yeah, software has bugs. This is IMHO not at all a reason to not mark some specific modules as experimental. Just imagine if we would mark system module as experimental :P

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

Heh :)

fabianx’s picture

fabianx commented

#14: Agree, it is at least beta level stability.

Just imagine if we would mark system module as experimental :P

Only if we also integrate it directly with systemd :D.

xjm’s picture

xjm
she/her
Primary language English
commented
Status: Active » Needs review

#2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits looks like it is running into a sticky testbot issue. In case that doesn't land before 8.2.0-rc1 (release window freeze starts 1200 UTC Tuesday), I think we have two options:

  1. Move the issue to 8.3.x, and consider BigPipe beta or RC now, as well as leaving open the possibility to further raise its stability prior to 8.2.0's release. (Could maybe even be made totally stable during RC i.e. moved out of the Experimental package, if we are ready to provide full support and BC. Since during RC stable things are also in RC and it's just a one-line revert if there turns out to be a critical.)
  2. Continue to work on the issue during RC, and increase BigPipe's stability to beta once it lands. BigPipe could be marked RC in a subsequent patch release, but would need to be moved to stable in 8.3.x only.

We don't need to decide now; I just wanted to lay out the options as I see them since there's a good chance this might be the first experimental module to graduate out of alpha. :)

catch’s picture

catch
he/him
Primary language English
commented

@Wim yeah wasn't saying it makes big pipe unstable, just pointing out it existed ;)

Both options in #17 seem reasonable. For me it seems easier to commit #2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits either before or at beta for bigpipe, and then move bigpipe to RC, rather than do RC first and have to worry about bc layers for the patch given it's close already.

xjm’s picture

xjm
she/her
Primary language English
commented

It turns out that #2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits relied on adding some functionality to stable core, which is now (correctly) in a separate issue at #2795209: Allow streamed responses to be cached by Page Cache. As of right now I'm pretty sure that issue will need to be 8.3.x only, which makes me thing option 1 in #17 is probably what we should consider doing.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Status: Needs review » Reviewed & tested by the community

Indeed.

Pinged Fabian, to ask whether he still stands by what he wrote 1 month ago in #8:

+1 to beta stability during beta and potentially stable for 8.2.0

We think it's best to mark BigPipe as beta-level stability. Because #2657684: Refactor BigPipe internals to allow a contrib module to extend BigPipe with the ability to stream anonymous responses and prime Page Cache for subsequent visits would introduce a change in behavior for anonymous users, and if BigPipe remains experimental (but beta-level stability), then we aren't required to opt in new sites but opt out existing sites (which would require either a container parameter or configuration, BigPipe currently has neither, and preferably we'd keep it that way, because it simplifies long-term maintenance).

In any case, beta-level stability (instead of marking BigPipe non-experimental) leaves us with necessary freedom in that regard.

xjm’s picture

xjm
she/her
Primary language English
commented
Title: Mark BigPipe as beta-level stability (instead of alpha), or even non-experimental » Mark BigPipe as beta-level stability (instead of alpha)
Status: Reviewed & tested by the community » Fixed
Issue tags: +8.2.0 release notes

Thanks @Wim Leers! I updated https://www.drupal.org/core/experimental, and we will also make sure this is reflected in the 8.2.0 release notes.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

WOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOT!

(I'd use emoji, but drupal.org doesn't support those :P)

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented

This was for 8.2. For 8.3, we now have #2797169: Mark BigPipe as stable/non-experimental.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.