BigPipe is a primary user of the Ajax API. We will remove the dependency between big_pipe.js and ajax.js as beginning of decoupling from the Ajax API.
We create a limited command and command queue system in big_pipe module JavaScript that only provides the small set of commands needed by the existing BigPipe implementation.
The existing solution requires themes to implement javascript methods if the theme changes the markup for system messages. The related issues capture the desire by many to return message formatting to standard template system so that the HTML can be overridden in the usual way. This work will need to be completed before we can refactor the PHP dependencies on the Ajax classes.
We may also wish to explore stacking two sequential templates, one with CSS/JS asset markup and one with the placeholder markup.
No UI features in BigPipe.
None
TBD
TBD
The Big Pipe module does not need jQuery anymore. We are using htmx to handle content replacement for bigpipe.
| Comment | File | Size | Author |
|---|
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
compare changes, plain diff MR !12576changes, plain diff MR !12295 / changes, plain diff MR !12255changes, plain diff MR !12230compare compare compare 
Comments
Comment #5
nod_It's green:)
Comment #6
fathershawnAll green is super - nice work! This is direction really simplifies the changes in big_pipe.js and that’s a real plus.
Because bigPipe as we have it re-uses existing javascript from the ajax system, to remove the ajax system we need to replace that.
@nod_ had this list in a Slack discussion:
I think we need to identify the pieces of this list that htmx is not going to do, and implement those in general scope. Then the items that htmx does just fine in an ajax context, we need to have as helper methods in big_pipe.js and leverage the htmx api where we can.
I'm going to take them in order.
MessageCommand
In the ajax context, I would have implimented this in HTMX using a header and a custom event handler that calls our
Drupal.Messagefunction. This is actually the example in the trigger headers documentation.One thing that
Drupal.Messagedoes client-side is determine the selector for the message container. If there is a way to do that server-side then this could also be implemented as an extra out-of-band swap on the response without any additional javascript.We do need a helper method for this that we can use in big_pipe though.
RedirectCommand
HTMX has both the HX-Redirect and HX-Location response headers. We don't have a response so we need a function for that in bigPipe.
ReplaceCommand
We also just need the bigPipe use case for bigPipe. This is one of the core competencies of htmx in an ajax request context.
addCSS and addJS
In the ajax context using htmx, these feel like out of band swaps as well.
Comment #9
fathershawnComment #10
fathershawnHere's an MR into @nod_'s branch moving the bigPipe command JS into bigPipe scope. It looks like the test setup scripts don't like MR's into a feature branch. The tests generally pass locally - 2 failures involving noJS exceptions. Not sure if that's a false negative as all of these changes are in JS.
If this is an acceptable direction then I think the next level of refactoring is to create the needed bigPipe Command classes to replace the AjaxCommands and update BigPipe.php to work without the Ajax API classes.
Comment #11
fathershawnComment #13
fathershawnI still need to deal with Messages and Redirects, but this approach is simpler and over a 10 request sample results in a mean time of Load of 178.6 ms compared to 185.4 ms for the existing 11.x branch.
Comment #15
nod_I'll close my branch that replace commands execution code with a htmx-powered code. This issue is way too late for 11.2 so let's try to do it properly for 11.3 :)
Comment #16
fathershawnComment #17
fathershawnComment #18
fathershawnComment #19
fathershawnBigPipeRegressionTest::testMessages_2712935is now passingComment #20
fathershawnBigPipeRegressionTestis now completely passing.Comment #21
fathershawnAll tests green!
If this approach is good we should open an issue to theme individual messages to provide a server side way for themes to specify message markup
Comment #22
catchHaven't reviewed the MR yet but there's a longstanding existing issue with AJAX messages here #3396318: AJAX MessageCommand markup and styling differs from Theme default where either a drupalSettings or
templateapproach is being looked at to allow server-side rendering of the markup. BigPipe is currently the most common way that messages get added to the page by MesssagesCommand (since 10.3) which caught out a lot of themes that weren't theming js messages (#3456176: 10.3 upgrade now missing status-message theme suggestions).Comment #23
fathershawnThanks for the issue link in #22 @catch! I'll go there and chime in!
Comment #24
fathershawnPlease add credit for @jrockowitz and @richgerdes who collaborated on test refactoring at the DrupalNYC Contribution Day
Comment #27
nod_credit
Comment #28
fathershawnI posted in #3396318: AJAX MessageCommand markup and styling differs from Theme default and #3456176: 10.3 upgrade now missing status-message theme suggestions proposals that would simplify the work in this issue.
Comment #31
fathershawnComment #32
fathershawnI think the only unresolved feedback relates to message theming and swapping. I would propose that those are handled in the related issues on message theming.
Comment #33
larowlanI took a look at the current MR here, and also MR 12295. With the current MR, I'm concerned about the number of changes needed in the BigPipe PHP code, but also the need to recreate the message handling.
Looking at MR 12295 it looks much more isolated to just the handling of AJAX commands.
Is it an option to split this work into smaller, more-reviewable chunks? And if so could we start by re-opening12295 and getting that in? I think that reduces the amount of changes needed and increases our chances of moving forward here.
Thanks for working on this
Comment #36
fathershawn@larowan I've updated the issue summary to give an overview of how we got to where we are and the scope of MR-12295, which is our current MR. This MR combines work that @nod_ and I each did.
\Drupal\big_pipe\Render\BigPipeand clientside code are strongly connected so it is difficult to tease this apart. However, there is strong test coverage for BigPipe and we went through iterations to pass the tests.Comment #37
fathershawnComment #39
fathershawnMR-12476 communicates to me that I did not understand the limited scope of this issue. I understood that our goal here was remove the dependency between BigPipe and the existing Ajax API. @larowan has stated that to be too much change and @nod_ proposes to leave the dependencies between BigPipe and the four Ajax classes.
My question then is what is the plan and time line to remove these dependencies?
Comment #40
godotislateI haven't been following this issue closely, but speaking generally: often in situations like this, remaining work gets split off into follow ups. Reducing scope in individual issues makes the diffs smaller, easier to review, and more likely to be committed.
Comment #41
nod_@fathershawn you didn't misunderstand, I think I didn't explained it clearly enough.
My initial plan was quick and dirty: do the replacement of the ajax commands only so we can get that done quickly and move on. When you added your MR that went with a more HTMX way of doing it, it looked interesting: changing ajax commands for a meta header and raw HTML looked like a great improvement, and you were really involved in making it work so I though it'd be good to try it out. It started to get hard with asset handling and messages. To make it work in that new way the changes and compromises started to become too much for comfort. After chatting with larolwan and catch we figured the less user-disrupting way to go about it was to do the initial quick and dirty implementation. That way if something breaks, we know it comes from a mismatch between what we expect and how htmx does things, because it's the only thing we changed.
Bigpipe is not broken, if we change the data format, how the data is sent or processed we risk breaking it in ways that are not obvious. Check out the history on the big_pipe.js file, there has been many fixes after we moved to the mutation observer handling, it took many years to find the bugs and fix them. I don't want to add more work on this area when there are other things broken we could work on.
Dealing with how messages are handled is going to be easier and less stressful when the change is by itself, not blocking a big feature MR. I second @godotislate assessment the smaller the diff the more likely it is to get committed quickly.
For the timeline, when we'll get to removing the ajax commands from the PHP side is when we'll change how things are done in bigpipe. Until then there isn't much more to do on the bigpipe side of things.
Comment #42
fathershawnThanks @nod_ for the clear explanation. :)
I'm happy to have learned about the internals of our BigPipe implementation. It was also worthwhile to understand how to remove the dependencies on the PHP side and that if we are to move to an HTML based approach, we need a way of rendering individual messages.
+1 for removing the ajax dependency on the client-side. I'm going to update the issue summary with some of what you wrote and change the issue title to reflect the scope.
Comment #43
fathershawnComment #44
catchWe should open an explicit follow-up to un-AJAXify the PHP side. I think #3396318: AJAX MessageCommand markup and styling differs from Theme default might cover the messages issue, but does that need an HTMX equivalent or should try to fix that for AJAX and then use the same technique for HTMX?
Comment #46
fathershawnFollow-up issue to "un-AJAXify the PHP side" created and linked in the sidebar
Comment #47
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #48
nod_Comment #49
nod_Comment #50
fathershawnI think this is ready
Comment #51
catchI'm not qualified to review the js changes here, but +1 on the overall change. We'll be able to remove some of the bigpipe-specific logic when we have more of the AJAX to HTMX conversion in place, but this is a good incremental step. It's also a tangible front end performance improvement reducing the page weight when big pipe is loaded - which is often the only thing on the page that needs the AJAX library at all.
Comment #53
pdureau commentedCommitted 6fba210 and pushed to 11.x. Thanks!
Comment #54
catchComment #56
wim leersCame here via https://tresbien.tech/blog/drupal-bigpipe-now-uses-htmx/ — this looks like an awesome iteration to keep BigPipe's code healthy and maintainable! 🤩
Comment #58
drumlin44 commentedHello -- I did notice that, if there are multiple placeholders with identical IDs, only one seems to get replaced. The
insert()command inbig_pipe.commands.jscurrently only performs its action for a single target matching the selector. Is this intentional behavior? I believe it is a change from what happened before.Comment #59
andypost@drumlin44 please file new issue
Comment #60
drumlin44 commentedThank you @andypost -- I posted an issue at https://www.drupal.org/project/drupal/issues/3564880.