big_pipe sometimes fails to load blocks

What does "fail to load fully" mean, technically?

I have a similar issue. A views block will not appear on the first page load after a drush cr but will appear on subsequent loads. If I disable big_pipe, the issue is resolved.

Any JS errors in your browser's console?

Same error.
Use paragraph type > Reference View > Need to display all entities(more then 1000).

Showing only this

<span data-big-pipe-placeholder-id="callback=Drupal%5Cviewsreference%5CPlugin%5CField%5CFieldFormatter%5CViewsReferenceFieldFormatter%3A%3AlazyBuilder&amp;args%5B0%5D=10_foot_club&amp;args%5B1%5D=stats&amp;args%5B2%5D=a%3A5%3A%7Bs%3A5%3A%22pager%22%3BN%3Bs%3A8%3A%22argument%22%3BN%3Bs%3A6%3A%22offset%22%3BN%3Bs%3A5%3A%22title%22%3BN%3Bs%3A5%3A%22limit%22%3BN%3B%7D&amp;args%5B3%5D=a%3A0%3A%7B%7D&amp;args%5B4%5D=1&amp;args%5B5%5D=paragraph&amp;args%5B6%5D=15913&amp;args%5B7%5D=field_reference_view&amp;token=oeycRco-pkt2qUgNdMJ9n-NfJQO3ZDlqgl2v0GMJnlE">

No errors, no loading, nothing

REMARK: This error uppear then used in paragraph type > Field (Views Reference Field) > add large view.

Bumping to critical since this potentially renders a site unusable (i.e. if critical content just doesn't load).

How to reproduse other situation.

I have a node with custom field (structure: value, value_old, target_type, target_id)

Attached this field to node and add 150 row.

Page get some time to load but 3-5s, but the block: block-local-tasks-block is not loading at all.

<span data-big-pipe-placeholder-id="callback=Drupal%5Cblock%5CBlockViewBuilder%3A%3AlazyBuilder&amp;args%5B0%5D=adminimal_theme_primary_local_tasks&amp;args%5B1%5D=full&amp;args%5B2%5D&amp;token=3w6tD5gBLvpMvrRJQu4RJQdcYxhi3Ol7JBsSTj7VCMQ"></span>

Module bigpipe and big_pipe_sessionless enabled. (If disable big_pipe_sessionless, no changes.)

Running into this as well and the set up is the same as #6. I don't think we get to the 1000 items in a view (tho it's got 100s of rows), but the behaviour is the same.
The span is rendered with the big-pipe attributes and nothing happens, no JS error, call or anything.

--

Update: it may or may not be related, but sometimes, upon a hard refresh, sometimes I get the content to load, but I get an error like this

Symfony\Component\HttpKernel\Exception\BadRequestHttpException: Invalid contextual ID specified. in Drupal\contextual\ContextualController->render() (line 78 of /app/web/core/modules/contextual/src/ContextualController.php).

It might be due to the headers size. I remember having issues with varnish in the past if the payload or headers were too big, but I don't know if this can be related.

Update, the view being rendered that contained a few hundreds of rows was trying to render contextual links for each individual node, as well as some of the elements within it. For example, each node was rendering tags, which was also offering contextual links.

This meant that it was trying to render +1K contextual links and either the server, or BigPipe, or both didn't like it, as they were being requested as a big payload (with all node IDs and tag IDs).

As a workaround (not a real fix), in my case, I used this hook to get rid of them, so we wouldn't even request them to the server:

/**
 * Implements hook_entity_view_alter().
 */
function YOURMODULE_entity_view_alter(array &$build, EntityInterface $entity, EntityViewDisplayInterface $display) {
  // "if" with any condition that can isolate the entity in a known context. ie: view_mode only used on a view.
  if (in_array($build['#view_mode'], ['listing', 'icon'])) {
    unset($build['#contextual_links']);
  }
}

This might not be the same for the users that reported it, but I'd try to debug in an environment that works (locally?) and inspect the headers or payloads from the requests and see if that could be the reason.

@fjgarlin could you try with #3196973: Use Mutation observer for BigPipe replacements reverted? And also with #3387039: Large placeholders are not processed?

#10 I have also problem also with other situation where i not use View. For exemple block local_task.

This isn't only occurring with Views (which I am experiencing BTW with Views with both large and small numbers of rows).

In my case, the more critical issue that I am experiencing is with Commerce blocks. On product pages, sometimes the Variations/Add to Cart form does not appear - even for anonymous users (I am using Layout Builder to add the Variations block to the right column of a 2 column layout). To me, this definitely makes this issue critical and it renders Commerce useless if users can't add a product to their cart. It also sometimes prevents the Cart block (link with number of items in your cart) from rendering. So even if a user does manage to add a product to a cart, they can't click on the Cart block to view its content and/or Checkout!

Like other users above, when the block fails to load, I can see

Is this a regression as compared to the past?

If it is, this could relate to changes in Drupal Core or in perhaps, in changes web servers or some other element.

I’m not seeing this issue on D9.5, only D10. And at that, the only sites I’ve upgraded to 10 are all running 10.1, so not sure if it affects 10.0.

Also, all sites are on the same server set-up. The only difference between the D9.5 sites and the D10.1 sites is the PHP version. My D9.5 sites are on PHP 8.0 and the D10 sites are on PHP 8.1.

Not sure what anyone else’s setup is like.

Also, just to note, the Stack Exchange article referenced above mentions an out of date jQuery file causing the issues in that users theme. My themes don’t use any jQuery.

I'm pretty sure it's due to #3196973: Use Mutation observer for BigPipe replacements but I don't have a site experiencing the problem to test with.

#3387039: Large placeholders are not processed is a nearly RTBC fix to that logic, but at least one person on this issue claimed it didn't fix the problem for them.

So ideally we need people experiencing the issue to test with a revert of #3196973: Use Mutation observer for BigPipe replacements and separately with applying #3387039: Large placeholders are not processed and see whether either or both fix the issue.

#3387039: Large placeholders are not processed fixed the problem for me. I could not find the comment saying the fix doesn't work. They mention fix from #2 (on that issue), and the MR contains all the changes from #2 plus comments and tests. I marked it as RTBC.

I did not try the revert of #3196973: Use Mutation observer for BigPipe replacements.

I've just committed #3387039: Large placeholders are not processed.

The next scheduled patch release for Drupal 10 is beginning of December, in the meantime, sites can apply the diff from the MR.

If you've been experiencing this issue, please test with that MR applied, and report back here especially if it doesn't work for you, but also with clear steps to reproduce.

Moving this to 'needs more info' until the above is done.

#3387039: Large placeholders are not processed fixed the problem for us as well!

My Tabs still missing.
Not always loading ...

I'm facing the same issue, this just appeared on the Drupal 10 version, most of the times is the local tasks block that is not rendered, i don't see any js errors on the console or bad requests.

If someone has an idea of how to fix or create a workaround please share it.

I Have the last drupal 10 version 10.1.6
@seanB what i have to do to fix the issue?

Thanks,

Hi all,

facing the same issue with my views block where I also use better exposed filters. On the first page load the view disappears however after second refresh it comes back.

I also noticed that sometimes the view would load but then I get this javascript error when I try to interact with the filters.

https://ibb.co/CsQp02K

Once I disabled the module, my view was displaying correct and the ajax error was gone.

This only happened for logged in users.

Is there any fix for this please or workaround?

Thank you :)

@seixas for now you could download and apply the patch from #3387039: Large placeholders are not processed using composer to temp fix it until the next Drupal 10.1.x release.

#3387039: Large placeholders are not processed is in 10.1.6, so if you're on that version, you shouldn't be hitting that specific bug. That doesn't rule out that there could be an additional bug, but please be precise if you're reporting on here which version you're using.

Also, if the issue is that bigpipe placeholders aren't being replace, you should be able to still see them as code comments when inspecting the HTML. This will help to differentiate a placeholder being replaced, vs a block not rendering for other reasons - which does sometimes happen (incorrect cache contexts and etc.).

Hi,

Thanks SeanB and Catch, when I found this bug I had version 10.1.5, after updating to 10.1.6 it seems to be more difficult to replicate this bug, only appears once every 20+ refreshes, I'm using the "Claro 10.1.6" administration theme and the only block where i detected this issue is the "local tasks" one.

Another thing worth mentioning is that I can only replicate this problem if I create for example a new "Node" and have the cache completely turned off, if I turn on the normal Drupal cache I cannot replicate it anymore, for i will let the cache enabled (Maybe this can also work for you @mohithasmukh)

the only block where i detected this issue is the "local tasks" one.

And have you verified that there's an un-replaced big pipe placeholder for this block?

Another thing worth mentioning is that I can only replicate this problem if I create for example a new "Node" and have the cache completely turned off, if I turn on the normal Drupal cache I cannot replicate it anymore, for i will let the cache enabled

Which cache are you disabling?

Hi Catch,

To give you more information about this bug that I am getting -

Core version - 10.1.6
Php version - 8.1.18

In my view block I am using better exposed filters as part of the view. The view results are 100 plus. In the better exposed filter settings I have turned on the show only used terms checkbox.

Let me know if you require more information please. I am also happy to take this offline and show you the bug.

Thanks.

We were seeing missing blocks after we updated to D10 (10.1.5) for some roles. We have several blocks that have complicated processing and interactions with the group module for access.

I updated to 10.1.6 and we are not seeing the issue now. We still see a noticeable lag between initial page load and all the page elements appearing, but they all show up now. Our site is 100% authorized users -- none of the blocks are LARGE, but it appeared whatever is taking longer with processing was causing the problem.

We still see a noticeable lag between initial page load and all the page elements appearing,

That's the point of BigPipe: that most of the page is available and not waiting for the slow parts 😄

none of the blocks are LARGE, but it appeared whatever is taking longer with processing was causing the problem.

Interesting, that suggests that it was not just size-related like #3387039: Large placeholders are not processed made it seem, but also timing-related? 🤔 Perhaps it was primarily timing-related, and that just correlates which large size very visibly ("takes more time to generate a boatload of markup")?

So now we have one user report (thanks @pixlkat in #28!) saying it's completely fixed in 10.1.6, and two (@seixas in #25 and @mohithasmukh in #27) saying it is still a problem in 10.1.6. @seixas said it's now only ~5% of page loads, @mohithasmukh did not specify.

That makes it sound more likely that there's still some unknown race condition. To get a sense of how long things are taking here, could you perhaps provide a screenshot of this information for a page load where the bug (missing block) occurred? 🙏

@seixas @mohithasmukh please also provide the HTML output (screenshot from dev tools is fine) of a page where this bug as surfaced - as far as I can see, it's not actually confirmed that the blocks in question are missing because bigpipe placeholders weren't replaced. The placeholders will be in the markup as HTML comments.

@wim, I had to point out that they may see a delay until all content is loaded to my end users; until we upgraded to D10, the page would take a long time to load, but I don't remember seeing the slow loading of the different blocks, so I don't know if/how big pipe was actually working then. We have always had it enabled.

I'm attaching two screenshots from our production environment where we have not upgraded yet (i.e., still on 10.1.5). Two blocks were missing in this case, one is a graph that should not be a very large payload, the other is a block with a view that should also not have a large payload, but may have some views interaction with the group module.

Timing from dev tools:

List of JS files received with size:

We are facing the same issue with Custom Block Plugins, the Block we have has a rendered form within, the output halts after every other refresh. We discovered a temp fix which is to add a random parameter to the URL Example: http://www.example.com/?param=RANDOM_STRING. Perhaps that disables any caching mechanism involved and treats each page request as unique? Not sure.

Drupal Version: 10.1.4
PHP Version: 8.1.13

Reproduction steps:

- Add a Block to any region of the page.
- Refresh multiple times. (Incongnito after first refresh Block disappears).

We have done/observed the following:

- Disabled Cache entirely on the page, disabled cache modules.
- Disabled caching on Twig.
- Cleared Cache tables at every page reload using killswitch service.
- Attempted to disable big_pipe by using hook setting #create_placeolder to false.
- Re-creating the Block.
- No logs reported on watchdog/server logs.
- Disabled Javascript via Chrome Tools, issue still persists.

Would rather have the Drupal core team have a look at this, as it's critical and deemed Drupal 10 sites as unusable due to the nature of this bug making the UX experience unreliable.

@inteeka-help, if you are on 10.1.4, then you are probably still seeing #3387039: Large placeholders are not processed which was fixed in 10.1.6 - you only get core fixes if you update to the version that includes the fixes - asking us to fix things we already fixed does not make them appear retrospectively in 10.1.4.

If you are on 10.1.6 or higher and still getting issues, as above, we still need the information asked for in #30.

--

@pixlkat see #30 - we need to confirm whether this is actually big pipe blocks not being replaced, or whether it's a different issue causing the block not to be rendered, the way to do this is to check the HTML output of the page with the missing block, and look for an un-replaced bigpipe placeholder. If you find that, then confirming that (and showing the markup on this issue) would be helpful.

If we can confirm that, one possible approach would be to roll the bigpipe js all the way back to how it was prior to #3196973: Use Mutation observer for BigPipe replacements and then try that issue again later, but I'm not keen to do that until we actually verify there's still a problem post 10.1.6.

@inteeka-help A maintainer asked for specific information in comment 30.

Everyone affected by this issue: provide the information the maintainer asked for in comment #30 above.

Hi, I should have included a screenshot of the markup when I added my other screenshots. When my blocks were missing, there were big pipe placeholders appearing where the block should have been (it's how I found this issue).

I don't have an environment running right now that I can use to reproduce, and I'm in the middle of prepping for a new production release. I'll go back to 10.1.5 when I have a moment and get a screenshot for you.

@pixlkat if you are on 10.1.5, please update to 10.1.6 or higher first. 10.1.6 already fixed #3387039: Large placeholders are not processed so we cannot fix that twice. If someone is still having issues on 10.1.6 or higher then that is valid, otherwise, it is counterproductive posting about bugs in 10.1.5 or lower on this issue.

@catch, I am on 10.1.6 now. We have not seen the issue since updating core. Sorry if that wasn't clear previously. You'd asked for confirmation if when I was seeing it, I saw unreplaced big pipe placeholders, and I did.

And, I take that back. Our stage environment is running Drupal 10.1.6. We have received no reports of this happening on production, but load is probably pretty low right now.

While running a drush command that did a lot of content updates in the background, I logged in to our stage environment as a user with admin role and had the block that contained a view missing. My assumption is that the drush command was using enough resources to slow down the return of the page. See the attached screenshot for the markup from using the browser inspector. The bib pipe placeholder span is where a view should appear.

Hi, I wasn't able to replicate the issue again. #30

the big pipe module seems to be causing some of the blocks on the current site I'm working on to disappear as well.

This problem is appearing on a site running Drupal 10.2.0.
-
In this example, the block is - basic block - containing just copyright declaration text.
When logged in, the block is replaced by a following span element

<div class="region region-sub-footer-first">
	<span data-big-pipe-placeholder-id="callback=Drupal%5Cblock%5CBlockViewBuilder%3A%3AlazyBuilder&amp;args%5B0%5D=cscs_block_subfooter_copyright&amp;args%5B1%5D=full&amp;args%5B2%5D&amp;token=5lVI4mHWyZIwLY6wWm5pTSMuev7kcD5R5ljc1PnhoYU">
	</span>
</div>

When logged out (anonymous mode) the block displays as it should, as follows:

<div class="region region-sub-footer-first">
	<div id="block-cscs-block-subfooter-copyright" class="clearfix block block-block-content block-block-content4e8b2554-2f43-45f8-b73a-fb2ac44c0812">
		<div class="content">
			<div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item">
				<div class="text-md-start text-lg-left">Copyright © 2023 - <a style="text-decoration:none;" href="https://www.someurl.net/">www.placeholderurl.com</a> - All rights reserved
				</div>
			</div>
		</div>
	</div>
</div>

Is there a way to repair this problem?

[ screenshot - - logged in - big pipe span only - block missing ]

[ screenshot - logged out - the block appears as it should ]

I've just run into this problem after upgrading from D9 to D10.2. I get the same sort of results as #41 where the big pipe placeholder span is where the blocks should be. Doesn't seem to matter if i clear the cache or not.

I'm seeing it mostly with Facet blocks on a search_api views page but there are other, less obvious unrendered placeholders on the page. Certain facets seem to come and go as I keep hitting refresh. No errors in browser console or watchdog. Just silently failing to replace the placeholders.

Interestingly, there are script tags at the bottom of the page that appear to have the correct ajax commands to replace the placeholders with block markup.

<script type="application/vnd.drupal-ajax" data-big-pipe-replacement-for-placeholder-with-id="callback=Drupal%5Cblock%5CBlockViewBuilder%3A%3AlazyBuilder&amp;args%5B0%5D=groupname&amp;args%5B1%5D=full&amp;args%5B2%5D&amp;token=PMm7HXc3h6xwp7fdT2QeK23AgPgG_mZIph3vrx5pDhQ">
... AJAX commands JSON .. 
<script>

Is it possible there is a race condition where the first thing rendered is invalidating subsequent tokens somewhere along the line?

That's enough additional reports to move this back to active.

I think we need an MR here which reverts the bigpipe JavaScript to the way it was prior to #3196973: Use Mutation observer for BigPipe replacements - that issue caused the original (fixed) bug but there might be other cases we're not handling yet. Ideally people could then test that MR on their sites that are having this issue to see if it resolves it for them. If that's the case, I think we should roll back and start again on that issue.

I had a bit of a play around to see better what is happening. If you investigate big_pipe.js there is a processReplacement function which is the bit that extracts the JSON and creates that AJAX commands to perform the replacement.

When it fails it is because this fails to get any text content:

content = replacement.textContent.trim();

Based on the issue mentioned in #33 and debugging, I wonder if there is a problem with the mutation observer change..

Edit: Probably should reload my page before posting next time. Bit redundant now given the previous comment. ;-)

Hi. After updating to 10.2.0 on my site for logged in users, blocks began to appear in random order. If the block is not displayed, the code <span data-big-pipe-placeholder-... appears instead.

I found a solution for myself here https://drupal.stackexchange.com/questions/308541/webform-block-shows-fo...

I turned on the cache lifetime to 1 minute, the default value was 0. All blocks are displayed as they should be.

If this is truly caused by #3196973: Use Mutation observer for BigPipe replacements, then I'm wondering if there's a browser-specific problem at play here.

#41, #42, #45: can you please let us know which browser you observed the problem in, and whether you can reproduce it in another browser? 🙏 Thank you!

I tried reproducing this with a simple test case using requests for /big_pipe_test_large_content and having Chrome dev tools throttle the response, but I was unsuccessful - the content was replaced as expected. I work with @pixlkat, but I'm not currently on her project, so maybe when she's back from the holiday, we can debug what she's seeing in #39.

In the meantime, I had a thought about whether, instead of the solution from #3387039: Large placeholders are not processed to observe the character data child nodes and keep checking whether the JSON is valid, an approach like this might work a little better:

1. Wrap the replacement <script> tags, including the start/stop in a <div>. So send a chunk with an open div tag before sending the scripts, then send a chunk with the closing div tag after the stop script
2. Use MutationObserver to observe the childlList of the div, without subtree or characterData
3. In the callback, on each added element, see if the JSON is valid and replace if so. Also, check whether the previousSibling was replaced and replace it if not. This would also cover the last replacement script, since when the stop script tag is added to the DOM, its previous sibling should be ensured to be replaced

This approach means it's possible that it will take two scripts to be added to the DOM before 1 replacement is done, but it also means MutationObserver doesn't have to observe the document.body and its subtree, to reduce the chance there are Drupal behaviors or other observers running that could possibly interfere.

Hi all,
We've recently upgraded to 10.2.0 and we're now experiencing this issue with our search form block. The issue does seem to go away when I am not logged in, but when i am logged in, the block only shows about 20% of the time.

When the block is missing, it's HTML is replaced with the following:

<span data-big-pipe-placeholder-id="callback=Drupal%5Cblock%5CBlockViewBuilder%3A%3AlazyBuilder&amp;args%5B0%5D=spicy_nextopiasearchblock&amp;args%5B1%5D=full&amp;args%5B2%5D&amp;token=jS-7dOtSNj9P4Sz8UWKNnZ1REPBxwc_mLbSlHVEpb64">

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'big_pipe_interface_preview' -->
<!-- FILE NAME SUGGESTIONS:
   * big-pipe-interface-preview--block--full.html.twig
   * big-pipe-interface-preview--block--spicy-nextopiasearchblock.html.twig
   * big-pipe-interface-preview--block.html.twig
   x big-pipe-interface-preview.html.twig
-->
<!-- BEGIN OUTPUT from 'core/modules/big_pipe/templates/big-pipe-interface-preview.html.twig' -->


<!-- END OUTPUT from 'core/modules/big_pipe/templates/big-pipe-interface-preview.html.twig' -->

</span>

I am currently using Chrome 120.0.6099.110

@rsnyd you should switch Twig debug output off - that definitely won't help.

Following up on my own comment in #47: a <div> wrapper can't be placed around the replacement scripts, because unless it's loaded async, big_pipe.js will run before the <div> chunk is sent and becomes available in the DOM.

The other part about observing only the childList in the mutation observer and checking whether the previousElementSibling has been replaced is workable, but it's inconclusive that the change will fix anything since I'm yet not able to reproduce unreplaced blocks.

One more thought is that instead of processing all the outstanding mutations on DOMContentLoaded, maybe it's possible to loop through the remaining keys in drupalSettings.bigPipePlaceholderIds and processing outstanding replacements against those.

@catch,
Thank you. This is from our non-production site. All future comments will not contain the twig debug comments.

I am able to consistently reproduce this, which seems to be one of the big problems on this issue. I'm happy to follow guidance to make changes, but unable to troubleshoot myself.

I have the same issue after update Drupal from 10.1 to 10.2. Some placeholders randomly were not rendered. Example:

<span data-big-pipe-placeholder-id="callback=profile.lazy_load_callbacks%3ArenderMarkup&amp;args%5B0%5D=getFirstName&amp;args%5B1%5D&amp;token=WE7nT_ET1Ho6EgZoIU13pAeumXWjEVSpayPEjH2qttw">
<p class="card-text placeholder-glow">
  <span class="placeholder col-2"></span>
  <span class="placeholder col-5"></span>
</p>
</span>

There are no any additional error in watchdog or console.

It is connected with 10.2 update - 'PHP Fibers support was added to BigPipe and the Renderer, which allows Drupal to potentially run different code while it's waiting for an asynchronous operation to return.' ?

@catch,
You were reporter for both: (https://www.drupal.org/node/3383449 and https://www.drupal.org/node/3377570)

Do you think either of the those two 'fixed' issues are related to this?

@rsynd no I think it's #3196973: Use Mutation observer for BigPipe replacements although this brings up a good point.

People who can reproduce this - on the same page that you find the unreplaced big pipe placeholder, can you check the end of the HTML document and look for the actual replacements too? If the replacement is there, but it's just not put into the right place, then we know it's a JavaScript issue and probably #3196973: Use Mutation observer for BigPipe replacements.

@catch, I checked and found that:

<script type="application/vnd.drupal-ajax" data-big-pipe-replacement-for-placeholder-with-id="callback=profile.lazy_load_callbacks%3ArenderMarkup&amp;args%5B0%5D=getFirstName&amp;args%5B1%5D&amp;token=WE7nT_ET1Ho6EgZoIU13pAeumXWjEVSpayPEjH2qttw">
[{"command":"insert","method":"replaceWith","selector":"[data-big-pipe-placeholder-id=\u0022callback=profile.lazy_load_callbacks%3ArenderMarkup\u0026args%5B0%5D=getFirstName\u0026args%5B1%5D\u0026token=WE7nT_ET1Ho6EgZoIU13pAeumXWjEVSpayPEjH2qttw\u0022]","data":"RXXXXX","settings":null}]
</script>

is here, in my case RXXXXX should be replace with palceholder. So data coma from backend but was not put in correct place.

I've been able to reliably reproduce the issue on my local computer with these steps:

1. Install latest 11.x-dev Drupal with standard profile
2. Disable CSS/JS Aggregation. I don't know if this makes a difference, but documenting in case
3. Set $settings['extension_discovery_scan_tests'] = TRUE; so that test modules can be enabled
4. Enable big_pipe_regression_test module
5. Place the Recent Content block in the Content region of the default theme (olivero) below the main content block. Don't need to create any nodes
6. Visit /big_pipe_test_large_content
7. After the page loads, after all the "boings", the unreplaced element for the views block persists and looks like:
   

   <span data-big-pipe-placeholder-id="callback=Drupal%5Cblock%5CBlockViewBuilder%3A%3AlazyBuilder&amp;args%5B0%5D=olivero_views_block__content_recent_block_1&amp;args%5B1%5D=full&amp;args%5B2%5D&amp;token=D8eu9554J7yRwNmrp0zO-JbnU1s0ATBmtdMMFxK83uI">
   </span>

8. Note that reloading the page makes the block appear, probably because it's been cached. But the first page visit after clearing the cache with drush cr has the unreplaced <span> again

I have not been able to reproduce this with an automated test - probably needs some combination of other JS/blocks provided by Standard that are not present in a minimal test config.

Anyway, apologies for burying the lede, because I think I've found the root cause. As discussed in #3387039: Large placeholders are not processed, it is possible for the mutation observer to observe incomplete script nodes added to the DOM. The solution provided observes the subtree for characterData changes until the text node contains valid JSON. However, after debugging the JS, I've found that it is possible that the <script> can be observed first with completely empty content. The mutation observer subsequently observes the child text node with the JSON, but it can be a complete node at this point, meaning that the detection for a characterData mutation fails. It's whole child text node that is added.

I'm attaching two patches. One is a minimal patch that will act on the child text node parent when it is added: 3390178-check-parent-node-try-2.patch

The second patch instead contains changes I mentioned in #47 and #50: mutation observer only observes the childList of the body for new nodes. Incomplete nodes are addressed by re-checking the previous element sibling of new nodes: 3390178-check-previous-element-sibling-try-2.patch.

I think the second approach is better because it reduces the scope of the observer.

Additionally, I wonder if adding a sledgehammer approach like this on window load (because apparently there are (edge?) cases when DOMContentLoaded doesn't fire?) might also prevent replacements from not happening at all:

  window.addEventListener('load', () => {
    // This is a fallback in case replacements were somehow not completed, or
    // DOMContentLoaded was not triggered. Any remaining keys in
    // drupalSettings.bigPipePlaceholderIds represent IDs of content not
    // replaced, so find the matching script and do the replacement.
    observer.disconnect();
    for (const key in drupalSettings.bigPipePlaceholderIds) {
      const replacement = document.querySelector(`script[data-big-pipe-replacement-for-placeholder-with-id="${key}"]`);
      if (replacement) {
        processReplacement(replacement);
      }
    }
  });

Oops, small mistakes with both patches. Trying again.

Thanks @godotislate for both patches.

The first patch worked fine for me, but the second patch only fixed the issue on smaller pages with less content. On larger pages, the issue still appears, although it did reduce the issue significantly.

Given how user-facing this bug is and how hard it's been to reproduce consistently at all, I think I'd be fine with committing a fix without automated tests here - we can open a follow-up to figure that bit out. Especially the first option in #58 is very minimal.

Converted #58.1 to an MR including the prettier cs fix (I hope). See if it passed the existing bigpipe test coverage to start with.

@godotislate

I have not been able to reproduce this with an automated test - probably needs some combination of other JS/blocks provided by Standard that are not present in a minimal test config.

It's OK to use the standard profile in the test, like protected $profile = 'standard'; - if we can get a failing test with that, we can always work backwards to the testing profile from there.

To try to test for multiple missing replacements, I went another way with the test and just rendered a ton of node entities with lazy builders. With enough nodes (4000), this seems to reliably create some (~10) failures. See test only results.

Pushed up commit to PR and rebased against 11.x.

Left a comment about using the ticket number as the function name. Can we use something else?

Addressed MR comments and rebased.

Hiding patches for clarity.

Seems feedback has been addressed. Tests are green and changes seem fine.

Pushed a change to try to make the test faster, will revert if it doesn't work

So the test change fails, even with the fix. Does that mean we're still missing something?

I think it's failing because big pipe requires a logged in session to work, and that was removed from the test.

Saw your comment about the session, that probably explains the fail, pushed fix

Ok, that's looking better failing/passing as expected but a lot faster than creating nodes

Ready for review

Damn, that test passes on gitlabci, fails locally - probably a performance thing?

Ok, bumping it to 3k failed as expected, and ran in 6s which is faster than the 45s the node version took

Ready for review again

RTBC +1

Left 1 small comment.

Was a bit concerned about creating that many nodes, the revised test coverage looks great, and really, really good that we found a reproducible test case for this.

I pinged @smustgrave because I'm not sure which comment #78 refers to.

Moving this to RTBC - I can commit this tomorrow if no-one beats me to it and there's no more feedback.

Thanks everyone for the reports, this was hard to track down in combination with the large placeholder issue, but hopefully it's fully covered now.

Committed/pushed to 11.x and cherry-picked to 10.2.x, thanks!

Excellent work here! 👏

This has caused quite a few of our JS tests to start failing (in CI only for whatever reason) with

TypeError: Cannot read properties of null (reading 'nodeType') at checkMutation

I assume this is something to do with node.parentNode being NULL.

EDIT: Created #3417672: big_pipe JS checkMutation can throw type errors if parentNode is NULL to track this.

I'm still having this problem on 10.2.2. but my console errors (anonymized) read:

An error occurred during the execution of the Ajax response: LoadJS
  (anonymous) @ js_[hash].js?scope=footer&delta=0&language=en&theme=mytheme&include=[hash]:58
  Promise.catch (async)
  Drupal.Ajax.success @ js_[hash].js?scope=footer&delta=0&language=en&theme=mytheme&include=[hash]:58
  processReplacement @ js_[hash].js?scope=footer&delta=2&language=en&theme=mytheme&include=[hash]:67
  checkMutationAndProcess @ js_[hash].js?scope=footer&delta=2&language=en&theme=mytheme&include=[hash]:67
  (anonymous) @ js_[hash].js?scope=footer&delta=2&language=en&theme=mytheme&include=[hash]:67
  processMutations @ js_[hash].js?scope=footer&delta=2&language=en&theme=mytheme&include=[hash]:67

For what it's worth, the problem only seems to occur on pages that use Layout Builder. It's not a problem when the page was generated in Views.

Disabling Big Pipe clears the problem and it comes back when I re-enable it.

I'm still having this bug. I'm use Drupal Version 10.2.6 with Commerce. Block with commerce products disappeared.
html:

<span data-big-pipe-placeholder-id="callback=Drupal%5Cblock%5CBlockViewBuilder%3A%3AlazyBuilder&amp;args%5B0%5D=views_block__frontpage_products_block_1&amp;args%5B1%5D=full&amp;args%5B2%5D&amp;token=ASqtYUsg34Tt0QgwBdweGeBB2vj6r50LC9lj4QaMtMk">

<!-- THEME DEBUG -->
<!-- THEME HOOK: 'big_pipe_interface_preview' -->
<!-- FILE NAME SUGGESTIONS:
   * big-pipe-interface-preview--block--full.html.twig
   * big-pipe-interface-preview--block--views-block--frontpage-products-block-1.html.twig
   * big-pipe-interface-preview--block.html.twig
   x big-pipe-interface-preview.html.twig
-->
<!-- BEGIN OUTPUT from 'core/modules/big_pipe/templates/big-pipe-interface-preview.html.twig' -->


<!-- END OUTPUT from 'core/modules/big_pipe/templates/big-pipe-interface-preview.html.twig' -->

</span>

Good morning i am having similar problòems too,
on the homepage of the website i have to display 3 variants of the same view but only one is rendered while the others aren't resolved by bigpipe
diving into the requests looks like bigpipe forces loadjs to load the same css 3 times and this cause an exception breaking bigpipe.
This happens only when i am logged in, anonymous user isn't affected by this

Same problem as #91 here. Content is not rendered when logged and properly rendered for anon users.

Same problem, Content is not rendered when logged and properly rendered for anon users.
I use `attach_library` to load library using Vite in block language, the block is not rendered when logged and rendered for anonymous users.

If i add cache context to this core language block, it's ok (https://www.drupal.org/files/issues/2232375-54_0.patch)
If i disable vite on this library without add cache context it's also ok.

I get the same problem as #91. I have multiple views in my content area but only the first one is rendered when logged in. When not logged in, they render properly.

I'm having this problem too. At first, I did not notice that this problem only occurs when one is logged in. That helps, because my users do not log in. Nevertheless, would be great if a solution could be found.

A question: I read above that disabling bigpipe also solve this problem. I have a lightweight website, mostly only text. Would it then be okay to disable bigpipe, or does that lead to other problems?

Best regards,
Mariska

Hello, I am also facing same issue. The language switcher block I have added in header section. It shows for anonymous users but not for logged in user.
Getting issue with the big_pipe module.
Other than disabling core big_pipe module.
Could any one help with other solution.

Thanks in advance!!

For those, having this JavaScript error as mentioned by @tonka67 in #89 and others:

An error occurred during the execution of the Ajax response: LoadJS

I don't think that problem is directly related to this and this issue is closed, so I created a separate issue here:
#3463875: Ensure uniqueBundleId is unique in LoadJS

That new issue includes a patch that fixes the bug.

I just upgraded to 10.3.1 from 10.2.7 and views blocks on a page disappeared. I just saw big pipe placeholders. I can probably live without it, but I wonder if there is a better solution.

Same as #91

Anon and administrator get full content. A lesser permission level gets 'span data-big-pipe-placeholder-id=....' on the second and third variants of views blocks

Drupal 10.3.2

Hello, I am facing same issue as well. On 10.3.1 with Group 3.2.2 I have a view which shows group contents of a given group (via contextual filter). The view page works fine, the view block (limited to 10 nodes) rendered on the group page doesn't show, there is just the BigPipe placeholder. When disabling Big Pipe the block is rendered correctly. This is for admin user. I cannot provide information for anonymous user as I haven't configured group permissions for that yet.

Instead of fully disabling/uninstalling Big Pipe it also works to disable it just for the affected blocks (if that is deterministic). in my case I just disabled it for all views blocks:

/**
 * Implements hook_block_build_BASE_BLOCK_ID_alter().
 */
function mymodule_block_build_views_block_alter(array &$build, \Drupal\Core\Block\BlockPluginInterface $block) {
  // Disable placeholdering of this block.
  // @todo Remove this when "big_pipe sometimes fails to load blocks" is solved. See: https://www.drupal.org/project/drupal/issues/3390178#comment-15742673
  $build['#create_placeholder'] = FALSE;
}

I am currently experiencing the next issue, if I have exposed filter or a custom block plugin with a form and it contains "select" form element, the block will not be displayed.
1) removing or replacing a form element with "radios" solves the issue.
2) Disabling big_pipe modules solves the issue
3) Solution from #101, also solves the issue

Drupal 10.4.8

Why is this issue closed when many people are still having the problem?

If you're still experiencing this problem, please open a new issue and link to it from this one.