Invalidating cache tags inline can cause deadlocks, as per the D8 core issue.

Fabian and I are experimenting with a core change to allow callbacks to run after a root transaction completes, so we can give cache tag invalidation its own transaction and run the invalidations in a deterministic order.

Comments

bdragon created an issue. See original summary.

bdragon’s picture

Status: Active » Needs work
StatusFileSize
new2.98 KB

Initial version of patch.

Still needs a fallback to cope with running without the core patch.

bdragon’s picture

Status: Needs work » Needs review
StatusFileSize
new4.28 KB

Add backwards compatibility code that falls back to the old implementation when callback support is missing.

fabianx’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +scalability, +release notes

RTBC - I'd like to get this in, so that people can start testing the core patch

stevector’s picture

Status: Reviewed & tested by the community » Needs work

Hi Fabian and Brandon! Thanks for working on this issue.

I think _d8cache_has_transaction_callbacks() is returning the opposite boolean value that it should.

Instead of

function _d8cache_has_transaction_callbacks($connection)
  if (method_exists($connection, 'addRootTransactionEndCallback')) {
    return FALSE;
  }
  return TRUE;
}

Shouldn't it be

function _d8cache_has_transaction_callbacks($connection)
  if (method_exists($connection, 'addRootTransactionEndCallback')) {
    return TRUE;
  }
  return FALSE;
}

or just

function _d8cache_has_transaction_callbacks($connection)
   return method_exists($connection, 'addRootTransactionEndCallback');
}

With these patches applied (and the boolean flipped) I did not get deadlocks.

To replicate the deadlocks I made this branch in Pantheon Advanced Page Cache: https://github.com/pantheon-systems/pantheon_advanced_page_cache/compare... It reuses a previous Behat step I had written to test very long headers. The step saves nodes that reference hundreds of taxonomy terms (just with the free tagging field on the article content type in the Standard profile). By invalidating cache tags for those taxonomy terms and running Behat in two window simultaneously I can reliably replicated the deadlocks.

fabianx’s picture

Great catch and thanks for verifying that this indeed solves the deadlocks.

  • bdragon authored 96cf80f on 7.x-1.x
    Issue #3004429 by bdragon, Fabianx, stevector: Invalidate cache tags...
bdragon’s picture

Status: Needs work » Active
Issue tags: +Needs documentation

Apologies for the delay, this is in now.
Back to active for documentation.

  • 0677a65 committed on 7.x-1.x
    Update README.txt for #3004429 interim workaround.
    
bdragon’s picture

Status: Active » Postponed

Readme updated in 0677a657872141aef5e7239e2e44fd9361b53637 for the interim workaroud.

I'm setting this to Postponed because it will need another update once the native support for post transaction callbacks has landed.

bdragon’s picture

Issue tags: -Needs documentation
fabianx’s picture

Title: Invalidate cache tags after transaction » [PP-1] Invalidate cache tags after transaction

Adding that this is postponed just on one issue.

bdragon’s picture

Status: Postponed » Needs review
StatusFileSize
new1.27 KB

Reopening temporarily to handle an update to the API.

fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC - Excellent work!

  • bdragon authored 4fb8f3f on 7.x-1.x
    Issue #3004429 addendum by bdragon, Fabianx, stevector: Update API usage...
bdragon’s picture

Status: Reviewed & tested by the community » Postponed

Committed API update.

Back to postponed.

fabianx’s picture

Status: Postponed » Active

I decided to revert the PR again and support the original API idea instead.

bdragon’s picture

Status: Active » Needs review
StatusFileSize
new1.47 KB

Rework again for readding the $committed parameter.

fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC - We decided to change this back, because we got meanwhile consensus in D8 to add the $committed parameter.

Thanks!

  • bdragon authored bd1b78e on 7.x-1.x
    Issue #3004429 by bdragon, Fabianx, stevector: [PP-1] Invalidate cache...
bdragon’s picture

Status: Reviewed & tested by the community » Active

Committed, back to active.

wim leers’s picture

Title: [PP-1] Invalidate cache tags after transaction » Invalidate cache tags after transaction
wim leers’s picture

Status: Active » Needs work