Update to 3.5 replaces purge_drush but does not disable it

Same for us - similar problem.

PHP Fatal error:  Uncaught Error: Call to a member function prepare() on null in /var/www/web/core/lib/Drupal/Core/Database/StatementWrapper.php:51
Stack trace:
#0 /var/www/web/core/lib/Drupal/Core/Database/Connection.php(630): Drupal\Core\Database\StatementWrapper->__construct()
#1 /var/www/web/core/modules/mysql/src/Driver/Database/mysql/Insert.php(42): Drupal\Core\Database\Connection->prepareStatement()
#2 /var/www/web/modules/contrib/purge/src/Plugin/Purge/Queue/DatabaseQueue.php(60): Drupal\mysql\Driver\Database\mysql\Insert->execute()
#3 /var/www/web/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php(235): Drupal\purge\Plugin\Purge\Queue\DatabaseQueue->createItem()
#4 /var/www/web/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php(201): Drupal\purge\Plugin\Purge\Queue\QueueService->commitAdding()
#5 /var/www/web/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php(345): Drupal\purge\Plugin\Purge\Queue\QueueService->commit()
#6 /var/www/web/modules/contrib/purge/modules/purge_drush/src/Commands/PurgeDrushCommands.php(1131): Drupal\purge\Plugin\Purge\Queue\QueueService->destruct()
#7 [internal function]: Drupal\purge_drush\Commands\PurgeDrushCommands->__destruct()

We resolved it by downgrading to previous version 3.4.0.
Site runs on the latest Drupal 9 version

I agree with @pasive. I'm on Drupal 10 and the issue happened when I upgraded the module to version 3.5. Downgrading to the previous version resolved the issue for me.

Huh interesting.... the reason why its not removed is that when you delete a submodule it will cause a WSOD without manual db intervention. Therefore for the entire 3.x branch, the module needs to remain, even if we have it hidden.

The reason why we didn't automatically disable it is because sites could have been relying on the old namespaces, and disabling the module will kill their code. But in theory it should have worked along side the new modules even if enabled.

For API purposes, it should be using the libraries within purge itself. So its odd that its failing. I'll prioritize some time in the next weeks to look at it.

In an effort of trying to clean up the big picture a bit let me summarize the situation:

1. Between the dates when versions 3.4 (Sep 14, 2022) and 3.5 (Oct 12, 2023) were released more than an entire year has passed with lots of changes (commit history). Therefore not fully clear to me why @nicxvan highlighted only those 4 commits in #13 out of the 14 in total. Maybe he is right but I still suspect all changes until getting a deeper understanding of the issue.
2. Also, I feel unsure whether the issue reported in #9 and joined in #10 stems from the same root as the originally reported one. @pasive and @alfattal could you please test and confirm that upgrading to version 3.5, then manually disabling the purge_drush submodule (as it worked for @nicxvan in #15) eliminates the error on your environments as well? If it does then we can keep believing that your issue relates to the originally reported one.
3. Do we agree that any other contrib projects (eg. varnish_purge as mentioned above) are involved in causing the issue? Or was it only an early guess from OP which became obsolete to the current state of our knowledge?
4. Finally, in order to be able to tackle the issue (or issues? – as keeping the question open if we're facing the same bug) we definitely need to fixture the exact circumstances, and then compose a list of steps for reproducing the exception.

@all if you could paste here in a comment the result of the $ drush pdia command that would help a lot to see in which combination are our Purge configurations set up.

Intending to ease the situation, I separated all these changes into the same folder structure on a per-file basis.

@Balu Ertl In my case, I can confirm that disabling purge_drush and then, upgrade to 3.5 eliminated the error. I tried first to upgrade and then, manually disable purge_drush, but that DIDN'T eliminate the error.

Update: I started to get different errors when attempt to sync the database

PHP Fatal error:  Uncaught TypeError: Drupal\Core\Database\StatementWrapperIterator::__construct(): Argument #2 ($clientConnection) must be of type object, null given, called in /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Connection.php on line 527 and defined in /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php:53
Stack trace:
#0 /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Connection.php(527): Drupal\Core\Database\StatementWrapperIterator->__construct()
#1 /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Connection.php(787): Drupal\Core\Database\Connection->prepareStatement()
#2 /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Query/Select.php(525): Drupal\Core\Database\Connection->query()
#3 /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/DatabaseQueue.php(112): Drupal\Core\Database\Query\Select->execute()
#4 /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php(473): Drupal\purge\Plugin\Purge\Queue\DatabaseQueue->numberOfItems()
#5 /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php(209): Drupal\purge\Plugin\Purge\Queue\QueueService->numberOfItems()
#6 /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php(345): Drupal\purge\Plugin\Purge\Queue\QueueService->commit()
#7 /home/ffattal/mndor/docroot/modules/contrib/purge/modules/purge_drush/src/Commands/PurgeDrushCommands.php(1131): Drupal\purge\Plugin\Purge\Queue\QueueService->destruct()
#8 [internal function]: Drupal\purge_drush\Commands\PurgeDrushCommands->__destruct()
#9 {main}
  thrown in /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php on line 53

Fatal error: Uncaught TypeError: Drupal\Core\Database\StatementWrapperIterator::__construct(): Argument #2 ($clientConnection) must be of type object, null given, called in /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Connection.php on line 527 and defined in /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php on line 53

TypeError: Drupal\Core\Database\StatementWrapperIterator::__construct(): Argument #2 ($clientConnection) must be of type object, null given, called in /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Connection.php on line 527 in /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php on line 53

Call Stack:
    3.2812   70395624   1. Drupal\purge_drush\Commands\PurgeDrushCommands->__destruct() /home/ffattal/mndor/docroot/modules/contrib/purge/modules/purge_drush/src/Commands/PurgeDrushCommands.php:0
    3.2812   70395624   2. Drupal\purge\Plugin\Purge\Queue\QueueService->destruct() /home/ffattal/mndor/docroot/modules/contrib/purge/modules/purge_drush/src/Commands/PurgeDrushCommands.php:1131
    3.2812   70395624   3. Drupal\purge\Plugin\Purge\Queue\QueueService->commit($sync_stat = ???) /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php:345
    3.2812   70395624   4. Drupal\purge\Plugin\Purge\Queue\QueueService->numberOfItems() /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php:209
    3.2812   70395624   5. Drupal\purge\Plugin\Purge\Queue\DatabaseQueue->numberOfItems() /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/QueueService.php:473
    3.2813   70399152   6. Drupal\Core\Database\Query\Select->execute() /home/ffattal/mndor/docroot/modules/contrib/purge/src/Plugin/Purge/Queue/DatabaseQueue.php:112
    3.2813   70399304   7. Drupal\Core\Database\Connection->query($query = 'SELECT COUNT(*) AS "expression"\nFROM\n{purge_queue} "purge_queue"', $args = [], $options = []) /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Query/Select.php:525
    3.2813   70399744   8. Drupal\Core\Database\Connection->prepareStatement($query = 'SELECT COUNT(*) AS "expression"\nFROM\n{purge_queue} "purge_queue"', $options = ['fetch' => 5, 'allow_delimiter_in_query' => FALSE, 'allow_square_brackets' => FALSE, 'pdo' => []], $allow_row_count = ???) /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Connection.php:787
    3.2813   70400000   9. Drupal\Core\Database\StatementWrapperIterator->__construct($connection = class Drupal\mysql\Driver\Database\mysql\Connection { protected $target = 'default'; protected $key = 'default'; protected $logger = NULL; protected $transactionLayers = []; protected $driverClasses = ['Schema' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Schema', 'Condition' => 'Drupal\\Core\\Database\\Query\\Condition', 'Select' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Select', 'Upsert' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Upsert', 'Truncate' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Truncate', 'Update' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Update', 'Merge' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Merge', 'Delete' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Delete', 'Insert' => 'Drupal\\mysql\\Driver\\Database\\mysql\\Insert']; protected $statementWrapperClass = 'Drupal\\Core\\Database\\StatementWrapperIterator'; protected $transactionalDDLSupport = FALSE; protected $connection = NULL; protected $connectionOptions = ['database' => 'drupal', 'username' => 'drupal', 'password' => 'drupal', 'host' => 'localhost', 'port' => '3306', 'namespace' => 'Drupal\\mysql\\Driver\\Database\\mysql', 'driver' => 'mysql', 'prefix' => '', 'autoload' => 'core/modules/mysql/src/Driver/Database/mysql/', 'pdo' => [...], 'init_commands' => [...]]; protected $schema = class Drupal\mysql\Driver\Database\mysql\Schema { protected $connection = ...; protected $placeholder = 4; protected $defaultSchema = 'public'; protected $uniqueIdentifier = '6555187ca72bb7.24493336'; protected $mysqlStringTypes = [...] }; protected string $prefix = ''; protected array $tablePlaceholderReplacements = [0 => '"', 1 => '"']; protected $prefixes = []; protected $prefixSearch = []; protected $prefixReplace = []; protected $unprefixedTablesMap = []; protected $escapedTables = ['cache_container' => 'cache_container', 'config' => 'config', 'key_value' => 'key_value', 'cache_config' => 'cache_config', 'cache_default' => 'cache_default', 'cache_discovery' => 'cache_discovery', 'cache_bootstrap' => 'cache_bootstrap', 'cache_data' => 'cache_data', 'router' => 'router', 'path_alias' => 'path_alias', 'sessions' => 'sessions', 'users_field_data' => 'users_field_data', 'cachetags' => 'cachetags', 'webform_submission' => 'webform_submission', 'webform_submission_data' => 'webform_submission_data', 'purge_queue' => 'purge_queue', 'semaphore' => 'semaphore']; protected $escapedFields = ['' => '', 'table_schema' => '"table_schema"', 'table_name' => '"table_name"', 'collection' => '"collection"', 'name' => '"name"', 'config.name' => '"config"."name"', 'cid' => '"cid"', 'expire' => '"expire"', 'created' => '"created"', 'tags' => '"tags"', 'checksum' => '"checksum"', 'data' => '"data"', 'serialized' => '"serialized"', 'base_table.status' => '"base_table"."status"', 'base_table.path' => '"base_table"."path"', 'uid' => '"uid"', 'mail' => '"mail"', 'init' => '"init"', 'pass' => '"pass"', 'tag' => '"tag"', 'invalidations' => '"invalidations"', 'item_id' => '"item_id"', 'value' => '"value"', 'q.item_id' => '"q"."item_id"', 'q.data' => '"q"."data"']; protected $escapedAliases = ['name' => '"name"', 'config' => '"config"', 'expression' => '"expression"', 'base_table' => '"base_table"', 'cachetags' => '"cachetags"', 'item_id' => '"item_id"', 'data' => '"data"', 'q' => '"q"', 'purge_queue' => '"purge_queue"', 'key_value' => '"key_value"']; protected $rootTransactionEndCallbacks = []; protected $identifierQuotes = [0 => '"', 1 => '"']; private array ${Drupal\Core\Database\Connection}enabledEvents = []; protected $needsCleanup = FALSE; private $serverVersion = NULL }, $clientConnection = NULL, $query = 'SELECT COUNT(*) AS "expression"\nFROM\n"purge_queue" "purge_queue"', $options = [], $rowCountEnabled = FALSE) /home/ffattal/mndor/docroot/core/lib/Drupal/Core/Database/Connection.php:527

I try to fix the issue by relocating the update hook enforcing the cache clear from the sub-module to the parent if it's being invoked with a higher chance during an update process.

@all would you mind testing on your environments if this solves the exceptions?

I think this issue is tangentially related: https://www.drupal.org/i/3391383 -- we need to handle disappearing plugins better. Maybe some of the work in the patch there can be used to make handling the missing drush plugins uninstall correctly?

That makes sense.

Just to clarify how I fixed it on my site.
1. Remove processor options from my purgers for all env (using config split)
2. Uninstall purge_drush
3. Add the drush processor back to all env (using config split)
4. Export config and save for all env

This fixed the issue for my site.

Can you explain what "config split" means and why this was neccessary. Thanks!!!

Config split is a module that allows to have different config files applying for different environment (ex: local/test/production environment). If you don't use this module then you just make import and export of the same config files with no environment specificity.

@balu ertl

@all would you mind testing on your environments if this solves the exceptions?

The answer in my case was no, drupal_flush_all_caches(); it didn't solve it.

And unfortunately the manual solution of @nicxvan (#25) isn't something I can do on each of my environment. I can only do that on local.
So I'm still looking for a solution at the moment.

Anywhere we're creating a new instance of the plugin, ($this->pluginManager->createInstance) we're likely needing to wrap them:

• IteratingServiceBaseTrait.php
• InvalidationsService.php
• PurgersService.php --> note the Akami patch already does this
• QueueService.php
• PluginTestBase.php

Although @japerry said in #14:

“The reason why we didn't automatically disable it is because sites could have been relying on the old namespaces, and disabling the module will kill their code […]”

After a couple of days of exploration, I would revisit the above statement. In light of the number of people who had expressed their difficulties due to keeping the sub-module still installed, I believe that those who still rely on a piece of non-functional placeholder code should be a smaller minority. So I concluded that playing by Drupal's own terms is the safest we can do here. Just as ticket title suggests my solution disables the then-non-existent purge_drush sub-module during the update process.

Steps as I tested:

1. Get the fresh source code:
   • for you, it would be $ composer update drupal/purge
   • for me during testing it was $ composer require "drupal/purge:dev-3397227-update-to-3.5 as 3.x-dev" -W
2. Ensure that Drupal's internal mechanisms are notified about the changes: $ drush updb OR $ drush devel:reinstall purge
3. Try the original Drush command if it works (should! – for me at least does): $ drush p:queue-work

My last commit technically automates the steps that @nicxvan has shared in #27 with the difference that you don't need to manually disable the sub-module before the update process.

As it turned out during our recent call with Jakob, I was missing one key information: the reason why he would prefer avoiding uninstalling the sub-module. If some custom or contrib code depends on these Drush commands, then that would cause them a functionality loss. Of course, we can not know if any custom code relies on them among the 30k installation base. However, regarding the contrib space: I did minimal research and went through the modules appearing on the Purge ecosystem list. Only two projects, akamai and purge_control offer any Drush command plugins (here and here) but none of them has anything to do with our sub-module.

In case module maintainers prioritize backward compatibility then such a code refactoring should be taken as a breaking change. Then this MR could be merged onto a newly opened 4.x branch I think.

@nicxvan Now tested again. Performing the steps above (and $ drush cc to empty the plugin cache) leaves the processor plugin instance intact. The only change in site configuration you should see is the removal of this line:

diff --git a/config/sync/core.extension.yml b/config/sync/core.extension.yml
index 7038ab9..d1646f8 100644
--- a/config/sync/core.extension.yml
+++ b/config/sync/core.extension.yml
@@ -44,7 +44,6 @@ module:
   path: 0
   path_alias: 0
   purge: 0
-  purge_drush: 0
   purge_processor_cron: 0
   purge_processor_lateruntime: 0
   purge_purger_http: 0

Just clearing cache after deploying 3.5 to the server does resolve the issue for us. We did not disable purge_drush module.

I spent some time on this today and there are a few things to point out:
1) The original 3.5 code enabled ALL drush plugins by default. Only 3 were supposed to be 'enable_by_default':

drush_purge_queue_add
drush_purge_queue_work
drush_purge_invalidate

I don't -think- enabling the other plugins by default would cause the errors in the database query, but its untested, so who knows that could be it.

2) The 'enable_by_default' is sorta messy with purge. It means plugins are enabled by the mere existence of the module. This means that for those using the above drush commands (and associated plugins), they will DISAPPEAR if you disable purge_drush. You must manually re-enable the modules via the other drush commands (p:processor-add drush_purge_queue_work) or via purge ui.

3) The purge_drush module, with the above PR (purge_drush_fix) only serves to enable those three plugins by default.

4) The plugin cache gets automatically rebuilt when you run updb and have purge_drush enabled or if you manually disable purge_drush and run manually run updb. The classes are pointing to the purge core src directory. The command classes in purge_drush effectively do nothing, with the exception of those who may have extended them in their own custom commands and added annotations.

I'll give this a few weeks (since its christmas 2023) and then commit it if I don't see any responses.

I've tested the two PRs and still getting the error message when running drush cron or drush p:queue-work.

@balu-ertl Regarding your comment in #18, I think you're right. After revisiting this issue, I realized that it was caused by an old patch that is still sitting in my composer.jason file. After removing this patch, I was able to upgrade to version 3.5 without any issue.

I'm getting the same database connection errors as pointed above:

Fatal error: Uncaught TypeError: Drupal\Core\Database\StatementWrapperIterator::__construct(): Argument #2 ($clientConnection) must b  
  e of type object, null given, called in /var/www/site/web/core/lib/Drupal/Core/Database/Connection.php on line 557 and defined in /va  
  r/www/site/web/core/lib/Drupal/Core/Database/StatementWrapperIterator.php on line 54                                                   
                                                                                                                                         
  TypeError: Drupal\Core\Database\StatementWrapperIterator::__construct(): Argument #2 ($clientConnection) must be of type object, null  
   given, called in /var/www/site/web/core/lib/Drupal/Core/Database/Connection.php on line 557 in /var/www/site/web/core/lib/Drupal/Cor  
  e/Database/StatementWrapperIterator.php on line 54 

As far as i can tell, this error started happening at some point and i canñt get it go go away.
I've tried downgrading to purge 3.4 from 3.6 but i'm still getting te error. Also applying patches in this post has not solved the issue.
The error comes up during config-import or any operation invlving my processor, which in my case is varnish_purger.
If i disabel the Varnish Purger processor i get no error. As soon as i enable it via UI or config import i get the error.

I'm clearing the purge cache on node update via a node_update hook, so in fact this is blocking content editing.

Step debugging the code everything seems normal, i can't find the exact spot where this exception happens.
It also happens on drush cr.

Seems to be some query on cache_config table that is passing a null connection. I guess it has to do with the purge_drush module as it seems very similar. But I'm lost here.

protected readonly Drupal\Core\Database\Connection $connection = ... }; protected $needsCleanup = FALSE; private $serverVersion = NULL }, $clientConnection = NULL, $query = 'SELECT "cid", "data", "created", "expire", "serialized", "tags", "checksum" FROM "SITE-PREXIX_cache_config" WHERE "cid" IN ( :cids__0 ) ORDER BY "cid"', $options = [], $rowCountEnabled = FALSE) /var/www/site/web/core/lib/Drupal/Core/Database/Connection.php:557