Problem/Motivation

When running various drush commands using Drupal core v10.6, e.g. drush updatedb:status, I run into the following error:

Error: Cannot use object of type Doctrine\Common\Lexer\Token as array in /var/www/html/web/core/lib/Drupal/Component/Annotation/Doctrine/DocParser.php on line 616 #0 /var/www/html/web/core/lib/Drupal/Component/Annotation/Doctrine/DocParser.php(335): Drupal\Component\Annotation\Doctrine\DocParser->Annotations()
#1 /var/www/html/web/core/lib/Drupal/Component/Annotation/Doctrine/SimpleAnnotationReader.php(104): Drupal\Component\Annotation\Doctrine\DocParser->parse()
#2 /var/www/html/web/core/lib/Drupal/Component/Annotation/Doctrine/SimpleAnnotationReader.php(146): Drupal\Component\Annotation\Doctrine\SimpleAnnotationReader->getClassAnnotations()
#3 /var/www/html/web/core/lib/Drupal/Component/Annotation/Plugin/Discovery/AnnotatedClassDiscovery.php(145): Drupal\Component\Annotation\Doctrine\SimpleAnnotationReader->getClassAnnotation()
#4 /var/www/html/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php(119): Drupal\Component\Annotation\Plugin\Discovery\AnnotatedClassDiscovery->getDefinitions()
#5 /var/www/html/web/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php(213): Drupal\Core\Entity\EntityTypeManager->findDefinitions()
#6 /var/www/html/web/core/lib/Drupal/Component/Plugin/Discovery/DiscoveryCachedTrait.php(22): Drupal\Core\Plugin\DefaultPluginManager->getDefinitions()
#7 /var/www/html/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php(135): Drupal\Core\Plugin\DefaultPluginManager->getDefinition()
#8 /var/www/html/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php(260): Drupal\Core\Entity\EntityTypeManager->getDefinition()
#9 /var/www/html/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php(195): Drupal\Core\Entity\EntityTypeManager->getHandler()
#10 /var/www/html/web/core/lib/Drupal/Core/Datetime/DateFormatter.php(93): Drupal\Core\Entity\EntityTypeManager->getStorage()
#11 [internal function]: Drupal\Core\Datetime\DateFormatter->__construct()
#12 /var/www/html/vendor/symfony/dependency-injection/ContainerBuilder.php(1142): ReflectionClass->newInstanceArgs()
#13 /var/www/html/vendor/symfony/dependency-injection/ContainerBuilder.php(586): Symfony\Component\DependencyInjection\ContainerBuilder->createService()
#14 /var/www/html/vendor/symfony/dependency-injection/ContainerBuilder.php(1260): Symfony\Component\DependencyInjection\ContainerBuilder->doGet()
#15 /var/www/html/vendor/symfony/dependency-injection/ContainerBuilder.php(1212): Symfony\Component\DependencyInjection\ContainerBuilder->doResolveServices()
#16 /var/www/html/vendor/symfony/dependency-injection/ContainerBuilder.php(1112): Symfony\Component\DependencyInjection\ContainerBuilder->doResolveServices()
#17 /var/www/html/vendor/symfony/dependency-injection/ContainerBuilder.php(586): Symfony\Component\DependencyInjection\ContainerBuilder->createService()
#18 /var/www/html/vendor/symfony/dependency-injection/ContainerBuilder.php(531): Symfony\Component\DependencyInjection\ContainerBuilder->doGet()
#19 /var/www/html/vendor/drush/drush/src/Runtime/LegacyServiceInstantiator.php(289): Symfony\Component\DependencyInjection\ContainerBuilder->get()
#20 /var/www/html/vendor/drush/drush/src/Runtime/LegacyServiceInstantiator.php(253): Drush\Runtime\LegacyServiceInstantiator->resolveFromContainer()
#21 [internal function]: Drush\Runtime\LegacyServiceInstantiator->resolveArgument()
#22 /var/www/html/vendor/drush/drush/src/Runtime/LegacyServiceInstantiator.php(223): array_map()
#23 /var/www/html/vendor/drush/drush/src/Runtime/LegacyServiceInstantiator.php(193): Drush\Runtime\LegacyServiceInstantiator->resolveArguments()
#24 /var/www/html/vendor/drush/drush/src/Runtime/LegacyServiceInstantiator.php(167): Drush\Runtime\LegacyServiceInstantiator->instantiateObject()
#25 /var/www/html/vendor/drush/drush/src/Runtime/LegacyServiceInstantiator.php(117): Drush\Runtime\LegacyServiceInstantiator->create()
#26 /var/www/html/vendor/drush/drush/src/Runtime/LegacyServiceInstantiator.php(54): Drush\Runtime\LegacyServiceInstantiator->instantiateServices()
#27 /var/www/html/vendor/drush/drush/src/Boot/DrupalBoot8.php(243): Drush\Runtime\LegacyServiceInstantiator->loadServiceFiles()
#28 /var/www/html/vendor/drush/drush/src/Boot/DrupalBoot8.php(216): Drush\Boot\DrupalBoot8->addDrupalModuleDrushCommands()
#29 /var/www/html/vendor/drush/drush/src/Boot/BootstrapManager.php(211): Drush\Boot\DrupalBoot8->bootstrapDrupalFull()
#30 /var/www/html/vendor/drush/drush/src/Boot/BootstrapManager.php(352): Drush\Boot\BootstrapManager->doBootstrap()
#31 /var/www/html/vendor/drush/drush/src/Boot/BootstrapManager.php(304): Drush\Boot\BootstrapManager->bootstrapToPhaseIndex()
#32 /var/www/html/vendor/drush/drush/src/Boot/BootstrapHook.php(36): Drush\Boot\BootstrapManager->bootstrapToPhase()
#33 /var/www/html/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php(44): Drush\Boot\BootstrapHook->initialize()
#34 /var/www/html/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php(36): Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->doInitializeHook()
#35 /var/www/html/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php(29): Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->callInitializeHook()
#36 /var/www/html/vendor/consolidation/annotated-command/src/CommandProcessor.php(145): Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->initialize()
#37 /var/www/html/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(378): Consolidation\AnnotatedCommand\CommandProcessor->initializeHook()
#38 /var/www/html/vendor/symfony/console/Command/Command.php(292): Consolidation\AnnotatedCommand\AnnotatedCommand->initialize()
#39 /var/www/html/vendor/symfony/console/Application.php(1106): Symfony\Component\Console\Command\Command->run()
#40 /var/www/html/vendor/drush/drush/src/Application.php(201): Symfony\Component\Console\Application->doRunCommand()
#41 /var/www/html/vendor/symfony/console/Application.php(324): Drush\Application->doRunCommand()
#42 /var/www/html/vendor/symfony/console/Application.php(175): Symfony\Component\Console\Application->doRun()
#43 /var/www/html/vendor/drush/drush/src/Runtime/Runtime.php(113): Symfony\Component\Console\Application->run()
#44 /var/www/html/vendor/drush/drush/src/Runtime/Runtime.php(40): Drush\Runtime\Runtime->doRun()
#45 /var/www/html/vendor/drush/drush/drush.php(140): Drush\Runtime\Runtime->run()
#46 /var/www/html/vendor/bin/drush.php(119): include('...')
#47 {main}
Error: Cannot use object of type Doctrine\Common\Lexer\Token as array in Drupal\Component\Annotation\Doctrine\DocParser->Annotations() (line 616 of /var/www/html/web/core/lib/Drupal/Component/Annotation/Doctrine/DocParser.php).
 [warning] Drush command terminated abnormally.

From what I could gather, this seems to be an incompatibility with the now in core included doctrine/annotations code (see https://www.drupal.org/project/drupal/issues/3550917) and the latest doctrine/lexer versions. Given the former as external dependency did not declare v3 as compatible, this is probably the root cause.

Steps to reproduce

I do not have a full overview of all involved parts, but relevant for my case are:
- drupal/core v10.6.0
- drush/drush v13.7.0
- doctrine/lexer >=3.0.0

Proposed resolution

Achieving compatibility would be ideal, but as a (temporary?) solution declaring a conflict "doctrine/lexer": ">=3.0.0" in composer.json seems to be a perfect workaround on my system, that probably can be applied to core as well.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3563642

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

jnoordsij created an issue. See original summary.

jnoordsij’s picture

jnoordsij commented
Issue summary: View changes
jnoordsij’s picture

jnoordsij commented
Related issues: +#3429602: Annotation component has an undeclared dependency on doctrine/lexer 2, +#3429849: Make doctrine/lexer:^3.0 compatible with \Drupal\Component\Annotation\Doctrine.

On looking a bit further this seemingly was tackled for 11.x in https://www.drupal.org/project/drupal/issues/3429602 and https://www.drupal.org/project/drupal/issues/3429849, but (intentionally) skipped for backport to 10.x. As mentioned I'm not exactly sure of the conditions that are involved, but probably backporting is warranted?

Closer inspection of my composer.lock file seems to indicate doctrine/lexer is loaded as indirect dependency through egulias/email-validator (core dependency).

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Title: Apparent incompatibility of Drupal 10.6 + Drush with doctrine/lexer >= v3.0 » Drupal 10.6 incompatible with doctrine/lexer 3
Component: other » composer
Priority: Normal » Critical

Thanks for spotting this. I think this affects all users who install drupal/core 10.6 without using drupal/core-recommended.

When we removed doctrine/annotations we should have added an explicit dependency on doctrine/lexer, but we did not; we did not notice this because we were saved by the transitive dependency via egulias/email-validator. Core's composer.lock still contains doctrine/lexer 2.1.1 so we run tests against that version, and it is also the version added to the drupal/core-recommended metapackage.

Because egulias/email-validator allows v2 or v3 of doctrine/lexer, users will likely get the later version installed if they do not depend on drupal/core-recommended, but we are not actually compatible with v3.

We should do one or both of these things:

godotislate’s picture

godotislate
he/him
commented

I don't think we can do the second thing of adding a dependency on doctrine/lexer 3 to 10.6.x, because in #3550917: doctrine/annotations is abandoned, the forked version of doctrine/annotations is based on doctrine/annotations 1.14.4, which has a version constraint of "doctrine/lexer": "^1 || ^2".

godotislate opened merge request !14125

godotislate’s picture

godotislate
he/him
commented
Status: Active » Needs review

MR 14125 with explicit dependency on doctrine/lexer: ^2 ready for review.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Status: Needs review » Reviewed & tested by the community

Fine by me: in 10.5 we would not have allowed doctrine/lexer 3 for the same reason and given 10.6 is only a maintenance minor we should aim to make as few changes as possible, so only allowing doctrine/lexer 2 is the better option anyway.

a.dmitriiev’s picture

a.dmitriiev commented

Explicit dependency on doctrine/lexer: ^2 helped for my project on Drupal 10.6 too. Thank you. +1 approvement for MR.

catch closed merge request !14125

  • catch committed fe25d043 on 10.6.x 
    feat: #3563642 Drupal 10.6 incompatible with doctrine/lexer 3

By:...
catch’s picture

catch
he/him
Primary language English
commented
Status: Reviewed & tested by the community » Fixed

Yes sticking to ^2 makes sense, one year to go now and the security surface here is extremely low so the likelihood of having to deal with backporting an SA should be close to zero.

Committed/pushed to 10.6.x, thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented

Added credit for @jurgenhaas for bringing this up and discussing the solution in Slack.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.