Problem/Motivation

The original reason for the issue (see below) was fixed in #2631202: Doctrine no longer supports SimpleAnnotationReader, incorporate a solution into core. However, Chi pointed out in #46 that two annotations, internal and property, are still discovered. Those need to be fixed.

--- original problem/motivation ---
Add a class loader like this:

spl_autoload_register( function ($class_name) {
  file_put_contents(__DIR__ . '/classes.txt', $class_name . "\n", FILE_APPEND);
  },
  FALSE, TRUE);

It generates a classes.txt file.

Look at that file like this:

sort classes.txt | uniq -c

You'll see stuff like this:

   2 Drupal\Core\Annotation\PluginID
   3 Drupal\Core\Annotation\Translation
   5 Drupal\Core\Annotation\code
   3 Drupal\Core\Annotation\end
   3 Drupal\Core\Annotation\end_code
   5 Drupal\Core\Annotation\endcode
  25 Drupal\Core\Annotation\ingroup
   1 Drupal\Core\Annotation\property
  11 Drupal\Core\Annotation\see
   2 Drupal\Core\Annotation\todo

This means that the classloader is being asked to autoload, for instance, Drupal\Core\Annotation\ingroup 25 times in a front page load.

That's because the annotation system isn't filtering out known-bad annotations.

At absolute best this means that the autoloader has to waste time dealing with this stuff, and at worst has to do a file_exists() for each of these requests.

Proposed resolution

Add internal, property and event to the list of ignored annotations.

Remaining tasks

Review
Commit

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Comments

Mile23 created an issue. See original summary.

mile23’s picture

Title: Plugin annotation system tries to PSR-0 @see and other annotations. » Plugin annotation system tries to resolve @see and other annotations as classes.
Status: Active » Needs review
StatusFileSize
new1.18 KB

This patch illustrates a proof-of-concept solution.

Obviously we don't want to use reflection when we have a good solution in Doctrine\Common\Annotations\AnnotationReader already, which has an API for ignoring some annotations.

Add the autoloader to index.php, as in the original issue, and then observe the classes.txt file to see that these annotation-based classes are no longer being requested.

neclimdul’s picture

Yuck, I wonder when this happened.

It might be a lot of work but it seems like we might want to try and just grab as many of the PSR-5 annotations as possible just to protect this in the future.

tim.plunkett’s picture

I'd bet money it was #2090353: Don't require to put the use statement into plugin classes.

Previously we'd use \Doctrine\Common\Annotations\AnnotationReader::addGlobalIgnoredName() like $reader->addGlobalIgnoredName('file'); to filter out @file, but when we switched to SimpleAnnotationReader, we lost that ability.

tim.plunkett’s picture

Issue tags: +Needs tests
StatusFileSize
new8.75 KB

If only Doctrine had used more traits, or less private, this would be a lot less easy.
Is this worth an upstream patch?

tim.plunkett’s picture

+++ b/core/lib/Drupal/Component/Annotation/Plugin/Discovery/AnnotationReader.php
@@ -0,0 +1,28 @@
+    $this->parser->setIgnoredAnnotationNames(static::$globalIgnoredNames);

Not sure if we have to do this in each method each time (like the upstream AnnotationReader), or if one call here is fine.

We still need addGlobalIgnoredName calls for endlink and file, like before.

mile23’s picture

OK, so maybe reflection isn't such a bad idea. :-)

The only upstream change needed is that $parser be protected instead of private. We can subclass to our needs after that.

tim.plunkett’s picture

Yes, though it would be nice to have that upstream list of names to ignore.

xano’s picture

OK, so maybe reflection isn't such a bad idea. :-)

To get the list without having to hardcode it elsewhere? That would have the advantage of not having to look after it.

I had to look it up, but the reason we switched to SimpleAnnotationReader is stated in #2090353-2: Don't require to put the use statement into plugin classes.

dawehner’s picture

StatusFileSize
new1.47 KB

So yeah, SimpleAnnotationReader seemed to be the only way to be able to skip the use statements ... which is IMHO worth to do.

Did we considered to just do the following?

Status: Needs review » Needs work

The last submitted patch, 10: 2557433-10.patch, failed testing.

Status: Needs work » Needs review

dawehner queued 10: 2557433-10.patch for re-testing.

mile23’s picture

Status: Needs review » Needs work

I had to look it up, but the reason we switched to SimpleAnnotationReader is stated in #2090353-2: Don't require to put the use statement into plugin classes.

The namespace addition lives on DocParser, and SimpleAnnotationReader just sets it. And of course AnnotationReader doesn't give us an accessor for this.

If $parser were protected (in either SimpleAnnotationReader or AnnotationReader, or both) we could just subclass and change it, while also getting the benefits of the superclass.

+++ b/core/lib/Drupal/Component/Annotation/Plugin/Discovery/AnnotatedClassDiscovery.php
@@ -71,6 +73,13 @@ protected function getAnnotationReader() {
     if (!isset($this->annotationReader)) {
       $this->annotationReader = new SimpleAnnotationReader();
 
+      AnnotationReader::addGlobalIgnoredName('see');
+      AnnotationReader::addGlobalIgnoredName('code');
+      AnnotationReader::addGlobalIgnoredName('endcode');
+      AnnotationReader::addGlobalIgnoredName('ingroup');
+      AnnotationReader::addGlobalIgnoredName('property');
+      AnnotationReader::addGlobalIgnoredName('todo');

I'm pretty sure SimpleAnnotationReader tells DocParser *not* to use those globally ignored names. Also, the global ignored names array already contains all our tags so we don't have to set them.

So basically it's either an upstream fix or we write our own version of AnnotationReader.

The upstream fix is one of two things: Give AnnotationReader a way to set the namespace, or make $parser protected so we can subclass to our own needs.

However, if you can apply the patch in #10 and then try to repro with the autoloader in the issue summary and get no requests for \Drupal\whatever\see then yay.

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new2.68 KB

Here's a test. It will fail. :-)

Add more unsupported annotations as needed.

Status: Needs review » Needs work

The last submitted patch, 14: 2557433_14.patch, failed testing.

mile23’s picture

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new10.63 KB

This patch creates a new class called DrupalAnnotationReader. It's mostly a copy-paste of Doctrine's SimpleAnnotationReader, with all the extra stuff we want.

It includes the test from #14, the ignore list from AnnotationReader, and some of @alexpott's modifications from https://github.com/doctrine/annotations/pull/56

That PR for Doctrine has been around since late April, so it doesn't look like they're moving very quickly.

Will do coding standards stuff if this comes back green and there's some consensus.

Status: Needs review » Needs work

The last submitted patch, 17: 2557433_17.patch, failed testing.

Ocramius’s picture

Sigh, I really don't understand how this drupal tracker works. Trying again:

- I just rejected https://github.com/doctrine/annotations/pull/56
- see http://www.doctrine-project.org/jira/browse/DCOM-295

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new4.15 KB

Here's the test in #14 combined with the solution in #10, which fails.

Status: Needs review » Needs work

The last submitted patch, 21: 2557433_21.patch, failed testing.

dawehner’s picture

Well, the alternative is that Drupal just implements our own Annotation Reader

mile23’s picture

That's #17. :-)

tim.plunkett’s picture

So let's get that green.

mile23’s picture

Version: 8.0.x-dev » 8.2.x-dev
StatusFileSize
new18.15 KB

I thought I might try to tract the intractable again.

There are still a few fails. The most interesting one is Drupal\system\Tests\Plugin\Discovery\AnnotatedClassDiscoveryTest which finds the @file annotation and thinks it's a plugin annotation.

mile23’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 26: 2557433_26.patch, failed testing.

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new3.14 KB

Hi, I'm Mr. Testing. But I neglected to just run the test, which passes. :-)

Re-running the experiment in the issue summary I see no non-plugin annotations slipping in to the autoload, so I think this issue is done. But if anyone wants the test, here it is.

mile23’s picture

Title: Plugin annotation system tries to resolve @see and other annotations as classes. » Verify that the plugin annotation system does not try to resolve @see and other annotations

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mile23’s picture

Version: 8.3.x-dev » 8.2.x-dev
StatusFileSize
new645 bytes
new3.04 KB

Valid for 8.2.x beta because it's a test improvement.

Removed @file docblock.

Test still passes.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

borisson_’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

Removing the needs tests tag, this is just a test and it still makes sense to add this. Moving this issue to RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new1.71 KB
new3.13 KB

Patch in #32 is not testing anything. The VFS setup doesn't result in any plugins actually being scanned. Here's a test that proves we still have a problem.

Status: Needs review » Needs work

The last submitted patch, 37: 2557433-37.patch, failed testing. View results

mile23’s picture

+++ b/core/tests/Drupal/Tests/Component/Plugin/Discovery/AnnotatedClassDiscoveryTest.php
@@ -68,7 +71,7 @@
-      ['\\DrupalTest\\TestClass' => [vfsStream::url('test_class/DrupalTest')]], '\\DrupalTest\\Component\\Annotation\\'
+      ['\\DrupalTest\\TestClass' => [vfsStream::url('root/DrupalTest')]], '\\DrupalTest\\Component\\Annotation\\'

Good catch.

1) Drupal\Tests\Component\Plugin\Discovery\AnnotatedClassDiscoveryTest::testAutoloadBadAnnotations
Attempted to autoload a non-plugin annotation: addtogroup

So it looks like we do try to autoload all annotation.

Do we want to go back to the earlier attempts at rolling our own reader?

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new3.21 KB

Converted to a dataprovider so we can test all the testcases.

They all still fail except for { and }.

mile23’s picture

StatusFileSize
new3.7 KB

Forgot the interdiff.

Status: Needs review » Needs work

The last submitted patch, 40: 2557433_40.patch, failed testing. View results

mile23’s picture

I just opened this PR: https://github.com/doctrine/annotations/pull/211

I opened it because DocParser *never* ignores annotations when you also have namespaces, so therefore we can't even roll our own annotation reader.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

bradjones1’s picture

Version: 8.6.x-dev » 9.1.x-dev
chi’s picture

provideBadAnnotations needs @internal and @property tags as they also trigger autoloader. Though they probably need to be included to SimpleAnnotationReader::$ignoredAnnotations first.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Status: Needs work » Closed (duplicate)
Issue tags: +Bug Smash Initiative

This has been fixed, it was done in #2631202: Doctrine no longer supports SimpleAnnotationReader, incorporate a solution into core. The patch here contains one test file and that was added in the other issue.

Closing as a duplicate.

chi’s picture

@quietone I was checking this after #2631202: Doctrine no longer supports SimpleAnnotationReader, incorporate a solution into core has been resolved. See note #46.

andypost’s picture

@Chi do you mean that issue needs re-title and update summary instead of closing?

chi’s picture

I would like someone else confirmed #46 first.

quietone’s picture

Issue summary: View changes
Status: Closed (duplicate) » Needs review
StatusFileSize
new664 bytes
new1.36 KB

@Chi, thanks for pointing out my error.

I have updated the IS and provided a fail and success patch.

quietone’s picture

Title: Verify that the plugin annotation system does not try to resolve @see and other annotations » Add internal and property to the list of ignored annotations in the plugin annotation system

I forgot to make a more accurate title.

The last submitted patch, 53: 2557433-53-fail.patch, failed testing. View results

quietone’s picture

Issue summary: View changes
StatusFileSize
new746 bytes
new1.52 KB
new746 bytes

I looked at the list of API documentation and comment standards tag references and I think @event needs to be added to this list. Are there any others?

The last submitted patch, 56: 2557433-56-fail.patch, failed testing. View results

chi’s picture

Having @event listed there makes sense to me. Technically, any other tag can trigger the autoloader. There is little we can do about this. The way Doctrine parses annotations is really odd.

quietone’s picture

Title: Add internal and property to the list of ignored annotations in the plugin annotation system » Add internal, event, and property to the list of ignored annotations in the plugin annotation system

OK. Updating title

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Triggering for 10.1

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

10.1 failure in #56 seems to be random ckeditor5

alexpott’s picture

Status: Reviewed & tested by the community » Needs review

I think we need to be careful here - I think we'll break this contrib module https://git.drupalcode.org/project/bcubed/-/blob/8.x-1.x/src/Annotation/...

Maybe the right thing here is to just wait for annotations to pass into history?

smustgrave’s picture

Is there a ticket we can postpone this on then?

chi’s picture

smustgrave’s picture

Status: Needs review » Postponed
Related issues: +#3252386: Use PHP attributes instead of doctrine annotations

Good find!

Of course if anyone thinks this shouldn't be postponed please move back to review

berdir’s picture

Status: Postponed » Closed (won't fix)

Postponed isn't really the right status I'd say. Once that issue and all the follow-ups to convert all annotation based plugins to attributes, this is simply a won't fix this.

I'd be fine to just close it as won't fix/outdated now, the chances that we'll remember to get back to this and do that then seems quite slim.

Doing that, if someone disagrees they can reopen.