Problem/Motivation

$ git lg package_manager/package_manager.api.php
* 115bcad - Hotfix spelling error in Package Manager API documentation. (9 months ago) <Phéna Proxima>
* ec507ef - Issue #3248976 by phenaproxima: Add API documentation for Package Manager (9 months ago) <phenaproxima>

yet:

$ git lg --since "9 months ago" -- package_manager | wc -l
     156

It looks like package_manager.api.php is so general that it is not wrong, but it certainly is incomplete. For example, there's no mention of ReadinessCheckEvent nor StatusCheckEvent.

And a few days ago, #3304365: Do not check excluded folders for symlinks landed which deprecated PreCreateEvent::excludePath(), but it's still mentioned in package_manager.api.php as the official API.

This is necessary to meet the documentation gate: https://www.drupal.org/about/core/policies/core-change-policies/core-gat....

Steps to reproduce

N/A

Proposed resolution

Particular items that need documenting

  1. re #3328978: Do we want to Throw an Exception if a git directory unknown to composer is added in Stage? we should clearly document package manager expects the stage to be only operated on via composer stager, i.e. if you run other file operations on the stage that are not composer operations you are on your own and it is not how the system is intended to work.

COMMIT NOTE:

Please remember to omit credit for #4, which did not make any change to the documentation (or code).

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

Wim Leers created an issue. See original summary.

wim leers’s picture

Issue summary: View changes
rohit.rawat619’s picture

Assigned: Unassigned » rohit.rawat619
rohit.rawat619’s picture

Assigned: rohit.rawat619 » Unassigned
Status: Active » Needs review
StatusFileSize
new6.7 KB
wim leers’s picture

Status: Needs review » Needs work

There's nothing to review in #4? 😅

No new documentation at all. Only out-of-scope changes.

wim leers’s picture

Title: package_manager.api.php is outdated » Define the Package Manager API (package_manager.api.php is outdated)
Assigned: Unassigned » tedbow
Priority: Normal » Critical
Related issues: +#3323003: Mark FailureMarker @internal and document ComposerUtility, PathLocator and ValidationResult

This touches on a crucial other subject too: not only is the documentation incomplete, it's also unclear what actually is part of the Package Manager API and what is not.

At #3323003-3: Mark FailureMarker @internal and document ComposerUtility, PathLocator and ValidationResult, I proposed closing that issue in favor of this one.

tedbow’s picture

Issue tags: +sprint
phenaproxima’s picture

Adjusting credits. The patch in #4 didn't seem to add anything useful so I don't think it should get credit.

phenaproxima’s picture

tedbow’s picture

re #3328978: Do we want to Throw an Exception if a git directory unknown to composer is added in Stage? we should clearly document package manager expects the stage to be only operated on via composer stager, i.e. if you run other file operations on the stage that are not composer operations you are on your own and it is not how the system is intended to work.

wim leers’s picture

Quoting #3331168-10: Limit trusted Composer plugins to a known list, allow user to add more:

Question

The issue summary seems to imply that a better solution still would be to flip the architecture around entirely, and switch from exclusions to inclusions. If we are to only support specific composer plugins, and each of those would require explicit vetting, then that would be possible. That'd of course be a drastic change, with big consequences.

But in security, in general, exclusions are considered a bad practice. So why was the architectural decision made to include everything, but allow exclusions, instead of the other way around? 🤔 This is not documented in package_manager.api.php and should be. Will update #3318306: Define the Package Manager API (package_manager.api.php is outdated).

wim leers’s picture

Component: Code » Documentation
tedbow’s picture

Issue summary: View changes
tedbow’s picture

create #3336243: Update Package Manager event documentation in package_manager.api.php as a child issue so we can tackle a small part of this

wim leers’s picture

Title: Define the Package Manager API (package_manager.api.php is outdated) » [PP-1] Define the Package Manager API (package_manager.api.php is outdated)
Assigned: tedbow » Unassigned
Status: Needs work » Postponed

👍 Let's avoid conflicting with that issue! 😄

wim leers’s picture

Title: [PP-1] Define the Package Manager API (package_manager.api.php is outdated) » Define the Package Manager API (package_manager.api.php is outdated)
Status: Postponed » Active
wim leers’s picture

phenaproxima’s picture

Component: Documentation » Package Manager
tedbow’s picture

Version: 8.x-2.x-dev » 3.0.x-dev
wim leers’s picture

Title: Define the Package Manager API (package_manager.api.php is outdated) » [PP-1] Define the Package Manager API (package_manager.api.php is outdated)
Status: Active » Postponed
phenaproxima’s picture

Title: [PP-1] Define the Package Manager API (package_manager.api.php is outdated) » Define the Package Manager API (package_manager.api.php is outdated)
Status: Postponed » Active

Blocker is in!

phenaproxima’s picture

Trying to adjust credit given that the patch in #4 makes no actual changes to the documentation.

phenaproxima’s picture

phenaproxima’s picture

phenaproxima’s picture

Issue summary: View changes
wim leers’s picture

Assigned: wim leers » Unassigned
Status: Active » Reviewed & tested by the community
StatusFileSize
new7.51 KB

Mirrored the structure of core/modules/ckeditor5/ckeditor5.api.php, the corresponding output of which can be seen at https://api.drupal.org/api/drupal/core%21modules%21ckeditor5%21ckeditor5....

wim leers’s picture

Self-review for context

+++ b/package_manager/package_manager.api.php
@@ -152,13 +167,16 @@
+ * @section sec_validators_excluders API: excluders vs validators & status checks

@@ -185,23 +203,32 @@
- * \Drupal\package_manager\Event\StatusCheckEvent
- * Dispatched to check the status of the Drupal site and whether Package Manager
- * can function properly. These checks can be performed anytime, so this event
- * may be dispatched multiple times.
...
+ *
+ * Virtually every validator should react not only to
+ * \Drupal\package_manager\Event\PreCreateEvent and/or
+ * \Drupal\package_manager\Event\PreApplyEvent but also
+ * \Drupal\package_manager\Event\StatusCheckEvent. This event is dispatched to
+ * check the status of the Drupal site and whether Package Manager can function
+ * properly.
+ *
+ * The Package Manager module does not trigger this event because it does not
+ * have a UI. Modules building on top of Package Manager are likely to dispatch
+ * this event to prevent the user from starting an operation that would fail.
+ *

These are arguably the most contentious bits.

See #3341469-7: Create StageBase::stageDirectoryExists() for improved DX to see if stage directory exists. and preceding discussion.

I tried to clarify here what I thought in #3341469-2: Create StageBase::stageDirectoryExists() for improved DX to see if stage directory exists. to be wrong, which @phenaproxima said at #3341469-4: Create StageBase::stageDirectoryExists() for improved DX to see if stage directory exists. to be correct, but then my detailed investigation at #3341469-7: Create StageBase::stageDirectoryExists() for improved DX to see if stage directory exists. showed that it was actually wrong, just in a pretty subtle way.

I tried to make this clear once and for all by documenting it like this — hope y'all like it 😊

phenaproxima’s picture

This looks pretty good to me, but I'm not sure I should give the final sign-off. For the simple reason that I'm too close to this code. I've been living and breathing Package Manager for over a year. Almost any explanation of its internals will make sense to me, because my mind will automatically fill in the gaps.

I think I want this to be reviewed by someone with much less familiarity with the internals before I commit it.

wim leers’s picture

Assigned: Unassigned » tedbow

Almost any explanation of its internals will make sense to me, because my mind will automatically fill in the gaps.

Looks like I missed my opportunity to get lorem ipsum committed as official docs 🤣🤣🤣

I'm glad that at least you found nothing wrong! 🥳

But if you can't commit it, then only @tedbow can IMHO. Given his role on this project, I think that probably makes sense anyway 👍

phenaproxima’s picture

Don't get me wrong: I'm happy to commit it. I did read the diff and the changes made sense.

I merely want a second pair of eyes to review it first, a set of eyes that hasn't had my level of exposure to Package Manager.

chrisfromredfin’s picture

Hi all, just chiming in here. I downloaded 3.0.x, applied the patch, and just read package_manager.api.php from top to bottom.

Overall, this absolutely gives me an understanding of how Package Manager works. I had only three things come to mind as I read it:

  1. Near line 35: The piece about "only using the same class..." seems like an oddity to me, so might need some explanation why (although that documentation may not need to be in .api.php, but rather in a handbook page or something).
  2. Near line 120: I understand event propagation in the JS world, so stopping propagation in that sense makes sense to me. I don't know how to stop propagation, though (is there a Drupal API call EventSubscriber::stopPropagation() or something?), so pointing more specifically to a code example would be good. Also, if you are expecting a consumer/user of Package Manager to have to do the stopping, this is even more important.
  3. ...and for the stupid part... lines 160, 170, and 231 go past 80 characters. :)
phenaproxima’s picture

Status: Reviewed & tested by the community » Needs work

This is great feedback.

Near line 35: The piece about "only using the same class..." seems like an oddity to me, so might need some explanation why (although that documentation may not need to be in .api.php, but rather in a handbook page or something).

I believe you're referring to:

Only the owner can perform
 * operations on the stage directory, and only using the same class (i.e.,
 * \Drupal\package_manager\Stage or a subclass) they used to create it.

I can see why this would seem odd, and maybe we can clarify it. What it comes down to is that, for Package Manager, "ownership" consists of two things -- a random token, and the fully qualified name of the Stage that you called create() on. You need both of these things in order to reclaim the stage in subsequent requests and do things to it.

Near line 120: I understand event propagation in the JS world, so stopping propagation in that sense makes sense to me. I don't know how to stop propagation, though (is there a Drupal API call EventSubscriber::stopPropagation() or something?), so pointing more specifically to a code example would be good. Also, if you are expecting a consumer/user of Package Manager to have to do the stopping, this is even more important.

Yeah, we would expect code that integrates with Package Manager (a custom validator or event subscriber) to stop propagation if they encounter some catastrophic condition that's going to break any further subscribers. Symfony Event objects have a stopPropagation() method we would expect them to call, so let's add a reference to that.

And while we're at it, let's fix the longer-than-80-characters lines. :)

wim leers’s picture

Assigned: tedbow » wim leers

Addressing the feedback, and adding the docs that #3334994: Add new InstalledPackagesList which does not rely on Composer API to get package info should've added.

wim leers’s picture

Also removing the docs that #3344039 should have removed (see #3344039-20: Add a validate() method to ComposerInspector to ensure that Composer is usable).

wim leers’s picture

wim leers’s picture

Assigned: wim leers » phenaproxima

Adam said he'd take this on! 🥳

phenaproxima’s picture

Status: Needs work » Needs review

Went over this from top to bottom and I reorganized and pushed and pulled until I think it's both accurate and comprehensive.

I'd like review from Wim, since he's great at clarity and accuracy...and I'd also like a review from a consumer of Package Manager (that means @bnjmnm or @chrisfromredfin) before we commit.

phenaproxima’s picture

Assigned: phenaproxima » wim leers
wim leers’s picture

Assigned: wim leers » phenaproxima
Status: Needs review » Needs work

Review posted.

I don't think we need +1s from Package Manager consumers because this clearly is a massive leap forward. Any further improvements can happen at a later time — this is actively blocking the core alpha-experimental MR!

phenaproxima’s picture

Assigned: phenaproxima » wim leers
Status: Needs work » Needs review

🏓

wim leers’s picture

Assigned: wim leers » Unassigned
Status: Needs review » Reviewed & tested by the community

A huge leap forward, go for launch! 🚀

phenaproxima’s picture

  • phenaproxima committed 07dce485 on 3.0.x
    Issue #3318306 by phenaproxima, Wim Leers: Define the Package Manager...
phenaproxima’s picture

Status: Reviewed & tested by the community » Fixed

Whew!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.