Postponed on #3284945: Install endpoints that leverage Package Manager + core APIs

Problem/Motivation

Once the UI install controllers have landed, we need to update the Svelte UI to interact with them.

Steps to reproduce

Proposed resolution

Remaining tasks

  • ✅ File an issue about this project
  • ☐ Manual Testing
  • ☐ Code Review
  • ☐ Accessibility Review
  • ☐ Automated tests needed/written?
Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

bnjmnm created an issue. See original summary.

bnjmnm’s picture

bnjmnm’s picture

Status: Active » Postponed
bnjmnm’s picture

Title: [PP] Svelte UI for intall controllers » [PP] Svelte UI for install controllers

bnjmnm’s picture

Title: [PP] Svelte UI for install controllers » Svelte UI for install controllers
Status: Postponed » Needs review
rkoller’s picture

Status: Needs review » Needs work
StatusFileSize
new188.71 KB
new23.33 KB
new35.47 KB

I've successfully applied merge request #171 to an install of Drupal 9.5-beta2 with PHP 8.1.10. A few thoughts and observations:

1. right after the merge request was applied i got a notification that the active directory contains symlinks:
notification on the project browser page that you are unable to download modules via the ui because active directory contains symlink.
i am well aware because i forgot to delete the node_modules folder. but i would suggest that the notification is styled as a regular error notice and moved probably up to the green status message?

2. I've clicked the view commands drop button of a module i havent installed yet. I've then clicked the download (experimental) option. The install worked flawlessly (tried with one or two other modules). Problem is if i click enable (experimental) for the previously successfully downloaded module nothing happens and the module isn't getting enabled. i can click it a second or third time. still the enabling isnt taking place.

3. in contrast i've tried the same with the download and enable (experimental) option. that way it successfully runs through.
no description for the successful download and enable modal for the VBO module
but somehow a description is missing what actually happened in contrast to the output of a successful download:
modal for the successful download of the redirect module

4. in regards of the terms "download" and "enable" i would suggest to follow the pattern summed up in #2888657: [meta] Less confusing and more consistent wording needed in module/theme add/install/update and use "add" instead of "download" and "install" instead of "enable"

5. when i visit /admin/modules i run into the following error:

Warning: Trying to access array offset on value of type null in Drupal\project_browser\Controller\InstallerController->inProgress() (line 211 of modules/contrib/project_browser/src/Controller/InstallerController.php).
Drupal\project_browser\Controller\InstallerController->inProgress('views_bulk_operations')
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 580)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 159)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 81)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 707)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

but it also happened on /admin/reports/dblog. usually with the module i last tried to install.

and in regards of the buttons and the communicated different module states i have to think a bit. will post another comment later tonight.

rkoller’s picture

StatusFileSize
new1.44 MB

another detail i've tested in between meetups a few minutes ago is the screenreader output. i've tried to download a module with voiceover on macos monterey activated.
currently nothing is announced what is currently happening during the download process of a module. it might be helpful to indicate the different states with drupal.announce(). cuz in the current example (see voiceover.mp4) when the process takes too long with no user input voiceover starts to announce something again and stated "you are currently in a group. to exit this group,...". that could be confusing for the screenreader users.
at the end of the process it sounds a bit distracting/confusing to get a "view password" web dialogue with two items and then a "download of view password complete". i wonder if just an announcement like view password was successfully downloaded/added? also a question for sighted users if it is necessary to have a modal, which requires an extra click to close it, in the case of a successful download?

srishtiiee made their first commit to this issue’s fork.

narendraR made their first commit to this issue’s fork.

rkoller’s picture

two more thoughts and observations.

1. when a module is added and or installed you currently have for one the prominent spinner in the center of the module card as the center of attention and at the same time you have the add/installation stage label in the lower right corner of the card replacing the View Commands drop button (which happens also to have animated periods at the end). you have two areas of interest getting updated/animated. me as the user, in a case where the adding of a module took unexpectedly longer, was therefore drawn between the spinner and the stage label at the bottom. my eyes had to switch between the spinner, to see if it is still spinning and the process is still running, and between the stage label to see if the label has changed, repeatedly. visually the more or less constant jump is demanding in my case.
i wonder if it would make sense to unify the two components by moving the rather large and prominent spinner from the center of the card down and prepend it inline to the stage label as well as removing the animation of the periods at the end of the stage label? that way the information would be more compact and in one place.

2. currently the state a module is in (not added, added, installed) is not communicated in every case with the card directly:

not added (not directly visible it could only be concluded via the available options in the drop button but you have to click the drop button first)
added but not installed yet (same as not added - not directly visible, you have to click the drop button first to see the list of available options)
installed (directly visible by the different button with the label installed - problem here the labelinstalled gives no indication that the button is clickable and that by clicking you are redirected to /admin/extend)

If you click the default for the drop button (View commands) the redirect modal is shown with the options download and install which are not really necessary anymore with this patch in. But if the default option has to stay in for the time the feature is experimental it might be an idea to rename the default button label to something like Available actions?
and in case the Views commands default could be removed a suggestion might be for modules not installed to the file system yet the default could be: +Add and as an option in the drop button select list Install. For modules that are already added the button could be changed from a drop button to a regular button named Install.
In regards of installed modules the question is how to rename the button. all other labels describe an action while installed describes a state. i wonder if it would make sense to communicate that the button is redirecting users to the extend page?
and building on that would it make sense to communicate the state the module is in somewhere else on the card alongside? that the users doesnt have to rely on the buttons and the available actions to conclude the state of a module but has a module clearly indicating the state the module is in?

p.s. and i have applied the change that was made in #13 and i can confirm that the issue with #11.2 is fixed now. thank you!

rkoller’s picture

rkoller’s picture

StatusFileSize
new70.55 KB

another detail I've noticed. if you access the Project Browser page in MacOS with VoiceOver active and open the Rotor the labels for the pointing down triangle and the View Commands default option are sort of redundant and are not providing any context.

form controls windows in voiceovers roto with many redundant form control labels for caret and view commands button

maybe add aria-label attribute appending the module name accordingly? Something like

becomes Available actions for [module name] (e.g Available actions for Pathauto)

View commands becomes View Commands <span class="visually-hidden">for [module name]</span> (e.g. View Commands <span class="visually-hidden">for Pathauto</span>)

tim.plunkett’s picture

Status: Needs work » Needs review
rkoller’s picture

I've applied the latest patch to a drupal 9.5.0-beta2 install. ran yarn install and yarn build. when ran drush cr i get:

$> ddev drush cr      
PHP Fatal error:  Trait "Drupal\package_manager\StatusCheckTrait" not found in /var/www/html/web/modules/contrib/project_browser/src/Controller/InstallReadinessController.php on line 18

Fatal error: Trait "Drupal\package_manager\StatusCheckTrait" not found in /var/www/html/web/modules/contrib/project_browser/src/Controller/InstallReadinessController.php on line 18
 [warning] Drush command terminated abnormally.
Failed to run drush cr: exit status 1

same if i try to flush caches via the admin_toolbar:

Fatal error: Trait "Drupal\package_manager\StatusCheckTrait" not found in /var/www/html/web/modules/contrib/project_browser/src/Controller/InstallReadinessController.php on line 18

Update:
woops i was too fast. apologies. i have set up a fresh install yesterday night and i havent had installed package manager yet. by installing package manager the cache clear worked.

rkoller’s picture

@bnjmnm You've updated the name from Enable to Install. i think for consistency reasons it would make sense to change Download to Add as well to be inline with #2888657: [meta] Less confusing and more consistent wording needed in module/theme add/install/update. there was also #3315858: Make the micro copy for adding and installing modules consistent with the general plan for Drupal Core in the context of project browser where the two terms were updated.

in the context of #17 i've tested the changes @srishtiiee applied. looks good. i've only forgot one detail in my suggestion. the Installed button is missing the form controls context as well. as soon as you have one or more modules installed you have Installed button, Installed button, Installed button. maybe apply the same pattern like used for available actions and view commands?

rkoller’s picture

as @tim.plunkett suggested in todays meeting i've opened four followup issues. they cover the rest of the issues i've mentioned in my previous comments. those aren't vital for the core functionality of the patch and cover general ux and a11y related issues and would just blocked the merge in the worst case. i've added them as child issues to the issue here:

#3318726: Reposition the card centric spinner for the Svelte UI for install controllers
#3318735: The different stages of the add and install process aren't announced in screenreaders for the Svelt UI for install controllers
#3318747: Change the default action for the Svelt UI for install controllers
#3318750: Remove the successful download/install modal from the Svelte UI for install controllers

rkoller’s picture

StatusFileSize
new44.79 KB

I've tested one scenario that "might" happen in the real world. I've applied the following steps:

1. selected the category features package.
2. click download and install the "varbase editor"
3. while the download starts immediately clicked download and install the "media hero slider". the following notification window turned up:
lock window for media hero slider install
i've just clicked the close the modal button and then just waited a bit.
4. the varbase editor was hanging in the in progress stage indefinitely.
5. i've reloaded the page and the "varbase editor" card was showing the view commands drop button again.
6. i've clicked download and install the "varbase editor" again. the install steps feel now much much slower compared to the fresh installs before. and the installation hangs at the in progress stage (now for a few minutes already)

as a user i was uncertain if clicking the unlock install stage link in the model would have been too early (or how long i would have had to wait). by closing the modal not clicking the unlock install stage link i have no apparent option to get to that functionality again to check if there is anything locked. at least it feels to me cuz of the indefinite hang in in progress.

rkoller’s picture

StatusFileSize
new28.04 KB

It recovered. Yesterday night after a restart of ddev the unlock install stage modal showed up again and i was able to get back to normal. but today after starting my ddev project i got the following warning on the welcome! front page in olivero:

Warning: Trying to access array offset on value of type null in Drupal\project_browser\Controller\InstallerController->inProgress() (line 211 of modules/contrib/project_browser/src/Controller/InstallerController.php).
Drupal\project_browser\Controller\InstallerController->inProgress('pathauto')
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 580)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 169)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 81)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 707)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

revisiting the page the warning was gone. and i got the following entries of the type project browser today (see phptuf.txt - perhaps that helps(?)) during yesterdays tests. there are also several page not found entries. all during the downtime caused by my test in #22

bnjmnm’s picture

Re #22: it turns out that Varbase Editor won't install for composer dependency related reasons. This should have resulted in an error explaining as much. I address the underlying issue preventing this error from appearing and stopping the install process. The module it's still not installable for reasons unrelated to project browser, but now that information is actually presented to the user.

#23: this error was due to an expectation that Package Manager is always available. This has also been addressed.

rkoller’s picture

thanks for all the fixes! I've tried to test the latest changes of the patch but since yesterday night the merge request diff fails to apply and is getting skipped with composer patches. is it possible that that is caused by the changes in #3318905: Address Package Manager deprecation warning that got merged yesterday or because of the tests being red for 50c0f3ec the latest commit?

chrisfromredfin’s picture

I have reviewed this to the best of my ability, but it has my approval. I have manually tested and think this is wonderful enough to release in beta. I would love it if someone with some better chops would also take a look, though, so leaving in NR.

rkoller’s picture

The merge request successfully applies again. thanks! I've then tested the case with installing two modules right after eachother once more. But tried this time instead of closing the unlock the install stage modal, keeping the modal open and let the install of the first module run through the process. After that i've clicked the unlock the install stage link. that worked and i was able to install the second module. But with one of the modules (simple xml sitemap) i've tried installing i ran into an error i haven't seen yet:

location: https://projectbrowser.ddev.site/admin/modules/project_browser/install-apply/drupal/simple_sitemap/ml295NIWns30Ab6bLQ7-7bhblwst82DLNoLs5vQXIOc
referrer: https://projectbrowser.ddev.site/admin/modules/browse
Message:
Drupal\package_manager\Exception\StageException: Malformed release data: Field [date]: This value should be of type numeric., Field [download_link]: This value should not be blank.. #0 /var/www/html/web/modules/contrib/project_browser/src/ComposerInstaller/Installer.php(26): Drupal\package_manager\Stage->dispatch(Object(Drupal\package_manager\Event\PreApplyEvent), Object(Closure)) #1 /var/www/html/web/modules/contrib/automatic_updates/package_manager/src/Stage.php(451): Drupal\project_browser\ComposerInstaller\Installer->dispatch(Object(Drupal\package_manager\Event\PreApplyEvent), Object(Closure)) #2 /var/www/html/web/modules/contrib/project_browser/src/ComposerInstaller/Installer.php(38): Drupal\package_manager\Stage->apply(600) #3 /var/www/html/web/modules/contrib/project_browser/src/Controller/InstallerController.php(477): Drupal\project_browser\ComposerInstaller\Installer->apply() #4 [internal function]: Drupal\project_browser\Controller\InstallerController->apply('drupal', 'simple_sitemap', 'ml295NIWns30Ab6...') #5 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array) #6 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(580): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #7 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure)) #8 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) #9 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(169): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #10 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(81): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1) #11 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #12 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #13 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #14 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true) #15 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #16 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #17 /var/www/html/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #18 /var/www/html/web/core/lib/Drupal/Core/DrupalKernel.php(707): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #19 /var/www/html/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request)) #20 {main}

bnjmnm’s picture

Re #27 as ugly as the error looks, that is the feature doing it's job properly. At least at the time I type this, the release data for version 2.4 of simple_sitemap is missing values for several required fields https://updates.drupal.org/release-history/simple_sitemap/8.x . The installer errs on the side of caution and will not install anything with unexpected/corrupt release data.

<release>
<name>simple_sitemap 8.x-2.4</name>
<version>8.x-2.4</version>
<tag>8.x-2.4</tag>
<version_major>2</version_major>
<version_patch>4</version_patch>
<status>published</status>
<release_link>https://www.drupal.org/project/simple_sitemap/releases/8.x-2.4</release_link>
<download_link/>
<date/>
<mdhash/>
<filesize/>
<files>
<file>
<url/>
<archive_type/>
<md5/>
<size/>
<filedate/>
</file>
<file>
<url/>
<archive_type/>
<md5/>
<size/>
<filedate/>
</file>
</files>
<terms>
<term>
<name>Release type</name>
<value>New features</value>
</term>
</terms>
<security covered="1">Covered by Drupal's security advisory policy</security>
<core_compatibility>8.x</core_compatibility>
</release>
rkoller’s picture

re #28 ah excellent and thanks for the explanation!
the only thing i wonder, do you think it would make sense to make the error message more comprehensible for less technical folks? Drupal\package_manager\Exception\StageException: Malformed release data: Field [date]: This value should be of type numeric., Field [download_link]: This value should not be blank. is kind of abstract and challenging. A user might not know what the best next step might be? maybe it would make sense to create a followup so the this issue isn't blocked?

bnjmnm’s picture

I established that the error in #27 is due to release info on Drupal.org missing required properties - so it not an issue with this feature, just an error relayed by the UI.

Looking further, I found that there was a single malformed release in Simple XML Sitemap. This was an older release - not the version being installed. The presence of one malformed release crashed the entire process despite all other releases being valid. I confirmed with @tedbow that if Package Manager filtered malformed releases from the list of candidates instead of killing the process entirely upon finding one was a safe & secure approach. This also happens to be a pattern that has been in the core update module for some time. With this approach there is still no risk of installing a malformed release, it just means that a malformed past release won't obstruct the installation of a perfectly valid one. I opened an issue in Automatic updates for this #3320558: When parsing releases, place ProjectRelease::createFromArray in a try block. Once addressed, this specific error shouldn't happen - and if all releases happen to be malformed there will be a more helpful error that says no compatible releases are available.

bnjmnm’s picture

Assigned: bnjmnm » Unassigned
hotwebmatter’s picture

Status: Needs review » Reviewed & tested by the community

Reviewed and tested at NEDCamp with @chrisfromredfin and @leslieg.

I encountered some hurdles with the add-project-as-symlink.sh, and I can document my workaround in a separate post.

For now, what's important is that this looks beautiful and works great. I can't wait to see it in Drupal 10!

Screenshots to follow.

/RTBC

phenaproxima’s picture

Status: Reviewed & tested by the community » Needs work

I found a few things in review that I'm wondering about, including potential random failure conditions. :( Sorry!

narendrar’s picture

I have admin_toolbar installed via composer locally. When I tried to install Admin Toolbar Content module through GUI, I get InstallException: The new package drupal/admin_toolbar_tools will be installed in the directory , which already exists but is not managed by Composer.. This is because Admin Toolbar Content has dependencies on admin_toolbar_tools which is a sub-module of admin_toolbar. 🥹

bnjmnm’s picture

Status: Needs work » Needs review

Addressed the final bit of @phenaproxima feedback. Setting back to NR.

narendrar’s picture

Feedback addressed, allow_ui_install added in constants.js and project_browser.admin_settings.yml

tim.plunkett’s picture

Assigned: Unassigned » chrisfromredfin
Status: Needs review » Reviewed & tested by the community

Thanks @narendraR for the changes! This is ready to go. Adding credit to those who gave feedback outside this issue (e.g. via Usability and Accessibility reviews)

chrisfromredfin’s picture

Status: Reviewed & tested by the community » Fixed

Squee! This is huge. I'm also going to roll beta2!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

tim.plunkett’s picture

Issue tags: +core-mvp