Problem/Motivation

Cases to cover:

  1. Metadata is unable to be fetched due to a transient error, such as a 5XX response.
  2. Metadata is unable to be fetched, but it requires administrator intervention, such as when API credentials are invalid.
  3. A metadata error is thrown during a user-interface action, like saving a media entity, and the entity save should be aborted or metadata queued for later.
  4. A metadata error is thrown while processing the queue. If it's a transient error, the item should be re-queued to the bottom of the queue. If it's a "permanent" error, do we delete the queue item?
  5. A thumbnail download fails, and we fall back to the default thumbnail. The queue should attempt to get the thumbnail again later.
  6. No error occurred, the metadata is empty (this is the one case currently handled by core).

Steps to reproduce

Proposed resolution

Remaining tasks

Depends on #2878119: Whether queued or not, update the media thumbnail and metadata before beginning the entity save database transaction.
Postponed on #2983456: Expose triggering update of media metadata + thumbnail to end users

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#3 2983728-review-not-to-test.patch3.89 KBvijaycs85

Comments

deviantintegral created an issue. See original summary.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

vijaycs85’s picture

vijaycs85
Location London, UK
commented
StatusFileSize
new 2983728-review-not-to-test.patch3.89 KB

We could try
a) throw an exception for cases that just need to be simple logging.
b) Dispatch an event for the cases that require additional actions - e.g. adding to queue.

Here is a (sample) patch to handle thumbnail update failure. This patch applied on top of #2878119-101: Whether queued or not, update the media thumbnail and metadata before beginning the entity save database transaction and just for review purpose.

deviantintegral’s picture

deviantintegral commented

I like the idea of dispatching an event for error handling. However, is there ever a reason why multiple subscribers would want to respond to the same event? It seems like there'd be close to a one-to-one mapping between media source plugins and subscriber implementations.

To me, what's missing currently is good default handling for common error conditions, especially related to network or HTTP errors. For example, if a thumbnail update times out, or 503s, it would be reasonable to transparently requeue the thumbnail. I think the same holds for metadata updates too. It feels a bit fragile to require contrib developers to duplicate this in most modules providing a media source.

So how about something like:

  • Core provides three exception classes, one for "remote data was not found", and one for "remote data could not be loaded and should be retried".
  • The contrib entry points would be an event subscriber or similar to alter the exception or remove it, such as changing a 404 (which should probably be dropped from the queue by core) to the "try this again later" exception.

Thoughts?

deviantintegral’s picture

deviantintegral commented
Related issues: +#1832818: Allow a queue item to be postponed

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

quietone commented
Issue summary: View changes
Issue tags: +Bug Smash Initiative

The issue this is postponed has been closed as won't fix. The last comments in that issue refer to continuing in #2983456: Expose triggering update of media metadata + thumbnail to end users so I am changing this issue to be postponed on that issue.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.