Problem/Motivation

When an oembed entity is fetched and there is an error fetching the resource a message is shown to the user saying "Could not retrieve the oEmbed resource", however, this happens on every visit, and this is shown also to anonymous users, and for them, it doesn't make sense.

Steps to reproduce

- Create an entity to be embedded with a valid URL
- Remove, hide or block the referenced entity (if it is a video mark it as private or delete it)
- Visit the page where the entity is supposed to be rendered.

Proposed resolution

Log the error instead of showing it in a message.

Issue fork drupal-3202896

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

  • 3202896-11.x Comparechanges, plain diff MR !9357
  • 10.3.x Comparecompare
  • 4 hidden branches
  • 3202896-oembed-error-message Comparecompare
  • 10.4.x Comparecompare
  • 11.x Comparecompare
  • 3202896-dont-display-oembed Comparecompare

Comments

BetoAveiga created an issue. See original summary.

betoaveiga’s picture

StatusFileSize
new774 bytes

Adding patch to log instead of sending a message on oEmbed fetch error.

cilefen’s picture

Status: Active » Needs review
joao.ramos.costa’s picture

StatusFileSize
new575 bytes
new750 bytes

Minimal change to #2.

joao.ramos.costa’s picture

joao.ramos.costa’s picture

StatusFileSize
new752 bytes
new575 bytes

Minimal change to #2.
(Sorry, a little problem in the previous patch.)

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev
marcoscano’s picture

Title: Oembed error message should be logged instead of shown » Don't display OEmbed error to anonymous visitors when resource stops being available
Version: 9.3.x-dev » 9.4.x-dev
Status: Needs review » Needs work
Issue tags: +Needs tests

I agree this issue makes sense and it's a bug, since the "Could not retrieve the oEmbed resource" being displayed to anonymous visitors when a remote video stops being available isn't ideal.

However, I do think that's a good indicator for editors. So my suggestion is that instead of always logging, we should check $media->access('update')
- If the user can edit the media item, show the error on screen, so they can take action to fix the problem
- If the user can't edit, it's likely a visitor, in which case we should just just log the error.

And yes, we should definitely return NULL in any case (as suggested by #6), otherwise the site will break hard when it tries to call methods on the non-existing $resource variable.

Also, we need tests! :)

phenaproxima made their first commit to this issue’s fork.

melonangie’s picture

StatusFileSize
new886 bytes

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

gerzenstl’s picture

Patch from #11 works fine.

Tested with Drupal 9.3.13

adrianodias’s picture

#11 is good for me too.

Tested with Drupal 9.4.7

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

martijn de wit’s picture

Added 10.1.x: PHP 8.1 & MySQL 5.7 to #11 to see how it works out.

adubovskoy’s picture

StatusFileSize
new1.3 KB

Rewrote a bit issue-3202896-6.patch, added link for editing media entity.

martijn de wit’s picture

Sounds like a good addition!

Only thing that is left is a proper test to move this issue forward.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

anybody’s picture

Should we perhaps close this issue and merge efforts in #2972846: Improve oEmbed exception logging to make fixing these issues and testing more easy? Both are touching the message details and are very close... What do you think?

recrit’s picture

To me the approach in this issue is better than #2972846: Improve oEmbed exception logging.

#2972846: Improve oEmbed exception logging: The change in core/modules/media/src/OEmbed/ResourceFetcher.php will now expose a lot of information to an anonymous user when the fetcher error message is displayed by \Drupal\media\Plugin\media\Source\OEmbed::getMetadata(). That is not desired on any website.

  throw new ResourceException('Could not retrieve the oEmbed resource: ' . $e->getMessage(), $url, [], $e);
msnassar’s picture

When indexing media with unavailable resource using search api, I see the following error in the log:
Warning: Undefined variable $resource_url in Drupal\media\Plugin\media\Source\OEmbed->getMetadata() (line 253 of core/modules/media/src/Plugin/media/Source/OEmbed.php)

Steps to reproduce:
- Create media entity using live stream video using e.g. youtube.
- From youtube, finish the steaming and make it unavailable.
- Use search api to index media.
- run search index.
- Check the log.

I think we should use media source field value to display the resource url as follows:

$this->logger->error(
  $e->getMessage() . ' Resource URL: %resource_url | Media ID: %media_id',
  ['%resource_url' => $media->getSource()->getSourceFieldValue($media), '%media_id' => $media->id()]
);

recrit changed the visibility of the branch 3202896-dont-display-oembed to hidden.

recrit changed the visibility of the branch 11.x to hidden.

recrit changed the visibility of the branch 10.3.x to hidden.

recrit changed the visibility of the branch 10.4.x to hidden.

recrit’s picture

StatusFileSize
new1.55 KB

@msnassar The Undefined variable $resource_url should be fixed in the latest MR.
I created an MR against 11.x with error message updated to the following. I removed the "|" from the message since system logs often are delimited with "|" which could cause issue with parsing the log.

      $this->logger->error(
       'An error occurred while fetching an oEmbed resource for Media ID: %media_id, Embed URL: %media_url, oEmbed Resource URL: %resource_url. Error: @error',
        [
          '%resource_url' => !empty($resource_url) ? $resource_url : 'unknown',
          '%media_url' => $media_url,
          '%media_id' => $media->id(),
          '@error' => $e->getMessage(),
        ]
      );

Attached is a static patch of the MR for composer builds. Please contrib to the MR for any further development.

antonín slejška’s picture

Status: Needs work » Needs review

The patch issue-3202896-7.patch and the diff drupal-3202896-MR9357-11.x--20240828-1.diff look good for me.

smustgrave’s picture

Status: Needs review » Needs work

Have not reviewed but was previously tagged for tests which still appear to be needed.

anybody’s picture

See #20 I still think it would make more sense to get that one finished instead...? (And yes, it's annoying for users ;))

erindarri changed the visibility of the branch 10.3.x to active.

joegl’s picture

StatusFileSize
new1.55 KB

Applied a patch from the Merge Request diff successfully to Drupal Core 10.3.13.

I reviewed the Merge Request in #2972846 first, but that issue does not add an access check before logging the error and does not appear to resolve the issue of this error showing to anonymous users. As another user commented in that issue, it appears the changes there are exposing even more detailed information to anonymous users which exacerbates the issue presented here.

loze’s picture

Maybe it could dispatch an event when the source isn't found?

I have a site with over 100,000 entities with media references, many of these have become old and the youtube urls are no longer active. I would like to be able to flag the posts so I can easily find them and remove them or take some appropriate action.

If i was able to listen for an event when there is an error I could do this.

joegl’s picture

I ran into a unique fatal error caused by the patch. In short, the media entity being used to generate the edit link for the error message will not always have an ID or valid edit link. I recommend adding a condition to check for the ID of the media entity before continuing.

In length, a content type with an oEmbed media entity will reference the media entity using an entity reference field. When using layout builder for the default display for the content type, the content preview for layout builder uses the generateSampleValue method on the entity reference item to generate a preview. The method attempts to find a random, valid entity to use as a sample value. If it cannot find one, it generates an entity on the fly. The entity generated on the fly is not saved, and does not have a valid ID. This throws a fatal error then when using this code because it attempts to generate an edit link on a media entity without an ID. I recommend adding a condition to check for the ID of the media entity before continuing.

Perhaps there's a better way to go about this then adding a check for the $media->id().

joegl’s picture

StatusFileSize
new1.59 KB

I added the conditional to the merge request and cut a new patch on 10.4.x

loze changed the visibility of the branch 3202896-oembed-error-message to hidden.

pameeela’s picture

Maybe it could dispatch an event when the source isn't found?

This sounds like a good follow up task as it expands the scope of this, and I think it would be better to fix the user-facing issue as originally reported first. This is definitely something I have noticed and agree it is a problem!

poker10’s picture

Re #20 and #31 - I think this issue should be fixed regardless of the #2972846: Improve oEmbed exception logging, because the error should not be visible to anonymous users. The other issue's MR does not fix the message displayed for anonymous users, so let's keep both issues open.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.