Add poster image to HTML5 media videos

If the image you want to show as a poster is tied to the video file uploaded, maybe you want to have the image field on the video media type instead? Then you can create a viewmode that shows the image, and other that shows the video, and render the entity using one or another depending on the context.

The FileVideoFormatter is provided by the file module. It would be nice if you can select an image field from the parent entity in the formatter (when the image module is installed). That way you could:

• Create a media type with a 'Video' source
• Add a image field to your media type
• Select the image field as 'poster' on the formatter of the video field
• Add the poster attribute in the formatter during rendering

Moving this to the file module since this is not a media specific problem. You could want the same functionality using a video file field and image field on a node.

I had the same problem. I may switch back to using video.js module as there's an issue/patch there to support poster with core media.

For now, I modified the twig file. I didn't try the views approach like you did. I added an image field to the media video type that can be optionally uploaded when creating the local video. The image has to have the same name (minus extension) of the video file.

Yea, not ideal to do this logic in twig perhaps, but I just want it to work without having to write code or something else.

{# Get poster from first src #}
{% set file = files|first %}
{% if 'src' in file.source_attributes|keys %}
  {% set parts =  file.source_attributes.src|split('.', -1) %}
  {% set poster = parts|join('.') ~ '.jpg' %}
{% endif %}

{# Add poster to attributes #}
{% if poster is defined %}
  {% set attributes = attributes.setAttribute('poster', poster) %}
{% endif %}

<video {{ attributes }}>
  {% for file in files %}
    <source {{ file.source_attributes }} />
  {% endfor %}
</video>
{{ devel_breakpoint() }}

Hi @all. I met with this problem on many projects. During implementing a Media thumbnail video module, I added an extended video formater that works with media video and attach media thumbnail image like a html5 video poster attribute. Let's try.

@lamp5
Can you provide more details please?
I need it for my project : i've changed the default video player with mediaelement player and i need video thumbnails. With the mediaelement video player there is a attribute "poster" for the tag = that s the thumbnails of the video.

i want that thumbnails to be generated automaticaly. Drupal 8 does that for the default video player, can i fetch that thumbnails? Or with the field "thumbnails" that i see in the display view of the media content type "video"? Or do i have to generate it with a tool, like a video formater as you say.

Try this module

Here is patch that adds the ability to specify a field on the media video entity to use as a poster file as well as a field to use as a transcript file. So, in order to add poster and transcript capabilities to your media video file entity you would:

1. Install this patch
2. Add an image field for the poster to the media->video_file bundle
3. Add a file field for the transcript (vtt) to the media->video_file bundle
4. Under manage display configure one or more display modes to use the two new fields you just created
5. Create a new video_file with poster and transcript
6. On a node using Add media add your media entity
7. Save and view node

1. +++ b/core/modules/file/src/Plugin/Field/FieldFormatter/FileVideoFormatter.php
   @@ -40,6 +46,27 @@ public static function defaultSettings() {
   +    $image_fields = [0 => $this->t('None')];
   

   We support #empty_option on #type select to define that, you don't need to put that in the array. And those empty options usually have '- None -' or so t o separate them from the other values.

2. +++ b/core/modules/file/src/Plugin/Field/FieldFormatter/FileVideoFormatter.php
   @@ -66,6 +93,18 @@ public function settingsForm(array $form, FormStateInterface $form_state) {
          ],
   +      'poster' => [
   +        '#type' => 'select',
   +        '#title' => $this->t('Poster field'),
   +        '#default_value' => $this->getSetting('poster'),
   +        '#options' => $image_fields,
   +      ],
   +      'transcript' => [
   +        '#type' => 'select',
   +        '#title' => $this->t('Transcript field'),
   +        '#default_value' => $this->getSetting('transcript'),
   +        '#options' => $file_fields,
   

   I'd say we should either hide those options if no candidates are available or include instructions on how to use them, for example as a description that says that a file/image field needs to be created.

3. +++ b/core/modules/file/src/Plugin/Field/FieldFormatter/FileVideoFormatter.php
   @@ -79,9 +118,46 @@ public function settingsSummary() {
        ]);
   +    $summary[] = $this->t('Poster field: %poster', ['%poster' => $this->getSetting('poster')]);
   +    $summary[] = $this->t('Transcript field: %transcript', ['%transcript' => $this->getSetting('transcript')]);
   

   this should only be added if not empty.

4. +++ b/core/modules/file/src/Plugin/Field/FieldFormatter/FileVideoFormatter.php
   @@ -79,9 +118,46 @@ public function settingsSummary() {
   +      $file = $entity->get($this->getSetting('poster'))[0];
   +      $file_uri = $file->entity->getFileUri();
   +      $poster = file_url_transform_relative(file_create_url($file_uri));
   +
   

   $file should be $file_item or so, as it is not a $file entity but a FileItem object.

   file_create_url() and file_url_transform_relative() are now deprecated in favor of the new file url generator service in Drupal 9.3

   I'd suggest using $item->entity->createFileUrl() which is an existing wrapper for that.

5. +++ b/core/modules/file/src/Plugin/Field/FieldFormatter/FileVideoFormatter.php
   @@ -79,9 +118,46 @@ public function settingsSummary() {
   +      $attributes = $elements[0]['#attributes'];
   +      $attributes->setAttribute('poster' , $poster);
   +      $elements[0]['#attributes'] = $attributes;
   

   attributes is an object, you don't need to set a reference to it back, it's also very common to rely on array access with it and just do [#attributes']['poster'] = ...

6. +++ b/core/modules/file/src/Plugin/Field/FieldFormatter/FileVideoFormatter.php
   @@ -79,9 +118,46 @@ public function settingsSummary() {
   +      $languages = \Drupal::languageManager()->getNativeLanguages();
   +      $elements[0]['#transcript'] = [
   +         'file' => $transcript,
   +         'srclang' => $langcode,
   +         'label' => $languages[$langcode]->label(),
   +      ];
   

   I don't think you can blindly assume that the language of that file is actually going to be $langcode, there might be a translation for it.

   We can't guarantee that this is correct, but it's more likely to work if you check if $entity has a translation for $langcode, and then a) get that translation to get the value from a possibly translated file field in the correct language and if that's not available, us the active language of the entity.

@berdir thank-you very much for the review, I will make the appropriate changes and re-roll. Cheers!

I have worked on few points from #17 comment like point 1,3,4,5.

Hopefully it wud be helpful for u

Thank-you for the help @dhirendra.mishra! I have addressed all of the comments from @berdir now with the exception of #5 which throws an error if you treat it as an array (see attached with dpm). That being said I think I improved how I added the poster attribute be using the "merge" method. I am not sure I handled the translations correctly for #6, I would appreciate any recommendations there!

Fixing test case failure

Fixing test case failure, adding srclang to dictionary.txt file

Thank-you @BS Pavan for your help!

Fixed failing test cases of SchemaIncompleteException.

Manually tested the poster functionality on a real site (dev only for now), and it's working nicely. Using image/file fields on the media type with configuration seems like a good approach (although also shows that we can't retire those fields entirely in favour of media, since we need them on media entities...).

Still needs automated test coverage, and didn't manually test the transcript yet.

> although also shows that we can't retire those fields entirely in favour of media, since we need them on media entities...

I'm still confused why anyone thinks that's a option :) Image/Document medias were always going to require image/file fields? Media is no replacement, it's a system on top of the existing features. All we can do is explain it better in the UI when you want to use what.

I think it's more having both media and file/image as options for configurable fields on nodes and similar entities, but yeah not sure how to restrict that either except via docs.

#26 works for me. It would be nice if you could select an image style.

Patch updated for 9.3

Adds test and fixes label handling in formatter without language module enabled:

$label = ($languages[$langcode] instanceof Language) ? ($languages[$langcode]->getName()) : ($languages[$langcode]->label());
 

Please review. Adds test and fixes label handling in formatter without language module enabled:

$label = ($languages[$langcode] instanceof Language) ? ($languages[$langcode]->getName()) : ($languages[$langcode]->label());
 

Patch #37 works great for us! Thanks!

Maybe this should be a separate issue, but it would be nice to implement a switch to add the attribute "playsinline". I needed to add that to our to get out hero video to autoplay on iPhones.

@maddentim, see #3046152: Video media needs playsinline option for the playsinline attribute.

I'll review this patch.

Thanks.

Rerolled patch.

Rerolled patch with git format.

@pbouchereau the patch needs to be based in the docroot, not within core

@sleitner
And without a missing file too... learning the hard way.
Thanks.

phpunit has been updated in DrupalCI to 9.6.1 and throws new deprecation warnings. Not related with your patch as far as I can see.

We should wait for the next 9.5.x branch testing run and rerun the test as soon the problem is solved.

Branch testing only failed temporarily.

Branch testing only failed temporarily again.

updated the issue summary.

Wondering if this needs a review / sign off from a 'subsystem' media maintainer. Already seeing some of them commenting.

Added some tags so hopefully it will reach more people, searching the issue queue :)

----

btw really liking this new feature

making clear the current patch is about mapping the fields at a media entity.

Branch testing only failed temporarily again.

Branch testing only failed temporarily again.

+++ b/core/modules/file/src/Plugin/Field/FieldFormatter/FileVideoFormatter.php
@@ -66,6 +92,22 @@ class FileVideoFormatter extends FileMediaFormatterBase {
+        '#type' => 'select',
+        '#title' => $this->t('Poster field'),
+        '#description' => empty($image_fields) ? $this->t('Please add an image field to this media entity to use the poster feature.') : NULL,
+        '#default_value' => $this->getSetting('poster'),
+        '#options' => $image_fields,

The text here shouldn't use 'Please', I think we have an issue around to remove please from the code base entirely. Also not sure 'poster feature' helps much.

Maybe:

'An image field to use as the source of the poster attribute'?

--

Also what happens if the poster image uploaded is wildly different dimensions to the video - does this need to support image styles? Will add some complexity to the patch to do so, but was also mentioned in #30.

We have tested this patch for over a month now and I agree with catch. without an image style it is not usable for content-editors.

For site builders the option list is getting really long now and with adding image style options inside this screen I think we need some extra structure like fieldsets/details to group some settings. (also used an other patch to add some other video settings)

As we're now running into this requirement, I'll create a MR from the latest patch and will try to work on this. Really helpful addition.

Yay, it basically works!! :)
Give it a try on the tugboat preview.

Now I'm wondering, if the implementation currently in viewElements() shouln't at least partially be moved into prepareAttributes() or what's the reason for the decision to put it into viewElements().
Also I'm not totally sure about the Cache tags implementation.

Would be nice to have another review, I think we're making progress here and I'm happy to learn details. Also we should have a look at the existing tests again, I didn't touch them yet.

Current status:

@Grevil please note the 9 upstream test fails in core, that will also fail here, until it's resolved in core.

I guess we have to wait for 10.1.x-dev branch to pass...

I added a few comments, otherwise LGTM!

Forgot to mention it. I debugged the cache tags and they also LGTM!

Thanks @Grevil, I fixed the image style test now, please review finally.

Switching back to 10.1.x so we hopefully get this in before stable as minor fix.

See https://www.drupal.org/about/core/blog/new-drupal-core-branching-scheme-... (linked in the comment above) 11.x is the target branch for everything now and we backport from there.

Thanks @catch!

GitLab rebase didn't work so let's create a fresh new MR against 11.x instead and leave the 10.1.x one if someone uses / needs it in the meantime? @Grevil could you do that?

So the old MR might still be reviewed in the meantime to speed things up.

Testing the new image style feature.

A user has I have now 3 options within the poster field:

• None
• Thumbnail - image field on my video media entity
• Image - image field on my video media entity

As showed below the thumbnail option and the image option are equal. With both a user can select an image style. So I was wondering what is the difference?

Thank you for the feedback @Martijn de Wit. This shows possible confusion for end-users.

As showed below the thumbnail option and the image option are equal.

It's not the same. While in the "Poster field" you pick the image field to display as poster.
In the "Poster field image style" field, you select the image style (Imagecache was the name in Drupal 7) to apply on the image.

You say the options are equal - but that's not the case and it doesn't look like this is true on your screenshots?

Perhaps the UX team should have a short look to improve the descriptions or field labels?

Added a new branch "2954834-add-poster-image-transcript-from-fields-11.x" and applied the current "MR !4010 " diff.

@Anybody, as you created the MR, you should (hopefully) be able to close it again. I get a 404 error, when trying to close the old MR.

A yeah, now I get it :)

I have 2 image fields on my video entity. one named 'image' and one named 'thumbnail' from previous tests.
Sorry for the confusing post.

Then it works as advocated, so it was a good test :)

Took a look and believe we will need an upgrade path for the new formatter settings.

Thanks @smustgrave! That would mean, we need an update path to set all values to empty on existing formatters:

      'poster' => '',
      'poster_image_style' => '',
      'transcript' => '',

Is that really needed? I guess if yes, it's because the keys have to be added?
Do you know any similar example we could use for the update hook?

Would it perhaps make sense to add a helper for such cases to make field (formatter / storage) settings updates more simple (DX)? Similar to: #937442: Field type modules cannot maintain their field schema (field type schema change C(R)UD is needed)

At least to me such things always feel quite hard to do it right, having to copy existing examples etc... we shouldn't open that side topic here, I guess, but perhaps this is a good starting point to proceed the discussion in the linked issue or a new meta issue?

May be a good example https://www.drupal.org/project/drupal/issues/3309016

Is that really needed?

Yes we need to update existing configuration so that if you update and do a config export, the exported configuration matches how things would look if you'd resaved. If we don't do this, then when you save formatter settings later manually, these keys would be added which gets very inconsistent/confusing.

The pattern is to put the logic for adding the new keys into a presave hook, and then using the actual post update function to just load and re-save configuration. Views (both in 10.1 and 9.5) has the most examples of this, but also a grep for ConfigEntityUpdater will find more.

I added an update hook, but I couldn't figure out, how to specifically target field instances using the "file_video" formatter. Please review and adjust accordingly!

May be worth asking #contribute channel.

But the version number wouldd be 101021

The update should use hook_post_update_NAME() rather than hook_update_N(), and the logic of actually updating the configuration should happen in hook_entity_presave(), not in the update function itself, so that it also runs when importing default config and similar.

Thank you, @catch!

I got the point on using the hook_post_update_NAME() hook rather than the hook_update_N() hook, but didn't understand the hook_entity_presave() part, as we are basically only updating a few "entity_form_display" setting values and do not change the config itself. So why exactly the "hook_entity_presave" hook? Am I missing something? Is there an example with a similar update in core?

For an example of another update, see:

/**
 * Implements hook_ENTITY_TYPE_presave() for entity_view_display.
 */
function media_entity_view_display_presave(EntityViewDisplayInterface $view_display): void {
  $config_updater = \Drupal::classResolver(MediaConfigUpdater::class);
  assert($config_updater instanceof MediaConfigUpdater);
  $config_updater->processOembedEagerLoadField($view_display);
}

Which is related to:

media_post_update_oembed_loading_attribute()

Grepping for ConfigEntityUpdater will find more post updates which will usually be tied to a presave hook in the respective module.

Looking at this again, I think we might need to do more with the transcript support though:

HTML allows multiple <track> elements, to support subtitles in different languages as well as captions, see the examples on https://developer.mozilla.org/en-US/docs/Web/HTML/Element/track

So... should we think about supporting a multiple value field for 'track'? But then this would ideally support the 'kind' attribute as well as language etc., it is starting to look like another media type with its own fields at that point though and then a multi-value media field on the video entity.

Tagging for needs subsystem maintainer review to hopefully get a second opinion on that last bit.

Thanks for the clear-up @catch! I'll have a look again, once everything is discussed with the relevant sub system maintainer! :)

I think this might be worth splitting into two issues - one for poster image for which I assume there should just be one.

And one for transcript/<track> where there are more possibilities - if we do split, that should probably be the new issue since there'll be more to figure out on there. Might be worth waiting to see what media maintainers say though.

@catch totally agree with #94, thought the same yesterday. And I think the poster image is much more important / widely used. Google Webmaster Tools complains, if it's missing, so we should get that one in soon.

Yes, let's wait for media maintainers feedback.

HTML allows multiple
elements, to support subtitles in different languages as well as captions, see the examples on https://developer.mozilla.org/en-US/docs/Web/HTML/Element/track

So... should we think about supporting a multiple value field for 'track'? But then this would ideally support the 'kind' attribute as well as language etc., it is starting to look like another media type with its own fields at that point though and then a multi-value media field on the video entity.

Since media entities are translatable, I don't think we need to support different tracks for different languages. You probably only want the track for the current language (like how you would handle translations for other entities).

The kind attribute seems interesting, although I'm not completely sure if we can properly explain the different use-cases to site editors. I think accessibility experts might have more insights on when / how to use the kind attribute properly. It would already help a lot to configure the kind attribute on the formatter. Site builders can add a field description on the field where editors can upload the .vtt files, to make sure their editors understand what kind of input is expected.

Since media entities are translatable, I don't think we need to support different tracks for different languages. You probably only want the track for the current language (like how you would handle translations for other entities).

I don't think this is right for subtitles. You can have a video in a single language, with subtitles in 15 languages, and as someone watching the video, you would want those subtitles options to be available via the video player, not by having to view different language versions of the entity. i.e. you can watch an English language film with English, English captions, French, or Arabic subtitles. Even if someone is browsing a site in French, and there is a video that only has English audio, they still might want to choose English subtitles instead of French (i.e. if they're trying to learn English and want to pause and read sometimes when it's hard to hear).

I guess you are right we probably need to support multiple subtitle files in each translation, since the source field could technically be different for each translation. So you could have a different video for each language, while still wanting subtitles in multiple languages for each translation.

Then we only need to decide how to deal with the kind attribute. I don't feel I'm the best person to make a decision here. At the moment, I'm thinking it is already helpful to be able to configure a single kind value on the formatter for all the files uploaded in the selected file field. That should also be relatively easy to implement. If we need to support multiple kind attributes, I think we need to be able to configure a different file field for each kind. This is a bit more complex, but if we need to support this I would probably vote to do it like this.

Thanks for the discussion! I think we should split this then to not block the poster image implementation, which Google Search Console complains about. This is the key feature most users need.

Implementing transcript is also helpful, but not that widely used, I think. So I'd suggest

1. We change the title of this issue to "Add poster image and transcript to HTML5 media videos"
2. We create a sibling issue: "Add poster image and transcript to HTML5 media videos" as follow-up and postpone it on this one
3. We split the code so that this can be reviewed and merged and work on the transcript feature can be proceeded in detail later on.

Would have been better we did this earlier, sorry.

Yes splitting is good, but I think we should use track for the new issue rather than transcript, since that covers the transcript vs. subtitle use-cases and the HTML element is the same for both.

Created the child issue here: #3372592: Add track to HTML5 media videos.

Alright, I removed all the transcript parts. Please review, in case I missed anything!

Also, do we still need to implement the hook_ENTITY_TYPE_presave(), as discussed in #90 and #91? Still unsure about that bit.

EDIT: Not a well describing commit message, my apologies.

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Rebased

Seems we are missing upgrade test coverage. Should be simple though to add.

Updated the patch from #48 to include media entity reference field for thumbnail image on Drupal 10.x

Updated the patch from #48 to display default image from selected poster image field, if no thumbnail is added to video media.

Updating the patch in #109
Fix issue if no default image is added to field.

Why using the patch from #48?

There is a merge request that is far more recent? containing solution after several discussions with good points ..

The only thing that are missing in the merge request are tests...

This patch only for poster attribute.
1. Once the patch applies, then add existing image field
2. Change the poster option as an image in Manage display in Video Media type

Resolved merge conflicts after #3420980: Convert FieldFormatter plugin discovery to attributes was committed.

Hiding all old patches

@todo:

1. Fix coding standards
2. Add tests