4:3 content does not display properly when playing or pausing. The player seems to be getting some kind of re-sizing information that incorrectly adds a letterbox effect. The best way to identify the problem is to show how the interlacing fields are crushed slightly, highlighting how the output is distorted. There is problem with the 16:9 content.

I have done a great deal of testing and cannot find the source of this issue. At first I thought it related to video.js or css, but at this point I believe it is a bug in the video field. I have verified that the transcoded content has the correct dimensions and, since the problem shows up when using mediaelement.js player, it can't be a video.js issue. I am using multiple content types, one for HD (16:9), one for NTSC (4:3), and one for PAL (4:3).

Attached are two screen shots and the poster frame for a PAL 4:3 example, and a screen shot for a NTSC 4:3 example.

For what it's worth, PAL clips have 7 pixels of black added to the top and bottom of the player, and NTSC clips have 5 pixels added to the top and 6 pixels added to the bottom.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Jorrit’s picture

For one of the videos that you sent me, I get the following output:

jorrit@jorrit-vm:~/Downloads$ ffmpeg -i PAL-uncompressed_8-bit_countdown_2_testPattern-euro_webm_pal_1347133940.webm 
ffmpeg version 0.8.3-4:0.8.3-0ubuntu0.12.04.1, Copyright (c) 2000-2012 the Libav developers
  built on Jun 12 2012 16:52:09 with gcc 4.6.3
[matroska,webm @ 0x12239a0] Estimating duration from bitrate, this may be inaccurate

Seems stream 0 codec frame rate differs from container frame rate: 1000.00 (1000/1) -> 25.00 (25/1)
Input #0, matroska,webm, from 'PAL-uncompressed_8-bit_countdown_2_testPattern-euro_webm_pal_1347133940.webm':
  Duration: 00:00:09.99, start: 0.000000, bitrate: N/A
    Stream #0.0(eng): Video: vp8, yuv420p, 768x576, PAR 131:128 DAR 131:96, 25 fps, 25 tbr, 1k tbn, 1k tbc (default)
    Stream #0.1(eng): Audio: vorbis, 48000 Hz, stereo, s16 (default)

I don't know if you recognize the PAR / DAR values, but I don't. From what I read on the internet, the DAR actually determines how the video is displayed. The SAR is 768:576 = 4:3 = 1.333. The DAR is 131:96 = 1.36. The PAR is DAR/SAR = 1.03 = 131:128 (see here). When converting 768x578 to the DAR, I get 768 x 562.8 and I think that that is the resolution that the browser is using.

So I think it comes down to the fact that 768x576 is not really the output resolution of the file, it is programmed to be around 768x563.

When I look at the ffmpeg info of the original file, I get:

jorrit@jorrit-vm:~/Downloads$ ffmpeg -i PAL-uncompressed_8-bit_countdown_2_testPattern-euro.mov 
ffmpeg version 0.8.3-4:0.8.3-0ubuntu0.12.04.1, Copyright (c) 2000-2012 the Libav developers
  built on Jun 12 2012 16:52:09 with gcc 4.6.3
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'PAL-uncompressed_8-bit_countdown_2_testPattern-euro.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2012-09-08 19:09:14
  Duration: 00:00:09.00, start: 0.000000, bitrate: 167430 kb/s
    Stream #0.0(eng): Video: rawvideo, uyvy422, 720x576, 165888 kb/s, PAR 59:54 DAR 295:216, 25 fps, 25 tbr, 25 tbn, 25 tbc
    Metadata:
      creation_time   : 2012-09-08 19:09:14
    Stream #0.1(eng): Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
    Metadata:
      creation_time   : 2012-09-08 19:09:14
    Stream #0.2(eng): Data: tmcd / 0x64636D74, 0 kb/s
    Metadata:
      creation_time   : 2012-09-08 19:09:15

So the original video is 720x576 displayed at 295:216, which is 720x527.2. Lots of strange numbers here, so I am not at all surprised that the browser doesn't display the video the right way. Maybe the video module should force the aspect ratio of the output file to be such that the DAR is equal to the SAR, meaning that the PAR is 1:1.

michael.k’s picture

Good sleuthing, Jorrit!

Indeed, there are lots of strange numbers, though some of those aren't quite right. I think the math occasionally doesn't work right because FFmpeg is rounding the PAR/DAR/SAR values so as not to create imbalanced output on the command line, so it tends to create a rough reference rather than a perfectly precise and decimally accurate number (eg this output for the same file: "Video: h264, yuv420p, 768x576 [PAR 295:288 DAR 295:216], 930 kb/s, PAR 38645:36864 DAR 38645:27648"). But I don't know the internals of FFmpeg's calculus, and I'm not a broadcast engineer, so I'm guessing.

In the case of the PAL file you're referencing and its transcoded relation, the source aspect ratio shows a correct non-square PAR value of 59:54, but I can't verify the DAR number. The files that I've analyzed after transcoding have the correct dimensions and work on their own in other players, or in stand-alone browser windows, just not when handled by the Video module.

Anyway, the whole point of these transcoding jobs is to create square pixel versions for web display, so I think you're on to something to force 1:1. The question is how to tell FFmpeg to do that, right? It wouldn't be good if the Video module is compensating for a badly distorted file. There is an optional flag one can set for -aspect (eg "-aspect 1.3333" or "-aspect 4:3"), so maybe that can be integrated into the module?

Note that it's important to develop a solution that doesn't break the interlacing fields, so one needs to distort the width value to get PAR 1:1, not the height. So 768x576 is correct for PAL; 640x480 for DV NTSC; and 648x486 for CCIR 601 NTSC. (Although some will argue that the non-square sources should be 704x576 and 704x480, but I think 704w is primarily used for broadcasting and not for cameras, though I could be wrong.)

Thanks for your work on this!

Jorrit’s picture

Well, I do not intend to distort the width or height at all. If you request the video to be 640x480, the idea is that I not only specify -s 640x480, but also -aspect 4:3. In this way, PAR will be 1:1 and SAR=DAR=4:3.

michael.k’s picture

It seems that method will reinforce PAR 1:1. But in theory it shouldn't be necessary to add the -aspect flag, right? If FFmpeg transcodes a file from non-square PAR to square PAR, SAR and DAR should follow.

Perhaps the key to the problem is the sources that begin as non-square PAR (eg 720x576 with non-square pixels). It should work to transcode them into a theoretical PAR of 1:1 by requesting the transcoded size be 768x576, which should make SAR=DAR=4:3. But to my knowledge, a PAR flag is not available in FFmpeg. Perhaps there is a subtle mathematical inconsistency in the value that is passed to the module so that it cannot verify or interpret 1:1 correctly, so thinks it must compensate with letterboxing or black bars?

This is just a wild guess, but maybe number_format should use decimals or somehow be more accurate in video.field.inc?

video.field.inc, line 926:

    else {
      // Now lets check our ratio's
      $ratio = number_format($wxh[0] / $wxh[1], 4);
      if ($ratio == $aspect_ratio['ratio']) {
        $options[$key] = $value . ' (' . t('Matches ratio of original file') . ')';
      }
michael.k’s picture

Some of the comments here are interesting to note:
http://us.php.net/number_format

Jorrit’s picture

The number format is only used to compare aspect ratios in order to add the "matches ratio..." text. It is not used by or for FFmpeg.

michael.k’s picture

I think I've solved the puzzle for this issue.

This is a SAR and DAR metadata issue. When non-square sources are transcoded to supposedly square outputs, it's messy, producing odd values. When that mathematically imprecise output is interpreted by the module, regardless of whether SAR and DAR are close or far away from 1:1 and [aspect:ratio], the display is wrong if they are not exact.

So for example, when videos have a true SAR of 1:1 and DAR of 4:3, like some iPhone videos, they play perfectly in the module because there is no change to SAR and DAR during transcoding. But when non-square sources have close values and transcoding changes them from non-square to *almost* 1:1 and 4:3, like [SAR 295:288 DAR 295:216], they still become distorted when interpreted for display.

Note that the files themselves look good in the browser or in stand-alone viewers. It's something in the Video module that distorts them significantly. One of the reasons I'm using interlaced tests is to show how the source is "raked" but the module makes the display look rippled or wavy.

So it seems that SAR / DAR rounding errors during transcoding require a way for the module to call the video filter and set scale and DAR values, probably within admin/config/media/video/presets/preset/name.

This test produced the correct display:

  1. Converted non-square source of 720x576 in the usual way through the module, selecting 768x576 output
  2. Got 768x576 file with [SAR 295:288 DAR 295:216], which produced a poor display with black bars on top and bottom
  3. Then I converted a file manually from the same non-square source, but with correct -vf values
  4. This created a similarly sized 768x576 file, but with [SAR 1:1 DAR 4:3] metadata that displayed correctly!

(Note: I needed to trick the browser to refresh the display by changing the name of the file, reloading to get black, then changing it back to get the file that displayed properly.)

# This was the command (the pad value is redundant in this case and can be deleted):
ffmpeg -i PAL-DV_countdown_2_testPattern-euro3.mov -vf "scale=768:576,pad=768:576:0:0,setdar=4:3" test.mp4

# Then I replaced the file used by the node:
mv test.mp4 PAL-DV_countdown_2_testPattern-euro3_mp4_1348297857.mp4
Jorrit’s picture

is it also outputting a file with PAR 1:1 when you specify -s 768x568 -aspect "4:3" instead of the -vf bit?

michael.k’s picture

No I tried that. I'll send you a comparison shortly.

michael.k’s picture

FileSize
5.08 KB
4.79 KB
4.93 KB

Attached is the output from three separate transcoding jobs.

Jorrit’s picture

I mistyped. I meant -s 768x576 -aspect "4:3". Sorry.

michael.k’s picture

Wow, please accept my sincerest apologies! Comments #2 and #3 were correct, and the -aspect flag is enough. Re-testing several videos with the correct square pixel dimensions in both -aspect "4:3" and "16:9" proves that approach and creates PAR=SAR=1:1.

In a freak coincidence, I seem to have mistyped the dimensions in my earlier testing as well, so I had misled myself down -vf lane. This is one of the hazards of working very late...

So there is probably only the need for a user to set -aspect "4:3" or "16:9" because other aspect ratios are likely to come from square pixel sources.

Jorrit’s picture

Well, I could just calculate the -aspect ratio from the destination dimensions. They just need to be equal. So if you want a 400x300 file, I can calculate that the aspect ratio should be 4:3. FFmpeg also accepts a decimal argument, which makes this even easier. For the most common aspect ratios I will add the "x:y" notation. If one can't be found, I can use the decimal notation.

michael.k’s picture

That sounds great!

I've seen extremely wide videos and other unusual aspect ratios on the net, so it will be nice to support people's creative expression. There are also cases where people want to create 2.40:1 and other decimal-based ratios from non-square sources, also by cropping not just squeezing with the camera's available settings. But I haven't tested those things. (I use 2.40:1 but my sources already have square pixels in HD.) For example, some DPs like to tape off their intended display ratios when shooting because they know they're going to crop, even though their video files have more information.

michael.k’s picture

I've just been testing the conversion of non-square sources into 2.40:1 and verified that the use of -aspect 2.40 produces [SAR 1:1 DAR 12:5] in the output file. Likewise, -aspect "240:100" also works perfectly.

I'm sure this was implied in your comment, but just to be thorough, -aspect "2.4:1" produces the error Invalid chars ':1' at the end of expression '2.40:1', so the -aspect flag requires integers when using the ratio format.

Jorrit’s picture

I don't have time to test it, but maybe you can: what happens with DAR and PAR when you specify the aspect ratio to be the same as the resolution: -s 768x576 -aspect 768:576.

michael.k’s picture

That gives you [SAR 1:1 DAR 4:3] because in this case the dimensions are whole pixel-accurate for 4:3.

Likewise, with -s 1024x576 -aspect 1024:576, you get [SAR 1:1 DAR 16:9].

But if you are going for derivatives of non-square pixel sources that don't have perfect math, things get a bit tricky. There is not a perfect answer for what is the correct pixel width for DV NTSC 16:9 sources, so this creates problematic results for DAR.

For a source with 720x480, 20633 kb/s, SAR 10:11 DAR 15:11:

-s 852x480 -aspect 852:480 >> [SAR 1:1 DAR 71:40]
-s 852x480 -aspect "16:9" >> (same as above)

-s 853x480 -aspect 853:480 >> SAR 10:11 DAR 15:11 [x264 ERROR, *see below]
-s 853x480 -aspect "16:9" >> (same error)

-s 854x480 -aspect 854:480 >> [SAR 1:1 DAR 427:240]
-s 854x480 -aspect "16:9" >> (same as above)

#

*853x480 produces this error:
[libx264 @ 0x10200fa00] width not divisible by 2 (853x480)

But for other formats like webm, ogv and flv, it works to create [SAR 1:1 DAR 853:480].

Interestingly, Apple Final Cut Pro documentation recommends 853x480 for DV NTSC 16:9. My whole pixel analysis shows that the more correct number should be 852x480:
http://documentation.apple.com/en/motion/usermanual/index.html#chapter=B...

Jorrit’s picture

I can't imagine that they really mean "853x480". I almost never see odd numbers in video resolutions. As your output shows, libx264 doesn't even support that.

michael.k’s picture

I've been mystified by that number for years because it's in their documentation (as above). But I've never been able to make a mathematical proof until now. And yes, FFmpeg has a nice way to support the argument for 852x640 by not supporting the number 853 for its own conversion algorithm.

There are some other numbers in the Apple manual that also don't make sense. Unfortunately, these kinds of questions are outside the typical use case for FCP video projects, which makes the percentage of users that care or want to care very small. And sadly Apple doesn't listen very well to its pro community, which is also small, as is evident by the roll out for FCP X.

Thanks for your feedback.

brycefisherfleig’s picture

Issue summary: View changes
Status: Active » Postponed (maintainer needs more info)

Skimming this thread, I'm not entirely sure what work needs to be done on this thread. I'm postponing until Jorrit or Michael.k can clarify for me. It sounds like this is very much edge case related to unusual video resolution edge cases.

michael.k’s picture

Thanks for following up on this thread, brycefisherfleig, and glad to see your support of the Video module. I think it will be worth dealing with this odd parsing issue when some of the other issues related to conversion and transcoding get hammered out. I'm finally able to get back to some of this testing after many months of downtime.