Problem/Motivation

We discovered a couple of bugs in format_size:

  1. #2583041-15: GD toolkit & operations should catch \Throwable to fail gracefully in case of errors : the calculation is giving wrong results in some cases:
    the current code does rounding on rounding on rounding which may lead to false results: add 4053371676 as a test, this is 3,774996545 GB but 3,78 is currently returned...
  2. it can take a negative $size as input and format it to say '-1024 bytes'. But to be consistent it should be -1 KB
  3. pow(1024, 9) is incorrectly determined to be 1 YB when it should be 1024 YB

Proposed resolution

  • Fix the calculation issues and add tests.
  • Preserve the sign.

Remaining tasks

User interface changes

none

API changes

none

Comments

mondrake created an issue. See original summary.

mondrake’s picture

StatusFileSize
new950 bytes

Test only patch. Actually it'd better if this test had a data provider instead, but that's yet another issue.

mondrake’s picture

Status: Active » Needs review

Status: Needs review » Needs work

The last submitted patch, 2: 3001398-2-test-only.patch, failed testing. View results

longwave’s picture

I think a negative size is OK. What if you had a diff system that as well as showing the changes in lines it showed the change in filesize? That could be a negative value.

mondrake’s picture

StatusFileSize
new1.63 KB
new2.03 KB

#5 good thinking!

It looks like Bytes::toInt is converting negative $size to positive... a bit messy here.

mondrake’s picture

Status: Needs work » Needs review
mondrake’s picture

StatusFileSize
new3.25 KB
new1.75 KB

Also, -1 is considered plural by formatPlural, so we get '-1 bytes' in place of '-1 byte'. But that's too far away from here.

Here a first step towards fix.

longwave’s picture

We could do something like

return ($size < 0 ? '-' : '') . \Drupal::translation()->formatPlural(abs($size), '1 byte', '@count bytes', [], ['langcode' => $langcode]);

but not sure if this is an improvement, maybe just wait for #1131926: format_plural returns a plural value for a negative singular

The last submitted patch, 6: 3001398-6-test-only.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 8: 3001398-8.patch, failed testing. View results

mondrake’s picture

Status: Needs work » Needs review
StatusFileSize
new3.54 KB
new4.29 KB

Did not know about that, thanks!

Let's keep the workaround here FTTB. This one should be close now.

Status: Needs review » Needs work

The last submitted patch, 12: 3001398-12.patch, failed testing. View results

mondrake’s picture

StatusFileSize
new4.29 KB
new818 bytes

Zettabytes and Yottabytes are kinda stretching PHP's type casting features... :)

mondrake’s picture

Status: Needs work » Needs review
mondrake’s picture

StatusFileSize
new860 bytes
new4.26 KB

Actually better.

The last submitted patch, 14: 3001398-14.patch, failed testing. View results

dawehner’s picture

  1. +++ b/core/tests/Drupal/KernelTests/Core/Common/SizeTest.php
    @@ -48,11 +53,8 @@ protected function setUp() {
    +        $this->assertSame($expected, $result, $expected . ' == ' . $result . ' (' . $input . ' bytes)');
    

    What is the reason for putting $expected == $result in here? PHPUnit provides this information out of the box.

  2. +++ b/core/tests/Drupal/KernelTests/Core/Common/SizeTest.php
    @@ -62,11 +64,8 @@ public function testCommonFormatSize() {
    -      $this->assertEqual(
    -        $size,
    -        ($parsed_size = Bytes::toInt($string = format_size($size, NULL))),
    -        $size . ' == ' . $parsed_size . ' (' . $string . ')'
    -      );
    +      $parsed_size = Bytes::toInt($string = format_size($size, NULL));
    +      $this->assertEquals($size, $parsed_size, $size . ' == ' . $parsed_size . ' (' . $string . ')');
    

    To be honest I think we should improve the readability of this test and assign $string outside.

mondrake’s picture

@dawehner I'd do that in a separate issue. This test class deserves some love, rejuvenating it to proper PHPUnit concepts - first and foremost introducing @dataproviders for making tests more atomic.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Fair :)

mondrake’s picture

alexpott’s picture

Status: Reviewed & tested by the community » Needs review

I'm not entirely convinced by #5. I'd rather deprecate support for negative amounts and leave it up to the caller on how to handle negative bytes. So we don't change behaviour in Drupal 8 but in Drupal 9 we through an exception of something. I think it would also be useful to have a look at other PHP libraries and see how they handle the negative case.

mondrake’s picture

Assigned: Unassigned » mondrake
Status: Needs review » Needs work

deprecate support for negative amounts and leave it up to the caller on how to handle negative bytes

agree, keep it simple. Will reroll then.

mondrake’s picture

mondrake’s picture

Assigned: mondrake » Unassigned
Issue summary: View changes
Status: Needs work » Needs review
StatusFileSize
new2.99 KB
new3.41 KB

Here it is. Updated IS with the deprecation decision.

neclimdul’s picture

+ if ($size < 0) {
+ @trigger_error('Using format_size with negative $size argument is deprecated in Drupal 8.7.0 and will throw an exception in Drupal 9.0.0. Instead, handle the case of negative quantity beforehand and always pass a positive value. See https://www.drupal.org/node/3004611', E_USER_DEPRECATED);
+ }

If I had a quota of 2Gb of space and used 2.1Gb would I not have -0.1Gb of space? I think the assertion that you can't have negative sizes is incorrect.

neclimdul’s picture

Status: Needs review » Needs work

In fact, I think that should definitely be removed. There may a features that can't allow negative sizes but we absolutely have to be able to format a string with a negative size. The alternative is that a developer basically has to write:

$negative = $value < 0 ? '-' : '';
$string = $negative . format_size(abs($value));

And that just doesn't make sense.

mondrake’s picture

Status: Needs work » Needs review

Let's have more reviews before jumping into patching... there are two opposite views ATM.

mondrake’s picture

BTW the behaviour expected in #26 i.e. "-0.1 GB" is not what happens currently... negative value is just broken.

longwave’s picture

Agree with @neclimdul that there will be use cases where negative values make sense. I don't see how it can harm anyone by supporting negative values here, and if we start throwing exceptions (in D9) then there will be edge cases where somehow someone has ended up with e.g. a negative quota, and we would error when displaying when we don't strictly need to.

The only contentious point I feel is "-1 byte" vs "-1 bytes" but that already has its own issue.

neclimdul’s picture

longwave described my intended point better than I did. The expected output wasn't as important as the fact that it wouldn't error unexpectedly. Correct output would be nice though ;)

alexpott’s picture

Issue summary: View changes
StatusFileSize
new3.09 KB
new2.5 KB

It's not broken as in producing an error - it just always outputs bytes...

>>> format_size(1000);
=> Drupal\Core\StringTranslation\PluralTranslatableMarkup {#1595
     markup: "1000 bytes",
   }
>>> format_size(-1000);
=> Drupal\Core\StringTranslation\PluralTranslatableMarkup {#1600
     markup: "-1000 bytes",
   }
>>> format_size(-10000000);
=> Drupal\Core\StringTranslation\PluralTranslatableMarkup {#1596
     markup: "-10000000 bytes",
   }
>>> format_size(10000000);
=> Drupal\Core\StringTranslation\TranslatableMarkup {#1583
     markup: "9.54 MB",
   }

I'm okay with fix the thing to do -9.54 MB but we should leave the -1 byte/s thing well alone. And fix/discuss that in the other issue.

Whilst working on this I found a bug in really really large sizes > 1024 YB fixed that as well.

I've not included any change to \Drupal\Component\Utility\Bytes::toInt since I think that should have it's own issue. The change in #15 for example will result in breakages of 15 P-bytes whereas before that'd work!?!?!

krzysztof domański’s picture

StatusFileSize
new2.69 KB
new3.38 KB

Small refactoring: I did not change the logic of code. I only removed else in the following code and IMO unnecessary switch statement. The code is a slightly cleaner now.

if ($size < Bytes::KILOBYTE) {
  return \Drupal::translation()->formatPlural($size, '1 byte', '@count bytes', [], ['langcode' => $langcode]);
}
else {
case 'KB':
  return new TranslatableMarkup('@size KB', $args, $options);
case 'MB':
  return new TranslatableMarkup('@size MB', $args, $options);

In addition, there were no s at the end of kilobyte

// Rounded to 1 MB - not 1000 or 1024 kilobyte
alexpott’s picture

Status: Needs review » Needs work
+++ b/core/includes/common.inc
@@ -251,42 +251,25 @@ function check_url($uri) {
-    switch ($unit) {
-      case 'KB':
-        return new TranslatableMarkup('@size KB', $args, $options);
-      case 'MB':
-        return new TranslatableMarkup('@size MB', $args, $options);
-      case 'GB':
-        return new TranslatableMarkup('@size GB', $args, $options);
-      case 'TB':
-        return new TranslatableMarkup('@size TB', $args, $options);
-      case 'PB':
-        return new TranslatableMarkup('@size PB', $args, $options);
-      case 'EB':
-        return new TranslatableMarkup('@size EB', $args, $options);
-      case 'ZB':
-        return new TranslatableMarkup('@size ZB', $args, $options);
-      case 'YB':
-        return new TranslatableMarkup('@size YB', $args, $options);
...
+  $args = [
+    '@size' => $rounded_size * $sign,
+    '@unit' => $unit
+  ];
+  $options = ['langcode' => $langcode];
+  return new TranslatableMarkup('@size @unit', $args, $options);

Changing this removes the ability to translate the units KB/MB/GB etc... which must be why that is done like this.

Removing the else makes sense.

krzysztof domański’s picture

Status: Needs work » Needs review
StatusFileSize
new3.27 KB
new3.95 KB

Ignore patch #33. Removing else and correction of the comment.

The last submitted patch, 33: 3001398-33.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 35: 3001398-35.patch, failed testing. View results

mondrake’s picture

@alexpott re #32

we should leave the -1 byte/s thing well alone. And fix/discuss that in the other issue

I do not think we'll fix that in the other issue, see #1131926-31: format_plural returns a plural value for a negative singular. Do we need another separate issue?

mondrake’s picture

Status: Needs work » Needs review

#36 and #37 are unrelated failures. I’d RTBC but cannot due to earlier work on the patch.

goodboy’s picture

-    $absolute_size = $absolute_size / Bytes::KILOBYTE;
+    $absolute_size /= Bytes::KILOBYTE;
voleger’s picture

StatusFileSize
new1.21 KB
new3.95 KB

Rerolled.
Addressed #40
Fixed CS errors: Reformatted the switch/case construction.

Should the case 'YB': value be replaced with default: to avoid "Missing return statement" warning in IDE?

mondrake’s picture

Status: Needs review » Reviewed & tested by the community

Let's get this in so we can work on the conversion to OO. Retracted the CR that was drafted in #24.

Should the case 'YB': value be replaced with default: to avoid "Missing return statement" warning in IDE?

I don't think so, the block of code covers all possibilities without needing a default - which is not mandatory anyway for PHP.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed d29f046424 to 8.7.x and 1d78e9d5b6 to 8.6.x. Thanks!

  • alexpott committed d29f046 on 8.7.x
    Issue #3001398 by mondrake, Krzysztof Domański, alexpott, voleger,...

  • alexpott committed 1d78e9d on 8.6.x
    Issue #3001398 by mondrake, Krzysztof Domański, alexpott, voleger,...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.