Use SI and/or IEC units for byte counts?

This was posted 2 days late, right?

For reference, not only Apple is using the SI standard correctly, Linux kernel is too, as you can read in units(7)

It is my opinion, Drupal should always be using the correct SI and IEC units, except when handling a interface that does not handle those units correctly

This makes no sense to me.

Primary hosting platform is linux. It uses IEC (unless you go out of your way not to).
IMO it would be more confusing to have Drupal say one thing and command line say something else.

I.e.
$ dd if=/dev/zero of=testfile.txt bs=1000 count=1
$ dd if=/dev/zero of=testfile2.txt bs=1024 count=1
$ ls -al testfile*
-rw-r--r-- 1 michael michael 1024 2012-01-27 18:11 testfile2.txt
-rw-r--r-- 1 michael michael 1000 2012-01-27 18:10 testfile.txt
$ ls -alh testfile*
-rw-r--r-- 1 michael michael 1.0K 2012-01-27 18:11 testfile2.txt
-rw-r--r-- 1 michael michael 1000 2012-01-27 18:10 testfile.txt

For me we should follow the international standards, rather than be like a IE6. Drupal = Early Adopters.

Extra whitespace at the end:

+    if ((($unit_length == 2 || $unit_length == 3) && $unit[1] == 'i') || 

Gubibytes? :)

+    $string = '76.24 Gubibytes'; // Misspeld text -> 76.24 GiB

The patch does not apply.

We are still using the word 'Gubibytes', sorry for my ignorance but, shouldn't it be 'Gibibytes'?

Oh, sorry! You're right, I had not seen this!

Maybe we need more opinions before the RTBC.

Confusing.

It's patch going to get drupal accepts every units, right ?

user input :
10 KiB = 10 x 1024
10 kB = 10 x 1000
10 KB = (error ?)

#19: si-units.patch queued for re-testing.

Fixing test!

GGrrrrr

Less than minor: for consistency I think we should always use a space between the number and its units:
7.4kB --> 7.4 kB, 5MB --> 5 MB, 40gibibytes --> 40 gibibytes, ...

+ *   optional SI or IEC binary unit prefix (e.g. 2, 3 K, 5MB, 10 G, 6GiB, 8
+ *   bytes, 9mbytes).

+ *    - SI (powers of 1000), e.g. 7.4kB, 8 MB, 42.9 megabytes, 9 G, 1terabyte.
+ *    - IEC (powers of 1024), e.g. 9 Kibibytes, 0.3 MiB, 40gibibytes.

+ *    - SI (powers of 1000) 7.4 kB, 8 MB, 42.9 MB, 9 GB, 1 TB.
+ *    - IEC (powers of 1024) 9 KiB, 0.3 MiB, 40 GiB.

+      // bigger than 1 kB so we'll grab the last one.

+  // Create a ~1kB sized struct.

For me it's RTBC

Go with this, 17 months reviewing without substantial changes it's sufficient time.

The argumentation for this change is a bit poor, most people find it very confusing?

I am fine with following the standard that better matches file storage. But we don't really allow changes in, with this argumentation because do we know - what standard people expect here?

This issue fixes Drupal to not lie about how many bytes there are in a kilobyte. 1 kilogram of cheese is also 1000 grams so I'd expect that normal (cheese eating) users expect. Some will expect the lie, others just don't care.

I'm putting it back as RBTC since it is in fact reviewed by the community ;)

I'm going to mark this back to needs review, the patch itself has had lots, but the implications definitely need more discussion.

@catch: Dries already said he was up for the 1000 bytes in a kilobyte back in 2008: http://drupal.org/node/151902#comment-858570

@MrHaroldA: what people said in an issue four years ago isn't the same as discussing it now.

I think getting a patch in for IEC units would be a whole lot easier than changing the calculated values. If there's any code out there relying on this, we're only making string changes, rather than API changes. My 2 cents.

@catch: How can we get an updated opinion from Dries on this?

Either IEC/SI unit by default. It's patch will bring a new problem to Windows Users.

For example,

Files must be less than 500 KB.

Select a file and check the size in windows explorer. It shows 500KB. and then upload it.. Oh ouch. Error message popup.

Add a note to recommend site builders use IEC unit would make the site friendly for most users.

Same goes for real KB/kB/KiB users: select a 500kB file and upload it. It then shows as 488.2kB in Drupal so the user may assume that something went wrong in the upload.

As long as there are different implementation of the "same" unit, there will be confusion so it's probably best to settle on 1 definition of a kB. Future-wise that definitely must be 1kB = 1000 Bytes (SI) as Drupal 8 will be around for quite a while.

@jbrown I also got confused with the standards! ;) Thanks for pointing out: all KB's replaced with kB's!

@jbrown: already fixed!

Need ... more ... coffee ...

#53: handle-si-iec-units.patch queued for re-testing.

Still rooting for this to be committed to D8 ;)

#65: handle-si-iec-units.patch queued for re-testing.

We still need a decision about using SI or IEC.

Currently we use IEC numbers with SI units. We only need to use IEC numbers/units or SI numbers/units for consistency.

Nobody can choose between SI or IEC for default formatting? 2 years for a simple decision is a lot of time O.o

I think this is a good example of code bloat. Adding hundreds of extra lines of code that has to be maintained, parsed and run on every page load on every Drupal sites, all for the sake of some a minor unit difference that makes no difference what so ever to the user.

Worse, I think the changes around the PHP memory size are actively harmful:

-const DRUPAL_MINIMUM_PHP_MEMORY_LIMIT = '32M';
+const DRUPAL_MINIMUM_PHP_MEMORY_LIMIT = '32 MiB';

"32M" is the value you need to set in php.ini. If you try to input "32 MiB", you will get a weird error.

And what's the great benefit of showing '31.5344424 MiB' (or whatever) on the user status page, rather than a simple '32M'?

Isn't there something more important to work on for Drupal 8?

Isn't there something more important to work on for Drupal 8?

Not being able to tell how much MiB or MB a file is, is an utter faillure for a serious framework like Drupal. Postponing this to D9 in 2015/2016 seems like a great loss for a relatively straight forward bugfix.

But I agree, there are other importent issues too ;)

I don't think this is a bugfix at all. The old way (where MB means 1024^2 bytes) is still the way most people are used to seeing it. Like it or not, but Windows is still the largest desktop OS out there, and what the huge majority of our end users are using. And be honest, when did you last use the word gibibyte in a sentence?

This change adds a whole lot of code weight, accomplishes nothing useful, and it violates the principle of least astonishment.

So if a populair operating system has the same bug, this isn't a bug?

People clearly need to be educated, not be kept stupid for thinking 1 kilo = 1024 grams.

adds a whole lot of code weight

This adds 40 lines of code outside of tests. Forty lines.

accomplishes nothing useful

Usefulness is in the eye of the beholder.

Forty lines.

…are forty too many.

This prefix mess is a mess, sure, but it's not our mess to fix, so the principle of least astonishment dictates that we do what the end users expect, namely the old way, even if it's not techically a correct use do SI prefixes.

It's still a bug. You can't mix IEC numbers with SI units. That's not an opinion, that's a fact.

Even if Apple isn't dominating the computer OS landscape entirely yet, they're dominating mobile, and they seem to be dominating in attracting web developers. It seems odd to me that there'd be any discussion of not following the lead of Apple so that at least the web developers, who I've interpreted as mostly using Apple products these days, can have a confirmation on the website of what they see in Finder. Otherwise, if you can't reliably use it as a confirmation of what file you've likely uploaded, that defeats half the point of showing the size in the first place.

The @params in the parse_byte_count() and format_byte_count() docblocks should have type hints I think, e.g. @param int|string $count, @param bool $force_iec.

Related discussion from a while back #267883: parse_size() and format_size() do not use same kilo standard but using proper units seems like the thing to do.

+++ b/core/includes/common.inc
@@ -1009,9 +1009,8 @@ function parse_byte_count($count, $force_iec = FALSE) {
-  // Make sure an integer is returned. Not cast to an integer type as this would
-  // limit the maximum to 2 GiB on 32-bit platforms.
-  return floor($quantity);
+  // Make sure an integer is returned.
+  return (int) floor($quantity);

Is this an intended change, considering the documentation?

#87 didn't apply for me anymore. File module stuff got moved around two days ago.

I re-rolled it and removed the (int) declaration noted in #88.

Okay, let's just stop for a second, and review how these units are currently used across the industry:

Windows

Consistently binary (the old way)

OS X/iOS

Inconsistent. Decimal used for hard disk space in the Finder (in accordance with SI), binary used for RAM (in accordance with JEDEC). Binary used almost anywhere else (ls -lah, for example).

Linux/FreeBSD

Inconsistent, depending on which graphical shell (if any) you're using. CLI tools still using binary.

I don't own any Android devices, so I haven't been able to test those.

Bottom line

I don't there's a really good solution here.

• If we swich to using SI prefixes (ie. convert the numbers), we will get weird numbers in the UI (max upload size, 5.242 MB).

  Additionally, we would be doing the wrong thing for memory sizes (ie. RAM limits), since RAM measurement follows the JEDEC standard, which specifies that RAM should be measured binarily.

• If we swich to using IEC prefixes (ie.
  mebibytes and gibibytes), we will introduce a unit for disk size measurements that our users have likely never seen before, thereby confusing them.

In summary, I think we should leave this alone. It's a nasty mess, and there's no solution that's correct everywhere.

I think it probably makes sense to have the memory limits set in php.ini match the output on the Drupal side. A max upload of 10M in php.ini ought show as 10M in Drupal, lest create confusion; same for memory_limit. I don't think displaying "32M" for memory_limit from php.ini, showing as "32M" on the status page, means we can't show SI prefixes for metadata on file uploads, though. That would seem to be a similar conclusion to what Apple already implemented.

#93: Don't try to skirt the issue here. There is no correct here. We can choose between following SI, IEC or JEDEC rules. I outlined the problems in #91.

The patch already renders the PHP memory limit as IEC, so there is no problem there.

No, that's actually incorrect. It should use JEDEC, ie. classic binary units.

We need to use the modern SI standard so that there isn't 1024 bytes in a kilobyte

We don't actually need to change anything. The current state of things works fine, and I've never heard of any problems.

Additonally, SI isn't particularly modern. ~200 years old. Classic appeal to authority fallacy.

The reason we need to use IEC for the PHP memory limit is because this value always has a base of 1024 and we need to distinguish it from SI.

So instead of just showing the actual value from the php.ini configuration, you want to use obscure prefixes that are not used on any major computing platforms.

I understand that you'd like Drupal to be more technically correct, but the goal of the Drupal project is not to educate end users on scientific measurement standards, and as such I don't think the added costs of code complexity and user confusion are justified.

Not complying with measurement standards is not necessarily a bug, especially not in the case when such standards are not already in widespread use.

The SI standards are already in widespread use.

Bit Rate: http://en.wikipedia.org/wiki/Bit_rate

Linux Kernel: http://man7.org/linux/man-pages/man7/units.7.html

Apple: http://support.apple.com/kb/TS2419

Ubuntu: https://wiki.ubuntu.com/UnitsPolicy

The SI standards are already in widespread use.

First of all, SI is not a standard, it is a unit system. Secondly, I don't think anyone's arguing that SI itself is not widely used.

However, as I outlined in #91, the majority of computing platforms still use binary units to some extent, and Windows still use binary units exclusively.

And the IEC kibibyte/mebibytes are not used on any system a normal Drupal end-user would have been exposed to.

Add bold.

The related issue, #451404: Incorrect abbreviations in format_size, was discussed in #bugsmash, 3 months ago and we (darvanen, lendude, borisson_, catch and myself) all forgot to update the issue. I am doing so now.

The comment by miki in #91 provides good reason to leave this alone. The people in the meeting agreed as well. That makes this issue a won't fix. However, instead maybe this issue should be used to update documentation to explain what we are doing.