strange decimal precision error on floating numeric fields [#662692]

I am not sure if this was ever mentioned, tried to search the issue list and found #480938: decimal precision but it's not quite the same.

I am using uc_views to generate sales report out of an ubercart site and eventually using a field handler that goes down to using the render() code of the numeric handler (initialized as float) with no precision set (or everything set as default).

The number that should be displayed is the following: 36.00100
but I was seeing: 36.000999999999998

So I went down to debugging the field handlers and I found the following in class views_handler_field_numeric, in the render() function:

    $value = $values->{$this->field_alias};
    if (!empty($this->options['set_precision'])) {
      $value = number_format($value, $this->options['precision'], $this->options['decimal'], $this->options['separator']);
    }
    else {
      $remainder = abs($value) - intval(abs($value));
      $value = $value > 0 ? floor($value) : ceil($value);
      $value = number_format($value, 0, '', $this->options['separator']);
      if ($remainder) {
        // The substr may not be locale safe.
        $value .= $this->options['decimal'] . substr($remainder, 2);
      }
    }

In particular the following line, which has nothing wrong, does give the wrong result:

      $remainder = abs($value) - intval(abs($value));

If you do a plain 36.001 - 36 in any PHP test code you actually get 0.000999999999998.

I then went to look at the PHP bugs and found the following: http://bugs.php.net/bug.php?id=41357

So there's no bug in PHP and nothing wrong with that line, so I wonder, couldn't what that code does be implemented differently so we don't run into this kind of trouble?

Note: If you do set precision on the actual field handler, number_format() function takes care of the rounding and the problem goes away, but It would be good to use a direct number from a database without any function and have it displayed just fine.

I hope I was clear.

a.=

Comment	File	Size	Author
#6	662692.views_.numeric-float-decimal-rounding.patch	996 bytes	joachim
#6	7.x-3.x: PHP 5.3 & MySQL 5.5, D7 217 pass PHP 5.6 & MySQL 5.5, D7 217 pass PHP 7.1 & MySQL 5.5, D7 222 pass

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

dawehner

German

CreditAttribution: dawehner commented 24 January 2010 at 13:39

Status:

Active

» Closed (won't fix)

[10 May 2007 6:40pm UTC] johannes@php.net

Floating point values have a limited precision. Hence a value might 
not have the same string representation after any processing. That also
includes writing a floating point value in your script and directly 
printing it without any mathematical operations.

If you would like to know more about "floats" and what IEEE
754 is read this:
http://docs.sun.com/source/806-3568/ncg_goldberg.html
 
Thank you for your interest in PHP.

Thats by design of floats. You have to store it at decimals.

Comment #2

hanoii

Spanish

🇦🇷UTC-3

CreditAttribution: hanoii commented 24 January 2010 at 16:22

Yes, I read the PHP bug and I totally get it's something ineherent to the float library.

And if by storing at decimals you mean mysql DECIMAL type, they are stored like that, if you meant otherwise, I am not sure I understood.

On another note, I am thinking if there should be on a numeric field an option to print the value just as it comes from the db, w/o any formatting, either this as a default with all the extra settings to allow all the current formatting, or the formatting to default as it is now and some checkbox to get the raw value. If interested I can try to provide a patch.

Not really necessary as I can set the round option myself.

a.=

Comment #3

essbee CreditAttribution: essbee commented 29 March 2011 at 02:06

Version:

6.x-2.8

» 7.x-3.x-dev

In the new Views 3 for D7 the calculated fields seem to suffer from this. Are these processed as float? Is there anyway to address this?

Comment #4

ShaneOnABike CreditAttribution: ShaneOnABike commented 22 August 2011 at 17:10

Couldn't we implement some functions to deal with this ... such as (taken from http://www.php.net/manual/en/language.types.float.php#89842)

function floatcmp($f1,$f2,$precision = 10) // are 2 floats equal
{
    $e = pow(10,$precision);
    $i1 = intval($f1 * $e);
    $i2 = intval($f2 * $e);
    return ($i1 == $i2);
}

function floatgtr($big,$small,$precision = 10) // is one float bigger than another
{
    $e = pow(10,$precision);
    $ibig = intval($big * $e);
    $ismall = intval($small * $e);
    return ($ibig > $ismall);
}

function floatgtre($big,$small,$precision = 10) // is on float bigger or equal to another
{
    $e = pow(10,$precision);
    $ibig = intval($big * $e);
    $ismall = intval($small * $e);
    return ($ibig >= $ismall);
}

I was actually think that all we'd need to do prior to doing comparisons on floats would be to just convert them into an integer with a high precision and then do whatever comparison you want on them?

Comment #5

joachim CreditAttribution: joachim commented 5 January 2015 at 14:47

Category:	Support request	» Bug report
Status:	Closed (won't fix)	» Active

With the numbers I've seen this with, the error arises in this line:

      $remainder = abs($value) - intval(abs($value));

AFAICT this is there to strip the decimal portion out of a number, hence 46.3 give us 0.3. The decimal part is tacked back on later, using the decimal marker that's set in the options.

What about extracting the decimal part with string manipulation instead? That wouldn't be affect by PHP's handling of floats.

Comment #6

joachim CreditAttribution: joachim commented 6 January 2015 at 17:11

Status:

Active

» Needs review

File	Size
662692.views_.numeric-float-decimal-rounding.patch	996 bytes
7.x-3.x: PHP 5.3 & MySQL 5.5, D7 217 pass PHP 5.6 & MySQL 5.5, D7 217 pass PHP 7.1 & MySQL 5.5, D7 222 pass

Here's a patch that extracts the decimal portion using string functions.

It's working fine for me on various kinds of values.

Comment #7

5t4rdu5t CreditAttribution: 5t4rdu5t commented 16 October 2015 at 00:55

@joachim your patch works wonderfully. Thank you!!

Comment #8

lunazoid CreditAttribution: lunazoid as a volunteer commented 11 December 2015 at 02:37

Status:

Needs review

» Reviewed & tested by the community

Confirming that #6 solved the problem for me.

Comment #9

colan

he/him

English

Toronto 🇨🇦

CreditAttribution: colan at Colan Schwartz Consulting for Chapter Three commented 14 January 2016 at 19:36

We've recently switched our testing from the old qa.drupal.org to DrupalCI. Because of a bug in the new system, #2623840: Views (D7) patches not being tested, older patches must be re-uploaded. On re-uploading the patch, please set the status to "Needs Review" so that the test bot will add it to its queue.

If all tests pass, change the Status back to "Reviewed & tested by the community". We'll most likely commit the patch immediately without having to go through another round of peer review.

We apologize for the trouble, and appreciate your patience.

Comment #10

colan

he/him

English

Toronto 🇨🇦

CreditAttribution: colan at Colan Schwartz Consulting for Chapter Three commented 21 January 2016 at 15:34

Status:

Reviewed & tested by the community

» Needs work

Comment #11

NWOM CreditAttribution: NWOM commented 11 July 2017 at 15:20

Status:

Needs work

» Reviewed & tested by the community

This still applies cleanly. Thank you!

Comment #12

NWOM CreditAttribution: NWOM commented 11 July 2017 at 15:28

Status:

Reviewed & tested by the community

» Needs work

Ah my mistake, it looks like the last patch needs to be re-uploaded for the testing bot (as explained in #9).

Comment #13

MustangGB CreditAttribution: MustangGB commented 17 July 2017 at 14:16

Status:

Needs work

» Reviewed & tested by the community

Tests can now be added manually after a patch has been uploaded, NWOM has done this already, so no need to re-upload.

Comment #14

22 April 2019 at 14:20

DamienMcKenna committed 8d86ce2 on 7.x-3.x authored by joachim

Issue #662692 by joachim, hanoii, ShaneOnABike, lunazoid, 5t4rdu5t:...

Comment #15

DamienMcKenna

NH, USA

CreditAttribution: DamienMcKenna at Mediacurrent commented 22 April 2019 at 14:20

Status:	Reviewed & tested by the community	» Fixed
Parent issue:		» #2960871: Plan for Views 7.x-3.23 release

Committed. Thanks.

Comment #16

6 May 2019 at 14:24

Status:

Fixed

» Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Comment #17

amorsent CreditAttribution: amorsent commented 11 September 2020 at 16:37

This fix can leave unnecessary trailing zeros.
For example - Ubercart stores prices as decimal 16,5 - eg: "28.00800". Since this fix deals with the decimal as a string, it ends up keeping the extra trailing zeros: "28.00800" instead "28.008".

Note: Views in core did a different fix for a similar issue that might be worth backporting instead.
#1952926 - NumericField.php does not support negative value rendering in range -0.xx.

Commit: https://git.drupalcode.org/project/drupal/-/commit/852558772d131e11c6080...