Forum OpenACS Q&A: Tcl weird behaviour with math

Collapse
Posted by Antonio Pisano on
a user of our web ERP software running on OpenAcs had an issue with the rounding of an invoice's amount.

After some fiddling I found out this strange behaviour, which affects tcl up version 8.6 (tested on production server, tcl 8.4, and my PC, 8.5 and 8.6)

If you try to evaluate this expression:

expr 2241.57 * 100 - 224157

which should return a 0 (or 0.0) value, what tcl returns is instead:

2.9103830456733704e-11

In my situation the value of 2241.57 * 100 was fed to the ceil function, so the resulting number was bigger than we expected...

Is somebody else able to reproduce this behaviour?

Collapse
Posted by Benjamin Brink on
Antonio,

First, check the value of tcl_precision to verify it is zero. See www.tcl.tk/man/tcl8.6/TclCmd/tclvars.htm about half way down page.

Second, this looks like a standard binary math issue that affects most any computer language. See: http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems

When making an application that calculates currency, caution should be used to round off to the lowest denomination at appropriate places to avoid this kind of error. Commonly, a smallest value of 0.01 is assumed (cents in USD for example), but some currencies have a smallest unit of 0.05 for example.

Avoid coding that exacerbates binary errors in calculation, such as using ceiling before completing a rounding error. This is also true if using a currency value for a logical comparison with the value of 0, such as:

if { expr { 2241.57 *100 - 224157 } == 0 } { ... }

as there will be faulty logic. Round to the currency's lowest transferable unit first. Or, in the case of a logical comparison of zero, one can see if the error in comparison is less than the smallest transferable unit of currency; for example, see ec_same_value in ecommerce/tcl at end of http://cvs.openacs.org/browse/OpenACS/openacs-4/packages/ecommerce/tcl/ecommerce-money-computations-procs.tcl?hb=true

cheers,
Benjamin

Collapse
Posted by Antonio Pisano on
Thanks for your fast and clear explanation Benjamin, I will update my procs so they truncate all digits beyond required precision before being ceiled and this should do the trick.

All the best

Antonio

Collapse
Posted by Gustaf Neumann on
The best is to calculate internally the only the smallest unit (i.e. cents) in integers, calculate with these, and format these as required to dollar, euro, etc.

In the more general case, you might want to look into the tcl decimal arithmetic package: http://wiki.tcl.tk/28305

In some simple cases, it might be sufficient to work with a comparison operator with an espilon fuzz value (e.g. withinEpsilon is http://wiki.tcl.tk/879)

best regards
-gustaf neumann

Collapse
Posted by Brian Fenton on
Also, always brace your expressions! http://wiki.tcl.tk/10225