Precision and 'abs'

[Graham Samuel]

I’m having some trouble comparing numbers with lots of decimal places - numbers that ought to be equal are slightly unequal. In the LC Dictionary entry for ‘numberFormat’ there’s a warning about this, and a suggested solution:

> 
> Note: Since LiveCode does not use decimal numbers for its internal calculations (for reasons of speed), the decimal representation of a number is sometimes slightly off the correct number. For example, 10^-1 is equal to 0.1, but is calculated (to eighteen decimal places) as 0.100000000000000006. Because of this, setting the numberFormat to specify many decimal places after the decimal point may produce unexpected results in a statement that tests for an exact number. To prevent this, either avoid setting the numberFormat to a value more precise than you need, or use the abs function instead of the = operator to test equality:
> 
>   set the numberformat to ".##################"
>   put 10^-1 = 0.1 -- reports false because of the decimal error
>   put abs((10^-1) - 0.1) = zero -- reports true

Well, this sounds good, but looking up the definition of 'abs', there is no suggestion that taking the abs of a number will in any way alter its precision, so what does this advice mean? I have done a test where two numbers differ only after the tenth decimal place, but the test

  set the numberformat to "0.##########"
  put abs(aa-bb) = 0

returns false, so it didn't help.

Is this just a documentation bug, or is there something funny going on in the LC7.0.2-rc-2?

Graham