Troubles with inequality

[Bob Sneidar]

So here’s what I came up with. Pretty simple. 

on mouseUp
   put 5.226 into myVar
   set the numberFormat to "0.00"
   add .001 to myVar
   put myvar & cr into myString
   set the numberFormat to "0.000"
   put myvar after myString
   put myString
end mouseUp

I get 
5.23
5.227

Clearly the value in the variable is not being affected at all. This makes me feel a lot better. The moral of this is, always set the numberFormat appropriately prior to calculations of any kind, and don’t use the contents of fields in calculations. 

Bob S

> On Feb 4, 2015, at 15:20 , J. Landman Gay <jacque at hyperactivesw.com> wrote:
> 
> It's obscure, yes, though the dictionary has this:
> 
> "Important! Changing the numberFormat does not automatically change the format of a number that's already in a container. It affects numbers only when they are calculated and then displayed or used as strings. Otherwise, the number retains its full numeric precision."
> 
> The "already in a container" should be more precise though and specify "in a variable". Fields are containers, but they only hold strings.
> 
> 
> On 2/4/2015 5:02 PM, Graham Samuel wrote:
>> Thanks Jacque, a gold mine of information as usual. But it’s pretty obscure, isn’t it? All this started for me because I wanted to test if two numbers were equal, knowing that they probably weren’t exactly equal to the last decimal place but nevertheless were ‘engineering equal’ as it were - say to six places of decimals. So I tried to truncate them to that length and then compare them. More fool me. But now I think I know what should be done, thanks to you - and to everyone else who replied.
>> 
>> Graham
>> 
>> 
>>> On 4 Feb 2015, at 23:44, J. Landman Gay <jacque at hyperactivesw.com> wrote:
>>> 
>>> On 2/4/2015 3:01 PM, Graham Samuel wrote:
>>>> So really, if I want an uncomplicated string of characters derived
>>>> from a number via setting the numberFormat and then doing a
>>>> calculation, how do I get it?
>>> 
>>> You can turn it into a string by putting empty after it:
>>> 
>>>  put 1.55555 into tVar -- number
>>>  set the numberformat to "0.00"
>>>  add 1 to tVar -- still a number
>>>  put tVar into fld 1 -- numberformat applied now, field contains a string "2.55"
>>>  add 1 to tVar -- still a number, contains 3.55555
>>>  put empty after tVar -- numberformat applies, tVar is a string
>>> 
>>> --
>>> Jacqueline Landman Gay         |     jacque at hyperactivesw.com
>>> HyperActive Software           |     http://www.hyperactivesw.com
>>> 
>>> _______________________________________________
>>> use-livecode mailing list
>>> use-livecode at lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
> 
> 
> -- 
> Jacqueline Landman Gay         |     jacque at hyperactivesw.com
> HyperActive Software           |     http://www.hyperactivesw.com