Re: strange rounding result
- To: mathgroup at smc.vnet.net
- Subject: [mg80954] Re: [mg80918] strange rounding result
- From: Carl Woll <carlw at wolfram.com>
- Date: Thu, 6 Sep 2007 05:27:58 -0400 (EDT)
- References: <200709050659.CAA27251@smc.vnet.net>
Chris Chiasson wrote:
>Round[9995,10]
>Round[999.5,1]
>Round[99.95,1/10]
>Round[9.995,1/100]
>Round[.9995,1/1000]
>
>The results of these commands are decreasing powers of ten, except the
>second to last one, which gives 999/100 on my machine. I assume it has
>something to do with the storage of floating point numbers in binary
>rather than decimal form, but I don't know. Anyway, it just seemed
>strange to me and I wanted to point it out.
>
>Maybe it has something to do with the "round to even" behavior? Meh.
>
>
>
I think the following explains it:
In[144]:= N[
FromDigits[#, 2] & /@
RealDigits[{999.5, 99.95, 9.995, .9995}, 2], 20]
Out[144]= {999.50000000000000000, 99.950000000000002842,
9.9949999999999992184, 0.99950000000000005507}
Carl Woll
Wolfram Research
- References:
- strange rounding result
- From: "Chris Chiasson" <chris@chiasson.name>
- strange rounding result