MathGroup Archive 2007

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: strange rounding result

  • To: mathgroup at smc.vnet.net
  • Subject: [mg80954] Re: [mg80918] strange rounding result
  • From: Carl Woll <carlw at wolfram.com>
  • Date: Thu, 6 Sep 2007 05:27:58 -0400 (EDT)
  • References: <200709050659.CAA27251@smc.vnet.net>

Chris Chiasson wrote:

>Round[9995,10]
>Round[999.5,1]
>Round[99.95,1/10]
>Round[9.995,1/100]
>Round[.9995,1/1000]
>
>The results of these commands are decreasing powers of ten, except the
>second to last one, which gives 999/100 on my machine. I assume it has
>something to do with the storage of floating point numbers in binary
>rather than decimal form, but I don't know. Anyway, it just seemed
>strange to me and I wanted to point it out.
>
>Maybe it has something to do with the "round to even" behavior? Meh.
>
>  
>
I think the following explains it:

In[144]:= N[
 FromDigits[#, 2] & /@
  RealDigits[{999.5, 99.95, 9.995, .9995}, 2], 20]

Out[144]= {999.50000000000000000, 99.950000000000002842, 
9.9949999999999992184, 0.99950000000000005507}

Carl Woll
Wolfram Research


  • Prev by Date: Labelled matrix plot
  • Next by Date: Re: How to compile this module?
  • Previous by thread: Re: strange rounding result
  • Next by thread: Re: strange rounding result