MathGroup Archive 2007

[Date Index] [Thread Index] [Author Index]

Search the Archive

strange rounding result

  • To: mathgroup at smc.vnet.net
  • Subject: [mg80918] strange rounding result
  • From: "Chris Chiasson" <chris at chiasson.name>
  • Date: Wed, 5 Sep 2007 02:59:41 -0400 (EDT)

Round[9995,10]
Round[999.5,1]
Round[99.95,1/10]
Round[9.995,1/100]
Round[.9995,1/1000]

The results of these commands are decreasing powers of ten, except the
second to last one, which gives 999/100 on my machine. I assume it has
something to do with the storage of floating point numbers in binary
rather than decimal form, but I don't know. Anyway, it just seemed
strange to me and I wanted to point it out.

Maybe it has something to do with the "round to even" behavior? Meh.

-- 
http://chris.chiasson.name/


  • Prev by Date: Re: Unevaluated hypergeometric functions
  • Next by Date: Re: Fwd: correction to: more than 1.5 hrs of trying: Grid Alignment problem ?
  • Previous by thread: Re: How to obtain numberical value from an InverseFunction?
  • Next by thread: Re: strange rounding result