strange rounding result
- To: mathgroup at smc.vnet.net
- Subject: [mg80918] strange rounding result
- From: "Chris Chiasson" <chris at chiasson.name>
- Date: Wed, 5 Sep 2007 02:59:41 -0400 (EDT)
The results of these commands are decreasing powers of ten, except the
second to last one, which gives 999/100 on my machine. I assume it has
something to do with the storage of floating point numbers in binary
rather than decimal form, but I don't know. Anyway, it just seemed
strange to me and I wanted to point it out.
Maybe it has something to do with the "round to even" behavior? Meh.
Prev by Date:
Re: Unevaluated hypergeometric functions
Next by Date:
Re: Fwd: correction to: more than 1.5 hrs of trying: Grid Alignment problem ?
Previous by thread:
Re: How to obtain numberical value from an InverseFunction?
Next by thread:
Re: strange rounding result