Re: strange rounding result
- To: mathgroup at smc.vnet.net
- Subject: [mg80960] Re: [mg80918] strange rounding result
- From: Curtis Osterhoudt <cfo at lanl.gov>
- Date: Thu, 6 Sep 2007 05:31:02 -0400 (EDT)
- Organization: LANL
- References: <200709050659.CAA27251@smc.vnet.net>
- Reply-to: cfo at lanl.gov
Strange to me (although consistent with the documentation for Round, which states that "At midpoints, Round rounds towards even integers"). This is consistent with the "round-to-even" method (which I just learned about on Wikipedia); the new "standard". Perhaps my education, which stressed "bankers' rounding" needs to be updated! On Wednesday 05 September 2007 00:59:41 Chris Chiasson wrote: > Round[9995,10] > Round[999.5,1] > Round[99.95,1/10] > Round[9.995,1/100] > Round[.9995,1/1000] > > The results of these commands are decreasing powers of ten, except the > second to last one, which gives 999/100 on my machine. I assume it has > something to do with the storage of floating point numbers in binary > rather than decimal form, but I don't know. Anyway, it just seemed > strange to me and I wanted to point it out. > > Maybe it has something to do with the "round to even" behavior? Meh. -- ========================================================== Curtis Osterhoudt cfo at remove_this.lanl.and_this.gov PGP Key ID: 0x4DCA2A10 Please avoid sending me Word or PowerPoint attachments See http://www.gnu.org/philosophy/no-word-attachments.html ==========================================================
- References:
- strange rounding result
- From: "Chris Chiasson" <chris@chiasson.name>
- strange rounding result