[Date Index] [Thread Index] [Author Index]

Re: strange rounding result

• To: mathgroup at smc.vnet.net
• Subject: [mg80960] Re: [mg80918] strange rounding result
• From: Curtis Osterhoudt <cfo at lanl.gov>
• Date: Thu, 6 Sep 2007 05:31:02 -0400 (EDT)
• Organization: LANL
• References: <200709050659.CAA27251@smc.vnet.net>
• Reply-to: cfo at lanl.gov

Strange to me (although consistent with the documentation for Round, which 
states that "At midpoints, Round rounds towards even integers"). This is 
consistent with the "round-to-even" method (which I just learned about on 
Wikipedia); the new "standard". 

   Perhaps my education, which stressed "bankers' rounding" needs to be 
updated!

On Wednesday 05 September 2007 00:59:41 Chris Chiasson wrote:
> Round[9995,10]
> Round[999.5,1]
> Round[99.95,1/10]
> Round[9.995,1/100]
> Round[.9995,1/1000]
>
> The results of these commands are decreasing powers of ten, except the
> second to last one, which gives 999/100 on my machine. I assume it has
> something to do with the storage of floating point numbers in binary
> rather than decimal form, but I don't know. Anyway, it just seemed
> strange to me and I wanted to point it out.
>
> Maybe it has something to do with the "round to even" behavior? Meh.



-- 
==========================================================
Curtis Osterhoudt
cfo at remove_this.lanl.and_this.gov
PGP Key ID: 0x4DCA2A10
Please avoid sending me Word or PowerPoint attachments
See http://www.gnu.org/philosophy/no-word-attachments.html
==========================================================

• References:
  • strange rounding result
    • From: "Chris Chiasson" <chris@chiasson.name>