MathGroup Archive 2007

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: strange rounding result


On 9/5/07, Carl Woll <carlw at wolfram.com> wrote:
<snip/>
> I think the following explains it:
>
> In[144]:= N[
>  FromDigits[#, 2] & /@
>   RealDigits[{999.5, 99.95, 9.995, .9995}, 2], 20]
>
> Out[144]= {999.50000000000000000, 99.950000000000002842,
> 9.9949999999999992184, 0.99950000000000005507}


^^^
Woll FTW (again!)

Thanks!










(FTW is slang for, "For The Win")

-- 
http://chris.chiasson.name/


  • Prev by Date: Re: What is $MaxNumber on a 64 bit Computer?
  • Next by Date: Re: Grid
  • Previous by thread: strange rounding result
  • Next by thread: Re: strange rounding result