MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: A strange precision behavior?

  • To: mathgroup at smc.vnet.net
  • Subject: [mg90401] Re: A strange precision behavior?
  • From: Jean-Marc Gulliet <jeanmarc.gulliet at gmail.com>
  • Date: Tue, 8 Jul 2008 02:27:56 -0400 (EDT)
  • Organization: The Open University, Milton Keynes, UK
  • References: <g4smba$2g$1@smc.vnet.net>

Aaron Fude wrote:

> The following two commands result in different values:
> 
> SetPrecision[0.01, 50]
> 0.01`50
> 
> Is that because in the first case, 0.01 is first represented as a
> double and then passed to SetPrecision, while in the second case,
> Mathematica gets a chance to interprete 0.01 as 1/100?

Correct.

Note that in the first case, first a machine-precision number is 
created, then the precision of this number is set (converted) to 50. In 
the second case, a number is created to a precision of fifty out of the 
box, i.e. no conversion from lower to higher precision.

In[1]:= SetPrecision[0.01, 50]
x = 0.01;
RealDigits[x]
Precision[x]
$MachinePrecision
SetPrecision[x, 50]
y = 0.01`50
RealDigits[y]
Precision[y]
SetPrecision[y, 50]

Out[1]= 0.010000000000000000208166817117216851329430937767029

Out[3]= {{1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, -1}

Out[4]= MachinePrecision

Out[5]= 15.9546

Out[6]= 0.010000000000000000208166817117216851329430937767029

Out[7]= 0.010000000000000000000000000000000000000000000000000

Out[8]= {{1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 0}, -1}

Out[9]= 50.

Out[10]= 0.010000000000000000000000000000000000000000000000000

Regards,
-- Jean-Marc



  • Prev by Date: Re: Confused about precision vs accuracy
  • Next by Date: Re: distribution fitting
  • Previous by thread: Re: A strange precision behavior?
  • Next by thread: Re: A strange precision behavior?