Re: A strange precision behavior?

• To: mathgroup at smc.vnet.net
• Subject: [mg90401] Re: A strange precision behavior?
• From: Jean-Marc Gulliet <jeanmarc.gulliet at gmail.com>
• Date: Tue, 8 Jul 2008 02:27:56 -0400 (EDT)
• Organization: The Open University, Milton Keynes, UK
• References: <g4smba\$2g\$1@smc.vnet.net>

```Aaron Fude wrote:

> The following two commands result in different values:
>
> SetPrecision[0.01, 50]
> 0.01`50
>
> Is that because in the first case, 0.01 is first represented as a
> double and then passed to SetPrecision, while in the second case,
> Mathematica gets a chance to interprete 0.01 as 1/100?

Correct.

Note that in the first case, first a machine-precision number is
created, then the precision of this number is set (converted) to 50. In
the second case, a number is created to a precision of fifty out of the
box, i.e. no conversion from lower to higher precision.

In[1]:= SetPrecision[0.01, 50]
x = 0.01;
RealDigits[x]
Precision[x]
\$MachinePrecision
SetPrecision[x, 50]
y = 0.01`50
RealDigits[y]
Precision[y]
SetPrecision[y, 50]

Out[1]= 0.010000000000000000208166817117216851329430937767029

Out[3]= {{1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, -1}

Out[4]= MachinePrecision

Out[5]= 15.9546

Out[6]= 0.010000000000000000208166817117216851329430937767029

Out[7]= 0.010000000000000000000000000000000000000000000000000

Out[8]= {{1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0}, -1}

Out[9]= 50.

Out[10]= 0.010000000000000000000000000000000000000000000000000

Regards,
-- Jean-Marc

```

• Prev by Date: Re: Confused about precision vs accuracy
• Next by Date: Re: distribution fitting
• Previous by thread: Re: A strange precision behavior?
• Next by thread: Re: A strange precision behavior?