Re: A strange precision behavior?

• To: mathgroup at smc.vnet.net
• Subject: [mg90433] Re: A strange precision behavior?
• From: Szabolcs Horvát <szhorvat at gmail.com>
• Date: Wed, 9 Jul 2008 04:54:10 -0400 (EDT)
• References: <g4smba\$2g\$1@smc.vnet.net>

```Aaron Fude wrote:
> The following two commands result in different values:
>
> SetPrecision[0.01, 50]
> 0.01`50
>
> Is that because in the first case, 0.01 is first represented as a
> double and then passed to SetPrecision, while in the second case,
> Mathematica gets a chance to interprete 0.01 as 1/100?
>

You already got several answers to this, so I'd just like to add a
little illustration.  First, note that Mathematica, like most computer
programs, represents floating point numbers in binary.  Some numbers
that are exactly representable in decimal are not exactly representable
in binary.  Think about calculating 1/7 in decimal ... you get

0.142857142857142857

where 14287 is repeated infinitely many times.  No matter how many
digits you calculate, you never get an exact number ... those digits
just keep repeating infinitely.  Similarly, 0.01 is not exactly
representable in binary.

When you write 0.01, it is represented with 53 digits in binary on most
computers (actually only 52 digits need to be stored because the first
one is always 1).  You can see the digits with RealDigits:

In[1]:= RealDigits[0.01, 2]

Out[1]= {{1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0,
1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1,
0, 0, 0, 1, 1, 1, 1, 0, 1, 1}, -6}

Here the sequence 10100011110101110000 should be repeated infinitely,
but of course it is repeated only as many times as can be fit in 53 bits.

If you now increase the precision to 50 decimal digits, Mathematica just
adds some zeros at the end ...

In[2]:= RealDigits[SetPrecision[0.01, 50], 2]

Out[2]= {{1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0,
1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1,
0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, -6}

But if you tell Mathematica to represent 0.01 with enough binary digits
so that it will be precise to 50 decimal digits, that sequence will be
repeated for longer:

In[3]:= RealDigits[0.01`50, 2]

Out[3]= {{1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0,
1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1,
0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0,
0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1,
1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1,
0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0,
1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1,
1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1}, -6}

I hope that this explanation was not too long winded and redundant ...
sorry about that.

```

• Prev by Date: Re: Display[ ] renewed?
• Next by Date: Re: PlotRange Trouble
• Previous by thread: Re: A strange precision behavior?
• Next by thread: Confused about precision vs accuracy