[Date Index] [Thread Index] [Author Index]

Precision & InputForm perils

• To: mathgroup at smc.vnet.net
• Subject: [mg13629] Precision & InputForm perils
• From: Ersek_Ted%PAX1A at mr.nawcad.navy.mil
• Date: Fri, 7 Aug 1998 03:08:30 -0400
• Sender: owner-wri-mathgroup at wolfram.com

I was surprised to find the output from InputForm[x]  in the line below 
doesn't show the exact value used for (x).

In[1]:=
x=N[Pi];
InputForm[x]

Out[1]//InputForm=
3.141592653589793


If you want InputForm to show all the digits used you have to evaluate 
($NumberMarks=True) as in the line below.

In[2]:=
$NumberMarks=True;
InputForm[x]

Out[2]//InputForm=
3.14159265358979311`


By default Precision returns an integer (to be consistent with earlier 
versions of Mathematica).   In the line below we are told (x) has
sixteen  digits of precision.  Well it turns out the precision (on my
computer) is a  bit less than sixteen digits.

In[3]:=
Precision[x]

Out[3]=
16


If you want Precision to give you a better idea what the precision is
you  have to evaluate the trap door (SetPrecision[Round, False]) as in
the line  below.

In[4]:=
SetPrecision[Round,False];
Precision[x]

Out[4]=
15.9546


In the line above Precision gave us a better idea how precise (x) is,
but  that isn't exactly right either.  You can get a better answer if
you look at  the InputForm of Precision[x] as below.

In[5]:=
Precision[x]//InputForm

Out[5]//InputForm=
15.9545897701910028`


Actually the precision above is also an approximation.  My computer
stores  machine numbers with a 53 bit mantissa, so the precision of (x)
is exactly (  Log[10,  2^53] ).  In the line below we see the precision
indicated by  Precision[x] is very close to the exact value.

In[6]:=
Precision[x] - Log[10, 2^53]

Out[6]=
1.77636`*^-15


Actually it's hardly ever important to know what the precision is to
many  decimal places.  That's because each binary bit corresponds to
about 0.30103  decimal digits.  I just thought I would point out that
what you normally see  isn't precisely the truth.

Ted Ersek