[Date Index] [Thread Index] [Author Index]

When Is Precision[ ] $MachinePrecision, And When Is It Not?

• To: mathgroup at smc.vnet.net
• Subject: [mg47440] When Is Precision[ ] $MachinePrecision, And When Is It Not?
• From: Harold.Noffke at wpafb.af.mil (Harold Noffke)
• Date: Mon, 12 Apr 2004 03:44:54 -0400 (EDT)
• Sender: owner-wri-mathgroup at wolfram.com

$Version "5.0 for Microsoft Windows [2000] (November 18, 2003)"

MathGroup:

Some follow-up to Peng Yu's 2004-04-09 post on numerical precision.

On my machine, I get ...

	In[1]:= $MachinePrecision
	Out[1]= 15.9546

	In[2]:= NumberForm[2./3.]
	Out[2]//NumberForm= 0.666667

	In[3]:= NumberForm[2./3., $MachinePrecision]
	NumberForm::iprf:
	   Formatting specification 15.9546
	     should be a positive integer or a pair of positive integers.
	Out[3]//NumberForm= 0.666667

So, it seems obvious I can't display numbers to $MachinePrecision
exactly, because $MachinePrecion is not an integer.  However ...

	In[2]:= NumberForm[2./3., 15]
	Out[2]//NumberForm= 0.666666666666667

	In[3]:= NumberForm[2./3., 15] // Precision
	Out[3]= MachinePrecision

	In[4]:= NumberForm[2./3., 16]
	Out[4]//NumberForm= 0.6666666666666666

	In[5]:= NumberForm[2./3., 16] // Precision
	Out[5]= MachinePrecision
	
	In[6]:= NumberForm[2./3., 20] // Precision
	Out[6]= MachinePrecision
	
At this point, it looks like 15-digit numbers get rounded, as I
expect.  However, 16- and larger-digit numbers do not get rounded, and
Mathematica continues to print "MachinePrecision" instead of
"Arbitrary-Precision".  What I expect here is for rounding to
continue, and for Mathematica to print "Arbitrary-Precision" instead
of "MachinePrecision" when the number of digits specified become 16 or
greater.

Clearly I am missing one of Mathematica's syntax subtilties.  Can
someone please clarify this?

Thanks.
Harold