       Problems with numerical precision

• To: mathgroup at smc.vnet.net
• Subject: [mg13221] Problems with numerical precision
• From: Joel Cannon <cannon at alpha.centenary.edu>
• Date: Mon, 13 Jul 1998 07:43:10 -0400
• Sender: owner-wri-mathgroup at wolfram.com

```I was surprised by the following behavior. I have a large, compicated 39
term exact expression which I will paste at the bottom of this message
(actually it is one element of a matrix that needs to be evaluated).
The terms vary in magnitude from approx. 10^(21) to 10^(7). The sum of
the terms is ~.1 so (in retrospect) the answer would be unreliable
until precision was at least greater than 22 digits. What surprised me
was that when I innocently used standard precision when converting to a
number, mathematica gave an erroneous answer but indicated that this
answer had normal precision. As soon as I used a precision sufficient
to obtain a reliable number, mathematica returned a reasonable value
for precision.

(* I innocently evaluate the expression with 16 bit precision and
allegedly get a 16 digit answer *)

In:=
N[exact]

Out=
-387994.

In:=
Precision[%]

Out=
16

(* Now I evaluate by mapping N to all levels of the expression with 16
and 32 digit  precision.  An erroneous answer is returned with 16 which
indicates full precision and the answer I believe to be correct is
returned with 32 digit precison mapped to all levels, with an
appropriately reduced precision. *)

MapAll[N[#,16]&,exact]

Out=
-256921.5135808811

In:=
Precision[%]

Out=
16

(I

MapAll[N[#,32]&,exact]

Out=
-0.13585158

In:=
Precision[%]

Out=
8

(* Finally, here is a table which shows the precision which is mapped to
all levels, the number that is returned and the precision of the number
indicated by mathematica. At a precision above 24, the answers appear
to become reliable. Sorry for the mess. I couldn't figure out how to
paste from the notebook. *)

Table[ {j,a=MapAll[N[#,j]&,exact],Precision[a]},{j,16,48,4}]
//ColumnForm

Precision::"mnprec":
"Value \!\(-4\) would be inconsistent with \$MinPrecision;
bounding by \

Out=
\!\(\*
InterpretationBox[GridBox[{
{
RowBox[{"{",
RowBox[{"16", ",",
RowBox[{"-",
StyleBox["256921.513580881066`",
StyleBoxAutoDelete->True,
PrintPrecision->16]}], ",", "16"}], "}"}]},
{\({20, \(-0.12362454`-3.9052\), 0}\)},
{\({24, \(-0.135849898159`0.1358\), 0}\)},
{\({28, \(-0.1358515670090712`4.1358\), 4}\)},
{\({32, \(-0.13585157663154997479`8.1358\), 8}\)},
{\({36, \(-0.135851576631518434076851`12.1358\), 12}\)},
{\({40, \(-0.1358515766315184402124864022`16.1358\), 16}\)},
{\({44, \(-0.13585157663151844021244701292576`20.1358\),
20}\)},
{\({48, \(-0.135851576631518440212447012352254477`24.1358\),
24}\)}
},
GridBaseline->{Baseline, {1, 1}},
ColumnAlignments->{Left}],
ColumnForm[ {{16, -256921.51358088106, 16}, {20, 0.*^3, 0}, {24,
0.*^-1,
0}, {28, -0.1359, 4}, {32, -0.13585158, 8}, {36,
-0.135851576632, 12}, {
40, -0.1358515766315184, 16}, {44, -0.13585157663151844021,
20}, {
48, -0.135851576631518440212447, 24}}],
Editable->False]\)

Can someone explain what is happening here, and if there is a way I can
avoid being harmed by this behavior other than simply maintaining
vigilence (never a bad idea)?

I am leaving on holidays so, if possible, copy any posts to me at
cannon at centenary.edu.

Thanks,

Here is the exact expression:

exact  = -98932079226263562173399040*Sqrt[60697/(158260343003171*Pi)] +
(167631926666382868224147456*Sqrt[106981/(89790972595727*Pi)])/5 +
26878975514241750849945600*Sqrt[268801/(35736206484587*Pi)] -
(183643178376764530781847552*Sqrt[290377/(33080884640531*Pi)])/7 +
(4165863466523953112285184*Sqrt[392863/(24451088647349*Pi)])/11 +
3058948678399888982016*Sqrt[1038407/(9250638756541*Pi)] +
(1654985324340900265984*Sqrt[1905803/(5040357287329*Pi)])/35 +
(849492869112715601248256*Sqrt[3774901/(2544683433887*Pi)])/5 -
(203986427460336461836976128*Sqrt[6678671/(1438299332197*Pi)])/185 -
(68003308870639942631424*Sqrt[9104173/(1055112643319*Pi)])/5 +
(2920584358349766656*Sqrt[13340621/(720051041047*Pi)])/45 +
(175270642332778551574528*Sqrt[13499291/(711587596657*Pi)])/55 +
(1268747540898097659904000*Sqrt[15389981/(624167634727*Pi)])/3 +
280763897614182480355328*Sqrt[16396861/(585839450567*Pi)] -
28800062220197167104*Sqrt[22342499/(429939732313*Pi)] -
97720275491918237743104*Sqrt[33359131/(287954984177*Pi)] +
(41966530783971421192192*Sqrt[154770941/(62065449607*Pi)])/21 -
(357808048101469978624*Sqrt[1606415629/(5979727703*Pi)])/45 -
(633057454176539218880*Sqrt[18380881181/(522604327*Pi)])/9 +
9676270103070556320*Sqrt[74937438661/(128185967*Pi)] +
(47810693140721427456*Sqrt[312474980077/(30741431*Pi)])/7 +
(2330163324805222383*Sqrt[628373454919/(15286973*Pi)])/44 -
295629902583470424*Sqrt[1273936457237/(7540351*Pi)] -
(52622736437562021*Sqrt[3860008365931/(2488577*Pi)])/16 +
(914195162323708076*Sqrt[8917555200659/(1077193*Pi)])/45 -
(132856119246638149*Sqrt[18222830192651/(527137*Pi)])/30 -
(610534248521959563*Sqrt[24451088647349/(392863*Pi)])/10240 -
(108242785354494755*Sqrt[27327687311743/(351509*Pi)])/32768 +
(20044562352647247*Sqrt[35736206484587/(268801*Pi)])/160 -
(51225637784351357*Sqrt[99463930741931/(96577*Pi)])/20971520 -
(33196959658197917*Sqrt[847158306664033/(11339*Pi)])/3932160 +
(68764704878583*Sqrt[2287670407064413/(4199*Pi)])/134217728 +
(1136338473977591*Sqrt[3509655841893851/(2737*Pi)])/2621440 +
(5988802116986951*Sqrt[13472549844689299/(713*Pi)])/184320 -
(156103631329335*Sqrt[38890396920095021/(247*Pi)])/738197504 -
(5520209566513*Sqrt[43465737734223847/(221*Pi)])/1610612736 -
(134206564095*Sqrt[105559648783115057/(91*Pi)])/8589934592 +
(24339742449*Sqrt[738917541481805399/(13*Pi)])/1073741824 +
(1437739131061*Sqrt[9605928039263470187/Pi])/105553116266496

------------------------------------------------------------------------------
Joel W. Cannon                 |   (318)869-5160          Dept. of
Physics               |   (318)869-5026  FAX    Centenary College of
Louisiana |       P. O. Box 41188		       |
Shreveport, LA 71134-1188      |

```

• Prev by Date: Re: Integrate Bug?
• Next by Date: Win95 to Win98 issues
• Previous by thread: Re: help with trig(?) calculation
• Next by thread: Re: Problems with numerical precision