Re: RE:Working Precision

*To*: mathgroup at smc.vnet.net*Subject*: [mg24030] Re: [mg23928] RE:Working Precision*From*: Richard Fateman <fateman at cs.berkeley.edu>*Date*: Tue, 20 Jun 2000 03:07:42 -0400 (EDT)*Organization*: University of California, Berkeley*References*: <8ihv28$led@smc.vnet.net>*Sender*: owner-wri-mathgroup at wolfram.com

for those who are not tired of this thread.... David Withoff wrote: > > > Richard Fateman showed how arbitrary precision arithmetic can > > produce strange results. I have some more strange behavior. > > These examples are really just different ways of disguising some > basic issues in numerical analysis, most of which are not specific > to Mathematica. All of these behaviors are entirely reasonable. > Any system of arithmetic can be made to show behaviors that might > be surprising to some people, and machine arithmetic can be a lot > more amusing in this regard than Mathematica arithmetic. 16-digit > machine arithmetic, for example, routinely must make up digits to > fill in the required 16 digits, a fact which can lead to all sorts > of mathematically indefensible effects. Of course this depends on who is doing the defending! Most people find it upsetting to learn that a brief computation produces a number x such that x==x+1, and worse. ><snip> > > To understand the behavior of this example it is useful to recall > that inexact numbers correspond to intervals. With 3-digit decimal > arithmetic, for example, the number 1.23 is the best available > 3-digit decimal representation for any number between 1.225 and > 1.235, so mathematically this number represents the interval between > those points. For any inexact number, the numerical "roundoff > error" of the number gives the width of the corresponding interval. This recollection is of something called "significance arithmetic" which is not widely used, and has the counter-intuitive results exhibited. That's why it isn't used. Another view is that the number 1.23 is a particular exact number. On a decimal machine it is 1.23000000000000000000000.... or perhaps 123/100. On a binary machine, one must in general commit a rounding error to the nearest binary number. But then it is exact. Here's an easily defended rule: Do all arithmetic exactly. When you must store the result into a finite sized memory location, round it to the nearest representable number exactly representable in that memory location. In case of a tie, round to even. > > The Do[x=2*x-x, {100}] example generates a value of x that corresponds > to an interval around 1 and with a width of approximately 2. This > interval arises through application of standard propagation of error > in which the error in the result is estimated by accumulating upper > bounds for the error in each step, with all errors assumed to be > uncorrelated. In this example, which repeatedly adds and subtracts > the same number, all of the errors are perfectly correlated, so this > calculated upper bound to the error is significantly larger than the > actual error. On a binary machine with a binary exponent, multiplication by 2 is always exact. Think about it: it adds one to the exponent, leaving the mantissa alone. There is no error on such a machine operation. Consider the subtraction. The first time 2*x-x is computed, perhaps 1 bit is lost. The next 99 times, the answer is unchanged. > Most practical calculations aren't like this. In > most practical calculations, the assumption of uncorrelated errors > is a good assumption, or at least it is better than any realistic > alternative assumption. > I don't know how to define practical calculations as a class, but there are very uncomfortable consequences of the default behavior of Mathematica, the system. A serious system could be constructed that actually computed error bounds with some rigor, without massive overestimation, at least for some important classes of problems. I guess we can agree to disagree on whether Mathematica has the best approach possible in the context of numeric or symbolic/numeric systems. Richard Fateman <snip> ...... > > Although Mathematica has perhaps given wider exposure to the > topic of reliable arithmetic, this has been an active area of > research for decades, and there is extensive literature on the > subject. If you are interested in this subject you might try > exploring the literature in a good technical library. There are > very few questions about the behavior of this aspect of Mathematica > that can't be answered by a review of The Mathematica Book, study > of the above mentioned Numerics Report, and a few minutes of > careful thought. > > > -------------------- > > Regards, > > Ted Ersek > > Dave Withoff > Wolfram Research