Re: RE:Working Precision
- To: mathgroup at smc.vnet.net
- Subject: [mg24030] Re: [mg23928] RE:Working Precision
- From: Richard Fateman <fateman at cs.berkeley.edu>
- Date: Tue, 20 Jun 2000 03:07:42 -0400 (EDT)
- Organization: University of California, Berkeley
- References: <8ihv28$led@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
for those who are not tired of this thread....
David Withoff wrote:
>
> > Richard Fateman showed how arbitrary precision arithmetic can
> > produce strange results. I have some more strange behavior.
>
> These examples are really just different ways of disguising some
> basic issues in numerical analysis, most of which are not specific
> to Mathematica. All of these behaviors are entirely reasonable.
> Any system of arithmetic can be made to show behaviors that might
> be surprising to some people, and machine arithmetic can be a lot
> more amusing in this regard than Mathematica arithmetic. 16-digit
> machine arithmetic, for example, routinely must make up digits to
> fill in the required 16 digits, a fact which can lead to all sorts
> of mathematically indefensible effects.
Of course this depends on who is doing the defending! Most people
find it upsetting to learn that a brief computation produces a
number x such that x==x+1, and worse.
><snip>
>
> To understand the behavior of this example it is useful to recall
> that inexact numbers correspond to intervals. With 3-digit decimal
> arithmetic, for example, the number 1.23 is the best available
> 3-digit decimal representation for any number between 1.225 and
> 1.235, so mathematically this number represents the interval between
> those points. For any inexact number, the numerical "roundoff
> error" of the number gives the width of the corresponding interval.
This recollection is of something called "significance arithmetic"
which is not widely used, and has the counter-intuitive results
exhibited. That's why it isn't used.
Another view is that the number 1.23 is a particular exact number.
On a decimal machine it is 1.23000000000000000000000.... or
perhaps 123/100.
On a binary machine, one must in general commit a rounding error
to the nearest binary number. But then it is exact.
Here's an easily defended rule: Do all arithmetic exactly. When
you must store the result into a finite sized memory location,
round it to the nearest representable number exactly representable
in that memory location. In case of a tie, round to even.
>
> The Do[x=2*x-x, {100}] example generates a value of x that corresponds
> to an interval around 1 and with a width of approximately 2. This
> interval arises through application of standard propagation of error
> in which the error in the result is estimated by accumulating upper
> bounds for the error in each step, with all errors assumed to be
> uncorrelated. In this example, which repeatedly adds and subtracts
> the same number, all of the errors are perfectly correlated, so this
> calculated upper bound to the error is significantly larger than the
> actual error.
On a binary machine with a binary exponent, multiplication by 2 is
always exact. Think about it: it adds one to the exponent, leaving
the mantissa alone. There is no error on such a machine operation.
Consider the subtraction.
The first time 2*x-x is computed, perhaps 1 bit is lost. The next
99 times, the answer is unchanged.
> Most practical calculations aren't like this. In
> most practical calculations, the assumption of uncorrelated errors
> is a good assumption, or at least it is better than any realistic
> alternative assumption.
>
I don't know how to define practical calculations as a class, but
there are very uncomfortable consequences of the default behavior
of Mathematica, the system.
A serious system could be constructed that actually computed
error bounds with some rigor, without massive overestimation, at
least for some important classes of problems.
I guess we can agree to disagree on whether Mathematica has
the best approach possible in the context of numeric or
symbolic/numeric systems.
Richard Fateman
<snip>
......
>
> Although Mathematica has perhaps given wider exposure to the
> topic of reliable arithmetic, this has been an active area of
> research for decades, and there is extensive literature on the
> subject. If you are interested in this subject you might try
> exploring the literature in a good technical library. There are
> very few questions about the behavior of this aspect of Mathematica
> that can't be answered by a review of The Mathematica Book, study
> of the above mentioned Numerics Report, and a few minutes of
> careful thought.
>
> > --------------------
> > Regards,
> > Ted Ersek
>
> Dave Withoff
> Wolfram Research