MathGroup Archive 2000

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: RE:Working Precision

  • To: mathgroup at
  • Subject: [mg24030] Re: [mg23928] RE:Working Precision
  • From: Richard Fateman <fateman at>
  • Date: Tue, 20 Jun 2000 03:07:42 -0400 (EDT)
  • Organization: University of California, Berkeley
  • References: <8ihv28$>
  • Sender: owner-wri-mathgroup at

for those who are not tired of this thread....

David Withoff wrote:
> > Richard Fateman showed how arbitrary precision arithmetic can
> > produce strange results.  I have some more strange behavior.
> These examples are really just different ways of disguising some
> basic issues in numerical analysis, most of which are not specific
> to Mathematica.  All of these behaviors are entirely reasonable.
> Any system of arithmetic can be made to show behaviors that might
> be surprising to some people, and machine arithmetic can be a lot
> more amusing in this regard than Mathematica arithmetic.  16-digit
> machine arithmetic, for example, routinely must make up digits to
> fill in the required 16 digits, a fact which can lead to all sorts
> of mathematically indefensible effects.

Of course this depends on who is doing the defending! Most people
find it upsetting to learn that a brief computation produces a
number x such that x==x+1, and worse.

> To understand the behavior of this example it is useful to recall
> that inexact numbers correspond to intervals.  With 3-digit decimal
> arithmetic, for example, the number 1.23 is the best available
> 3-digit decimal representation for any number between 1.225 and
> 1.235, so mathematically this number represents the interval between
> those points.  For any inexact number, the numerical "roundoff
> error" of the number gives the width of the corresponding interval.

This recollection is of something called "significance arithmetic"
which is not widely used, and has the counter-intuitive results 
exhibited. That's why it isn't used.

Another view is that the number 1.23 is a particular exact number.
On a decimal machine it is 1.23000000000000000000000.... or
perhaps 123/100.
On a binary machine, one must in general commit a rounding error
to the nearest binary number. But then it is exact.

Here's an easily defended rule:  Do all arithmetic exactly. When
you must store the result into a finite sized memory location, 
round it to the nearest representable number exactly representable
in that memory location. In case of a tie, round to even.

> The Do[x=2*x-x, {100}] example generates a value of x that corresponds
> to an interval around 1 and with a width of approximately 2.  This
> interval arises through application of standard propagation of error
> in which the error in the result is estimated by accumulating upper
> bounds for the error in each step, with all errors assumed to be
> uncorrelated.  In this example, which repeatedly adds and subtracts
> the same number, all of the errors are perfectly correlated, so this
> calculated upper bound to the error is significantly larger than the
> actual error. 

On a binary machine with a binary exponent, multiplication by 2 is
always exact.  Think about it:  it adds one to the exponent, leaving
the mantissa alone.  There is no error on such a machine operation.
Consider the subtraction.
The first time 2*x-x is computed, perhaps 1 bit is lost.  The next
99 times, the answer is unchanged.

> Most practical calculations aren't like this.  In
> most practical calculations, the assumption of uncorrelated errors
> is a good assumption, or at least it is better than any realistic
> alternative assumption.

I don't know how to define practical calculations as a class, but 
there are very uncomfortable consequences of the default behavior
of Mathematica, the system.
  A serious system could be constructed that actually computed
error bounds with some rigor, without massive overestimation, at
least for some important classes of problems.

I guess we can agree to disagree on whether Mathematica has
the best approach possible in the context of numeric or
symbolic/numeric systems.

 Richard Fateman

> Although Mathematica has perhaps given wider exposure to the
> topic of reliable arithmetic, this has been an active area of
> research for decades, and there is extensive literature on the
> subject.  If you are interested in this subject you might try
> exploring the literature in a good technical library.  There are
> very few questions about the behavior of this aspect of Mathematica
> that can't be answered by a review of The Mathematica Book, study
> of the above mentioned Numerics Report, and a few minutes of
> careful thought.
> > --------------------
> > Regards,
> > Ted Ersek
> Dave Withoff
> Wolfram Research

  • Prev by Date: RE: Re: Functional Expression Meaning (was:A Functional Expression Trick)
  • Next by Date: Win 98 compatibility With Ver. 2.2.2
  • Previous by thread: Re: RE:Working Precision
  • Next by thread: Re: RE:Working Precision