Re: Why Mathematica does not issue a warning when the calculations
- To: mathgroup at smc.vnet.net
- Subject: [mg117646] Re: Why Mathematica does not issue a warning when the calculations
- From: Daniel Lichtblau <danl at wolfram.com>
- Date: Tue, 29 Mar 2011 06:52:57 -0500 (EST)
----- Original Message -----
> From: "Richard Fateman" <fateman at eecs.berkeley.edu>
> To: "Daniel Lichtblau" <danl at wolfram.com>
> Cc: "Richard Fateman" <fateman at cs.berkeley.edu>, mathgroup at smc.vnet.net
> Sent: Thursday, March 24, 2011 3:54:13 PM
> Subject: Re: [mg117570] Re: Why Mathematica does not issue a warning when the calculations
> On 3/24/2011 12:38 PM, Daniel Lichtblau wrote:
> > ----- Original Message -----
> >> From: "Richard Fateman"<fateman at eecs.berkeley.edu>
> >> To: "Daniel Lichtblau"<danl at wolfram.com>
> >> Cc: "Richard Fateman"<fateman at cs.berkeley.edu>,
> >> mathgroup at smc.vnet.net
> >> Sent: Wednesday, March 23, 2011 10:44:11 PM
> >> Subject: Re: [mg117570] Re: Why Mathematica does not issue a
> >> warning when the calculations
> >> Perhaps
> >>
> >> Table[FractionalPart[(6 + Sqrt[SetPrecision[2, n]])^20], {n, 15,
> >> 25}]
> >> // Column
> >>
> >> is a better illustration of the software-precision inheritance
> >> capability
> >> of Mathematica. It is like running the same computation but with
> >> numbers
> >> 2.000000
> >> 2.0000000
> >> 2.00000000
> >> etc, and seeing what happens.
> >>
> >> the fact that 2.0000==2.000000000 is True (etc) means that a==b and
> >> both
> >> a and b are numbers, does not mean that f[a]==f[b]. I view this as
> >> problematical,
> >> even if most people don't ever even notice it.
> > I think this same "non-homomorphicity" (or whatever) phenomenon will
> > show up, in one form or another, with any finite precision
> > semantics.
> >
> I guess I don't know what you mean by "finite precision semantics"
> unless you mean that "equality" doesn't
> mean equal, but merely "similar".
I mean whether one is using fixed pecision or significance arithmetic.
> We've discussed this before. If you
> have a true function f, that
> is a procedure whose results depend only on its argument(s), then when
> A
> is equal to B, f(A) should be
> equal to f(B). This is fundamental to programming and to mathematics,
> but is violated by Mathematica.
Probably by other mathematical programming languages as well. Especially in the operating world of finite precision arithmetic. It's something people often get used to (of necessity, if you work on this stuff).
> >> DL says...
> >>> Mathematica documentation is fairly clear about distinctions
> >>> between
> >>> machine vs. (Mathematica's implementation of) software arithmetic.
> >>> The
> >>> claims to which you refer are made for the latter and not the
> >>> former.
> >>> While I've not looked too hard of late, I think this is made
> >>> reasonably clear in the Documentation Center.
> >> the loop above does not involve machine numbers, but it does
> >> suggest a
> >> failure to increase precision automatically
> >> in the context of FractionalPart.
> > You want maybe Mathematica should play poker?
> Actually, I don't expect FractionalPart to win in this circumstance in
> Mathematica.
> > ("I see your 10 digits, and raise you five more"). To badly misquote
> > Einstein, "Mathematica does not play poker". (Original quote,
> > subject to spelling errors: "Gott werfel nicht").
> >
> > In slightly more detail: the precision would be degraded before
> > FractionalPart ever sees its argument. So your example is not
> > well-founded in a mathematical sense: it would require invention of
> > unknown digits.
> Mathematically speaking, I'm not inventing unknown digits. I am saying
> that all digits not
> specified are zero. This is the common understanding, that "2" is
> shorthand for
> 2.00000.....0000.....
>
> This is not the understanding in Mathematica, where (for software
> bigfloats)
> 2.000000000000000000 is more like shorthand for
> "all numbers between
> 1.9999999999999999995
> 2.0000000000000000005
> "
> (DL may be willing and able to explain what Mathematica REALLY does)
Something along the lines you suggest. Your alternative approach is fine for handling of integers. It does not work so well for inputs such as 1.3 which do not translate exactly to finite binary representations. But Mathematica semantics would still be violated by upping precision, and that in no way contradicts the documentation remarks about ratcheting precision internally, as they refer to handling of exact input (N[2,10] et al are not exact, and they are what FractionalPart sees).
> > That is not to say such semantics are always wrong. But generally
> > it is a bad idea to add unknown zero bits (what SetPrecision would
> > do) absent a compelling reason to do so.
> This is the place where DL and I disagree.
>
> ... snip
> > I'm missing your point here. My claim sort of implied Mathematica is
> > alone in use of significance arithmetic.
> Yes, it's not because no one else thought about it. It is because
> others who
> promoted it were either convinced that it didn't work so well, or
> perhaps
> were hooted out of the numerical analysis / programming languages
> community. Papers and implementations go back to 1964, but stop by
> about 1977.
I think there may be some since then. No matter. We find it works well for what we need. Your view, roughly, is that we picked it up off the garbage heap of numerics literature. Perhaps so. In all the conferences I've attended, I've only once heard it disparaged (not derided, but disparaged) as a bad method. More below. Anyway, it may be helpful to bear in mind that (unlike these endless threads), this usage is not academic to us. We rely on it for those other things, like putting kids through school. Maybe the folks who shied away from it in the literature had similar considerations? That would make for a humerous rift between theory and practice...
> I thought you were claiming that Mathematica had the
> lion's share of bigfloat usage.
No. Though I suspect it does, once you get past the 200 digit mark or so. This is to exclude the slightly high precision scientific computational apps, and those possibly ubiquitous cryptography apps, which will dominate any usage of Mathematica. (Caveat: with emergence of Wolfram|Alpha, this could change. Though I doubt W|A gets tremendous high precision usage at this time).
> > It is fairly obvious that nobody knows the extent to which bignum
> > arithmetic is used, either within or outside of Mathematica. My
> > claim is that those who use it in Mathematica seem to get along with
> > it.
> Except for the ones who post those annoying questions here:)
I'm not sure your plural is justified. Certainly the original post in this thread was about a limitation of machine arithmetic functionality. One that, I might add, is readily overcome with Mathematica's bignum arithmetic semantics.
> Anyway, assuming they use Mathematica, they do not have a choice of
> another bigfloat
> implementation.
Those who require fixed precision seem able to learn how to use that in Mathematica.
> I agree that people who avoid Mathematica probably do so for other
> reasons, and
> as I said earlier.
>
>
> > I will surmise that the user base for other such libraries is not
> > huge in comparison to Mathematica (might be larger, but not by an
> > order of magnitude), provided one discounts what amounts to software
> > emulation of low precision bignums e.g. quad precision.
> >
> > I will also surmise that the number of people who use other
> > libraries to avoid Mathematica significance arithmetic is probably
> > small by comparison to the number who avoid Mathematica for
> > geopolitical rather than scientific reasons (obviously I do not
> > really mean "geopolitical" but it seemed like an appropriate term
> > for the occasion). By using "avoid" I intentionally exclude those
> > who do not use Mathematica for more passive reasons, such as
> > preference for, or easier access to, competing programs.
> >
> > Summarizing the above, I might simply say words to the effect that
> > "Mathematica seems to be holding its own". Wait, those are the words
> > I said earlier.
> >
> >
> >> From a marketing perspective, it may be quite unimportant how
> >> Mathematica does bigfloats,
> >> or how it does intervals.
> > Maybe not quite right. We also use significance arithmetic, and it
> > has enabled some of the technology that perhaps is useful for
> > marketing.
>
> Maybe or maybe not.
> >
> >> If you did not have significance arithmetic, what then could you do
> >> if
> >> you wanted to have
> >> some reliable error estimates.
> > Absolutely nothing. We'd simply not have some of the symbolic and
> > numeric technology it has enabled.
> I think you are wrong, as I suggested 2 methods.
> > Why else do you think it is that, outside of Mathematica, one finds
> > no serious (i.e. production quality, externally usable)
> > implementation of numeric Groebner bases or CAD based on approximate
> > arithmetic?
> Perhaps WRI hired the people who were interested in doing this?
More a matter of seeing and seizing an opportunity. We were already here.
> >> You could use intervals instead of
> >> numbers. But you
> >> would have to tinker somewhat with any non-trivial program to get
> >> it
> >> to work
> >> with intervals.
> >>
> >> Or...
> >>
> >> .... Here's a heuristic to see if your answer is numerically
> >> correct.
> >> Do the computation in precision 20.
> >> Do the computation in precision 40.
> >> If the answers agree to (say) 18 places, believe it.
> >> You could even do the computation in precision 80, and see if that
> >> agrees
> >> with the precision 40 result to (say) 38 places.
> >>
> >> This is not entirely foolproof.
> >>
> >> But it may be a good deal faster than doing it one precision (say
> >> 40)
> >> in
> >> significance arithmetic,
> >> at least if you are using state-of-the-art arbitrary precision in
> >> the
> >> first place.
> > (1) I don't think this would gain one a huge amount of speed. See
> > item (2).
> We could test this, but I'm not sure it could be done entirely within
> Mathematica. The libraries
> used by Mathematica (GMP, MPFR??) can be accessed other ways.
> > (2) This idea is in use in one branch of the numeric Groebner basis
> > literature. It is not so good for the absolutely vital step of zero
> > detection. So a finite modulus (a la Groebner trace algorithm) is
> > also used. Starts getting complicated (especially when the input was
> > approximate to begin with). And I do't think they have it to the
> > point of being generally usable, though I might be not quite up to
> > date on this. Also, if memory serves, this methodology has not shown
> > itself to be remarkably fast.
> My impression is that the actual primary use of Grobner basis
> computations is to
> write papers on Grobner basis computations, and the other "uses" are
> "in principle" "demonstrations". This may have changed with a more
> numerical
> approach, however.
Your impression is off insofar as industrial usage goes. For example, we use them in NSolve, exact equation solving, some polynomial algebra, and probably a few places I am forgetting. Not an academic exercise, those applications. Also starting to make inroads in hybrid polynomial algebra computation e.g handling of low precision inputs. Not yet sure if that will fly, though.
> Are you claiming that now people are buying Mathematica to run GB
> computations?
> And are you claiming furthermore this could only be done in
> significance
> arithmetic?
Neither. And most definitely not the first.
I am claiming that this is extremely useful for the "bigger picture" technology it enables, a few parts of which I noted just above. Whether people buy Mathemntica for that stuff I would not know. Pretty sure it does not hurt though.
> > Curiously enough, the handling of approximate Groebner bases is an
> > area where it frequently odes make sense to invent digits when input
> > has low precision.
> >
> >
> >> ...
> >>> Not very many. One such person carries considerable weight in his
> >>> field (which is very much the appropriate term, as it happensy).
> >>> But
> >>> his field is something quite removed from computer arithmetic.
> >> (I think you mean "quite removed from computer algebra"? At least
> >> if
> >> we
> >> are thinking of
> >> the same heavyweight. And that still would not be very accurate.)
> >> RJF
> > I meant computer arithmetic, because that is closer to the topic of
> > this thread. But computer algebra would also apply. No idea whether
> > we have the same person in mind though.
> I was thinking of W. Kahan, who has been critical of Mathematica (in
> particular its arithmetic!)
> and was a principal architect of the IEEE 754 floating-point standard,
> and a key
> designer of the HP scientific and business calculators (with
> remarkably
> accurate
> algorithms). So I view him as being highly competent in computer
> arithmetic.
> Where he carries considerable weight...
>
> RJF
Wrong last name (and maybe wrong interpretation of a weak pun). I have quite high regard for Kahan in numerics, computer arityhmetic, etc. That said, I did one time tell him we had no intention of getting rid of significance arithmetic; he was the one to disparage that usage, when we discussed it in 2007.
There are some issues with IEEE arithmetic, by the way. That too is a work in progress (I'm sure this is not news to Kahan).
Daniel Lichtblau
Wolfram Research