Re: Re: Re: error with Sum and Infinity
- To: mathgroup at smc.vnet.net
- Subject: [mg102465] Re: [mg102443] Re: [mg102437] Re: error with Sum and Infinity
- From: DrMajorBob <btreat1 at austin.rr.com>
- Date: Mon, 10 Aug 2009 04:15:17 -0400 (EDT)
- References: <h5bk64$hlm$1@smc.vnet.net> <200908070932.FAA15211@smc.vnet.net>
- Reply-to: drmajorbob at bigfoot.com
I myself often "poke at the corpse" to see what's wrong with my code. That
mostly works, and I'm not completely sure there's a better way.
Despite all my complaints about Mathematica documentation, I seriously
doubt that anybody else does it greatly better.
Bobby
On Sun, 09 Aug 2009 17:19:26 -0500, Richard Fateman
<fateman at cs.berkeley.edu> wrote:
> George Woodrow III wrote:
>> Also including a private response from Richard:
>>
>> [me]
>>> You can get the performance you want by using Piecewise[]. That way,
>>> there is only one function definition.
>>>
>> [Richard]
>> Yes, one can often get the predetermined right answer by asking a
>> subtlely different question. Still, I think you agree the answer to
>> the original question was wrong...
>>
>> [my comment]
>> The point to this is that yes, the original 'naive' attempt at a sum
>> gave the wrong answer. The wrong answer should have prompted a journey
>> to find out why.
>
> Advising the user to read the manual is so common that there is a
> heavily-used acronym for it (RTFM). But in this case the manual probably
> does not provide enough guidance about what Sum does.
>
>>
>> The reason why the first attempt gave the wrong answer is that the
>> question was ill-posed for Mathematica to "always" do the right thing.
>
> You and I know that Mathematica didn't do the right thing. But there was
> no way for a user to determine if the problem was ill-posed in terms of
> Mathematica. Indeed, I can think of a technique that would allow
> Mathematica to solve such problems, and I have no reason to believe, a
> priori, that Mathematica does not have that technique. Furthermore,
> Mathematica may have such a technique in some future version.
> In particular, if Mathematica does not apparently intend to distinguish
> between a mathematical function composed of Sin Cos Pi + - * and a
> user-defined pattern-driven computer subroutine f[x_]:=If[ ...], then
> it should work regardless of how the expression is conveyed. The problem
> is WELL FORMED!
>
>> My solution (using Piecewise) and another poster's solution using
>> DiracDelta posed the question in a way that when Mathematica switched
>> from brute force (below SymbolicSumThreshold) to symbolic, it would do
>> the right thing.
>
> Another solution would be to do the problem in your head, and write down
> the answer, since it was obvious. This is making excuses for
> Mathematica, don't you think?
> <snip.. example>
>
>> The point of this is to show that you need to know some maths, and at
>> least a little bit about the conventions of Mathematica. An infinite
>> sum cannot be treated the same as a finite sum, and care needs to be
>> taken to be sure that Mathematica knows what the sequence really is.
>
> But in addition to knowing some maths, you also need to know things
> about Mathematica that it had no business expecting you to know.
> That's what I object to here.
>
>>
>> In the case of a sequence of all 0, except for a single 1, then the
>> function that defines the sequence needs to make the exception clear
>> in the definition.
>>
>> Could Mathematica be designed to always give the right answer, not
>> matter what the input? Perhaps. Most of the time, it would be a waste
>> of time.
>
> Should you make an excuse for Mathematica getting the wrong answer, even
> if it could, with some slight amount of additional cleverness get the
> right answer? No, I think you should make a fuss. Maybe just a slight
> amount of fuss, but some fuss. To show that you are paying attention. I
> make a fuss because I know that the program can be written properly.
> You think it would be a waste of time to write the program properly.
> That's a fundamental disagreement between us. I elaborate further below.
>
>>
>> The proper lesson to learn is that if you are going to be evaluating
>> infinite sums, then it would be worth finding out how Mathematica does
>> this and what needs to be done to make the code fool-proof.
>
> No, I think the lesson is that Mathematica documentation does not
> describe what it does with functions that are not "closed form" in
> infinite summations or even "large summations with SumThresholdWhatever
> terms", and that there is nothing you can do about such undocumented
> "features" except try to program around them (and in my case, also
> complain). Note that the default values or even the existence of
> SystemOptions can change without notice from version to version, and so
> you cannot possibly "find out" about such things all the time. It is
> simply a bad idea for Mathematica to behave this way, and for you to
> make the excuse that the user of Mathematica is responsible for making
> his code fool-proof in the light of this design is to place an enormous
> and largely unnecessary burden on someone who has paid good money for
> this program.
>
> I believe that Stephen Wolfram's perspective on this is that no one
> should believe the result of a computer calculation without independent
> verification, and therefore it is acceptable to take shortcuts which
> work most of the time. But not always. This is apparently OK for (some)
> physicists. It is generally unacceptable to computer scientists who do
> research in this area. It should be unacceptable to mathematicians,
> except perhaps those who were trained by physicists.
>
>
>>
>> In this case, one could learn that Mathematica will sometimes ignore
>> exceptional cases (separate definitions of the function at special
>> values) when computing sums, and if you want these cases to be
>> included in an infinite sum, you need to define the function so that
>> these cases are handled symbolically.
>
> You could learn it, but mostly by treating Mathematica, the computer
> program, as though it were a dead frog, and you were a curious child.
> You wonder if a frog, which lives in the water, breathes with gills or
> lungs. So you dissect it. The advantage of a computer program over a
> corpse is that you can get any number of fresh corpses to dissect, and
> so you can come to learn all kinds of things. Unfortunately, some of
> the things you learn may be incorrect generalizations from specific
> tests.
>
> I would prefer that Mathematica behave in a predictable, documented, and
> mathematically consistent way, and that I not be required to poke it and
> check that my programs still work when I install a new version, or when
> I run the program with different input parameters that might, unknown to
> me, exceed some threshold that I never heard of.
>
>>
>> You can be upset that mathematica does not do exactly what you expect
>> all the time, but it does no good to rant about it.
>
> It depends on your expectation being reasonable or not, don't you think?
>
>> The fact is, there are well documented ways to deal with this problem
>> so that mathematica *does* do the right thing, not just in this toy
>> example, but in more complex problems.
>
> Although I have hardly touched the vast array of contributions from
> users, I suspect that your claim is not true for this case or for some
> of the more complex problems. Where is it documented (much less "well
> documented") that sums above 1000000 [or whatever] are different from
> below that?
>
>> Every language has its way of interpreting code. Mathematica is
>> special in that there are a lot more ways to get the code 'almost'
>> right and usually a lot of different ways to get the code exactly
>> right. A similar coding error in c would simply give a compiler error
>> or crash.
>
> This is true, but Mathematica has some pretense to doing mathematics,
> and you are blaming the user for making a programming error when there
> is nothing wrong with the program. The error is that Mathematica is not
> correctly interpreting the code, which is both legal, perfectly clear,
> and unambiguous, by taking a shortcut that is incorrect. Think about
> what you are saying. If a C programmer used a compiler and found that
> it sometimes took a correct program and ran it incorrectly, would he
> say, oh, I can write a different program that gets the right answer?
> This is why I view Mathematica, from a computer science/ programming
> language perspective as, how shall we say,... disappointing.
>
> So what is the issue?
>
> 1. WRI does not define what its way of interpreting code really is, and
> reserves the right to change it, anyway.
> 2. The Mathematica user is dependent on the good will of the
> implementation programmers and, to some extent, luck, for his program to
> work.
> 3. While guided by mathematics generally, the computer system
> programmers deliberately take shortcuts that violate 100% correctness.
>
> You are, of course, free to try to use Mathematica to solve problems,
> and it may work just fine most of the time. But as a computer scientist
> who is familiar with mathematics generally, and quite familiar with
> computer algebra systems, I find it unfortunate that you (and others)
> make excuses for Mathematica getting incorrect answers, even when other
> computer programs might get them right, (and WRI may be convinced to fix
> the issues too) thinking that somehow the USER is at fault.
>
>
>>
>> As for your gratuitous comment about Andrzej's post, I think that
>> anyone familiar with any type of computer algebra system would be able
>> to guess, just in terms of practicality, that there is a threshold
>> from brute force to symbolic summation.
>
> Why can't Mathematica figure out that the summand is defined in a
> peculiar way and has to be handled carefully? Why does the user have to
> think about whether the summation is done by brute force or by closed
> formula? What if the user is building a package and does not know what
> the user of his package has done to define a program that is,
> eventually, used as a summand? The package-builder must do the analysis
> that WRI should have done in the first place, or else that package may
> fail mysteriously. This means the package writer must violate a precept
> of the WRI way, which is that a Mathematica user need not be concerned
> about the details of algorithms. Maybe you THINK you know about how to
> do summation, but do you know how Mathematica does it? Does your
> knowledge extend to do (say) polynomial factorization? Are you expected
> to know which classes of polynomials are hard to factor and therefore
> you should not ask Mathematica to factor them?
>
>> Otherwise, Mathematica could never compute an infinite sum. I have
>> been using Mathematica for 20 years, and I did not know the name for
>> the threshold, let alone its value, but for my purposes, I did not
>> need to know. However, I did know (and it was in the documentation)
>> that there is a point where one type of computation ends and another
>> begins.
>>
>> The function SystemOptions[] is defined and documented in version 7
>> (It was new in version 6).
>>
>> If you execute SystemOptions["*"], you get an impressive list of all
>> the options and their current values. Most appear to be
>> self-explanatory, but a few look like more explanation is needed. It
>> also appears that if your code depends on the values for these
>> options, the code may be fragile at a minimum.
>
> This is true. In fact I may have used SystemOptions[] at some point in
> the distant past (1984?) but did not connect it to Sum since there is no
> indication in the documentation for Sum that it depends on this
> threshold option. So there is no way, other than poking at the corpse,
> for you to tell if your code is fragile.
>
>>
>> Note that the original problem had a resolution that did not involve
>> special knowledge of these variables.
>
> Sure, there are lots of resolutions, including, as I said, the excellent
> one of the user just figuring out the answer. Face it, Mathematica got
> the wrong answer to a problem that is entirely well-posed. Stop making
> excuses!
>
>> I'd say that knowing that these options exist belongs to advanced
>> users of Mathematica. It might be nice to have more complete
>> documentation for some of these options -- and Tech Support has
>> provided this information when I have needed it, but this knowledge
>> would not have solved the original question.
>
> That is because the original problem was well posed, and Mathematica
> should have solved it!. Can you not conceive of a programming solution
> to the original problem, or at least a solution that determined whether
> this was a case that might lead to an error? {It might require going
> slightly outside the usual Mathematica techniques -- I don't know
> offhand how to decompose a function definition to see how many rules it
> includes, and do a kind of meta-analysis of the patterns. }
>
>
> ...<snip>
>
> RJF
--
DrMajorBob at bigfoot.com
- References:
- Re: error with Sum and Infinity
- From: Richard Fateman <fateman@cs.berkeley.edu>
- Re: error with Sum and Infinity