Re: Re: Re: error with Sum and Infinity

*To*: mathgroup at smc.vnet.net*Subject*: [mg102465] Re: [mg102443] Re: [mg102437] Re: error with Sum and Infinity*From*: DrMajorBob <btreat1 at austin.rr.com>*Date*: Mon, 10 Aug 2009 04:15:17 -0400 (EDT)*References*: <h5bk64$hlm$1@smc.vnet.net> <200908070932.FAA15211@smc.vnet.net>*Reply-to*: drmajorbob at bigfoot.com

I myself often "poke at the corpse" to see what's wrong with my code. That mostly works, and I'm not completely sure there's a better way. Despite all my complaints about Mathematica documentation, I seriously doubt that anybody else does it greatly better. Bobby On Sun, 09 Aug 2009 17:19:26 -0500, Richard Fateman <fateman at cs.berkeley.edu> wrote: > George Woodrow III wrote: >> Also including a private response from Richard: >> >> [me] >>> You can get the performance you want by using Piecewise[]. That way, >>> there is only one function definition. >>> >> [Richard] >> Yes, one can often get the predetermined right answer by asking a >> subtlely different question. Still, I think you agree the answer to >> the original question was wrong... >> >> [my comment] >> The point to this is that yes, the original 'naive' attempt at a sum >> gave the wrong answer. The wrong answer should have prompted a journey >> to find out why. > > Advising the user to read the manual is so common that there is a > heavily-used acronym for it (RTFM). But in this case the manual probably > does not provide enough guidance about what Sum does. > >> >> The reason why the first attempt gave the wrong answer is that the >> question was ill-posed for Mathematica to "always" do the right thing. > > You and I know that Mathematica didn't do the right thing. But there was > no way for a user to determine if the problem was ill-posed in terms of > Mathematica. Indeed, I can think of a technique that would allow > Mathematica to solve such problems, and I have no reason to believe, a > priori, that Mathematica does not have that technique. Furthermore, > Mathematica may have such a technique in some future version. > In particular, if Mathematica does not apparently intend to distinguish > between a mathematical function composed of Sin Cos Pi + - * and a > user-defined pattern-driven computer subroutine f[x_]:=If[ ...], then > it should work regardless of how the expression is conveyed. The problem > is WELL FORMED! > >> My solution (using Piecewise) and another poster's solution using >> DiracDelta posed the question in a way that when Mathematica switched >> from brute force (below SymbolicSumThreshold) to symbolic, it would do >> the right thing. > > Another solution would be to do the problem in your head, and write down > the answer, since it was obvious. This is making excuses for > Mathematica, don't you think? > <snip.. example> > >> The point of this is to show that you need to know some maths, and at >> least a little bit about the conventions of Mathematica. An infinite >> sum cannot be treated the same as a finite sum, and care needs to be >> taken to be sure that Mathematica knows what the sequence really is. > > But in addition to knowing some maths, you also need to know things > about Mathematica that it had no business expecting you to know. > That's what I object to here. > >> >> In the case of a sequence of all 0, except for a single 1, then the >> function that defines the sequence needs to make the exception clear >> in the definition. >> >> Could Mathematica be designed to always give the right answer, not >> matter what the input? Perhaps. Most of the time, it would be a waste >> of time. > > Should you make an excuse for Mathematica getting the wrong answer, even > if it could, with some slight amount of additional cleverness get the > right answer? No, I think you should make a fuss. Maybe just a slight > amount of fuss, but some fuss. To show that you are paying attention. I > make a fuss because I know that the program can be written properly. > You think it would be a waste of time to write the program properly. > That's a fundamental disagreement between us. I elaborate further below. > >> >> The proper lesson to learn is that if you are going to be evaluating >> infinite sums, then it would be worth finding out how Mathematica does >> this and what needs to be done to make the code fool-proof. > > No, I think the lesson is that Mathematica documentation does not > describe what it does with functions that are not "closed form" in > infinite summations or even "large summations with SumThresholdWhatever > terms", and that there is nothing you can do about such undocumented > "features" except try to program around them (and in my case, also > complain). Note that the default values or even the existence of > SystemOptions can change without notice from version to version, and so > you cannot possibly "find out" about such things all the time. It is > simply a bad idea for Mathematica to behave this way, and for you to > make the excuse that the user of Mathematica is responsible for making > his code fool-proof in the light of this design is to place an enormous > and largely unnecessary burden on someone who has paid good money for > this program. > > I believe that Stephen Wolfram's perspective on this is that no one > should believe the result of a computer calculation without independent > verification, and therefore it is acceptable to take shortcuts which > work most of the time. But not always. This is apparently OK for (some) > physicists. It is generally unacceptable to computer scientists who do > research in this area. It should be unacceptable to mathematicians, > except perhaps those who were trained by physicists. > > >> >> In this case, one could learn that Mathematica will sometimes ignore >> exceptional cases (separate definitions of the function at special >> values) when computing sums, and if you want these cases to be >> included in an infinite sum, you need to define the function so that >> these cases are handled symbolically. > > You could learn it, but mostly by treating Mathematica, the computer > program, as though it were a dead frog, and you were a curious child. > You wonder if a frog, which lives in the water, breathes with gills or > lungs. So you dissect it. The advantage of a computer program over a > corpse is that you can get any number of fresh corpses to dissect, and > so you can come to learn all kinds of things. Unfortunately, some of > the things you learn may be incorrect generalizations from specific > tests. > > I would prefer that Mathematica behave in a predictable, documented, and > mathematically consistent way, and that I not be required to poke it and > check that my programs still work when I install a new version, or when > I run the program with different input parameters that might, unknown to > me, exceed some threshold that I never heard of. > >> >> You can be upset that mathematica does not do exactly what you expect >> all the time, but it does no good to rant about it. > > It depends on your expectation being reasonable or not, don't you think? > >> The fact is, there are well documented ways to deal with this problem >> so that mathematica *does* do the right thing, not just in this toy >> example, but in more complex problems. > > Although I have hardly touched the vast array of contributions from > users, I suspect that your claim is not true for this case or for some > of the more complex problems. Where is it documented (much less "well > documented") that sums above 1000000 [or whatever] are different from > below that? > >> Every language has its way of interpreting code. Mathematica is >> special in that there are a lot more ways to get the code 'almost' >> right and usually a lot of different ways to get the code exactly >> right. A similar coding error in c would simply give a compiler error >> or crash. > > This is true, but Mathematica has some pretense to doing mathematics, > and you are blaming the user for making a programming error when there > is nothing wrong with the program. The error is that Mathematica is not > correctly interpreting the code, which is both legal, perfectly clear, > and unambiguous, by taking a shortcut that is incorrect. Think about > what you are saying. If a C programmer used a compiler and found that > it sometimes took a correct program and ran it incorrectly, would he > say, oh, I can write a different program that gets the right answer? > This is why I view Mathematica, from a computer science/ programming > language perspective as, how shall we say,... disappointing. > > So what is the issue? > > 1. WRI does not define what its way of interpreting code really is, and > reserves the right to change it, anyway. > 2. The Mathematica user is dependent on the good will of the > implementation programmers and, to some extent, luck, for his program to > work. > 3. While guided by mathematics generally, the computer system > programmers deliberately take shortcuts that violate 100% correctness. > > You are, of course, free to try to use Mathematica to solve problems, > and it may work just fine most of the time. But as a computer scientist > who is familiar with mathematics generally, and quite familiar with > computer algebra systems, I find it unfortunate that you (and others) > make excuses for Mathematica getting incorrect answers, even when other > computer programs might get them right, (and WRI may be convinced to fix > the issues too) thinking that somehow the USER is at fault. > > >> >> As for your gratuitous comment about Andrzej's post, I think that >> anyone familiar with any type of computer algebra system would be able >> to guess, just in terms of practicality, that there is a threshold >> from brute force to symbolic summation. > > Why can't Mathematica figure out that the summand is defined in a > peculiar way and has to be handled carefully? Why does the user have to > think about whether the summation is done by brute force or by closed > formula? What if the user is building a package and does not know what > the user of his package has done to define a program that is, > eventually, used as a summand? The package-builder must do the analysis > that WRI should have done in the first place, or else that package may > fail mysteriously. This means the package writer must violate a precept > of the WRI way, which is that a Mathematica user need not be concerned > about the details of algorithms. Maybe you THINK you know about how to > do summation, but do you know how Mathematica does it? Does your > knowledge extend to do (say) polynomial factorization? Are you expected > to know which classes of polynomials are hard to factor and therefore > you should not ask Mathematica to factor them? > >> Otherwise, Mathematica could never compute an infinite sum. I have >> been using Mathematica for 20 years, and I did not know the name for >> the threshold, let alone its value, but for my purposes, I did not >> need to know. However, I did know (and it was in the documentation) >> that there is a point where one type of computation ends and another >> begins. >> >> The function SystemOptions[] is defined and documented in version 7 >> (It was new in version 6). >> >> If you execute SystemOptions["*"], you get an impressive list of all >> the options and their current values. Most appear to be >> self-explanatory, but a few look like more explanation is needed. It >> also appears that if your code depends on the values for these >> options, the code may be fragile at a minimum. > > This is true. In fact I may have used SystemOptions[] at some point in > the distant past (1984?) but did not connect it to Sum since there is no > indication in the documentation for Sum that it depends on this > threshold option. So there is no way, other than poking at the corpse, > for you to tell if your code is fragile. > >> >> Note that the original problem had a resolution that did not involve >> special knowledge of these variables. > > Sure, there are lots of resolutions, including, as I said, the excellent > one of the user just figuring out the answer. Face it, Mathematica got > the wrong answer to a problem that is entirely well-posed. Stop making > excuses! > >> I'd say that knowing that these options exist belongs to advanced >> users of Mathematica. It might be nice to have more complete >> documentation for some of these options -- and Tech Support has >> provided this information when I have needed it, but this knowledge >> would not have solved the original question. > > That is because the original problem was well posed, and Mathematica > should have solved it!. Can you not conceive of a programming solution > to the original problem, or at least a solution that determined whether > this was a case that might lead to an error? {It might require going > slightly outside the usual Mathematica techniques -- I don't know > offhand how to decompose a function definition to see how many rules it > includes, and do a kind of meta-analysis of the patterns. } > > > ...<snip> > > RJF -- DrMajorBob at bigfoot.com

**References**:**Re: error with Sum and Infinity***From:*Richard Fateman <fateman@cs.berkeley.edu>

**Re: Re: Re: error with Sum and Infinity**

**Number of Words in a String**

**Re: Re: Re: error with Sum and Infinity**

**Re: Re: Re: Re: error with Sum and**