Re: Need features to make kernel and user functions bullet proof
- To: mathgroup at smc.vnet.net
- Subject: [mg84083] Re: Need features to make kernel and user functions bullet proof
- From: David Bailey <dave at Remove_Thisdbailey.co.uk>
- Date: Mon, 10 Dec 2007 04:30:28 -0500 (EST)
- References: <fjbceq$4mf$1@smc.vnet.net> <fjdsp3$j2m$1@smc.vnet.net>
Szabolcs Horvát wrote: > Hi, > > Here are my thoughts on this: > > Ted Ersek wrote: >> Need features to make kernel and user functions bullet proofWe can >> overload functions that are part of the Mathematica language, and this >> is good. However, we often hear in the MathGroup that you can easily >> break the kernel by changing kernel functions. This wouldn't be so risky >> if users could not indirectly change the way functions such as Solve, >> Simplify, NDSolve, etc. perform. I mean a user should be able to change >> what Simplify does by making a new definition for Simplify, but changing >> functions such as Thread, Map, or Apply should not change Solve, Simplify, >> or NDSolve. Besides that users who are writing their own functions should >> be able to ensure their functions cannot be broken inadvertently. > > But is this really necessary? I mean, is there any legitimate reason to > change symbols like Thread, Map, or Apply? If one needs a Map that > behaves a bit differently, shouldn't one just define a new myMap function? > > (I know that some people are changing these symbols ... see e.g. > > http://groups.google.com/group/comp.soft-sys.math.mathematica/browse_thread/thread/90da91b2fb43891a/4d26e4c1e1edcce3#4d26e4c1e1edcce3 > > But I always found these things extremely scary (and I think that it is > a bad practice to do this). As you mentioned, it is possible that some > built-in (or package) function uses the form Map[{a,b,c},d] (see the > referenced post), so it would be broken by this change to Map. However, > in this situation the user could have just defined his own listableMap[] > function, which would be safe. Actually after I read that message I > searched all the standard packages and the UpValues, OwnValues and > DownValues of symbols returned by Names[] for Map[_List, __], but found > nothing (though I'm not sure I searched the *built-in* definitions > correctly, and I could completely prevent evaluation of the symbols only > if I ran Mathematica without the Front End ...) ) > >> ------------------------------------------------------------------------- >> Also notice you can change the default options of a protected symbol >> without first calling Unprotect. As a result a user can seriously break >> a lot of functions with lines such as >> >> SetOptions[ReplaceRepeated,MaxIterations -> 1]; >> SetOptions[Map,Heads->True]; >> >> Mathematica needs a new attribute called OptionsProtected. >> OptionsProtected::usage="OptionsProtected is an attribute that prevents >> the default options of a symbol from being modified." > > > OptionsProtected could be useful. > > >> ------------------------------------------------------------------------- >> >> Now about preventing inadvertent changes to a functions definitions. One >> way WRI could provide this capability would be to have all symbols in the >> System` context also in a context called DefaultDefinitions`. All Symbols >> in context DefaultDefinitions` would have attributes {OptionsProtected, >> Protected, Locked}. That would make it impossible for a user to modify >> features in the DefaultDefinitions` context. >> >> Things that happen automatically behind the scenes should use symbols >> from the DefaultDefinitions` context to ensure everything always >> works correctly. So for example when definitions are automatically >> added to DownValues[f], UpValues[f], etc. they should be created using >> DefaultDefinitions`HoldPattern, and DefaultDefinitions`RuleDelayed. >> >> Imagine what could happen if you changed the meaning of RuleDelayed in >> the current Mathematica. >> >> -------------------------------------------------------------------------= >> - >> WRI could also give us a new function called SystemDefaults that would >> do the following. >> >> Attributes[SystemDefaults]={HoldRest}; >> >> SystemDefaults[All, expr] replaces all symbols in expr that are in >> the System` context with the same symbol from the DefaultDefinitions` >> context. Once the replacements are made the new form of expr evaluates. >> >> SystemDefaullts[{symb1, symb2, ...},expr] selects from {symb1, symb2, >> ....} Symbols in the System` context and replaces each instance of >> these symbols in expr with the same symbol from the DefaultDefinitions` >> context. Once the replacements are made the new form of expr evaluates >> .. >> >> ----------------------------------------------- >> >> Then I could write the following and ensure all system symbols below >> are from the DefaultDefinitions` context and it would not be possible >> to make Inadvertent changes to my function integrate. >> >> >> >> SystemDefaults[All, >> integrate[c_?NumericQ, f_, x_Symbol]:=c*integrate[f,x]; >> integrate[f_+g_,x_Symbol]:=integrate[f,x]+integrate[g,x]; >> integrate[x_Symbol,x_Symbol]:=1/2 x^2; >> integrate[_Symbol^n_?(NumericQ[#]&&TrueQ[# != -1]&), >> x_Symbol]:=1/(n+1) x^(n+1) >> ] >> >> ------------------------------------------------ >> Instead I might want to use the following to ensure >> symbols DefaultDefinitions`NumericQ, DefaultDefinitions`Power, >> DefaultDefinitions`SetDelayed are used in my definitions of integrate. No >> other symbol form the DefaultDefinitions` context would be used. >> >> >> >> SystemDefaults[{NumericQ,Power,SetDelayed}, >> integrate[c_?NumericQ f_,x_Symbol]:=c*integrate[f,x]; >> integrate[f_+g_,x_Symbol]:=integrate[f,x]+integrate[g,x]; >> integrate[x_Symbol,x_Symbol]:=1/2 x^2; >> integrate[_Symbol^n_?(NumericQ[#]&&TrueQ[#!=-1]&),x_Symbol]:=1/(n+1)x >> ^(n+1) >> ] >> >> In the above examples above use of SystemDefaults[_._] would not force >> the use of symbols from the DefaultDefinitions` context when f[t], or >> (t) in integrate[f[t], t] evaluates. >> > > Now about changing symbols like Simplify, NIntegrate, etc.: > > There might be reasons for changing these, but one could argue that a > legitimate change should extend these symbols in a mathematically > sensible way, so that other function which use them would not break > (i.e. Simplify would still simplify expression, but would be a bit > smarter, NIntegrate would still compute integrals numerically, possibly > with another method etc.) If the behaviour of these functions is > changed completely, then probably it would have been better to define > completely new functions instead. > > So I think that one should never change built-in symbols, except in the > case when it is really expected that the new definitions would be > applied even when the re-defined symbols are used inside some package > (or built-in) function. > > I see that the proposed SystemDefaults allows for excluding certain > symbols (more precisely: it allows for only including certain symbols > when rewriting them to the alternatives in DefaultDefinitions), but this > couldn't work correctly because of the nature of Mathematica: Suppose > that we're writing a new function that uses Simplify, and we want to > allow for the re-definition of Simplify, therefore we use > System`Simplify instead of DefaultDefinitions`Simplify. But if our > function "calls" a built-in function that also uses Simplify (I don't > know if there are actually any of these), the built-in function will use > DefaultDefinitions`Simplify and not the intended System`Simplify. So we > would end up with two different versions of Simplify being > inconsistently called in different situations. This sounds even more > dangerous and confusing than the current situation. > > In my (subjective) opinion, for those functions where re-definition > seems legitimate (like Simplify), WRI should provide a documented and > safe API that only allows sensible changes. And indeed they do for some > functions: Simplify has the TransformationFunctions option, NIntegrate > can be extended with new integration methods, NumericQ and Derivative > are directly assignable (NumericQ[n] = True and Derivative[1][f] = g > work without Unprotect[]ing these symbols), etc. > > To summarize: I think that one should never modify any Protected > built-in functions, except through documented APIs. But I like the idea > of OptionsProtected. > > What do other MathGroup users think about this? Are there situations > when it is legitimate to modify built-ins? > > I can give one example and show that it was a bad idea because it really > breaks things: when I was trying to get the DownValues of all symbols > returned by Names[], I changed its HoldAll attribute to HoldAllComplete > so that it would work on really strange symbols like 'x' if 'x' has the > definitions x[1]=1 and _[x] ^= x. (Question: Are there other ways of > doing this than changing the attributes of DownValues?) But with this > change, DownValues[Evaluate[sym]] ceases to work, and indeed I got lots > of error messages when I tried to open the Help Browser ... > > Szabolcs > > P.S. I hope that my broken English didn't make this long post > completely unintelligible. > > P.P.S. Your Mathematica Tricks website is really nice! > Maybe I read Ted's request a little differently. I think he is saying that however carefully you write a package for others to use, you can never make it bullet-proof because others may have made changes to built-in definitions, and then perhaps report spurious bugs in the package! So saying that people should not change built-in definitions rather misses the point. OptionsProtected, plus access to a list of any symbols that have been unprotected (regardless of whether they were re-protected) would help I think. David Bailey http://www.dbaileyconsultancy.co.uk