[Date Index] [Thread Index] [Author Index]

Re: Need features to make kernel and user functions bullet proof

• To: mathgroup at smc.vnet.net
• Subject: [mg84083] Re: Need features to make kernel and user functions bullet proof
• From: David Bailey <dave at Remove_Thisdbailey.co.uk>
• Date: Mon, 10 Dec 2007 04:30:28 -0500 (EST)
• References: <fjbceq$4mf$1@smc.vnet.net> <fjdsp3$j2m$1@smc.vnet.net>

Szabolcs Horvát wrote:
> Hi,
> 
> Here are my thoughts on this:
> 
> Ted Ersek wrote:
>> Need features to make kernel and user functions bullet proofWe can
>> overload functions that are part of the Mathematica language, and this
>> is good.  However, we often hear in the MathGroup that you can easily
>> break the kernel by changing kernel functions. This wouldn't be so risky
>> if users could not indirectly change the way functions such as Solve,
>> Simplify, NDSolve, etc. perform. I mean a user should be able to change
>> what Simplify does by making a new definition for Simplify, but changing
>> functions such as Thread, Map, or Apply should not change Solve, Simplify,
>> or NDSolve. Besides that users who are writing their own functions should
>> be able to ensure their functions cannot be broken inadvertently.
> 
> But is this really necessary?  I mean, is there any legitimate reason to 
> change symbols like Thread, Map, or Apply?  If one needs a Map that 
> behaves a bit differently, shouldn't one just define a new myMap function?
> 
> (I know that some people are changing these symbols ... see e.g.
> 
> http://groups.google.com/group/comp.soft-sys.math.mathematica/browse_thread/thread/90da91b2fb43891a/4d26e4c1e1edcce3#4d26e4c1e1edcce3
> 
> But I always found these things extremely scary (and I think that it is 
> a bad practice to do this).  As you mentioned, it is possible that some 
> built-in (or package) function uses the form Map[{a,b,c},d] (see the 
> referenced post), so it would be broken by this change to Map.  However, 
> in this situation the user could have just defined his own listableMap[] 
> function, which would be safe.  Actually after I read that message I 
> searched all the standard packages and the UpValues, OwnValues and 
> DownValues of symbols returned by Names[] for Map[_List, __], but found 
> nothing (though I'm not sure I searched the *built-in* definitions 
> correctly, and I could completely prevent evaluation of the symbols only 
> if I ran Mathematica without the Front End ...) )
> 
>> -------------------------------------------------------------------------
>> Also notice you can change the default options of a protected symbol
>> without first calling Unprotect. As a result a user can seriously break
>> a lot of functions with lines such as
>>
>>   SetOptions[ReplaceRepeated,MaxIterations -> 1];
>>   SetOptions[Map,Heads->True];
>>
>> Mathematica needs a new attribute called OptionsProtected.
>> OptionsProtected::usage="OptionsProtected is an attribute that prevents
>> the default options of a symbol from being modified."
> 
> 
> OptionsProtected could be useful.
> 
> 
>> -------------------------------------------------------------------------
>>
>> Now about preventing inadvertent changes to a functions definitions. One
>> way WRI could provide this capability would be to have all symbols in the
>> System` context also in a context called DefaultDefinitions`. All Symbols
>> in context DefaultDefinitions` would have attributes {OptionsProtected,
>> Protected, Locked}.  That would make it impossible for a user to modify
>> features in the DefaultDefinitions` context.
>>
>> Things that happen automatically behind the scenes should use symbols
>> from the DefaultDefinitions` context to ensure everything always
>> works correctly.  So for example when definitions are automatically
>> added to DownValues[f], UpValues[f], etc.  they should be created using
>> DefaultDefinitions`HoldPattern, and DefaultDefinitions`RuleDelayed.
>>
>> Imagine what could happen if you changed the meaning of RuleDelayed in
>> the current Mathematica.
>>
>> -------------------------------------------------------------------------=
>> -
>> WRI could also give us a new function called SystemDefaults that would 
>> do the following.
>>
>> Attributes[SystemDefaults]={HoldRest};
>>
>> SystemDefaults[All, expr] replaces all symbols in expr that are in
>> the System` context with the same symbol from the DefaultDefinitions`
>> context. Once the replacements are made the new form of expr evaluates.
>>
>> SystemDefaullts[{symb1, symb2, ...},expr]  selects from {symb1, symb2,
>> ....} Symbols in the System` context and replaces each instance of
>> these symbols in expr with the same symbol from the DefaultDefinitions`
>> context. Once the replacements are made the new form of expr evaluates
>> ..
>>
>> -----------------------------------------------
>>
>> Then I could write the following and ensure all system symbols below
>> are from the DefaultDefinitions` context and it would not be possible
>> to make Inadvertent changes to my function integrate.
>>
>>
>>
>> SystemDefaults[All,
>>   integrate[c_?NumericQ, f_, x_Symbol]:=c*integrate[f,x];
>>   integrate[f_+g_,x_Symbol]:=integrate[f,x]+integrate[g,x];
>>   integrate[x_Symbol,x_Symbol]:=1/2 x^2;
>>   integrate[_Symbol^n_?(NumericQ[#]&&TrueQ[# != -1]&), 
>> x_Symbol]:=1/(n+1) x^(n+1)
>> ]
>>
>> ------------------------------------------------
>> Instead I might want to use the following to ensure
>> symbols DefaultDefinitions`NumericQ, DefaultDefinitions`Power,
>> DefaultDefinitions`SetDelayed are used in my definitions of integrate. No
>> other symbol form the DefaultDefinitions` context would be used.
>>
>>
>>
>> SystemDefaults[{NumericQ,Power,SetDelayed},
>>   integrate[c_?NumericQ f_,x_Symbol]:=c*integrate[f,x];
>>   integrate[f_+g_,x_Symbol]:=integrate[f,x]+integrate[g,x];
>>   integrate[x_Symbol,x_Symbol]:=1/2 x^2;
>>   integrate[_Symbol^n_?(NumericQ[#]&&TrueQ[#!=-1]&),x_Symbol]:=1/(n+1)x
>> ^(n+1)
>> ]
>>
>> In the above examples above use of SystemDefaults[_._] would not force
>> the use of symbols from the DefaultDefinitions` context when f[t], or
>> (t) in integrate[f[t], t] evaluates.
>>
> 
> Now about changing symbols like Simplify, NIntegrate, etc.:
> 
> There might be reasons for changing these, but one could argue that a 
> legitimate change should extend these symbols in a mathematically 
> sensible way, so that other function which use them would not break 
> (i.e. Simplify would still simplify expression, but would be a bit 
> smarter, NIntegrate would still compute integrals numerically, possibly 
> with another method etc.)  If the behaviour of these functions is 
> changed completely, then probably it would have been better to define 
> completely new functions instead.
> 
> So I think that one should never change built-in symbols, except in the 
> case when it is really expected that the new definitions would be 
> applied even when the re-defined symbols are used inside some package 
> (or built-in) function.
> 
> I see that the proposed SystemDefaults allows for excluding certain 
> symbols (more precisely: it allows for only including certain symbols 
> when rewriting them to the alternatives in DefaultDefinitions), but this 
> couldn't work correctly because of the nature of Mathematica:  Suppose 
> that we're writing a new function that uses Simplify, and we want to 
> allow for the re-definition of Simplify, therefore we use 
> System`Simplify instead of DefaultDefinitions`Simplify.  But if our 
> function "calls" a built-in function that also uses Simplify (I don't 
> know if there are actually any of these), the built-in function will use 
> DefaultDefinitions`Simplify and not the intended System`Simplify.  So we 
> would end up with two different versions of Simplify being 
> inconsistently called in different situations.  This sounds even more 
> dangerous and confusing than the current situation.
> 
> In my (subjective) opinion, for those functions where re-definition 
> seems legitimate (like Simplify), WRI should provide a documented and 
> safe API that only allows sensible changes.  And indeed they do for some 
> functions:  Simplify has the TransformationFunctions option, NIntegrate 
> can be extended with new integration methods, NumericQ and Derivative 
> are directly assignable (NumericQ[n] = True and Derivative[1][f] = g 
> work without Unprotect[]ing these symbols), etc.
> 
> To summarize:  I think that one should never modify any Protected 
> built-in functions, except through documented APIs.  But I like the idea 
> of OptionsProtected.
> 
> What do other MathGroup users think about this?  Are there situations 
> when it is legitimate to modify built-ins?
> 
> I can give one example and show that it was a bad idea because it really 
> breaks things:  when I was trying to get the DownValues of all symbols 
> returned by Names[], I changed its HoldAll attribute to HoldAllComplete 
> so that it would work on really strange symbols like 'x' if 'x' has the 
> definitions x[1]=1 and _[x] ^= x.  (Question: Are there other ways of 
> doing this than changing the attributes of DownValues?)  But with this 
> change, DownValues[Evaluate[sym]] ceases to work, and indeed I got lots 
> of error messages when I tried to open the Help Browser ...
> 
> Szabolcs
> 
> P.S.  I hope that my broken English didn't make this long post 
> completely unintelligible.
> 
> P.P.S.  Your Mathematica Tricks website is really nice!
> 
Maybe I read Ted's request a little differently. I think he is saying 
that however carefully you write a package for others to use, you can 
never make it bullet-proof because others may have made changes to 
built-in definitions, and then perhaps report spurious bugs in the 
package! So saying that people should not change built-in definitions 
rather misses the point. OptionsProtected, plus access to a list of any 
symbols that have been unprotected (regardless of whether they were 
re-protected) would help I think.

David Bailey
http://www.dbaileyconsultancy.co.uk