Re: Re: Add syntax highlighting to own command

*To*: mathgroup at smc.vnet.net*Subject*: [mg101917] Re: [mg101899] Re: Add syntax highlighting to own command*From*: Leonid Shifrin <lshifr at gmail.com>*Date*: Wed, 22 Jul 2009 06:24:53 -0400 (EDT)*References*: <200907090600.CAA17547@smc.vnet.net> <h3766u$f9h$1@smc.vnet.net>

Hi Bastian, Some comments: 1. About variable renaming in SetDelayed: this happens sometimes if SetDelayed in lexically scoped inside some other scoping constructs (Module, With, Function). For example: In[1] = Module[{a,b},ff[a_,b_]:=a^2+b^2] In[2] = ?ff Out[2] = Global`ff ff[a_,b_]:=a^2+b^2 No renaming in this case - the formal parameters a,b in SetDelayed completely shadowed the Module variables, and the r.h.s does not reference any other local variables. Now: In[3] = Module[{a,b,c},ff[a_,b_]:=a^2+b^2+c^2] In[4] = ?ff Out[4] = Global`ff ff[a$_,b$_]:=a$^2+b$^2+c$39213^2 Mathematica seems to be rather aggressive in variable renaming, doing it also in cases when it is not strictly necessary, like the last one - but this should be ok as long as we don't assume that variable names in scoping constructs will keep exactly those symbolic values we give them. This makes sense to me, since they are really "dummy" variables and reliance on their exact symbolic name seems conceptually wrong to me (this does not refer to Block). Let me repeat once again, that SetDelayed (or, more correctly, RuleDelayed) is different from other scoping constructs in that it will not rename its variables in they conflict with local variables in scopes *inner* to SetDelayed (RuleDelayed) - or at least this is my current understanding of it. 2. Your workaround: Verbatim[ SetDelayed ][ l_, Let[ v_, Verbatim[ Condition ][ r_, c_ ] ] ] ^:= Let[ v, SetDelayed[ #, r /; c ] & @ ToExpression @ # ] & @ ToString[ l, InputForm ] is interesting and quite inventive, but still not completely satisfactory in my view. The point is that it solves one problem - how to "hide" the content of the l.h.s. until the variable collision resolution is done - and thus your variables will not be renamed. By the way, let me digress a bit: since the heart of this code is in hiding the argument of SetDelayed from the outer With (which will be there once one iteration of Let gets executed), the following does exactly the same and does not require ToString-ToExpression cycle: Verbatim[SetDelayed][l_, LetB[v_, Verbatim[Condition][r_, c_]]] ^:= Module[{a}, LetB[v, Unevaluated[SetDelayed[a, r /; c]] /. a :> l]]; Anyways, while this does solve one problem, it does not solve another, more subtle one: it is difficult to implement UpValues with SetDelayed and Condition ourselves. There are internal heads RuleCondition and $ConditionHold, which get executed when we execute scoping constructs with conditions attached - we don't want this to happen prematurely. Taking your code (I renamed Let to LetB): In[5] = ClearAll[LetB]; SetAttributes[LetB, HoldAll]; SyntaxInformation[LetB] = {"ArgumentsPattern" -> {_, _}}; Verbatim[SetDelayed][l_, LetB[v_, Verbatim[Condition][r_, c_]]] ^:= LetB[v, SetDelayed[#, r /; c] &@ToExpression@#] &@ ToString[l, InputForm]; LetB[{}, expr_] := expr; LetB[{head_}, expr_] := With[{head}, expr]; LetB[{head_, tail__}, expr_] := With[{head}, LetB[{tail}, expr]]; In[6] = Clear[f]; f[x_,y_]:=LetB[{xl=x,yl=y+xl+1},xl^2+yl^2/;(xl+yl<15)]; f[x_,y_]:=x+y; In[7] = ?f Out[7] = Global`f f[x_,y_]:=x^2+(1+x+y)^2/;x+(1+x+y)<15 f[x_,y_]:=x+y So far so good, although the nested With-s are gone. Now: In[8] = Clear[f]; f[x_,y_]:=LetB [{xl=x,yl=y+xl+1},Print["*"];(xl^2+yl^2/;(xl+yl<15))]; f[x_,y_]:=x+y; In[9] = ?f Out[9] = Global`f f[x_,y_]:=x+y Not good, in my view. My version does this IMO more consistently: In[10] = ClearAll[LetL]; SetAttributes[LetL, HoldAll]; LetL /: Verbatim[SetDelayed][lhs_, rhs : HoldPattern[LetL[{__}, _]]] := Block[{With}, Attributes[With] = {HoldAll}; lhs := Evaluate[rhs]]; LetL[{}, expr_] := expr; LetL[{head_}, expr_] := With[{head}, expr]; LetL[{head_, tail__}, expr_] := Block[{With}, Attributes[With] = {HoldAll}; With[{head}, Evaluate[LetL[{tail}, expr]]]]; In[11] = Clear[f]; f[x_,y_]:=LetL [{xl=x,yl=y+xl+1},Print["*"];(xl^2+yl^2/;(xl+yl<15))]; f[x_,y_]:=x+y; In[12] = ?f Out[12] = Global`f f[x_,y_]:=With[{xl=x},With[{yl=y+xl+1},Print[*];xl^2+yl^2/;xl+yl<15]] f[x_,y_]:=x+y The reason why my version works is that, with the use of the Block trick, LetL is really a macro and works at "compile - time" - it is exactly equivalent to enter the nested With by hand - it really writes code for us. So, when SetDelayed is called, the system sees nested With with condition attached, and keeps both defs. 3. ToString - ToExpression cycles - my personal opinion: I went through the stage when I was using them extensively as a cheap substitute for more complex expressions involving Evaluate, Unevaluated, Hold and the like. I try not to do it anymore in systematic programming. I think it is not a good style, in particular it undermines possible introspection of the code, and a few other reasons. If you need to keep code unevaluated just once, use Unevaluated. If you need to insert it held in some expression, use, for example, a combination of local Hold-like wrapper and rules, something like Module[{myHold}, Attributes[myHold] = HoldAll With[{heldcode = myHold[code]}, f[g[h[q[heldcode]]]]/.myHold[x__]:>x]] Or wrap everything in Hold and massage your expression into the form you want, releasing Hold at the end. This is cleaner. Also, while I did not do systematic benchmarks to investigate this particular issue, on general grounds I would expect a performance hit when using ToString - ToExpression cycles, as compared to non-standard evaluation constructs. This is not to say that I am not using this technique - it is quite useful at times. I just try to use it only when it has a clear advantage in terms of code length, development time, readability etc. 4. Block trick This is a very powerful but also potentially dangerous use of dynamic scoping provided by Block. It makes virtually every Mathematica symbol, even built-in protected symbols, to temporarily "forget" many of global properties associated with it, such as global rules of various kind and attributes. The interesting part is that it does this even with built-ins, whose global rules we can not otherwise affect, other than defining our own on top of them. Symbols with Locked attribute can not be Blocked. Block trick is dangerous since it will block your symbol for an entire execution stack enclosed in Block, that is, in all functions that may be called during this execution. One has to control very well the execution stack of the computation on which Block trick is used, to use it safely. This is especially true if some system symbol is Blocked, since part of code for the system functions is written in Mathematica itself, and we don't know what internal code will be modified by Block in this case. You can even corrupt Mathematica kernel in subtle ways for a given session by doing this carelessly - this happened to me a few times, I can only wish good luck with debugging for a poor person who gets into this situation. Probably the most common use of Block trick is to change the order of evaluation in ways complex enough that the standard Unevaluated - Evaluated - Hold - etc machinery becomes cumbersome. I tend to consider Block trick, used in this manner, as a prototyping tool only, in all cases when the Block-ed head is known at "compile - time" - in all such cases it is possible (although may be not as easy) to convert the code to Unevaluated - Evaluated - Hold - etc style. There are also less trivial uses for it, for which I think it is more justified. In my version of Let, its usefulness is magnified by recursion, where it guarantees that With will "forget about itself" in function definitions until the r.h.s of the def is formed, no matter how many levels of With are nested. But certainly this could have been accomplished without Block as well. Hope this helps. Regards, Leonid On Tue, Jul 21, 2009 at 12:52 AM, Bastian Erdnuess <earthnut at web.de> wrote: > Leonid Shifrin <lshifr at gmail.com> wrote: > > > Hi Bastian, > > > > If I understand you correctly and the last rule you added > > > > ( Let /: ( lhs_ := Let[ vars_, expr_ /; cond_ ] ) := > > Let[ vars, lhs := expr /; cond ]) > > > > is to allow shared local variables in function definitions (like with > > standard With, Module or Block), then your code does not work for me. > > Yes, you are (and understood me) right. > > > Consider the following definition: > > > [...] > > > f[x_, y_] := > > Let[{xl = x, yl = y + xl + 1}, xl^2 + yl^2 /; (xl + yl < 15)]; > > I tried first something different than I actually posted. I thought > then I could write it different. But somehow I must have forgotten to > test it correctly. > > However, also with my original version (something like > > f_[ l_, Let[ r_ ] ] := bla[ l := r ] /; f == SetDelayed ) > > I would have gotten about > > f[ x$_, y$_ ] := x^2 + (1 + x + y)^2 /; 1 + 2x + y < 15 . > > I didn't recognized that the local variables in f get renamed. > > > I can think of several reasons why this does not work, and will expand > this > > discussion if you will be interested, but here is the version that I > > believe does work: > > Now, I can too. But if you want to explain some of your reasons, i'll > hear. > > I also found a trick to get around the renaming problem (using your > Verbatim instead of mine f_[...] /; f == ...): > > Verbatim[ SetDelayed ][ l_, > Let[ v_, Verbatim[ Condition ][ r_, c_ ] ] > ] ^:= > Let[ v, > SetDelayed[ #, r /; c ] & @ ToExpression @ # > ] & @ ToString[ l, InputForm ] > > This passes basically the left hand side as string around the issue with > the renaming. > > The last days I often figured out that that what I think of as "syntax > transformation" or Simon called "passing unevaluated arguments around" > works best by passing the things around as strings. Are there > disadvantages doing that? I could imagine that this might be not the > best if one wants to compile the source -- but so far I'm far away from > starting to try to understand Mathematicas compiler. > > > [ Block trick ] > > > This uses the Block trick. It is actually a true macro, since > > it expands the code before SetDelayed acts on it: > > That seems to be a nice trick. Thank you for showing that. I need to > think a little bit about that to decide if I like it for this situation. > > Regards, > Bastian > >

**References**:**Add syntax highlighting to own command***From:*earthnut@web.de (Bastian Erdnuess)

**Re: Thoughts on a Wolfram|Alpha package for**

**Re: Re: Thoughts on a Wolfram|Alpha package for**

**Re: Add syntax highlighting to own command**

**Jens-Peer Kuska**