RE: Re: Readability confuses mathematica? - Notation Package

*To*: mathgroup at smc.vnet.net*Subject*: [mg44588] RE: [mg44565] Re: Readability confuses mathematica? - Notation Package*From*: Jason Harris <jasonh at wolfram.com>*Date*: Mon, 17 Nov 2003 03:38:19 -0500 (EST)*Sender*: owner-wri-mathgroup at wolfram.com

Hi David, >I would like to put some comments in here concerning the Notation package. I >think the package tackles a very useful and a very complicated problem. I >doubt if there are many people who could carry out what you have done. Thanks. >Still I find severe problems with the package. It is difficult to put my >finger accurately on them but I will make an attempt. I have, on a number of >occasions, spent many hours and even days trying to use the package and have >always given up in frustration. Perhaps my intellectual capability just >isn't up to it. Its a non-trivial topic :) >User interface and documentation are very important for a package. Definitely. >Let's talk about user interface first. The first thing that will happen when >a user is trying to learn a package like this is that he will make mistakes. >You can count on it. But there appears to be no easy method to undo the >mistakes. If a specific notation or Symbol has been entered incorrectly, >there doesn't seem to be any good way to get rid of it without killing the >kernel and starting over. When I tried to use RemoveNotation or RemoveSymbol >it did not get rid of the underlying built_up symbols so errors or warning >messages were generated when I tried again to define a notation. One should >be able to easily undo one's mistakes and continue on. It would be nice. I'll add it to my list, but... its not that absolutely high up there. When you are developing these notations, you can easily use ClearNotations[] and that will clean out the notations. Let's not kid ourselves though; Notations are a non-trivial area. As mathematicians and physicists etc, we chop and change between various different forms and we perform semantic context-sensitive interpretations that are somewhat incompatible with a rigid and codified system of rules. In practice to create simple notations like * subscript G for a group multiplication or something is pretty trivial. However for complex notations, (e.g. bra's and ket's in physics), this means that we have to negotiate a pre-established set of conventional typesetting rules, then overload our own behaviors into them, all the while worrying about evaluation (of boxes and kernel expressions). This makes the whole process somewhat non-trivial. Beginning Mathematica users should be able to utilize notations others have created, but probably it is a bit much to expect them to know the intricacies of creating their own complex notations. That is really an experienced user's game. But to reiterate creating simple notations is simple. >It would be very helpful if one did not have to use the palette to enter a >notation or symbol. People do seem to have trouble in selecting the correct >box. It might be better if one could type in the command to establish a >notation and perhaps use Headers to distinguish special input. Myself, if the underlying form is quite complex I usually type out the FullForm and then just convert to standard form if the underlying box structure has hidden tag boxes, etc. (like for Bra's and Ket's, tensors, etc.) Or as a designer you should create palettes for your users, or create aliases. Both of these are of course fully supported by the Notation package. >Much of the Mathematica pattern capability appears to be lost in the >Notations package. Or else it isn't very clear how to use it. Can we use the >general patterns, for example multiple arguments, arguments with specified >heads, or conditional patterns when establishing Symbols and Notations? Absolutely. >Can we use Rule based programming with entities established with Symbolize or >Notation? Definitely. I use this feature very heavily! >Without the use of general patterns I think that Notations is too >severely limited. I'm hoping that you will tell me that I am wrong here and >one can make general use of patterns! But it should be as easy as using >general patterns in regular Mathematica. This is a good point. I have been toying with the dilemma of this perception for a while. Its not too difficult for the user to enter complex notations and use the full gamut of normal Mathematica patterns, although they might not do exactly what the user wants, since it is matching on boxes on one side and patterns on the other. Anyway it's all there in the help files... Basically, you add a complex pattern wrapper. When designing the package I had two choices, make the patterns act like patterns, or make the patterns act like notations. I chose notations since this lead to totally unequivocal and predictable results. However, I have thought about this in the back of my mind for a while (a slow burning problem) and I think for the next major revision, time permitting, I'll have a stab at making it "as easy as a normal pattern". The problem is side effect evaluation. More than anything, Notations have to be extremely careful to not have unwanted evaluation. How this will gel with simple user expectations is another matter. I can go into more detail but it gets kind of heavy... :) I'll see how some of my investigations pan out. >The modification of the notebook style sheet may be necessary and harmless >but it certainly does raise eyebrows with the user. It probably needs more >explanation in the documentation. Does it add a style sheet to the notebook >or does it modify the underlying style sheet? This is why I still question >if the palette is the best method for entering notations. It "instantiates" the style sheet if not already instantiated and then adds the styles. In some future version, we want to have cascading style sheets so this kind of thing will be unnecessary. It has been a long standing feature I have desired. Hopefully it will be present in the next major release. >I still think that the documentation is not good enough. The very first >thing that a new user might try is the first example under Symbolize. I defy >anyone to freshly type that example in, and not just evaluate the example in >the Help notebook. It is definitely not What You See Is What You Type! If a >user can't get beyond that example what chance does he have? Ahh, well there you have it. The whole mechanism behind notations sometimes requires that you don't have What You See Is What You Type. Why is this? Well, consider the example of the domain partial to which you referred. That is really a symbol times an operator which is turning around and acting as a symbol. So what you would normally see is a symbol times an operator. But we, being clever humans, automatically just flip this in our mind semantically and think "ohh that's an operator." Thus for our computer codification, we need some extra information around. (Humans provide this information in our own complex context sensitive ways. E.g. a physicist will see a p^2 / (2 m) and "auto-magically" know that p is an operator and m is an identifier... etc.) Indeed, for a complex mathematical expression, in practice you often can't have What You See Is What You Type. Unfortunately there is just too much variation on what we mean by a particular textbook notation. Currently to achieve parsing of some structures the underlying boxes at present have to be quite complex. Eg take a look at the traditional form for one of our Hypergeometrics, eg. ToBoxes @ TraditionalForm @ Hypergeometric1F1[a, b, z] >As you explain below, the user has to be instructed about the use of the >underlying cell expressions. And once these are brought in the Notations >package is not quite the shortcut that users might have been hoping for. It >may be all right but it has to be explained and there have to be numerous >and COMMON examples. > >In summary >1) I would like to see a better user interface and usability. I have toyed with having an auto-generated palette which is automatically updated and includes all of the notations that have been entered in the given session. Would you find such a thing useful? I / we are actively working on several features related to making box structures cleaner, simpler, and more powerful. >2) This is a difficulty subject and requires extra good documentation. I agree. >3) We need to know how to use general patterns and rule based programming >with notations. See the response above. You can do this all right now, it's just not always the most intuitive... >I would dearly love to be able to use Notations. Renan Cabrera, >Jean-Francois Gouyet and I have a Tensorial package where we use Format >statements to format tensor output, and a number of other things such as >absolute derivatives, Lie derivatives or colored indices. It looks nice and >works, but the output cannot be copied and pasted. I was able to write some >MakeBoxes and MakeExpression statements that did allow the copying of output >and tab entry on indices, but it was delicate and version dependent so we >didn't include it in the package. It would be very nice to make Notations >work on something like this. I have got a system for doing this. I have been meaning to put the final touches on it so that it is usable to the community. As part of my PhD thesis, I came up with an algorithm for Canonicalizing tensorial expressions which was much faster than other algorithms at the time. It is a true algorithm in that it uses computational group theory, and other tricks, not pattern matching which gets exponentially slower as the number of terms rise. (It still is the fastest as far as I am aware, unless there have been other advances which I am unaware of. I know Portugal was also working on similar problems. He implements largely in another system though, whereas I implement in Mathematica.). I really meant to add this to 5.0 but due to other commitments it slipped. I really want to see this as part of the next major release. Many users have requested this. One of the big things holding it up is ironically not enough polished documentation... Cheers, Jason ------------- Jason Harris Wolfram Research >David Park >djmp at earthlink.net >http://home.earthlink.net/~djmp/ > > > > >From: Jason Harris [mailto:jasonh at wolfram.com] To: mathgroup at smc.vnet.net > >Hi Andrzej, > >[Notation example snipped] > >> On my system this example just fails miserabley (the >> definiton of transmissionCoefficient does not parse) although it once >> used to work. (Actually, I would like to hear from other people, >> because either the Notation package is no longer working or something >> is wrong with my installation). > >I know these things can appear frustrating at times, but the Notation >package is working fine. In the example you have given, you have >managed to include the box wrapper twice in your Symbolize statement. >(You likely did this through copying the k subscript including box >wrapper and pasted it into a new Symbolize statement.) > > >> Anyway, if this works on your system than the problem is solved. You >> can symbolize subscripted variables and use them as if they were >> symbols. On the other hand, in the past, when this package used to work >> on my system, I got myself into a huge mess when I tried to evaluate >> again a notebook that had previously worked correctly. > >If there is some notebook corruption going on then we definitely want >to know about this. In the example you have given it is extremely >likely that you inadvertently caused the error with a copy and paste >from one symbolize statement to another. (I know notation statements >can be tricky and it is not all that hard to "hang yourself".) > > >> Because of that I finally decided that relying on a package like this is >> just too risky if you are doing any serious work. > >Hmm... Actually I personally think its just the opposite. If you are >really doing serious work you need to know about typesetting and how >to enter expressions in notations which are familiar to >Mathematicians, physicists, and other users and have these notations >function correctly. > >First a general comment: >When you are troubleshooting problems with setting up notations it is >often necessary to look at the underlying boxes representing the >typeset expression. You can do this through either the command key >shortcut of cmd-shift-E (OSX) and I think cntrl-shift-E (Win), or >through the menu item "Format" -> "Show Expression..." > >Looking at the underlying boxes shows you how the typeset expression >is represented. As a Mathematica programmer you no doubt use FullForm >at times to examine how a certain pattern is structured. Its really >not too different with typesetting in that Mathematica functions through >MakeBoxes and MakeExpression which operate on these box structures. If >you don't have the correct structures, the boxes will not be >interpreted as they should be. > >In a Symbolize statement waiting to be filled in the underlying >structure is > >Cell[BoxData[ > RowBox[{"Symbolize", "[", > TagBox["\[Placeholder]", > NotationBoxTag, > TagStyle->"NotationTemplateStyle"], "]"}]], "Input"] > >To see this load the notation package, type Esc-symb-Esc and then show >the underlying expression. The box structure to be symbolized must be >the first argument of this TagBox. > >The tag box wrapper, NotationBoxTag, is necessary so you can enter >typeset expressions into Mathematica that are not currently >syntactically valid. Then once the Notation package gets a hold of >them, it compiles / translates these into corresponding rules for >MakeExpression and MakeBoxes that do what you instructed. The tag box >wrapper inertizes the box structure so it becomes syntactically valid >Mathematica input. > >You can see this by copying out the box wrapper into a new cell and >entering some syntactically invalid input into the wrapper, say "x" >followed by "*", and then viewing how it is interpreted by >Mathematica. E.g. after loading the notation package paste the >following cell into Mathematica and interpret it: > >Cell[BoxData[ > TagBox[ > RowBox[{"x", "*"}], > NotationBoxTag, > TagStyle->"NotationTemplateStyle"]], "Input"] > >Then evaluate this interpreted cell. Then start a new cell and >evaluate FullForm[%]. The answer will be > >NotationBoxTag[RowBox[List["x", "*"]]] > >Thus the TagBox wrapper, NotationBoxTag, has allowed us to enter >something that is syntactically invalid and get the box structure into >the kernel. (Incidentally you can programmatically create symbolize >and notation statements this way.) The TagBox wrapper NotationBoxTag >can do this miraculous feat because it itself has a corresponding >notation. (The TagStyle option is set so you can get some visual >indication of where these tag box wrappers occur.) > >So anyway getting back to your example if you look at the underlying >boxes you will see that instead of having > >Cell[BoxData[ > RowBox[{"Symbolize", "[", > TagBox[ > SubscriptBox["k", "_"], > NotationBoxTag, > TagStyle->"NotationTemplateStyle"], "]"}]], "Input"] > >You had > >Cell[BoxData[ > RowBox[{"Symbolize", "[", > TagBox[ > TagBox[ > SubscriptBox["k", "_"], > NotationBoxTag, > TagStyle->"NotationTemplateStyle"], > NotationBoxTag, > TagStyle->"NotationTemplateStyle"], "]"}]], "Input"] > >i.e. the wrapper was in there twice. Fixing this mistake resolves your >problem. > >Cheers, > Jason > >------------- >Jason Harris >Wolfram Research