Efficiency in Mathematica. Part 2
- To: mathgroup at yoda.ncsa.uiuc.edu
- Subject: Efficiency in Mathematica. Part 2
- From: dan at chem.bu.edu (Dan Dill)
- Date: Thu, 27 Sep 90 12:56:13 -0400
Richard (fateman at ucbarpa.Berkeley.EDU) comments that the essence of the speed up is to have accessed, by whatever means, internal algorithms that take advantage of the list structure of the data. David Jacobson (jacobson at cello.hpl.hp.com) sent me a suggestion that may do just this, in the context of "traditional" subscripting. By itself David's method reduced the time from 870 seconds to 9.6 seconds, only about twice as slow as the fastest scheme I have been able to achieve. I include David's text below, but the idea is to prevent Mathematica from knowing it is dealing with a list until all subscripting is done: SetAttributes[HoldList,HoldAll]; ne = 50; nkmat = 18; (* The values were used earlier, too *) oldmodhold := Block[{i,j}, data = Apply[HoldList, Apply[HoldList,data,4]]; For[i = 1, i < ne + 1, i++, For[j = 1, j < nkmat + 1, j++, data[[i,2,j,1]] = N[Mod[data[[i,2,j,1]], 1], 8] ] ]; data = Apply[List, Apply[List,data,4]]; ]; Timing[oldmodhold;] Out[137]= {9.58333 Second, Null} Recall that oldmod differs from the above by the absence of the Apply[...] before and after the double loop, and the timing is then 870 seconds. So David must be onto something. Here's David's message: ------------------------------- >From jacobson at cello.hpl.hp.com Tue Sep 25 13:45:14 1990 Return-Path: <jacobson at cello.hpl.hp.com> Received: from hplms2.hpl.hp.com by chem.bu.edu (5.61+++/JLK-4.0.3) id AA11645; Tue, 25 Sep 90 13:45:10 -0400 Received: from cello.hpl.hp.com by hplms2.hpl.hp.com with SMTP (15.11.1.3/15.5+IOS 3.20) id AA17661; Tue, 25 Sep 90 10:47:20 pdt Received: from localhost by cello.hpl.hp.com with SMTP (15.11/15.5+IOS 3.14) id AA25443; Tue, 25 Sep 90 10:46:41 pdt To: dan at chem.bu.edu (Dan Dill) Subject: Re: Efficiency in Mathematica In-Reply-To: Your message of "Thu, 20 Sep 90 14:03:52 PDT." <9009250532.AA00115 at yoda.ncsa.uiuc.edu> Date: Tue, 25 Sep 90 10:46:35 PDT Message-Id: <25441.654284795 at cello.hpl.hp.com> From: jacobson at cello.hpl.hp.com Status: RO I've also noticed that programs that munge on just part of a list using subscripting run like molassas. I'm suspicious that a good part of the performance problem lies in the subscript operation. It evaluates both arguments, and the first is the whole giant list. So, in principle, the whole list gets scanned on each reference to see if there is anything that will evaluate. Internally, Mathematica seems to have some bunch of flags to avoid multiple evalutations, but when you are using subscripting on the left hand side, it may clear the flag. I've found that the following trick will often speed such things up by an order of magnitude. First do this. SetAttributes[HoldList,HoldAll] HoldList will be a "function" that works about like List, except that it does not evaluate its arguments. You can easily replace the outmost List with HoldList with Apply[HoldList,<<<the old list>>>] Then you do all the subscripting, then let it all evaluate at one swoop with Apply[List,<<<the munged HoldList>>>]. In your case, you might want to use HoldList at 2 levels. If you see fit, I'd appreciate you trying this trick and reporting to the mathgroup what the performance was. -- David Jacobson