Memory leak in function using dynamic programming

*To*: mathgroup at smc.vnet.net*Subject*: [mg18341] Memory leak in function using dynamic programming*From*: jsobel at sdav01.seinf.abb.se (Jarl R Sobel)*Date*: Wed, 30 Jun 1999 14:13:16 -0400*Organization*: ABB Automation Products*Sender*: owner-wri-mathgroup at wolfram.com

The following was executed on a Macintosh 9600/350 under Mac OS 8.6 running Mathematica 4.0.0 The problem occured when I was doing some heavy simulation work in Mathematica, and I thought I should share my results with the readers of this newsgroup. The function memTest1 below is the skeleton of a more complicated function. The function p[j] is in reality very complicated to compute, and it is called several times with the same argument in different computations. Therefore cashing its values using dynamic programming saves a lot of time. In[1]:= memTest1[] := Module[ {p, aj}, p[j_] := p[j] = Array[Random[]&, 1000]; aj[j_] := (p[j].p[j]); Plus@@Table[aj[j], {j, 1, 20}] ] Calling this function consumes 166kBytes of memory each time the function is called. In[2]:= mem1 = MemoryInUse[]; In[3]:= SeedRandom[11]; memTest1[] Out[3]= 6670.45 In[4]:= MemoryInUse[] - mem1 Out[4]= 166596 It seems that Mathematica cannot retrieve the memory used in the dynamic assignment when the function which uses dynamic programming is mentioned in the definition of another function (the function aj[j_] in the above example). If the value of this function is cleared before the function exits, the memory occupied by the local variables is retrieved as expected. In[5]:= memTest2[] := Module[ {p, a, aj}, p[j_] := p[j] = Array[Random[]&, 1000]; aj[j_] := (p[j].p[j]); a = Table[aj[j], {j, 1, 20}]; Clear[aj]; Plus@@a ] In[6]:= mem2 = MemoryInUse[]; In[7]:= SeedRandom[11]; memTest2[] Out[7]= 6670.45 In[8]:= MemoryInUse[] - mem2 Out[8]= 1396 The function memTest1 could of course be written without using the function aj[j_], in which case no memory is lost. Mathematica 3.0.1 performs in the same way, only worse, since 2-3 times more memory is used up. Jarl R Sobel