Re: Memory Leak with KSubsets?
- To: mathgroup at smc.vnet.net
- Subject: [mg38385] Re: [mg38275] Memory Leak with KSubsets?
- From: Rob Pratt <rpratt at email.unc.edu>
- Date: Fri, 13 Dec 2002 04:10:01 -0500 (EST)
- Sender: owner-wri-mathgroup at wolfram.com
On Tue, 10 Dec 2002, Arny wrote: > Dearest group > > Has anyone noticed this before? I would like to use KSubsets and > RandomKSubsets a few million times in a proggy, but after executing it a few > thousand times Mathematica crashes, running out of memory. Watching my Taskman (Win > 2K, Mathematica 4.1) we see a constant increase in memory used, until its all soaked > up. I have this problem with both the Combinatorica.m that shipped with > Mathematica, and the new one that has been written, which I believe is now the one > in Mathematica 4.2. I am not really sure what it means to 'leak memory', but this > seems to fit the bill. > > Here is a crashing-demo. Executing "memorySwallower[3,100]", for example, > repeatedly just soaks up more and more memory. Pourquoi? I believe the > same thing happens if I use KSubsets also. > > In[1]:= > MemoryInUse[] > Get["DiscreteMath`Combinatorica`"] > MemoryInUse[] > Out[1]= > 1398624 > Out[3]= > 1956536 > > memorySwallower[S_,n_]:= > Module[{data,subsamples},Print[ToString[MemoryInUse[]]]; > Table[data=Table[{i,j},{i,1000},{j,1+m}]; > subsamples=Table[RandomKSubset[data,S],{100}],{m,n}]; > Print[ToString[MemoryInUse[]]]] > > > memorySwallower[3,100] > > 15497936 > 29833672 > > Regards, > B > _____________________________ > Bernard Gress > Department of Economics > University of California, Riverside > 1150 University Ave. > Riverside, CA 92521-0247 > Fax: (909) 787-5685 > Phone: (909) 778 9813 > BGRESSatMAILdotUCRdotEDU > ICQ: 9083461 > http://www.economics.ucr.edu/people/candidates.html I forwarded this message to Steven Skiena, the developer of Combinatorica, who received the following reply from wri: --- Well, I can confirm that the problem isn't Combinatorica, since the following: memorySwallower[S_, n_] := Module[{data, subsamples}, Print[ToString[MemoryInUse[]]]; Table[data = Table[{i, j}, {i, 1000}, {j, 1 + m}]; subsamples = Table[Take[data, S], {100}], {m, n}]; Print[ToString[MemoryInUse[]]]] has the same memory consumption behavior. (Replaced RandomKSubset with Take, which returns a list of the same length...) You can at least pass that much along to the user to exonerate yourself... :-) Actually, looking at this example some more, the problem is simply that the result (the full table, which is 13 MB or so in size) is getting cached in the command output history. At the time of the second MemoryInUse call, the table has not been deleted; and because Print returns Null, the value of the table is left behind to get cached in Out (the value that gets stored in Out is the last non-Null value from the compound expression). So, for example, if you were to evaluate memorySwallower[3, 100]; dummy; instead of just the memorySwallower command, 'dummy' gets put into the output history instead of the table, and successive calls to this show the memory does not continue increasing. (The second call is still 15 MB higher than the first call, since the table hasn't been deleted at that point...) Equivalently, if you write e.g.: memorySwallower[S_, n_] := Module[{data, subsamples}, Print[ToString[MemoryInUse[]]]; Table[data = Table[{i, j}, {i, 1000}, {j, 1 + m}]; subsamples = Table[Take[data, S], {100}], {m, n}]; dummy; Print[ToString[MemoryInUse[]]]] the non-null value is sufficient to release the memory required by the Table. --- Rob Pratt Department of Operations Research The University of North Carolina at Chapel Hill rpratt at email.unc.edu http://www.unc.edu/~rpratt/