MathGroup Archive 2002

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Memory Leak with KSubsets?

  • To: mathgroup at smc.vnet.net
  • Subject: [mg38385] Re: [mg38275] Memory Leak with KSubsets?
  • From: Rob Pratt <rpratt at email.unc.edu>
  • Date: Fri, 13 Dec 2002 04:10:01 -0500 (EST)
  • Sender: owner-wri-mathgroup at wolfram.com

On Tue, 10 Dec 2002, Arny wrote:

> Dearest group
>
> Has anyone noticed this before?  I would like to use KSubsets and
> RandomKSubsets a few million times in a proggy, but after executing it a few
> thousand times Mathematica crashes, running out of memory.  Watching my Taskman (Win
> 2K, Mathematica 4.1) we see a constant increase in memory used, until its all soaked
> up.  I have this problem with both the Combinatorica.m that shipped with
> Mathematica, and the new one that has been written, which I believe is now the one
> in Mathematica 4.2.  I am not really sure what it means to 'leak memory', but this
> seems to fit the bill.
>
> Here is a crashing-demo.  Executing "memorySwallower[3,100]", for example,
> repeatedly just soaks up more and more memory.  Pourquoi?  I believe the
> same thing happens if I use KSubsets also.
>
> In[1]:=
> MemoryInUse[]
> Get["DiscreteMath`Combinatorica`"]
> MemoryInUse[]
> Out[1]=
> 1398624
> Out[3]=
> 1956536
>
> memorySwallower[S_,n_]:=
>   Module[{data,subsamples},Print[ToString[MemoryInUse[]]];
>     Table[data=Table[{i,j},{i,1000},{j,1+m}];
>       subsamples=Table[RandomKSubset[data,S],{100}],{m,n}];
>     Print[ToString[MemoryInUse[]]]]
>
>
> memorySwallower[3,100]
>
> 15497936
> 29833672
>
> Regards,
> B
> _____________________________
>   Bernard Gress
>   Department of Economics
>   University of California, Riverside
>   1150 University Ave.
>   Riverside, CA 92521-0247
>   Fax: (909) 787-5685
>   Phone: (909) 778 9813
>   BGRESSatMAILdotUCRdotEDU
>   ICQ: 9083461
>   http://www.economics.ucr.edu/people/candidates.html


I forwarded this message to Steven Skiena, the developer of Combinatorica,
who received the following reply from wri:

---

Well, I can confirm that the problem isn't Combinatorica, since
the following:

memorySwallower[S_, n_] :=
  Module[{data, subsamples}, Print[ToString[MemoryInUse[]]];
    Table[data = Table[{i, j}, {i, 1000}, {j, 1 + m}];
      subsamples = Table[Take[data, S], {100}], {m, n}];
    Print[ToString[MemoryInUse[]]]]

has the same memory consumption behavior. (Replaced RandomKSubset
with Take, which returns a list of the same length...) You can
at least pass that much along to the user to exonerate yourself... :-)

Actually, looking at this example some more, the problem is simply
that the result (the full table, which is 13 MB or so in size)
is getting cached in the command output history. At the time of the
second MemoryInUse call, the table has not been deleted; and because
Print returns Null, the value of the table is left behind to get
cached in Out (the value that gets stored in Out is the last non-Null
value from the compound expression). So, for example, if you were
to evaluate

memorySwallower[3, 100]; dummy;

instead of just the memorySwallower command, 'dummy' gets put into
the output history instead of the table, and successive calls to this
show the memory does not continue increasing. (The second call is
still 15 MB higher than the first call, since the table hasn't been
deleted at that point...) Equivalently, if you write e.g.:

memorySwallower[S_, n_] :=
  Module[{data, subsamples}, Print[ToString[MemoryInUse[]]];
    Table[data = Table[{i, j}, {i, 1000}, {j, 1 + m}];
      subsamples = Table[Take[data, S], {100}], {m, n}];
    dummy;
    Print[ToString[MemoryInUse[]]]]

the non-null value is sufficient to release the memory required
by the Table.

---

Rob Pratt
Department of Operations Research
The University of North Carolina at Chapel Hill

rpratt at email.unc.edu

http://www.unc.edu/~rpratt/



  • Prev by Date: Re: Question on factor group calculations
  • Next by Date: Re: Re: functions inside Module
  • Previous by thread: Re: Memory Leak with KSubsets?
  • Next by thread: Re: Memory Leak with KSubsets?