MathGroup Archive 2011

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Memory leak or flawed garbage collector

  • To: mathgroup at smc.vnet.net
  • Subject: [mg120588] Re: Memory leak or flawed garbage collector
  • From: "Oleksandr Rasputinov" <oleksandr_rasputinov at hmamail.com>
  • Date: Sat, 30 Jul 2011 06:00:36 -0400 (EDT)
  • Delivered-to: l-mathgroup@mail-archive0.wolfram.com
  • References: <j0u7mo$fc1$1@smc.vnet.net>

Dear Fabrice

Mathematica's garbage collection is based on reference counting. Thus, if  
one creates a reference to a Temporary symbol, whether directly or  
indirectly, the symbol cannot be garbage-collected until the reference is  
removed. Here is an example:

In[1] :=
Module[{a}, a]

Out[1] =
a$592

In[2] :=
Names["Global`*"]

Out[2] =
{a, a$592}

Thus we see that the symbol Out has a DownValue (i.e. Out[1]) which  
references the internal symbol a$592. This reference is counted and thus  
prevents garbage collection. To clear it up we may do the following:

In[3] :=
Unprotect[Out];
Clear[Out];
Protect[Out];

In[6] :=
Names["Global`*"]

Out[6] =
{a}

Of course, it is quite possible that the references to your symbols  
responsible for their not being garbage-collected are somewhere other than  
the DownValues of In or Out, but if you find and remove these, you should  
find that the symbols are garbage-collected (unless for some reason they  
have had their Temporary attribute removed during the course of  
execution). If this does not occur after all references have been removed,  
then indeed a bug in the garbage collector would indeed be a possibility.  
(It is conceivable that references are not counted immediately, as would  
be the case in a mark-sweep garbage collector, but as far as I know such  
mechanisms are not in fact employed as far as the interpreter is  
concerned; they would likely be rather difficult to implement in a  
term-rewriting language such as Mathematica.)

Best,

O. R.

On Fri, 29 Jul 2011 13:05:44 +0100, Fabrice P. Laussy  
<fabrice.laussy at gmail.com> wrote:

> Dear Group,
>
> I have a problem of garbage collection or of memory leak, with a
> module that will seemingly not discard its internal variables, thus
> leading to explosion of memory usage by repeated calls to this module.
>
> Opening the notebook, nothing else being done yet:
>
> 	In[1]:= MemoryInUse[]
> 	Out[1]= 18460176
>
> Then I define the said module, with private variables, etc. Before
> running the first instance of it, already there appears many
> variables, some defined globally, others with the $ of internal
> variables:
>
> 	?Global`*
> 	
> 	Global`
> 	<many symbols>
>
> After running the first instance of the module:
>
> 	In[32]:= MemoryInUse[]
> 	Out[32]= 26536288
>
> After running the second instance:
>
>         In[35]:= MemoryInUse[]
>         Out[35]= 32878688
>
> Etc., the memory will keep increasing, although each module runs a
> separate case and could be computed in separate notebooks, thus the
> amount of required memory should be the same.
>
> The variables indeed keep cluttering things from the various instances
> of the module:
>
> 	?Global`*
> 	
> 	Global`
>       <more symbols>
>
> Now, if we take for instance, RegMatrix$632, which appears in the list  
> above. It seems to take no memory:
>
>      	In[39]:= ByteCount[RegMatrix$632]
> 	Out[39]= 0
>
> But it clearly is stored in memory:
>
> 	?RegMatrix$632
> 	Global`RegMatrix$632
> 	Attributes[RegMatrix$632]={Temporary}
> 	
> 	RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,-1,0,0,0,0,0,0}]=0.
> 	RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,0,0,0,0,0,1,0}]=0
> 	RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,0,0,0,1,0,0,0}]=0
> 	
> 	[snip]
> 	
> (a lot of more stuff here), and if I remove it:
>
> 	In[47]:= MemoryInUse[]
> 	Out[47]= 32879624
> 	
> 	In[48]:= Remove[RegMatrix$632]
>
> 	In[49]:= MemoryInUse[]
> 	Out[49]= 29521760
>
> So clearly, along with all the other stuff lying around, it accounts
> for this filling up of the memory. It seems there is a problem with
> garbage collection and disposing of local variables defined within a
> module.
>
> I don't think there is anything special with the module itself. It
> calls other modules defined externally. At the same time the problem
> does not seem reproducible defining a toy-module and looking for the
> same behaviour. Does this problem seem familiar? Strange? What could
> be causing it?




  • Prev by Date: Re: FinancialData errors
  • Next by Date: Re: FinancialData errors
  • Previous by thread: Re: Memory leak or flawed garbage collector
  • Next by thread: Re: Memory leak or flawed garbage collector