Re: Memory leak or flawed garbage collector
- To: mathgroup at smc.vnet.net
- Subject: [mg120588] Re: Memory leak or flawed garbage collector
- From: "Oleksandr Rasputinov" <oleksandr_rasputinov at hmamail.com>
- Date: Sat, 30 Jul 2011 06:00:36 -0400 (EDT)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
- References: <j0u7mo$fc1$1@smc.vnet.net>
Dear Fabrice
Mathematica's garbage collection is based on reference counting. Thus, if
one creates a reference to a Temporary symbol, whether directly or
indirectly, the symbol cannot be garbage-collected until the reference is
removed. Here is an example:
In[1] :=
Module[{a}, a]
Out[1] =
a$592
In[2] :=
Names["Global`*"]
Out[2] =
{a, a$592}
Thus we see that the symbol Out has a DownValue (i.e. Out[1]) which
references the internal symbol a$592. This reference is counted and thus
prevents garbage collection. To clear it up we may do the following:
In[3] :=
Unprotect[Out];
Clear[Out];
Protect[Out];
In[6] :=
Names["Global`*"]
Out[6] =
{a}
Of course, it is quite possible that the references to your symbols
responsible for their not being garbage-collected are somewhere other than
the DownValues of In or Out, but if you find and remove these, you should
find that the symbols are garbage-collected (unless for some reason they
have had their Temporary attribute removed during the course of
execution). If this does not occur after all references have been removed,
then indeed a bug in the garbage collector would indeed be a possibility.
(It is conceivable that references are not counted immediately, as would
be the case in a mark-sweep garbage collector, but as far as I know such
mechanisms are not in fact employed as far as the interpreter is
concerned; they would likely be rather difficult to implement in a
term-rewriting language such as Mathematica.)
Best,
O. R.
On Fri, 29 Jul 2011 13:05:44 +0100, Fabrice P. Laussy
<fabrice.laussy at gmail.com> wrote:
> Dear Group,
>
> I have a problem of garbage collection or of memory leak, with a
> module that will seemingly not discard its internal variables, thus
> leading to explosion of memory usage by repeated calls to this module.
>
> Opening the notebook, nothing else being done yet:
>
> In[1]:= MemoryInUse[]
> Out[1]= 18460176
>
> Then I define the said module, with private variables, etc. Before
> running the first instance of it, already there appears many
> variables, some defined globally, others with the $ of internal
> variables:
>
> ?Global`*
>
> Global`
> <many symbols>
>
> After running the first instance of the module:
>
> In[32]:= MemoryInUse[]
> Out[32]= 26536288
>
> After running the second instance:
>
> In[35]:= MemoryInUse[]
> Out[35]= 32878688
>
> Etc., the memory will keep increasing, although each module runs a
> separate case and could be computed in separate notebooks, thus the
> amount of required memory should be the same.
>
> The variables indeed keep cluttering things from the various instances
> of the module:
>
> ?Global`*
>
> Global`
> <more symbols>
>
> Now, if we take for instance, RegMatrix$632, which appears in the list
> above. It seems to take no memory:
>
> In[39]:= ByteCount[RegMatrix$632]
> Out[39]= 0
>
> But it clearly is stored in memory:
>
> ?RegMatrix$632
> Global`RegMatrix$632
> Attributes[RegMatrix$632]={Temporary}
>
> RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,-1,0,0,0,0,0,0}]=0.
> RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,0,0,0,0,0,1,0}]=0
> RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,0,0,0,1,0,0,0}]=0
>
> [snip]
>
> (a lot of more stuff here), and if I remove it:
>
> In[47]:= MemoryInUse[]
> Out[47]= 32879624
>
> In[48]:= Remove[RegMatrix$632]
>
> In[49]:= MemoryInUse[]
> Out[49]= 29521760
>
> So clearly, along with all the other stuff lying around, it accounts
> for this filling up of the memory. It seems there is a problem with
> garbage collection and disposing of local variables defined within a
> module.
>
> I don't think there is anything special with the module itself. It
> calls other modules defined externally. At the same time the problem
> does not seem reproducible defining a toy-module and looking for the
> same behaviour. Does this problem seem familiar? Strange? What could
> be causing it?