Memory leak or flawed garbage collector
- To: mathgroup at smc.vnet.net
- Subject: [mg120576] Memory leak or flawed garbage collector
- From: "Fabrice P. Laussy" <fabrice.laussy at gmail.com>
- Date: Fri, 29 Jul 2011 08:03:01 -0400 (EDT)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
Dear Group,
I have a problem of garbage collection or of memory leak, with a
module that will seemingly not discard its internal variables, thus
leading to explosion of memory usage by repeated calls to this module.
Opening the notebook, nothing else being done yet:
In[1]:= MemoryInUse[]
Out[1]= 18460176
Then I define the said module, with private variables, etc. Before
running the first instance of it, already there appears many
variables, some defined globally, others with the $ of internal
variables:
?Global`*
Global`
A coef Disa$ g2 K\[Sigma]3$ L\[Sigma]1 mybasis n\[Sigma]3 re$ X \[Gamma] \[Nu]\[Nu] \[Omega]3$
AllM coefa Dis\[Sigma]1 g3 L L\[Sigma]1$ mybasis$ n\[Sigma]3$ solMean Xsol \[Gamma]1 \[Nu]\[Nu]\[Nu] \[Omega]a
AllM$ coefa$ Dis\[Sigma]1$ i La L\[Sigma]2 myvar n$ solMean$ Xsol$ \[Gamma]2 \[Nu]\[Nu]\[Nu]$ \[Omega]a$
Av coef\[Sigma]1 eig im La$ L\[Sigma]2$ m\[CapitalDelta] order Spea x$ \[Gamma]3 \[Nu]\[Nu]$ \[Omega]max
Average coef\[Sigma]1$ eigV im$ lcor L\[Sigma]3 m$ out Spea$ X$ \[Gamma]a \[Nu]$ \[Omega]min
Average$ coef\[Sigma]2 eigV$ Ind lcor$ L\[Sigma]3$ M$ out$ Spectra3dots y \[CapitalDelta] \[CapitalSigma]1
Av$ coef\[Sigma]2$ eig$ Ind$ lmean L$ n P Spe\[Sigma]1 y$ \[CapitalDelta]1 \[CapitalSigma]1$
A$ coef\[Sigma]3 eme i$ lmean0 m na P1 Spe\[Sigma]1$ \[Alpha] \[CapitalDelta]2 \[CapitalSigma]2
broad coef\[Sigma]3$ emeB j lmean0$ M na$ P2 Spe\[Sigma]2 \[Alpha]a \[CapitalDelta]3 \[CapitalSigma]2$
broad$ coef$ emeB$ j$ lmean$ M1 nmax P3 Spe\[Sigma]2$ \[Alpha]a$ \[Delta]\[Omega] \[CapitalSigma]3
c Corr eme$ Ka Lora M1$ nn Pa Spe\[Sigma]3 \[Alpha]\[Sigma]1 \[Mu] \[CapitalSigma]3$
case CorrSpec ene Ka$ Lora$ M2 nn$ re Spe\[Sigma]3$ \[Alpha]\[Sigma]1$ \[Mu]\[Mu] \[Omega]
Change CorrSpec$ ene$ K\[Sigma]1 Lor\[Sigma]1 M2$ nt regmat Tota \[Alpha]\[Sigma]2 \[Mu]\[Mu]\[Mu] \[Omega]1
ChangeBack Corr$ f1 K\[Sigma]1$ Lor\[Sigma]1$ Me n\[Sigma]1 RegMat3dots Tota$ \[Alpha]\[Sigma]2$ \[Mu]\[Mu]\[Mu]$
\[Omega]1$
ChangeBack$ Co$ f1$ K\[Sigma]2 ls Me$ n\[Sigma]1$ RegMatrix Tot\[Sigma]1 \[Alpha]\[Sigma]3 \[Mu]\[Mu]$ \[Omega]2
Change$ c$ fs K\[Sigma]2$ ls$ mm n\[Sigma]2 RegMatrix$ Tot\[Sigma]1$ \[Alpha]\[Sigma]3$ \[Mu]$ \[Omega]2$
Co Disa g1 K\[Sigma]3 l\[CapitalDelta]1 mm$ n\[Sigma]2$ regmat$ x \[Alpha]$ \[Nu] \[Omega]3
After running the first instance of the module:
In[32]:= MemoryInUse[]
Out[32]= 26536288
After running the second instance:
In[35]:= MemoryInUse[]
Out[35]= 32878688
Etc., the memory will keep increasing, although each module runs a
separate case and could be computed in separate notebooks, thus the
amount of required memory should be the same.
The variables indeed keep cluttering things from the various instances
of the module:
?Global`*
Global`
A coef\[Sigma]1 eig$ i$ lcor$ m n P3 Spe\[Sigma]1 \[Alpha]a$ \[Mu]\[Mu]\[Mu] \[Omega]1$632
AllM coef\[Sigma]1$ eme i$646 lcor$646 M na Pa Spe\[Sigma]1$ \[Alpha]\[Sigma]1 \[Mu]\[Mu]\[Mu]$ \[Omega]1$647
AllM$ coef\[Sigma]2 emeB j lmean M1 na$ re Spe\[Sigma]2 \[Alpha]\[Sigma]1$ \[Mu]\[Mu]$ \[Omega]1$648
Av coef\[Sigma]2$ emeB$ j$ lmean0 M1$ nmax regmat Spe\[Sigma]2$ \[Alpha]\[Sigma]2 \[Mu]$ \[Omega]2
Average coef\[Sigma]3 eme$ Ka lmean0$ M2 nn RegMat3dots Spe\[Sigma]3 \[Alpha]\[Sigma]2$ \[Nu] \[Omega]2$
Average$ coef\[Sigma]3$ ene Ka$ lmean$ M2$ nn$ RegMatrix Spe\[Sigma]3$ \[Alpha]\[Sigma]3 \[Nu]\[Nu] \[Omega]2$630
Av$ coef$ ene$ Ka$629 Lora Me nt RegMatrix$ Tota \[Alpha]\[Sigma]3$ \[Nu]\[Nu]\[Nu] \[Omega]2$632
A$ Corr f1 Ka$646 Lora$ Me$ n\[Sigma]1 RegMatrix$630 Tota$ \[Alpha]$ \[Nu]\[Nu]\[Nu]$ \[Omega]2$647
broad CorrSpec f1$ K\[Sigma]1 Lor\[Sigma]1 mm n\[Sigma]1$ RegMatrix$632 Tot\[Sigma]1 \[Gamma] \[Nu]\[Nu]$
\[Omega]2$648
broad$ CorrSpec$ fs K\[Sigma]1$ Lor\[Sigma]1$ mm$ n\[Sigma]2 RegMatrix$647 Tot\[Sigma]1$ \[Gamma]1 \[Nu]$
\[Omega]3
c Corr$ g1 K\[Sigma]2 ls mybasis n\[Sigma]2$ RegMatrix$648 x \[Gamma]2 \[CapitalSigma]1 \[Omega]3$
case Co$ g2 K\[Sigma]2$ ls$ mybasis$ n\[Sigma]3 regmat$ X \[Gamma]3 \[CapitalSigma]1$ \[Omega]a
Change c$ g3 K\[Sigma]3 l\[CapitalDelta]1 myvar n\[Sigma]3$ re$ Xsol \[Gamma]a \[CapitalSigma]2 \[Omega]a$
ChangeBack Disa i K\[Sigma]3$ L\[Sigma]1 m\[CapitalDelta] n$ re$629 Xsol$ \[CapitalDelta] \[CapitalSigma]2$ \[Omega]a$630
ChangeBack$ Disa$ im L L\[Sigma]1$ m$ order re$646 x$ \[CapitalDelta]1 \[CapitalSigma]3 \[Omega]a$632
Change$ Dis\[Sigma]1 im$ La L\[Sigma]2 M$ out solMean X$ \[CapitalDelta]2 \[CapitalSigma]3$ \[Omega]a$647
Co Dis\[Sigma]1$ im$629 La$ L\[Sigma]2$ M$630 out$ solMean$ y \[CapitalDelta]3 \[Omega] \[Omega]a$648
coef eig im$646 La$629 L\[Sigma]3 M$632 P Spea y$ \[Delta]\[Omega] \[Omega]1 \[Omega]max
coefa eigV Ind La$646 L\[Sigma]3$ M$647 P1 Spea$ \[Alpha] \[Mu] \[Omega]1$ \[Omega]min
coefa$ eigV$ Ind$ lcor L$ M$648 P2 Spectra3dots \[Alpha]a \[Mu]\[Mu] \[Omega]1$630
Now, if we take for instance, RegMatrix$632, which appears in the list above. It seems to take no memory:
In[39]:= ByteCount[RegMatrix$632]
Out[39]= 0
But it clearly is stored in memory:
?RegMatrix$632
Global`RegMatrix$632
Attributes[RegMatrix$632]={Temporary}
RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,-1,0,0,0,0,0,0}]=0.
RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,0,0,0,0,0,1,0}]=0
RegMatrix$632[{0,0,0,0,0,0,0,0},{-1,0,0,0,1,0,0,0}]=0
[snip]
(a lot of more stuff here), and if I remove it:
In[47]:= MemoryInUse[]
Out[47]= 32879624
In[48]:= Remove[RegMatrix$632]
In[49]:= MemoryInUse[]
Out[49]= 29521760
So clearly, along with all the other stuff lying around, it accounts
for this filling up of the memory. It seems there is a problem with
garbage collection and disposing of local variables defined within a
module.
I don't think there is anything special with the module itself. It
calls other modules defined externally. At the same time the problem
does not seem reproducible defining a toy-module and looking for the
same behaviour. Does this problem seem familiar? Strange? What could
be causing it?