WOLFRAM

Services & Resources / Wolfram Forums / MathGroup Archive

MathGroup Archive 2005

[Date Index] [Thread Index] [Author Index]

Search the Archive

[long] Mathematica 5.1 and memory: any garbage collection?

  • To: mathgroup at smc.vnet.net
  • Subject: [mg59697] [long] Mathematica 5.1 and memory: any garbage collection?
  • From: some.poster at use.net
  • Date: Thu, 18 Aug 2005 00:16:30 -0400 (EDT)
  • Sender: owner-wri-mathgroup at wolfram.com
Hi,

I've been using Mathematica v5.1 under XP for some time now and found it largely
improved over previous versions, at least for my problems.

Only recently did I begin to handle some real data: rather large List
objects. I have to routinely select objects within lists as large as, say,
46000 * 512 * 20 items, items themselves not being atomic.

Of course I never expected to be able to use functionnal style for dealing
with such large guys on a 1Gb RAM PC: an outer For loop hands one object
(out of 46000) at a time, then functionnal style handling follows. In this
process I create variable-sized intermediate objects that are either
accumulated in decimated lists or discarded altogether.

It turns out that my typical intermediate objects eat up to 2.5 Mb each.
This sounds a little of a memory hog, but anyway, I can live with that.

My problem lies with the fact that memory used at intermediate stage seems
never be freed, even whith explicit Clear, Remove or any assignment to {} or
. in whatever order.

How can one recover the memory freed (if some is ever effectively freed)? Is
there _any_ garbage collection in the guts of this gazillion dollars piece
of code?

I have no problem with the idea of regularly invoking some function to
compact the heap (should the kernel be unable to perform it by itself). The
question is: what's its name?

How do you reclaim memory, you serious users who do computations on large
objects with numerous intermediate values?


At the end, I tried this, from a fresh kernel load:

In[1]:=
$ProductInformation

Out[1]=
{ProductIDName\[Rule]Mathematica,
  ProductKernelName\[Rule]Mathematica 5.1 Kernel,
  ProductVersion\[Rule]5.1 for Microsoft Windows (October 25, 2004),
  ProductVersionNumber\[Rule]5.1}

In[2]:=
MemoryInUse[]

Out[2]=
2124904

In[3]:=
(* let's create a moderate-size object *)
try=Tuples[{a,b,c,d,e,f},7];
ByteCount[try]
{MemoryInUse[],MaxMemoryUsed[]}

Out[4]=
14556704

Out[5]=
{16684640,16685792}

In[6]:=
(* simple mapped function *)
res=Map[2*#&,try];
{MemoryInUse[],MaxMemoryUsed[]}

Out[7]=
{93949328,103186832}                    (* Ouch ! So much for a _simple_ test *)

In[8]:=
(* forgetting a large expression doesn't free _any_ memory *)
try={};
{MemoryInUse[],MaxMemoryUsed[]}

Out[9]=
{93951056,103186832}

In[10]:=
(* no more success this way *)
try=.;
{MemoryInUse[],MaxMemoryUsed[]}

Out[11]=
{93952648,103186832}

In[12]:=
(* ... neither a sweep does *)
Clear[try];
{MemoryInUse[],MaxMemoryUsed[]}

Out[13]=
{93954320,103186832}

In[14]:=
(* ... nor a complete wash *)
Remove[try];
{MemoryInUse[],MaxMemoryUsed[]}

Out[15]=
{93955944,103186832}

In[16]:=
(* freeing the result also seems beyond possible *)
res={};
{MemoryInUse[],MaxMemoryUsed[]}

Out[17]=
{93958112,103186832}

In[18]:=
(* still no luck, dad *)
res=.;
{MemoryInUse[],MaxMemoryUsed[]}

Out[19]=
{93958952,103186832}

In[20]:=
(* ahem, what _is_ Clear supposed to do? *)
Clear[res];
{MemoryInUse[],MaxMemoryUsed[]}

Out[21]=
{93960448,103186832}

In[22]:=
(* ahem, what _is_ Remove supposed to do? *)
Remove[res];
{MemoryInUse[],MaxMemoryUsed[]}

Out[23]=
{93961936,103186832}

In[24]:=
(* sharing common subexpressions somehow works, slow as hell *)

(* please note: even after removing all values and symbols in all
   possible ways, try and res are _still_ somewhere in memory since
   Share _really_ does something! *)

(* Share should share nothing as we don't have any object stored, safe
perhaps symbols a..f *)

(* when we use larger objects or more intermediate results like res
   we get the kernel to shutdown very quickly at this rate *)
Share[]
{MemoryInUse[],MaxMemoryUsed[]}

Out[24]=
62824872

Out[25]=
{31138184,103186832}

(* did you expect to see about the same value as on the first call above? *)
  • Prev by Date: Re: About Simplify
  • Next by Date: Re: Re: How to specify boundary conditions on all 4 sides of a plate for a steady state heat equation (PDE) using NDSolve? (Laplace equation)
  • Previous by thread: Re: FindRoot for the determinant of a matrix with a varying size
  • Next by thread: Re: [long] Mathematica 5.1 and memory: any garbage collection?