Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: Critical memory leak with J/Link

• To: mathgroup at smc.vnet.net
• Subject: [mg47573] Re: [mg47530] Critical memory leak with J/Link
• From: "J. McKenzie Alexander" <jalex at lse.ac.uk>
• Date: Fri, 16 Apr 2004 05:20:54 -0400 (EDT)
• References: <200404150739.DAA11638@smc.vnet.net> <3235.129.27.203.84.1082029697.bloek@pwebmail.utanet.at>
• Sender: owner-wri-mathgroup at wolfram.com

>> Memory is not freed with calls to JavaGC[].  It is only freed with
>> QuitJava[] or quitting the kernel.
>
> JavaGC[] won't be of much help, if there is still a valid reference
> to the Object; in your case, the "g" expression still points to the
> MemoryHolder, which in turn still references the large Graph Expression
> that you passed in. This is, as far as I see it, correct behaviour
>  (Mathematica or J/Link doesn't know that you don't intend to use
> the "g" expression/reference anymore, so it keeps it).
>
> You can use either JavaBlock[] (wrap your code inside it) or one
> of the Release* Expressions (ReleaseObject[],...) to get rid of the
> reference (possibly followed by a JavaGC[] to make sure the GC frees 
> the
> memory).
>
> I suppose that should solve your problem (unless I'm missing
> something else).

Thanks for your reply, but I'm afraid the problem is a bit more subtle 
than it first appear. The problem doesn't concern how Mathematica keeps 
references to returned Java objects unless explicitly cleared with 
Scan[ReleaseJavaObject,PeekObjects[]] or something similar.  The Java 
object returned in the body of the Do[...] loop is the *same* Java 
object each time.  Mathematica realises this and does not make a copy 
of the object or continue to increase the reference count each time.  
If you look at the class definition of  MemoryLeaker, you'll see that 
it stores a reference to whatever single Expr object is passed to its 
constructor when it is created, and repeated calls to 
memoryLeaker@getExpr[] simply return a reference to that same object.

The problem is that, out of two *very* similar but slightly different 
sequences of commands, one causes a huge leak but the other doesn't. 
Compare the following sequence of commands:

	In[1]:=
	Needs["JLink`"];
	<<DiscreteMath`Combinatorica`

	In[3]:=
	AddToClassPath["/Users/jalex/Source/tmp/"];

	In[4]:=
	LoadJavaClass["MemoryLeaker"];

	In[5]:=
	g=GridGraph[50,50];

	In[6]:=
	memoryLeaker=JavaNew["MemoryLeaker",g];

	In[7]:=
	Do[
	    memoryLeaker@getExpr[],{50}
	    ];

	In[8]:=
	PeekObjects[]//Length

	Out[8]=
	1

State of J/Link process at end
=======================
Process	Name	%CPU	Real memory	Virtual memory
1490	 J/Link	0.00		20.71 MB		313.33 MB


Ok - fine.  In my experience this is normal memory usage.  Compare this 
with the following. The kernel was restarted between the above run and 
this run, and the cell tags indicate that the exact same sequence of 
command evaluation was followed.  Note that the *only* difference is at 
In[6], where instead of passing the GridGraph to the constructor, we 
pass the output of the Combinatorica function NormalizeVertices.


	In[1]:=
	Needs["JLink`"];
	<<DiscreteMath`Combinatorica`

	In[3]:=
	AddToClassPath["/Users/jalex/Source/tmp/"];

	In[4]:=
	LoadJavaClass["MemoryLeaker"];

	In[5]:=
	g=GridGraph[50,50];

	In[6]:=
	memoryLeaker=JavaNew["MemoryLeaker",
	      NormalizeVertices[g]
	      ];

	In[7]:=
	Do[
	    memoryLeaker@getExpr[],{50}
	    ];

	In[8]:=
	PeekObjects[]//Length

	Out[8]=
	1


State of J/Link process at end
=======================
Process	Name	%CPU	Real memory	Virtual memory
1492	 J/Link	0.50		279.81 MB	823.50 MB	


See the difference? That's a *huge* leak for no apparent reason.  And 
this isn't a bug in Apple's JVM because it's perfectly reproducible on 
a Windows XP box running Mathematica (as noted in my original message).

One may suspect that something funny is going on with the fact that we 
passed NormalizeVertices[g] to the constructor instead of a graph 
object itself.  Well, compare the second sequence of commands (the one 
we just saw) with the following.  Note that the only difference between 
the above and the below is that In[6] assigns the output of 
NormalizeVertices[g] to g2, and then we pass g2 to the MemoryLeaker 
constructor.  As before, the kernel was restarted between the above run 
and this one.  (Quitting the kernel destroys the J/Link connection and 
the memory is freed.)


	In[1]:=
	Needs["JLink`"];
	<<DiscreteMath`Combinatorica`

	In[3]:=
	AddToClassPath["/Users/jalex/Source/tmp/"];

	In[4]:=
	LoadJavaClass["MemoryLeaker"];

	In[5]:=
	g=GridGraph[50,50];
	g2=NormalizeVertices[g];

	In[7]:=
	memoryLeaker=JavaNew["MemoryLeaker",g2];

	In[8]:=
	Do[
	    memoryLeaker@getExpr[],{50}
	    ];

	In[9]:=
	PeekObjects[]//Length

	Out[9]=
	1


State of J/Link process at end
=======================
Process	Name	%CPU	Real memory	Virtual memory
1500	 J/Link	0.40		299.09 MB	823.50 MB


Any solutions or suggestions would be appreciated.  This phenomenon is 
killing my J/Link app.

Cheers,

Jason
--
Dr J. McKenzie Alexander
Department of Philosophy, Logic and Scientific Method
London School of Economics and Political Science
Houghton Street, London WC2A 2AE

• References:
  • Critical memory leak with J/Link
    • From: "J. McKenzie Alexander" <jalex@lse.ac.uk>