Re: Critical memory leak with J/Link
- To: mathgroup at smc.vnet.net
- Subject: [mg47603] Re: [mg47530] Critical memory leak with J/Link
- From: Todd Gayley <tgayley at wolfram.com>
- Date: Sat, 17 Apr 2004 02:31:43 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
At 02:39 AM 4/15/2004, J. McKenzie Alexander wrote:
>I've encountered a curious, and completely disabling, memory leak when
>I use J/Link. The memory leak is such that a program can easily
>consume over 1.5GB of virtual memory and will crash the system if left
>to run for an extended period of time. The leak occurs under both
>Windows XP and Mac OS X (Panther), with both machines running
>Mathematica 5.0 (J/Link version 2.1.1). The XP box has Sun Java
>1.4.1_03 (build 2) installed, and the OS X box has java 1.4.2_03 (build
>1.4.2_03-117.1) installed.
>
>Here's the simplest example I've found.
>
> // MemoryLeaker.java
> import com.wolfram.jlink.*;
> public class MemoryLeaker {
> Expr expr;
> public MemoryLeaker(Expr e) {
> expr = e;
> }
> public Expr getExpr() {
> return expr;
> }
> }
>
>Once MemoryLeaker is compiled, load it into Mathematica and use it as
>follows (adjusting the AddToClassPath command as necessary, of course).
>
> In[1]:=
> Needs["JLink`"];
> <<DiscreteMath`Combinatorica`
>
> In[3]:=
> AddToClassPath["/Users/jalex/Source/tmp/"];
>
> In[4]:=
> LoadJavaClass["MemoryLeaker"];
>
> In[5]:=
> g=GridGraph[50,50];
>
> In[6]:=
> memoryLeaker=JavaNew["MemoryLeaker",
> NormalizeVertices[g]
> ];
>
> In[7]:=
> Do[
> memoryLeaker@getExpr[],{50}
> ];
>
>The above run leaves J/Link, on my OS X box, in the following state:
>
>ID Name %CPU Real Memory Virtual Memory
>1287 J/Link 0.00 265.50 MB 823.50 MB
>
>Memory is not freed with calls to JavaGC[]. It is only freed with
>QuitJava[] or quitting the kernel.
>
>The real kicker is that if you replace line 6 with the following:
>
> In[6]:=
> memoryLeaker=JavaNew["MemoryLeaker", g ];
>
>no leak occurs. However, if you replace line 6 with
>
> In[6]:=
> g2 = NormalizeVertices[g];
> memoryLeaker = JavaNew["MemoryLeaker", g2];
>
>you still get the memory leak.
>
>Would someone please explain what I'm doing wrong in the above example
>or, if I'm not doing anything wrong, a work-around?
Jason,
Thanks for this bug report. We are studying the leak right now, but we know
that it is down inside MathLink itself and has to do with calling
MLTransferExpression() to move certain unusual types of data between links.
The leak can be easily duplicated in a C-language MathLink program.
Until this is fixed, there is a partial workaround in your J/Link program
that might be sufficient for you. You need to change the way the Expr is
stored internally so that MLTransferExpression() is not used to send it to
Mathematica. This can be done by calling any Expr method that "unwinds" the
expression data off of its highly optimized cache on an internal loopback
link. Any Expr method that inspects the expression in any way will perform
this unwinding; here is an example that uses length():
// MemoryLeaker.java
import com.wolfram.jlink.*;
public class MemoryLeaker {
Expr expr;
public MemoryLeaker(Expr e) {
expr = e;
// ADDED LINE. This has the side effect of unwinding the
// expression data off the internal loopback link:
expr.length();
}
public Expr getExpr() {
return expr;
}
}
This will make the movement of the data from Java to Mathematica not leak
memory (it will also make it a bit slower, but this will probably not be a
concern). Unfortunately, the initial send of the expression from
Mathematica to Java still uses MLTransferExpression() internally and this
cannot be changed. If you are only sending the data to Java once, and
retrieving it many times, the fix provided above will be enough for you,
but if you are sending expressions from Mathematica to Java as many times
as the other way around, then you will still have a problem.
If the above workaround is not sufficient for you, and you cannot wait for
a fix to show up in MathLink, then contact me directly and we can discuss
more complicated techniques, such as avoiding the Expr class entirely.
Todd Gayley
Wolfram Research