MathGroup Archive 2004

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Critical memory leak with J/Link

  • To: mathgroup at smc.vnet.net
  • Subject: [mg47603] Re: [mg47530] Critical memory leak with J/Link
  • From: Todd Gayley <tgayley at wolfram.com>
  • Date: Sat, 17 Apr 2004 02:31:43 -0400 (EDT)
  • Sender: owner-wri-mathgroup at wolfram.com

At 02:39 AM 4/15/2004, J. McKenzie Alexander wrote:
>I've encountered a curious, and completely disabling, memory leak when
>I use J/Link.  The memory leak is such that a program can easily
>consume over 1.5GB of virtual memory and will crash the system if left
>to run for an extended period of time.  The leak occurs under both
>Windows XP and Mac OS X (Panther), with both machines running
>Mathematica 5.0 (J/Link version 2.1.1).  The XP box has Sun Java
>1.4.1_03 (build 2) installed, and the OS X box has java 1.4.2_03 (build
>1.4.2_03-117.1) installed.
>
>Here's the simplest example I've found.
>
>         // MemoryLeaker.java
>         import com.wolfram.jlink.*;
>         public class MemoryLeaker {
>             Expr expr;
>             public MemoryLeaker(Expr e) {
>                 expr = e;
>             }
>             public Expr getExpr() {
>                 return expr;
>             }
>         }
>
>Once MemoryLeaker is compiled, load it into Mathematica and use it as
>follows (adjusting the AddToClassPath command as necessary, of course).
>
>         In[1]:=
>         Needs["JLink`"];
>         <<DiscreteMath`Combinatorica`
>
>         In[3]:=
>         AddToClassPath["/Users/jalex/Source/tmp/"];
>
>         In[4]:=
>         LoadJavaClass["MemoryLeaker"];
>
>         In[5]:=
>         g=GridGraph[50,50];
>
>         In[6]:=
>         memoryLeaker=JavaNew["MemoryLeaker",
>               NormalizeVertices[g]
>               ];
>
>         In[7]:=
>         Do[
>             memoryLeaker@getExpr[],{50}
>             ];
>
>The above run leaves J/Link, on my OS X box, in the following state:
>
>ID              Name    %CPU    Real Memory     Virtual Memory
>1287    J/Link  0.00            265.50 MB       823.50 MB
>
>Memory is not freed with calls to JavaGC[].  It is only freed with
>QuitJava[] or quitting the kernel.
>
>The real kicker is that if you replace line 6 with the following:
>
>         In[6]:=
>         memoryLeaker=JavaNew["MemoryLeaker", g ];
>
>no leak occurs.  However, if you replace line 6 with
>
>         In[6]:=
>         g2 = NormalizeVertices[g];
>         memoryLeaker = JavaNew["MemoryLeaker", g2];
>
>you still get the memory leak.
>
>Would someone please explain what I'm doing wrong in the above example
>or, if I'm not doing anything wrong, a work-around?


Jason,

Thanks for this bug report. We are studying the leak right now, but we know 
that it is down inside MathLink itself and has to do with calling 
MLTransferExpression() to move certain unusual types of data between links. 
The leak can be easily duplicated in a C-language MathLink program.

Until this is fixed, there is a partial workaround in your J/Link program 
that might be sufficient for you. You need to change the way the Expr is 
stored internally so that MLTransferExpression() is not used to send it to 
Mathematica. This can be done by calling any Expr method that "unwinds" the 
expression data off of its highly optimized cache on an internal loopback 
link. Any Expr method that inspects the expression in any way will perform 
this unwinding; here is an example that uses length():

     // MemoryLeaker.java
     import com.wolfram.jlink.*;
     public class MemoryLeaker {
         Expr expr;
         public MemoryLeaker(Expr e) {
             expr = e;
             // ADDED LINE. This has the side effect of unwinding the
             // expression data off the internal loopback link:
             expr.length();
         }
         public Expr getExpr() {
             return expr;
         }
     }

This will make the movement of the data from Java to Mathematica not leak 
memory (it will also make it a bit slower, but this will probably not be a 
concern). Unfortunately, the initial send of the expression from 
Mathematica to Java still uses MLTransferExpression() internally and this 
cannot be changed. If you are only sending the data to Java once, and 
retrieving it many times, the fix provided above will be enough for you, 
but if you are sending expressions from Mathematica to Java as many times 
as the other way around, then you will still have a problem.

If the above workaround is not sufficient for you, and you cannot wait for 
a fix to show up in MathLink, then contact me directly and we can discuss 
more complicated techniques, such as avoiding the Expr class entirely.


Todd Gayley
Wolfram Research 


  • Prev by Date: Re: Returning rules ...
  • Next by Date: plot combination
  • Previous by thread: Re: Critical memory leak with J/Link
  • Next by thread: DisplayForm changes precission