Re: Critical memory leak with J/Link

*To*: mathgroup at smc.vnet.net*Subject*: [mg47603] Re: [mg47530] Critical memory leak with J/Link*From*: Todd Gayley <tgayley at wolfram.com>*Date*: Sat, 17 Apr 2004 02:31:43 -0400 (EDT)*Sender*: owner-wri-mathgroup at wolfram.com

At 02:39 AM 4/15/2004, J. McKenzie Alexander wrote: >I've encountered a curious, and completely disabling, memory leak when >I use J/Link. The memory leak is such that a program can easily >consume over 1.5GB of virtual memory and will crash the system if left >to run for an extended period of time. The leak occurs under both >Windows XP and Mac OS X (Panther), with both machines running >Mathematica 5.0 (J/Link version 2.1.1). The XP box has Sun Java >1.4.1_03 (build 2) installed, and the OS X box has java 1.4.2_03 (build >1.4.2_03-117.1) installed. > >Here's the simplest example I've found. > > // MemoryLeaker.java > import com.wolfram.jlink.*; > public class MemoryLeaker { > Expr expr; > public MemoryLeaker(Expr e) { > expr = e; > } > public Expr getExpr() { > return expr; > } > } > >Once MemoryLeaker is compiled, load it into Mathematica and use it as >follows (adjusting the AddToClassPath command as necessary, of course). > > In[1]:= > Needs["JLink`"]; > <<DiscreteMath`Combinatorica` > > In[3]:= > AddToClassPath["/Users/jalex/Source/tmp/"]; > > In[4]:= > LoadJavaClass["MemoryLeaker"]; > > In[5]:= > g=GridGraph[50,50]; > > In[6]:= > memoryLeaker=JavaNew["MemoryLeaker", > NormalizeVertices[g] > ]; > > In[7]:= > Do[ > memoryLeaker@getExpr[],{50} > ]; > >The above run leaves J/Link, on my OS X box, in the following state: > >ID Name %CPU Real Memory Virtual Memory >1287 J/Link 0.00 265.50 MB 823.50 MB > >Memory is not freed with calls to JavaGC[]. It is only freed with >QuitJava[] or quitting the kernel. > >The real kicker is that if you replace line 6 with the following: > > In[6]:= > memoryLeaker=JavaNew["MemoryLeaker", g ]; > >no leak occurs. However, if you replace line 6 with > > In[6]:= > g2 = NormalizeVertices[g]; > memoryLeaker = JavaNew["MemoryLeaker", g2]; > >you still get the memory leak. > >Would someone please explain what I'm doing wrong in the above example >or, if I'm not doing anything wrong, a work-around? Jason, Thanks for this bug report. We are studying the leak right now, but we know that it is down inside MathLink itself and has to do with calling MLTransferExpression() to move certain unusual types of data between links. The leak can be easily duplicated in a C-language MathLink program. Until this is fixed, there is a partial workaround in your J/Link program that might be sufficient for you. You need to change the way the Expr is stored internally so that MLTransferExpression() is not used to send it to Mathematica. This can be done by calling any Expr method that "unwinds" the expression data off of its highly optimized cache on an internal loopback link. Any Expr method that inspects the expression in any way will perform this unwinding; here is an example that uses length(): // MemoryLeaker.java import com.wolfram.jlink.*; public class MemoryLeaker { Expr expr; public MemoryLeaker(Expr e) { expr = e; // ADDED LINE. This has the side effect of unwinding the // expression data off the internal loopback link: expr.length(); } public Expr getExpr() { return expr; } } This will make the movement of the data from Java to Mathematica not leak memory (it will also make it a bit slower, but this will probably not be a concern). Unfortunately, the initial send of the expression from Mathematica to Java still uses MLTransferExpression() internally and this cannot be changed. If you are only sending the data to Java once, and retrieving it many times, the fix provided above will be enough for you, but if you are sending expressions from Mathematica to Java as many times as the other way around, then you will still have a problem. If the above workaround is not sufficient for you, and you cannot wait for a fix to show up in MathLink, then contact me directly and we can discuss more complicated techniques, such as avoiding the Expr class entirely. Todd Gayley Wolfram Research