Memory leak - Table & Export
- To: mathgroup at smc.vnet.net
- Subject: [mg68740] Memory leak - Table & Export
- From: "Mark Van De Vyver" <mvyver at gmail.com>
- Date: Thu, 17 Aug 2006 04:18:52 -0400 (EDT)
- Reply-to: mvdv at spamcop.net
- Sender: owner-wri-mathgroup at wolfram.com
Hi,
Some of the following code is specific to Windows XP professional,
i.e. the use of <<"!tasklist...".
If the following behavior is not due to my ignorance of Mathematica I'll
submit a bug report to Wolfram.
I'd appreciate any suggestions about:
a) How to stop Mathematica leaking memory in the following code?
b) Why the MemoryInUse[] results are so vastly different from what I
see in the windows taskmanager, and how can I get get the 'real'
memory in use to be the number that the Mathematica function MemoryInUse[]
reports.
I came across the following behavior while importing some data files,
checking if they contained symbols and then exporting them. However
the following code shows the same type of behavior.
Some observations.
Share[] and ClearCache[] seem to have negligible effects.
MemoryInUse[] understaes the memory actually in use by the kernel;
initially by a factor of close to 3 which eventually grows to a factor
of close to 10!
This renders Mathematica unusable for data analysis in situations that involve
more than trivial sized datasets.
I've shown the first and last line of the Mathematica output and done the same
for the temp.txt file that is written to in the For[...] loop.
Thanks in advance
Regards
Mark
(* -------------- Start mma input lines -------------- *)
$HistoryLength = 0;
$Version
<< Developer`
<< Utilities`MemoryConserve`
SetDirectory["d:\\temp"];
<< "!tasklist /FI \"ImageName eq MathKernel.exe\">temp.txt";
For[idx =1,idx<=15,idx++,
data=Table[Random[],{i,1000},{j,50*idx}];
Export[ToString[idx]<>".csv",data,"CSV"];
Print[ MemoryInUse[]];
time=AbsoluteTiming[Share[];];
Print[time,", ", MemoryInUse[]];
time=AbsoluteTiming[ClearCache[];];
Print[time,", ", MemoryInUse[]];
<<"!tasklist /FI \"ImageName eq MathKernel.exe\">>temp.txt";
]
(* -------------- End mma input lines -------------- *)
(* -------------- Start mma output lines -------------- *)
Out[2]=
5.2 for Microsoft Windows (June 20, 2005)
[ snip ]
From In[8]:=
3651312
From In[8]:=
{0.0781250 Second,Null}\[InvisibleSpace], \[InvisibleSpace]3422968
From In[8]:=
{0. Second,Null}\[InvisibleSpace], \[InvisibleSpace]3425832
[ snip ]
From In[8]:=
9020464
From In[8]:=
{0.2500000 Second,Null}\[InvisibleSpace], \[InvisibleSpace]9020496
From In[8]:=
{0. Second,Null}\[InvisibleSpace], \[InvisibleSpace]9020496
(* -------------- End mma output lines -------------- *)
(* -------------- Start temp.txt output lines -------------- *)
Image Name PID Session Name Session# Mem Usage
========================= ====== ================ ======== ============
MathKernel.exe 276 Console 0 10,760 K
[ snip ]
Image Name PID Session Name Session# Mem Usage
========================= ====== ================ ======== ============
MathKernel.exe 276 Console 0 87,936 K
(* -------------- Start temp.txt output lines -------------- *)