Re: Memory leak - Table & Export
- To: mathgroup at smc.vnet.net
- Subject: [mg68771] Re: Memory leak - Table & Export
- From: Jens-Peer Kuska <kuska at informatik.uni-leipzig.de>
- Date: Fri, 18 Aug 2006 03:12:18 -0400 (EDT)
- Organization: A Customer of Tele2
- References: <ec1bl7$p65$1@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Hi,
just a guss -- Mathematica unpack the array for the Export[]
and forgott to repack it ??
Regards
Jens
Mark Van De Vyver wrote:
> Hi,
> Some of the following code is specific to Windows XP professional,
> i.e. the use of <<"!tasklist...".
>
> If the following behavior is not due to my ignorance of Mathematica I'll
> submit a bug report to Wolfram.
> I'd appreciate any suggestions about:
> a) How to stop Mathematica leaking memory in the following code?
> b) Why the MemoryInUse[] results are so vastly different from what I
> see in the windows taskmanager, and how can I get get the 'real'
> memory in use to be the number that the Mathematica function MemoryInUse[]
> reports.
>
> I came across the following behavior while importing some data files,
> checking if they contained symbols and then exporting them. However
> the following code shows the same type of behavior.
>
> Some observations.
> Share[] and ClearCache[] seem to have negligible effects.
> MemoryInUse[] understaes the memory actually in use by the kernel;
> initially by a factor of close to 3 which eventually grows to a factor
> of close to 10!
> This renders Mathematica unusable for data analysis in situations that involve
> more than trivial sized datasets.
>
> I've shown the first and last line of the Mathematica output and done the same
> for the temp.txt file that is written to in the For[...] loop.
>
> Thanks in advance
> Regards
> Mark
>
> (* -------------- Start mma input lines -------------- *)
>
> $HistoryLength = 0;
> $Version
> << Developer`
> << Utilities`MemoryConserve`
> SetDirectory["d:\\temp"];
> << "!tasklist /FI \"ImageName eq MathKernel.exe\">temp.txt";
>
> For[idx =1,idx<=15,idx++,
> data=Table[Random[],{i,1000},{j,50*idx}];
> Export[ToString[idx]<>".csv",data,"CSV"];
> Print[ MemoryInUse[]];
> time=AbsoluteTiming[Share[];];
> Print[time,", ", MemoryInUse[]];
> time=AbsoluteTiming[ClearCache[];];
> Print[time,", ", MemoryInUse[]];
> <<"!tasklist /FI \"ImageName eq MathKernel.exe\">>temp.txt";
> ]
>
> (* -------------- End mma input lines -------------- *)
>
> (* -------------- Start mma output lines -------------- *)
> Out[2]=
> 5.2 for Microsoft Windows (June 20, 2005)
>
> [ snip ]
>
> From In[8]:=
> 3651312
> From In[8]:=
> {0.0781250 Second,Null}\[InvisibleSpace], \[InvisibleSpace]3422968
> From In[8]:=
> {0. Second,Null}\[InvisibleSpace], \[InvisibleSpace]3425832
>
> [ snip ]
>
> From In[8]:=
> 9020464
> From In[8]:=
> {0.2500000 Second,Null}\[InvisibleSpace], \[InvisibleSpace]9020496
> From In[8]:=
> {0. Second,Null}\[InvisibleSpace], \[InvisibleSpace]9020496
>
> (* -------------- End mma output lines -------------- *)
>
> (* -------------- Start temp.txt output lines -------------- *)
>
> Image Name PID Session Name Session# Mem Usage
> ========================= ====== ================ ======== ============
> MathKernel.exe 276 Console 0 10,760 K
>
> [ snip ]
>
> Image Name PID Session Name Session# Mem Usage
> ========================= ====== ================ ======== ============
> MathKernel.exe 276 Console 0 87,936 K
>
> (* -------------- Start temp.txt output lines -------------- *)
>