Re: Huge memory leaks when importing two-dimensional array in Mathematica
- To: mathgroup at smc.vnet.net
- Subject: [mg103463] Re: Huge memory leaks when importing two-dimensional array in Mathematica
- From: "Alexey Popkov" <lehin.p at gmail.com>
- Date: Wed, 23 Sep 2009 07:24:37 -0400 (EDT)
- References: <h9927o$1nm$1@smc.vnet.net>
- Reply-to: "Alexey Popkov" <lehin.p at gmail.com>
I apologize because the code I have posted here for Mathematica 5.2 was
wrong. The real behavior in v.5.2 is as follows:
In[1]:=
MemoryInUse[]
t = Table[Random[Real, {-1, 1}], {1000000}, {2}];
MemoryInUse[]
SetDirectory["C:/"]
t >> temp.dat
MemoryInUse[]
MaxMemoryUsed[]
Pause[3]
Quit[]
Out[1]=2128760
Out[3]=18132392
Out[4]=C:\
Out[6]=18138208
Out[7]=91137872
In[1]:=MemoryInUse[]
SetDirectory["C:/"]
t = << temp.dat;
MemoryInUse[]
MaxMemoryUsed[]
Out[1]=2127368
Out[2]=C:\
Out[4]=70129064
Out[5]=120570008
So in really v.5.2 behaves like v.7.01 in this case. But nevertheless I can
import my two-dimensional array (file "2DArray1.dat" with size 554 482 863
bytes) only in v.5.2 but not in v.7.01. Interestingly, another 2D-array
(file "2DArray2.dat" with size 454 468 512) is imported in v.7 without
difficulties. The comparison follows.
In v.5.2:
In[1]:=
MemoryInUse[]
SetDirectory["C:/"]
t = << "2DArray1.dat";
MemoryInUse[]
MaxMemoryUsed[]
Out[1]=2127432
Out[2]=C:\
Out[4]=818129176
Out[5]=1463533184
In v.7.01:
In[1]:= MemoryInUse[]
SetDirectory["C:/"]
t = << "2DArray1.dat";
MemoryInUse[]
MaxMemoryUsed[]
Out[1]= 10039832
Out[2]= "C:\\"
During evaluation of In[1]:=
No more memory available.
Mathematica kernel has shut down.
Try quitting other applications and then retry.
Another (smaller) array:
In v.5.2:
In[1]:=
MemoryInUse[]
SetDirectory["C:/"]
t = << "2DArray2.dat";
MemoryInUse[]
MaxMemoryUsed[]
Out[1]=2127432
Out[2]=C:\
Out[4]=682129192
Out[5]=1237539808
In v.7.01:
In[1]:= MemoryInUse[]
SetDirectory["C:/"]
t = << "2DArray2.dat";
MemoryInUse[]
MaxMemoryUsed[]
Out[1]= 10039832
Out[2]= "C:\\"
Out[4]= 770058936
Out[5]= 1291636032
----- Original Message -----
From: "Alexey" <lehin.p at gmail.com>
Newsgroups: comp.soft-sys.math.mathematica
Sent: Tuesday, September 22, 2009 3:28 AM
Subject: [mg103463] Huge memory leaks when importing two-dimensional array in
Mathematica
> Hello,
> Consider the following (evaluated in Mathematica 7.01):
>
> In[1]:= MemoryInUse[]
> t = RandomReal[{-1, 1}, {1000000, 2}];
> MemoryInUse[]
> SetDirectory["C:/"]
> t >> temp.dat
> MemoryInUse[]
> Quit[]
> Out[1]= 10040344
> Out[3]= 26058000
> Out[4]= "C:\\"
> Out[6]= 26065008
>
> One may see that the matrix "t" takes less than 26Mb of memory. Now we
> try
> to import the file "temp.dat":
>
> In[1]:= MemoryInUse[]
> SetDirectory["C:/"]
> t = << temp.dat;
> MemoryInUse[]
> Out[1]= 10039112
> Out[2]= "C:\\"
> Out[4]= 86058504
>
> The same data takes more than 80Mb of memory! How this happens?
>
> The same code in Mathematica 5.2 works as expected:
>
> In[1]:=
> MemoryInUse[]
> t=RandomReal[{-1,1},{1000000,2}];
> MemoryInUse[]
> SetDirectory["C:/"]
> t>>temp.dat
> MemoryInUse[]
> Pause[2]
> Quit[]
>
> Out[1]=
> 2128352
>
> Out[3]=
> 2129928
>
> Out[4]=
> C:\
>
> Out[6]=
> 2134576
>
> In[1]:=
> MemoryInUse[]
> SetDirectory["C:/"]
> t=<<temp.dat;
> MemoryInUse[]
>
> Out[1]=
> 2127256
>
> Out[2]=
> C:\
>
> Out[4]=
> 2129344
>
> This memory leak is critical for me because I can not import backwards
> in
> Mathematica 7 my file exported in this way from Mathematica 7. The
> file size
> is 512Mb and it may be imported in Mathematica 5.2 without
> difficulties.
>
> Any ideas?
>