Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: ByteCount of imported machine-precision data matrix three times

  • To: mathgroup at smc.vnet.net
  • Subject: [mg92209] Re: ByteCount of imported machine-precision data matrix three times
  • From: dh <dh at metrohm.ch>
  • Date: Tue, 23 Sep 2008 07:31:23 -0400 (EDT)
  • References: <gb7odj$nij$1@smc.vnet.net>


Hi Gareth,

I can not explain what is hapening, but it has nothing to do with your 

file. Maybe Wolfram could give some insight. Consider:

dummyData=10^10 Table[       RandomReal[{-1,1}],{20},{14}];

ByteCount[dummyData]

dummyData=10^10 Table[10^10  RandomReal[{-1,1}],{20},{14}];

ByteCount[dummyData]

Daniel



Gareth Russell wrote:

> Hi,

> 

> I am encountering some strange memory-related behavior when importing 

> numerical data from a file. If anyone is interested, a (small) example 

> file is here:

> 

> http://web.njit.edu/~russell/Mathematica.html

> 

> It's a simple 2D array of numbers. The issue is that when imported, 

> ByteCount[] indicates that the resultant expression takes up more than 

> three times as much memory as an equivalent machine-precision matrix 

> generated within Mathematica. All diagnostics that I can think of 

> indicate that the imported expression is equivalent in precision. And 

> indeed, ByteCount applied to individual elements of each matrix returns 

> 16 as an answer. It's only the overall ByteCount which is hugely 

> different.

> 

> I discovered a workaround: if I generate a dummy matrix of 0. elements 

> (which has the smaller ByteCount), and add it to the imported matrix, 

> the result, while appearing identical (as it should), now also has the 

> smaller ByteCount.

> 

> Does anyone know what it going on here? Until I discovered the 

> workaround it was a problem, as I need to read in a large number of 

> much larger matrices all together, was encountering memory issues.

> 

> Thanks,

> 

> Gareth

> NJIT

> 

> 

> 

> In[1452]:= data = Drop[Import["12e.dat"], 6];

> 

> In[1453]:= Dimensions[data]

> 

> Out[1453]= {20, 14}

> 

> In[1454]:= {Min[Flatten[data]], Max[Flatten[data]]}

> 

> Out[1454]= {-1.1, 0.992655}

> 

> In[1455]:= MachineNumberQ[data[[1, 1]]]

> 

> Out[1455]= True

> 

> In[1456]:= ByteCount[data]

> 

> Out[1456]= 7384

> 

> In[1466]:= ByteCount[data[[1, 1]]]

> 

> Out[1466]= 16

> 

> In[1457]:= dummyData = Table[RandomReal[{-1.1, 1}], {20}, {14}];

> 

> In[1458]:= ByteCount[dummyData]

> 

> Out[1458]= 2360

> 

> In[1460]:= zeroData = Table[0., {20}, {14}];

> 

> In[1461]:= ByteCount[zeroData]

> 

> Out[1461]= 2360

> 

> In[1462]:= newData = data + zeroData;

> 

> In[1463]:= {Min[Flatten[newData]], Max[Flatten[newData]]}

> 

> Out[1463]= {-1.1, 0.992655}

> 

> In[1464]:= MachineNumberQ[newData[[1, 1]]]

> 

> Out[1464]= True

> 

> In[1465]:= ByteCount[newData]

> 

> Out[1465]= 2360

> 

> In[1467]:= ByteCount[newData[[1, 1]]]

> 

> Out[1467]= 16

> 





-- 



Daniel Huber

Metrohm Ltd.

Oberdorfstr. 68

CH-9100 Herisau

Tel. +41 71 353 8585, Fax +41 71 353 8907

E-Mail:<mailto:dh at metrohm.com>

Internet:<http://www.metrohm.com>




  • Prev by Date: Re: Finding Nash Equilibrium in a 2 person game with continuous action space
  • Next by Date: Re: mathematica 6.0.3 and mac os x 10.5.5 crashes?
  • Previous by thread: Re: ByteCount of imported machine-precision data matrix three times
  • Next by thread: about font of mathematica 6.03 in gentoo