       ByteCount of imported machine-precision data matrix three times equivalent generated matrix

• To: mathgroup at smc.vnet.net
• Subject: [mg92171] ByteCount of imported machine-precision data matrix three times equivalent generated matrix
• From: Gareth Russell <russell at njit.edu>
• Date: Mon, 22 Sep 2008 05:27:24 -0400 (EDT)

```Hi,

I am encountering some strange memory-related behavior when importing
numerical data from a file. If anyone is interested, a (small) example
file is here:

http://web.njit.edu/~russell/Mathematica.html

It's a simple 2D array of numbers. The issue is that when imported,
ByteCount[] indicates that the resultant expression takes up more than
three times as much memory as an equivalent machine-precision matrix
generated within Mathematica. All diagnostics that I can think of
indicate that the imported expression is equivalent in precision. And
indeed, ByteCount applied to individual elements of each matrix returns
16 as an answer. It's only the overall ByteCount which is hugely
different.

I discovered a workaround: if I generate a dummy matrix of 0. elements
(which has the smaller ByteCount), and add it to the imported matrix,
the result, while appearing identical (as it should), now also has the
smaller ByteCount.

Does anyone know what it going on here? Until I discovered the
workaround it was a problem, as I need to read in a large number of
much larger matrices all together, was encountering memory issues.

Thanks,

Gareth
NJIT

In:= data = Drop[Import["12e.dat"], 6];

In:= Dimensions[data]

Out= {20, 14}

In:= {Min[Flatten[data]], Max[Flatten[data]]}

Out= {-1.1, 0.992655}

In:= MachineNumberQ[data[[1, 1]]]

Out= True

In:= ByteCount[data]

Out= 7384

In:= ByteCount[data[[1, 1]]]

Out= 16

In:= dummyData = Table[RandomReal[{-1.1, 1}], {20}, {14}];

In:= ByteCount[dummyData]

Out= 2360

In:= zeroData = Table[0., {20}, {14}];

In:= ByteCount[zeroData]

Out= 2360

In:= newData = data + zeroData;

In:= {Min[Flatten[newData]], Max[Flatten[newData]]}

Out= {-1.1, 0.992655}

In:= MachineNumberQ[newData[[1, 1]]]

Out= True

In:= ByteCount[newData]

Out= 2360

In:= ByteCount[newData[[1, 1]]]

Out= 16

--
Gareth Russell
NJIT

```

• Prev by Date: Re: data source for demonstration
• Next by Date: Re: data source for demonstration
• Previous by thread: Re: Strange size changes in Manipulate with GeometricTransformation
• Next by thread: mathematica 6.0.3 and mac os x 10.5.5 crashes?