       Re: ByteCount of imported machine-precision data matrix three times

• To: mathgroup at smc.vnet.net
• Subject: [mg92205] Re: ByteCount of imported machine-precision data matrix three times
• From: Jean-Marc Gulliet <jeanmarc.gulliet at gmail.com>
• Date: Tue, 23 Sep 2008 07:30:38 -0400 (EDT)
• Organization: The Open University, Milton Keynes, UK
• References: <gb7odj\$nij\$1@smc.vnet.net>

```Gareth Russell wrote:

> I am encountering some strange memory-related behavior when importing
> numerical data from a file. If anyone is interested, a (small) example
> file is here:
>
> http://web.njit.edu/~russell/Mathematica.html
>
> It's a simple 2D array of numbers. The issue is that when imported,
> ByteCount[] indicates that the resultant expression takes up more than
> three times as much memory as an equivalent machine-precision matrix
> generated within Mathematica. All diagnostics that I can think of
> indicate that the imported expression is equivalent in precision. And
> indeed, ByteCount applied to individual elements of each matrix returns
> 16 as an answer. It's only the overall ByteCount which is hugely
> different.

*snip*

You will get the most compact form only when your data are made of
numeric values of same type (say, all machine integers or all
floating-point numbers) which is not the case for your imported dataset
(testing the first element only is not enough, see below). If it were
the case, Mathematica would use a packed array representation .

Something along the line

data = Developer`ToPackedArray[
6]]];

will do what you want.

Here is a step by step illustration of what is going on before and after
packed arrays are used.

In:= data =

In:= Dimensions[data]

Out= {20, 14}

In:= ByteCount[data]

Out= 7384

In:= Developer`PackedArrayQ[data]

Out= False

In:= MatrixQ[data, MachineNumberQ]

Out= False

In:= data = Developer`ToPackedArray[N[data]];

In:= ByteCount[data]

Out= 2360

In:= Developer`PackedArrayQ[data]

Out= True

In:= MatrixQ[data, MachineNumberQ]

Out= True

Regards,
- Jean-Marc

 /Performance of Linear Algebra Computation/
http://reference.wolfram.com/mathematica/tutorial/LinearAlgebraPerformance.html

```

• Prev by Date: Re: Re: the graphic of a function
• Next by Date: Re: Finding Nash Equilibrium in a 2 person game with continuous action space
• Previous by thread: Re: ByteCount of imported machine-precision data matrix three times
• Next by thread: Re: ByteCount of imported machine-precision data matrix three times