MathGroup Archive 2011

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Importing text file as machine-precision array

  • To: mathgroup at smc.vnet.net
  • Subject: [mg118047] Re: Importing text file as machine-precision array
  • From: David Bailey <dave at removedbailey.co.uk>
  • Date: Sat, 9 Apr 2011 17:59:55 -0400 (EDT)
  • References: <inpfu3$9as$1@smc.vnet.net>

On 09/04/2011 12:29, Sidney Cadot wrote:
> Hi all,
>
> I am trying to import a big text file containing non-sparse 2D
> numerical data into Mathematica. My aim is to get an array where the
> numbers are stored as machine-precision numbers. In other words,
> ByteCount[data] should be approximately equal to 8 times the number of
> table elements (assuming machine precision corresponds to 8-byte
> IEEE-754 numbers).
>
> If I do a straight data=Import["file","Table"], the file is parsed
> with all numbers represented as arbitrary precision floating point
> numbers; the data structure is ten times as big as it should be. If I
> do data=N[data] afterwards, the data is still four times as big as I
> would have expected.
>
> What is the appropriate way to accomplish what I want? Unfortunately,
> I cannot seem to find an option to Import[] to do this immediately
> when reading the data.
>
The only way that you will obtain an array as compact as you calculate 
(plus a small overhead) is to use the function Developer`ToPackedArray 
on the result after applying N.

You need to realise that arrays of reals (and some other basic types) 
come in two formats. That packed array format assumes that all elements 
are Real, or whatever. The other format can accommodate any type of 
object in any position, so it is less compact. Mathematica algorithms 
produce the same results on either format, but sometimes (as with 
Import) an array is not packed when it could be.

Packed arrays are also more efficient in terms of speed of processing.

David Bailey
http://www.dbaileyconsultancy.co.uk



  • Prev by Date: Re: CDF browser plugin and Wolfram library archive
  • Next by Date: Re: CDF browser plugin and Wolfram library archive
  • Previous by thread: Re: Importing text file as machine-precision array
  • Next by thread: Re: Importing text file as machine-precision array