Services & Resources / Wolfram Forums / MathGroup Archive

MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Efficient way of reading matrices

  • To: mathgroup at
  • Subject: [mg85243] Re: [mg85229] Efficient way of reading matrices
  • From: Sseziwa Mukasa <mukasa at>
  • Date: Mon, 4 Feb 2008 03:09:14 -0500 (EST)
  • References: <>

On Feb 2, 2008, at 11:32 PM, Szabolcs Horv=E1t wrote:

> I need to read in some very big integer matrices from text files, the
> biggest of which have a dimension of approx 200*250000.
> What is the most efficient way of doing this?
> The problem is that ReadList[#, Number, RecordLists->True] cannot do
> this without swapping (it takes more than 10 minutes to finish), but
> after the matrix has been read in and compacted with ToPackedArray, it
> only takes up 200 MBs, and I can work with it comfortably.
> Are there more efficient ways to read in such big files?  Is there any
> way to tell Mathematica that all the integers are machine-size and the
> matrix is rectangular (i.e. not a ragged array)?
> A last resort would be to try to write a MathLink program for reading
> the data but I would like to avoid this ...

If you are reading machine numbers then BinaryReadList seems more 

  • Prev by Date: Re: Fourier Series Expansions and it's Coefficients question revised tia
  • Next by Date: Re: Using "Limit" when the limit is a delta function
  • Previous by thread: Efficient way of reading matrices
  • Next by thread: Re: Efficient way of reading matrices