Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2008

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Efficient way of reading matrices

  • To: mathgroup at smc.vnet.net
  • Subject: [mg85329] Re: Efficient way of reading matrices
  • From: m.r at inbox.ru
  • Date: Wed, 6 Feb 2008 05:40:12 -0500 (EST)
  • References: <fo3gad$b3p$1@smc.vnet.net> <fo9h1r$t26$1@smc.vnet.net>

On Feb 5, 5:24=A0am, "Szabolcs Horv=E1t" <szhor... at gmail.com> wrote:
> In Feb 5, 2008 7:22 AM, =A0<m... at inbox.ru> wrote:
>
>
>
> > Hi Szabolcs,
>
> > maybe you can read and pack your matrix row by row:
>
> > WriteString["/zzz", Sequence @@ Riffle[#, " "], "\n"]& /@
> > =A0 RandomInteger[{-1000, 1000}, {200, 250000}]
> > Close["/zzz"]
> > =A0(*a test file*)
>
> > (m = Developer`ToPackedArray@ Read["/zzz",
> > =A0 Array[Number&, 250000]]& /@ Range[200];) // Timing
> > Close["/zzz"]
>
> Thanks you Maxim Rytin,
>
> Good and useful suggestion, as always. :-) =A0I have already written a
> MathLink matrix reader program to do the work, but your program
> (though slower) uses even less memory as it never needs to store two
> copies of the complete matrix in memory.
>
> Szabolcs Horv=E1t

The speed can be improved some because a lot of time is spent
evaluating Number&[arrayindex]:

In[1]:= (m = Developer`ToPackedArray@ Read["/zzz",
  Array[Number&, 250000]]& /@ Range[200];) // Timing

Out[1]= {50.204, Null}

In[1]:= Module[{f = ConstantArray[Number, 250000]},
  m = Developer`ToPackedArray@ Read["/zzz",
    f]& /@ Range[200]]; // Timing

Out[1]= {22.625, Null}

Maxim Rytin
m.r at inbox.ru


  • Prev by Date: Re: Domains
  • Next by Date: Re: Mathematica 6.0.0 no graphics output on Windows XP
  • Previous by thread: Re: Efficient way of reading matrices
  • Next by thread: How edit a saved palette?