Re: Memory conserving Input of big Ascii tables ?
- To: mathgroup at smc.vnet.net
- Subject: [mg35414] Re: Memory conserving Input of big Ascii tables ?
- From: Jens-Peer Kuska <kuska at informatik.uni-leipzig.de>
- Date: Fri, 12 Jul 2002 04:28:23 -0400 (EDT)
- Organization: Universitaet Leipzig
- References: <agjj2m$251$1@smc.vnet.net>
- Reply-to: kuska at informatik.uni-leipzig.de
- Sender: owner-wri-mathgroup at wolfram.com
Hi,
write a MathLink program !
Regards
Jens
Justus Heimann wrote:
>
> Hello,
>
> I'm seeking for a memory (i.e., Mathematica kernel and WindowsOS
> memory!) conserving way to read in big Ascii tables (in fact quadratic
> matrices of up to 10.000 x 10.000 elements) of double precision real
> numbers ? The processing time is of minor importance, memory saving is
> the key!
>
> What I tried with my Mathematica 4.0 Windows version is a simple
> sequentially procedure like:
>
> ----------
> funcReadSeqBigMatrices[filein_String]:=
> Module[{streamIn, DUMMY, MATRIX = {}},
>
> streamIn = OpenRead[filein];
>
> Do[{
> DUMMY =
> ReadList[streamIn, Array[Number&, 3], Ni][[All, 3]];
> AppendTo[MATRIX, DUMMY];
> Clear[DUMMY]; Unprotect[Out]; Clear[Out]; Protect[Out];
>
> }, {Nj}];
>
> Close[streamIn];
> MATRIX = Developer`ToPackedArray[MATRIX, Real];
> inMATRIX := Return[MATRIX];
> ];
> ----------
>
> in order to read in Ascii tables formated like (row index i, column
> index j, value) of the form:
>
> ----------
> 1 1 0.529597998598605E-04
> 1 2 0.530631874495232E-04
> 1 ..... .....................
> 1 10000 0.516990975201977E-04
> 2 1 0.529597998598605E-04
> 2 2 0.530631874495232E-04
> 2 ..... .....................
> 2 10000 0.532446901533145E-04
> ..... ..... .....................
> 10000 1 0.532674217089594E-04
> 10000 2 0.517207876590374E-04
> 10000 ..... .....................
> 10000 10000 0.516990975201977E-04
> ----------
>
> But, this isn't memory conserving enough in my case. Does anyone know
> a nice and memory conserving way to do this job ???
>
> Many many thanks!
> Justus