Re: Speeding up ReadList[]?
- To: mathgroup@smc.vnet.net
- Subject: [mg12057] Re: [mg12020] Speeding up ReadList[]?
- From: Sean Ross <seanross@worldnet.att.net>
- Date: Fri, 24 Apr 1998 01:52:27 -0400
- References: <199804170740.DAA00295@smc.vnet.net.>
John Sidles wrote:
>
> I use Mathematica to process Excel spreadsheets containing large arrays
> ({500,500}) of ASCII-coded floats. To read in the lists, I use the
> Mathematica function
>
> ReadList[stream, Number, RecordLists->True]
>
> Everything works fine, except that as each new array, is read and
> processed, ReadList[] gets slower and s l o w e r and
> s...l...o...w...e...r. The first ReadList[] takes only 40 seconds,
> while the second ReadList[] can take 400 seconds or longer, and I don't
> even know how long the third ReadList[] takes.
>
> Each file is processed independently, and no large arrays are retained
> in memory (they are stored as local variables inside Module[] blocks).
> VIrtual memory is not used, and both the Kernel and the Notebook retain
> plenty of unallocated memory.
>
> Each file does generate a large Graphics[] object, but surprisingly,
> Mathematica does not slow down while computing each graphic. Thus,
> ReadList[] is the only function evidencing a slowdown.
>
> Neither the MathSource nor the FAQ seem to address this problem. Has
> anyone else run into this slowdown, and how can it be avoided?
>
> Thanks ... JAS
It seems to me that the reading process slows down with the number of
elements and the complexity of the structure mathematica is to deal
with. I typically read in data files as a single string and then
process the data with string manipulation commands and using
ToExpression at the end.
--
Remove the _nospam_ in the return address to respond.
- References:
- Speeding up ReadList[]?
- From: sidles@u.washington.edu (John Sidles)
- Speeding up ReadList[]?