[Date Index] [Thread Index] [Author Index]

Re: Speeding up ReadList[]?

• To: mathgroup@smc.vnet.net
• Subject: [mg12057] Re: [mg12020] Speeding up ReadList[]?
• From: Sean Ross <seanross@worldnet.att.net>
• Date: Fri, 24 Apr 1998 01:52:27 -0400
• References: <199804170740.DAA00295@smc.vnet.net.>

John Sidles wrote:
> 
> I use Mathematica to process Excel spreadsheets containing large arrays
> ({500,500}) of ASCII-coded floats.  To read in the lists, I use the
> Mathematica function
> 
>       ReadList[stream, Number, RecordLists->True]
> 
> Everything works fine, except that as each new array, is read and
> processed, ReadList[] gets slower and s l o w e r and
> s...l...o...w...e...r.  The first ReadList[] takes only 40 seconds,
> while the second ReadList[] can take 400 seconds or longer, and I don't
> even know how long the third ReadList[] takes.
> 
> Each file is processed independently, and no large arrays are retained
> in memory (they are stored as local variables inside Module[] blocks).
> VIrtual memory is not used, and both the Kernel and the Notebook retain
> plenty of unallocated memory.
> 
> Each file does generate a large Graphics[] object, but surprisingly,
> Mathematica does not slow down while computing each graphic.  Thus,
> ReadList[] is the only function evidencing a slowdown.
> 
> Neither the MathSource nor the FAQ seem to address this problem.  Has
> anyone else run into this slowdown, and  how can it be avoided?
> 
> Thanks ... JAS


It seems to me that the reading process slows down with the number of
elements and the complexity of the structure mathematica is to deal
with.  I typically read in data files as a single string and then
process the data with string manipulation commands and using
ToExpression at the end.
-- 
Remove the _nospam_ in the return address to respond.

• References:
  • Speeding up ReadList[]?
    • From: sidles@u.washington.edu (John Sidles)