Re: Slow Import version 4
- To: mathgroup at smc.vnet.net
- Subject: [mg18091] Re: Slow Import version 4
- From: paulh at wolfram.com (P.J. Hinton)
- Date: Thu, 17 Jun 1999 12:26:32 -0400
- Organization: Wolfram Research, Inc.
- References: <7k4osl$co5@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
In article <7k4osl$co5 at smc.vnet.net>, Mark Fisher <mefisher at bellsouth.net> writes:
> I am puzzled by the slowness of Importing a "Table" of data in Version
> 4.
>
> Export["mat.dat", Table[Random[], {100},{100}]
> Timing[Import["mat.dat"];]
> Timing[ReadList["mat.dat", Number, RecordLists->True];]
>
> The Timing for Import is 43.89 Second, while for ReadList it's only 0.71
> Second. Any ideas as to what's going on here? (BTW, I'm using Win98.)
The extra time is the cost of having Mathematica determine the type
of each data field for you. Import[] for the "Table" format uses a
sizable number of heuristics to determine what types of data are in
each record (e.g. string, date, currency, etc.).
If you are working with a large data set which has a very regular
structure, ReadList[] is the better option. Import[] is more
appropriate when you have a text file which has an irregular
pattern (e.g. varying widths, text headers, etc.) and you would
have to actually scan the contents of the file and code up several
different Read[] statements to bring the data into the kernel.
--
P.J. Hinton
Mathematica Programming Group paulh at wolfram.com
Wolfram Research, Inc.
Disclaimer: Opinions expressed herein are those of the author alone.