Re: Import[filename,"CSV"] is so slooooooooow!
- To: mathgroup at smc.vnet.net
- Subject: [mg45935] Re: [mg45920] Import[filename,"CSV"] is so slooooooooow!
- From: Tomas Garza <tgarza01 at prodigy.net.mx>
- Date: Fri, 30 Jan 2004 04:16:03 -0500 (EST)
- References: <200401291036.FAA09853@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
You might try playing around with options NullWords and WordSeparators in
ReadList. I have a file with about 300,000 text records which is read in
about 8 seconds. If you have a numeric file, use Number instead of Word:
AbsoluteTiming[test = ReadList["c:\myfile.txt", Word,
NullWords -> True, WordSeparators -> {",", "\t"}, RecordLists ->
True]; ]
{8.7025136 Second,Null}
Tomas Garza
Mexico City
----- Original Message -----
From: "1.156" <rob at pi-overe.com>
To: mathgroup at smc.vnet.net
Subject: [mg45935] [mg45920] Import[filename,"CSV"] is so slooooooooow!
> Hi.
> I'm getting data in CSV files to analyze. With the new ver. 4.2 I can
> get these data in nicely using
> data = Import["filename", "CSV"]
>
> but it takes 30 seconds to get in a 100kB file. And I've got to do some
> much longer files.
>
> On the other hand, if I change all the comma delimiters to spaces, I can
> get the data in using
> inFile = OpenRead["filename"];
> y = ReadList[inFile, Number, RecordLists -> True];
> Close[inFile];
>
> This pulls in the same 100kB file in a second or so. But, doing this
> comma to space conversion is truly slow (using Word).
>
> So, can someone out there enlighten me with a suggestion on how to
> either speed up Import or get ReadList to handle comma delimiters?
>
> Thanks and Regards, Rob
>
- References:
- Import[filename,"CSV"] is so slooooooooow!
- From: "1.156" <rob@pi-overe.com>
- Import[filename,"CSV"] is so slooooooooow!