Slow Import of CSV files
- To: mathgroup at smc.vnet.net
- Subject: [mg79370] Slow Import of CSV files
- From: j.f.b.payne at tesco.net
- Date: Wed, 25 Jul 2007 02:20:43 -0400 (EDT)
Hello, Has anyone else looked at the speed of Import of CSV files? With an 8MB file (about 300,000 lines with three numbers per line) I was disappointed to find that Mathematica version 6.0 Import is about 3 times slower than version 5.2. Then I got an e-mail from info at wolfram.com saying Mathematica 6.0.1 contains: * Improved performance of various Import and Export converters * Enhancements to Table, CSV, TSV and MathML import So I tried version 6.0.1 and was more disappointed to find it is slower still. Maybe other files import quickly (but I can't guess why, I didn't think there was anything special about mine). Technical support suggest that Import does more work in version 6 than version 5.2, which may well be true in general. However, for the specific case of CSV files, perusal of version 6 Help (ref/format/CSV under Options|Import & Export Functions) and version 5.2 Help (Import) suggests that they handle the same things (numbers, strings, dates, currency symbols). Of course you can work round CSV Import with ReadList, but Import is a bit more convenient (if you don't have to wait 10 seconds for it to execute). Or is there an option to set for CSV Import to speed it up? I am just using Timing[rawXYZ5 = Import["Sample 5.prn", "CSV"];] John Payne