Re: Importing CSV (comma separated values) files?
- To: mathgroup at smc.vnet.net
- Subject: [mg13775] Re: Importing CSV (comma separated values) files?
- From: David Withoff <withoff>
- Date: Mon, 24 Aug 1998 05:07:24 -0400
- Sender: owner-wri-mathgroup at wolfram.com
> I have been trying to convince Mathematica 3.0.1 to ReadList an ordinary
> CSV spreadsheet file, without luck.
>
> The file contains 240 lines, each line containing 240 non-negative
> integer values separated by commas.
>
> raw2Ddata =
> ReadList[fullpath,Number,RecordLists->True (*,
> WordSeparators->{","}*)]; Read::"readn":
> "Syntax error reading a real number from \!\(\"FWB 4 GB:Gigabit \
> Ethernet:Of\" \\[Ellipsis] \"urs:2D Image Data:50M_OFL.IMG\"\)."
>
> All I can guess is that Mathematica expects numbers with decimals, and
> isn't getting the decimals. If that's true, it's a problem, because
> the instruments don't do decimal numbers. Is there an "Integer"
> declaration for ReadLists? "Integer" didn't work.
The problem is the commas, which aren't part of Number format (see the
documentation for Number). There are no problems related to decimals
or with reading integers.
Also, WordSeparators is meaningful only when reading Word format. It
doesn't do anything when reading Number format.
If the file consists of comma-separated numbers, you can read the file
in Word format, and convert the words into numbers after the file has
been read in to Mathematica:
data = ReadList["file", Word, RecordLists -> True, WordSeparators ->
","]; data = Map[ToExpression, data, {2}];
Without seeing your file or knowing exactly what you want, I can't be
sure that these inputs will work for you exactly as given, but it is
likely that something very similar will do what you want.
If you can save the file in a tab-separated format or a space-separated
format (most spreadsheet programs can do that), then you can just use
ReadList["file", Number]
to get a list of numbers, or
ReadList["file", Number, RecordLists -> True]
to get an array of numbers. Commas mean different things in different
programs, and so can be troublesome even if Mathematica isn't involved.
Your experience may of course be different, but tab-separated data is
more common in the applications that I have used.
> Also, a comment for Wolfram: Mathematica should be able to read and
> write Microsoft Excel "text" format, and also CSV formatted files
> directly and efficiently, using built-in functions, to enable access to
> the vast range of scientific instruments and analysis packages, and
> databases. The present manual process is very slow and clumsy.
Is the process above something that you would describe as clumsy? Would
it be an improvement to have functions such as
ReadCommaSeparatedNumberFile[p_String] :=
Map[ToExpression, ReadList[p, Word,
RecordLists -> True, WordSeparators -> ","], {2}]
???
> And, the error message needs some work. One easy and very helpful trick
> is to echo the offending line, and then put a line of ^^^ symbols under
> the offending token:
>
> Now is the t1me for ...
> ___________^^^^_________
> Number found where only letters were expected.
I have no disagreeement with your sentiment about error messages.
Implementing this is non-trivial, since it is necessary consider such
things as what to do if the offending line is a million characters
long, arranging for the
> Now is the t1me for ...
> ___________^^^^_________
trick to work sensibly even if the output doesn't use a mono-spaced font
(underlining? color?), anticipating other pathological situations,
finding time to do this given the myriad other usability features that
one might want, and so forth. Your suggestion is still appreciated,
and as mentioned before there is no disagreement with the sentiment
that this error message is important and can be improved.
Dave Withoff
Wolfram Research