Re: help! to input data...
- To: mathgroup at smc.vnet.net
- Subject: [mg4211] Re: help! to input data...
- From: ianc (Ian Collier)
- Date: Tue, 18 Jun 1996 03:25:06 -0400
- Organization: Wolfram Research, Inc.
- Sender: owner-wri-mathgroup at wolfram.com
In article <4pit53$jk5 at dragonfly.wolfram.com>, tcdoe+ at pitt.edu (todd c.
doehring) wrote:
> well, silly me.
> I thought i was pretty good at the basics of mathematica, but today i
> tried to input some numerical data that was comma delimited. i.e.
>
> 456,-45,21,0,5
> 43,25,3,66,65
> ...
> ...
> ...
>
> it is a large data file (10Mb+) with 5 numbers per line. i can't
> convert the commas to tabs easily.
> i tried simply using:
>
> data=ReadList["c:\micro.dat", Number, RecordLists->True];
>
> then:
>
> data=ReadList["c:\micro.dat", RecordLists->True,
> WordSeparators->{","}];
>
> and
>
> data=ReadList["c:\micro.dat", Number, RecordLists->True,
> TokenWords->","];
>
> but all i ever get is the first number and an error such as:
>
> Read::readn: Syntax error reading a real number from c:\micro.dat.
>
> would someone please tell me the best way to input this data file??
> it's driving me crazy that i can't figure out something so basic.
>
> thanks,
> todd
The following is taken from the Technical Support FAQ area of
Wolfram Research's web pages (http://www.wolfram.com/support/).
How do I read comma-delimited data into Mathematica?
Mathematica's ReadList command does not have any built-in options to
read comma-separated data. However, we can still use ReadList in
combination with other functions to do this. Here is an example data file.
In[1]:= !!data.txt
1,10,1000
2,20,2000
3,30,3000
4,40,4000
5,50,5000
6,60,6000
7,70,7000
8,80,8000
9,90,9000
10,100,10000
There are two methods for reading comma-separated data. The first
method involves reading each value as a number and each comma (or
newline character) as a string. You then discard the strings, and are left
with only the values.
We start with the ReadList command.
In[2]:= data = ReadList["data.txt",{Number,Character}]
Out[2]= {{1, ,}, {10, ,}, {1000, }, {2, ,}, {20, ,}, {2000, }, {3, ,},
> {30, ,}, {3000, }, {4, ,}, {40, ,}, {4000, }, {5, ,}, {50, ,}, {5000, },
> {6, ,}, {60, ,}, {6000, }, {7, ,}, {70, ,}, {7000, }, {8, ,}, {80, ,},
> {8000, }, {9, ,}, {90, ,}, {9000, }, {10, ,}, {100, ,}, {10000, }}
Notice that each number is paired up with a character following it (either a
comma or a newline character). We can use the InputForm command to
verify this.
In[3]:= InputForm[data]
Out[3]//InputForm=
{{1, ","}, {10, ","}, {1000, "\n"}, {2, ","}, {20, ","}, {2000, "\n"},
{3, ","}, {30, ","}, {3000, "\n"}, {4, ","}, {40, ","}, {4000, "\n"},
{5, ","}, {50, ","}, {5000, "\n"}, {6, ","}, {60, ","}, {6000, "\n"},
{7, ","}, {70, ","}, {7000, "\n"}, {8, ","}, {80, ","}, {8000, "\n"},
{9, ","}, {90, ","}, {9000, "\n"}, {10, ","}, {100, ","}, {10000, "\n"}}
To remove these extra characters, we need to take the first element of
every sublist. We can Map the First command to every sublist to extract the
values.
In[4]:= data = Map[First,data]
Out[4]= {1, 10, 1000, 2, 20, 2000, 3, 30, 3000, 4, 40, 4000, 5, 50, 5000, 6,
> 60, 6000, 7, 70, 7000, 8, 80, 8000, 9, 90, 9000, 10, 100, 10000}
We now have our data, but we need it in a matrix form. Since we know that
this data is in three columns, we can use the Partition command to partition
the data into equal sublists of length 3.
In[5]:= data = Partition[data,3]
Out[5]= {{1, 10, 1000}, {2, 20, 2000}, {3, 30, 3000}, {4, 40, 4000},
> {5, 50, 5000}, {6, 60, 6000}, {7, 70, 7000}, {8, 80, 8000},
> {9, 90, 9000}, {10, 100, 10000}}
The data is more clearly displayed using MatrixForm.
In[6]:= MatrixForm[data,TableSpacing->{0}]
Out[6]//MatrixForm= 1 10 1000
2 20 2000
3 30 3000
4 40 4000
5 50 5000
6 60 6000
7 70 7000
8 80 8000
9 90 9000
10 100 10000
The second method involves read each line as a string and then converting
the string into an expression.
We simply use the ReadList command with the String specification.
In[7]:= data = ReadList["data.txt",String]
Out[7]= {1,10,1000, 2,20,2000, 3,30,3000, 4,40,4000, 5,50,5000, 6,60,6000,
> 7,70,7000, 8,80,8000, 9,90,9000, 10,100,10000}
Note that the data looks like it has been read in as numbers. However, we
can see they are string by using the InputForm command.
In[8]:= InputForm[data]
Out[8]//InputForm=
{"1,10,1000", "2,20,2000", "3,30,3000", "4,40,4000", "5,50,5000, "6,60,\
6000", "7,70,7000", "8,80,8000", "9,90,9000", "10,100,10000"}
This functions convert a string (of a sequence of numbers) into a list of
numbers.
In[9]:= f[x_String] := ToExpression[StringJoin["{", x ,"}"]]
Now we Map this function to our data
In[10]:= data = Map[f,data]
Out[10]= {{1, 10, 1000}, {2, 20, 2000}, {3, 30, 3000}, {4, 40, 4000},
> {5, 50, 5000}, {6, 60, 6000}, {7, 70, 7000}, {8, 80, 8000},
> {9, 90, 9000}, {10, 100, 10000}}
to get our matrix.
This method does not requires you to know how many columns were in
your data. However, the first method has a slight speed advantage for
large data files. Here are both methods used on a 200k file (from an IBM
RS/6000 workstation).
In[11]:= Timing[ long1 = Partition[Map[First,
ReadList["long.txt",{Number,Character}]],3]; ]
Out[11]= {9.02 Second, Null}
In[12]:= Timing[ long2 = Map[ToExpression[StringJoin["{",#,"}"]]& ,
ReadList["long.txt",String]]; ]
Out[12]= {14.7 Second, Null}
In[13]:= long1 == long2
Out[13]= True
The specific URL for this question and answer is:
http://www.wolfram.com/support/InputOutput/ExternalFiles/CommaSeparatedData.html
I hope this helps.
--Ian
-----------------------------------------------------------
Ian Collier
Wolfram Research, Inc.
-----------------------------------------------------------
tel:(217) 398-0700 fax:(217) 398-0747 ianc at wolfram.com
Wolfram Research Home Page: http://www.wolfram.com/
-----------------------------------------------------------
==== [MESSAGE SEPARATOR] ====