Re: cleaning up imported data
- To: mathgroup at smc.vnet.net
- Subject: [mg84948] Re: cleaning up imported data
- From: Mark Fisher <particlefilter at gmail.com>
- Date: Mon, 21 Jan 2008 04:05:25 -0500 (EST)
- References: <fmv19h$2md$1@smc.vnet.net>
On Jan 20, 3:38 am, Tom Metcalf <thmetc... at mac.com> wrote:
> I'm importing some data files (into Mathematica 6) that a colleague
> prepared, where each line of the original data file represents a data
> point, with several parameters recorded for each data point. The
> problem is that for some of these data points, he used ditto marks to
> specify that the value of a particular field is the same as the one
> from the record above, and there are stretches with many ditto marks
> in a row.
>
> I want to convert the ditto marks to the real values, which I did with
> the following Do loop, but the Do loop seems so... un-Mathematica-ish
> and I'm wondering if there's a more elegant way to accomplish the
> task.
>
> hdA = Import[filename, "TSV"];
> dup = hdA[[3,2]] (* manually set the ditto mark character *)
> Do[hdA[[i, j]] =
> If[hdA[[i, j]] == dup, hdA[[i - 1, j]], hdA[[i, j]]], {i,
> Length[hdA]}, {j, Length[hdA[[2]]]}]
>
> --Tom Metcalf
Hi Tom,
Here's one way.
DuplicateDittos::usage = "DuplicateDittos[data, dittomark] replaces \
the dittomarks with values from \"above\". DuplicateDittos works on \
vectors and matrices."
DuplicateDittos[data_?VectorQ, ditto_] :=
Rest@FoldList[If[#2 === ditto, #1, #2] &, 0, data]
DuplicateDittos[data_?MatrixQ, ditto_] :=
Transpose[ DuplicateDittos[#, ditto] & /@ Transpose[data]]
ditto = "\"";
data1 = {5, ditto, ditto, ditto, 7, 2, ditto, 9};
data2 = {10, ditto, 1, 2, 5, ditto, 3, ditto};
data = Transpose[{data1, data2}];
DuplicateDittos[data, ditto]
--Mark