Re: Rearranging a data array containing calendrical as well as data entries.
- To: mathgroup at smc.vnet.net
- Subject: [mg54842] Re: [mg54827] Rearranging a data array containing calendrical as well as data entries.
- From: "Barthelet, Luc" <lucb at ea.com>
- Date: Fri, 4 Mar 2005 05:07:39 -0500 (EST)
- Sender: owner-wri-mathgroup at wolfram.com
This code does the trick. By the way, I forgot that transforming the
cell to input form removed the comments. Bummer. How does one copy paste
in an email and keep the comments?
pathtoactivenotebook =
First[Last[First[NotebookInformation[
EvaluationNotebook[]]]]];
DirectoryListToPath[a_List] := StringJoin @@
Flatten[{Table[StringJoin[ToString[a[[i]]],
$PathnameSeparator], {i, 1, Length[a] - 1}],
a[[-1]]}];
folderName = DirectoryListToPath[
pathtoactivenotebook];
If[StringTake[folderName, {1}] == "\\",
folderName = StringDrop[folderName, 1]];
theFileName = FileNames["*.dat", folderName][[1]];
thisFile = OpenRead[theFileName];
data = ReadList[thisFile, Number, RecordLists ->
True];
Close[thisFile];
betterData = (If[First[#1] > 3000,
Join[{Floor[#1[[1]]/100], Mod[#1[[1]], 100]},
Rest[#1]], #1] & ) /@ data;
cutHead[u_] := Module[{year, month, flows, n},
{year, month} = u[[1,{1, 2}]];
flows = Join @@ (Drop[#1, 3] & ) /@ u;
n = Length[flows]; Transpose[{Table[year, {n}],
Table[month, {n}], Range[n], flows}]];
cleanData = Flatten[cutHead /@ Split[betterData,
#1[[{1, 2}]] == #2[[{1, 2}]] & ], 1];
PrintData[{startYear_, startMonth_, startDay_},
{endYear_, endMonth_, endDay_}] :=
Module[{}, cheapStartDate = 1000*startYear +
40*startMonth + startDay; cheapEndDate =
1000*endYear + 40*endMonth + endDay;
(StringJoin[ToString[PaddedForm[#1[[3]], 2,
NumberPadding -> "0", NumberSigns ->
{"", ""}]], {"Jan", "Feb", "Mar", "Apr",
"May", "Jun", "Jul", "Aug", "Sep", "Oct",
"Nov", "Dec"}[[#1[[2]]]], ToString[#1[[1]]],
" ", ToString[PaddedForm[#1[[4]], {3, 2}]],
"\n"] & ) /@ Select[cleanData,
1000*#1[[1]] + 40*#1[[2]] + #1[[3]] >=
cheapStartDate && 1000*#1[[1]] +
40*#1[[2]] + #1[[3]] <= cheapEndDate & ]];
result = PrintData[{2001, 1, 13}, {2001, 3, 3}];
thisFile = OpenWrite[ToFileName[folderName,
"output.txt"]];
WriteString[thisFile, StringJoin @@ result];
Close[thisFile];
-----Original Message-----
From: Gilmar [mailto:gilmar.rodriguez at nwfwmd.state.fl.us]
To: mathgroup at smc.vnet.net
Subject: [mg54842] [mg54827] Rearranging a data array containing calendrical as
well as data entries.
Dear Mathematica User Friends:
I have a file containing flow data from the USGS, in the following
format:
1999 1 1 489.82 489.82 495.01 495.01 495.01 495.01 495.01 490.51
1999 1 2 490.51 490.51 490.51 490.51 490.38 490.38 490.38 490.38
1999 1 3 490.38 510.38 510.38 510.38 510.38 510.38 528.66 528.66
1999 1 4 528.66 528.66 528.66 501.68 501.68 501.68 501.68
1999 2 1 501.68 496.44 496.44 496.44 496.44 496.44 478.72 478.72
1999 2 2 478.72 478.72 478.72 452.82 452.82 452.82 452.82 452.82
1999 2 3 450.19 450.19 450.19 450.19 450.19 443.98 443.98 443.98
1999 2 4 443.98 443.98 440.14 440.14
1999 3 1 440.14 440.14 440.14 453.64 453.64 453.64 453.64 453.64
1999 3 2 503.98 503.98 503.98 503.98 503.98 500.84 500.84 500.84
1999 3 3 500.84 500.84 473.48 473.48 473.48 473.48 473.48 463.19
1999 3 4 463.19 463.19 463.19 463.19 457.54 457.54 457.54
This format is used by the USGS to compress their data records.
Each row contains:
Year, Month Number(1 to 12), Row Number (1 to 4), and data entries.
The first row:
1999 1 1 489.82 489.82 495.01 495.01 495.01 495.01 495.01 490.51
contains flow values corresponding to: January 1 to January 8,
of the year 1999.
The second row:
1999 1 2 490.51 490.51 490.51 490.51 490.38 490.38 490.38 490.38
contains flow values corresponding to: January 9 to January 16,
of the year 1999.
The third row:
1999 1 3 490.38 510.38 510.38 510.38 510.38 510.38 528.66 528.66
contains flow values corresponding to: January 17 to January 24,
of the year 1999.
The fourth row:
1999 1 4 528.66 528.66 528.66 501.68 501.68 501.68 501.68
contains flow values corresponding to: January 25 to January 31,
of the year 1999.
I think that you get the picture of how this data set is assembled.
What I need is a program that can turn the above mentioned horizontal
array, into a simple vertical array, containing two columns;
the first column contains the dates when the data was collected,
and the second column contains the flow values; i.e.
01Jan1999 489.82
02Jan1999 489.82
03Jan1999 495.01
etc.
If I give the program a starting date, and ending date for an
arbitrary record; the program should be able to allocate two
arrays to:
(1.) put the dates between the starting date, and ending date,
to form the first column of the vertical array.
(2.)match correctly those dates with the data to appear in
the second column of the vertical array.
The program should discern between regular years, and leap
years. Those of you that are still using FORTRAN, and have
experienced how difficult it is to deal with date functions
using FORTRAN, might sympathize with my request.
P.S. To get a larger set of USGS flow data to test your program
please download the following file:
http://www.gilmarlily.netfirms.com/download/flow.dat
Thank you for your help!