Re: Rearranging a data array containing calendrical as well as data entries.

*To*: mathgroup at smc.vnet.net*Subject*: [mg54842] Re: [mg54827] Rearranging a data array containing calendrical as well as data entries.*From*: "Barthelet, Luc" <lucb at ea.com>*Date*: Fri, 4 Mar 2005 05:07:39 -0500 (EST)*Sender*: owner-wri-mathgroup at wolfram.com

This code does the trick. By the way, I forgot that transforming the cell to input form removed the comments. Bummer. How does one copy paste in an email and keep the comments? pathtoactivenotebook = First[Last[First[NotebookInformation[ EvaluationNotebook[]]]]]; DirectoryListToPath[a_List] := StringJoin @@ Flatten[{Table[StringJoin[ToString[a[[i]]], $PathnameSeparator], {i, 1, Length[a] - 1}], a[[-1]]}]; folderName = DirectoryListToPath[ pathtoactivenotebook]; If[StringTake[folderName, {1}] == "\\", folderName = StringDrop[folderName, 1]]; theFileName = FileNames["*.dat", folderName][[1]]; thisFile = OpenRead[theFileName]; data = ReadList[thisFile, Number, RecordLists -> True]; Close[thisFile]; betterData = (If[First[#1] > 3000, Join[{Floor[#1[[1]]/100], Mod[#1[[1]], 100]}, Rest[#1]], #1] & ) /@ data; cutHead[u_] := Module[{year, month, flows, n}, {year, month} = u[[1,{1, 2}]]; flows = Join @@ (Drop[#1, 3] & ) /@ u; n = Length[flows]; Transpose[{Table[year, {n}], Table[month, {n}], Range[n], flows}]]; cleanData = Flatten[cutHead /@ Split[betterData, #1[[{1, 2}]] == #2[[{1, 2}]] & ], 1]; PrintData[{startYear_, startMonth_, startDay_}, {endYear_, endMonth_, endDay_}] := Module[{}, cheapStartDate = 1000*startYear + 40*startMonth + startDay; cheapEndDate = 1000*endYear + 40*endMonth + endDay; (StringJoin[ToString[PaddedForm[#1[[3]], 2, NumberPadding -> "0", NumberSigns -> {"", ""}]], {"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"}[[#1[[2]]]], ToString[#1[[1]]], " ", ToString[PaddedForm[#1[[4]], {3, 2}]], "\n"] & ) /@ Select[cleanData, 1000*#1[[1]] + 40*#1[[2]] + #1[[3]] >= cheapStartDate && 1000*#1[[1]] + 40*#1[[2]] + #1[[3]] <= cheapEndDate & ]]; result = PrintData[{2001, 1, 13}, {2001, 3, 3}]; thisFile = OpenWrite[ToFileName[folderName, "output.txt"]]; WriteString[thisFile, StringJoin @@ result]; Close[thisFile]; -----Original Message----- From: Gilmar [mailto:gilmar.rodriguez at nwfwmd.state.fl.us] To: mathgroup at smc.vnet.net Subject: [mg54842] [mg54827] Rearranging a data array containing calendrical as well as data entries. Dear Mathematica User Friends: I have a file containing flow data from the USGS, in the following format: 1999 1 1 489.82 489.82 495.01 495.01 495.01 495.01 495.01 490.51 1999 1 2 490.51 490.51 490.51 490.51 490.38 490.38 490.38 490.38 1999 1 3 490.38 510.38 510.38 510.38 510.38 510.38 528.66 528.66 1999 1 4 528.66 528.66 528.66 501.68 501.68 501.68 501.68 1999 2 1 501.68 496.44 496.44 496.44 496.44 496.44 478.72 478.72 1999 2 2 478.72 478.72 478.72 452.82 452.82 452.82 452.82 452.82 1999 2 3 450.19 450.19 450.19 450.19 450.19 443.98 443.98 443.98 1999 2 4 443.98 443.98 440.14 440.14 1999 3 1 440.14 440.14 440.14 453.64 453.64 453.64 453.64 453.64 1999 3 2 503.98 503.98 503.98 503.98 503.98 500.84 500.84 500.84 1999 3 3 500.84 500.84 473.48 473.48 473.48 473.48 473.48 463.19 1999 3 4 463.19 463.19 463.19 463.19 457.54 457.54 457.54 This format is used by the USGS to compress their data records. Each row contains: Year, Month Number(1 to 12), Row Number (1 to 4), and data entries. The first row: 1999 1 1 489.82 489.82 495.01 495.01 495.01 495.01 495.01 490.51 contains flow values corresponding to: January 1 to January 8, of the year 1999. The second row: 1999 1 2 490.51 490.51 490.51 490.51 490.38 490.38 490.38 490.38 contains flow values corresponding to: January 9 to January 16, of the year 1999. The third row: 1999 1 3 490.38 510.38 510.38 510.38 510.38 510.38 528.66 528.66 contains flow values corresponding to: January 17 to January 24, of the year 1999. The fourth row: 1999 1 4 528.66 528.66 528.66 501.68 501.68 501.68 501.68 contains flow values corresponding to: January 25 to January 31, of the year 1999. I think that you get the picture of how this data set is assembled. What I need is a program that can turn the above mentioned horizontal array, into a simple vertical array, containing two columns; the first column contains the dates when the data was collected, and the second column contains the flow values; i.e. 01Jan1999 489.82 02Jan1999 489.82 03Jan1999 495.01 etc. If I give the program a starting date, and ending date for an arbitrary record; the program should be able to allocate two arrays to: (1.) put the dates between the starting date, and ending date, to form the first column of the vertical array. (2.)match correctly those dates with the data to appear in the second column of the vertical array. The program should discern between regular years, and leap years. Those of you that are still using FORTRAN, and have experienced how difficult it is to deal with date functions using FORTRAN, might sympathize with my request. P.S. To get a larger set of USGS flow data to test your program please download the following file: http://www.gilmarlily.netfirms.com/download/flow.dat Thank you for your help!