Re: config. data from internet database
- To: mathgroup at smc.vnet.net
- Subject: [mg14747] Re: config. data from internet database
- From: paulh (P.J. Hinton)
- Date: Thu, 12 Nov 1998 02:17:45 -0500
- Organization: Wolfram Research, Inc.
- References: <72beid$k2c@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
In article <72beid$k2c at smc.vnet.net>, jim leddon <jleddon at cyberramp.net> writes: |> I have a question regarding how to extract and put into a list form, |> numbers from an internet science database file that I have already |> imported into my Mathematica 3.0 program using "ReadList". The problem |> is as follows: |> |> I want to extract from a file named "1997.ADJ" (which is a list of daily |> 10.7 cm solar flux values for the year 1997), certain columns of |> numbers from the period Sept. 1, 1997 to Dec. 31, 1997 which will form |> a list (time series) which I will use to plot, get a Fourier |> Transform, a power-spectrum, etc.. |> The file comes from the internet site: |> http://www.ngdc.noaa.gov/stp/stp.html and is found after getting into |> this site by the following steps: |> |> Click "Solar and Upper Atmosph." |> Click "Get Data" |> "" "Solar Radio" |> "" " Flux" |> "" "1997.ADJ" |> |> I am at this point where I have read in the file by using the ReadList |> function as follows: ReadList["a\1997.adj", String] My output is a |> table of average daily values for each month of 1997 with a header and |> footer section containing comments. I only want to extract the data |> values from four columns (Sep., Oct., Nov, Dec). I've tried using |> mathematica functions such as MemberQ, Character,and Select and have |> not been successful. At this point, I'd really appreciate some help! |> Got any ideas? |> |> Thanks for your time and consideration. I look forward to hearing from |> you. Because this table is slightly irregular, we'll have to do some fancy footwork to get the data you need: (* First, let's open the file as an InputStream[]. *) In[1]:= stmp = OpenRead["1997.ADJ"]; (* Skim past the five header lines. *) In[2]:= Do[Read[stmp, String], {5}] (* Now read in the 37 lines of data and and vertical spaces. *) In[3]:= daysString = Table[Read[stmp, String], {37}]; (* Then we map a function which converts each string into a stream and breaks the strings up into lists of individual strings. *) In[4]:= daysTable = Map[Block[{stmp = StringToStream[#], lst}, lst = ReadList[stmp, Word]; Close[stmp]; lst]&, daysString]; (* Close the data file. *) In[5]:= Close[stmp]; (* Any vertical spaces will produce empty lists, so we can throw those out. *) In[6]:= trimmedTable = DeleteCases[daysTable, {}]; (* Now we even out the array be entering rows 29 and 30 of Febrary. *) In[7]:= MapAt[Insert[#1, 0, 3] & , %, 29]; In[8]:= MapAt[Insert[#1, 0, 3] & , %, 30]; (* And do the same for day 31 of September, April, June, and November. *) In[8]:= finalTable = MapAt[ReplaceAll[#, {date_, jan_, mar_, may_, jul_, aug_, oct_, dec_} -> {date, jan, 0, mar, 0, may, 0, jul, aug, 0, oct, 0, dec}]&, %, 31] (* We can now get the last four columns and knock out the zero pads. *) In[9]:= desiredData = Map[Map[ToExpression, #]&, DeleteCases[Take[Transpose[finalTable], -4], 0, -1]] (* The nested Map operation is needed to turn the strings into numbers. *) Hope that helps. -- P.J. Hinton Mathematica Programming Group paulh at wolfram.com Wolfram Research, Inc. http://www.wolfram.com/~paulh/ Disclaimer: Opinions expressed herein are those of the author alone.