Re: Importing (and dealing with) multiple files

*To*: mathgroup at smc.vnet.net*Subject*: [mg60799] Re: [mg60778] Importing (and dealing with) multiple files*From*: "David Annetts" <davidannetts at aapt.net.au>*Date*: Thu, 29 Sep 2005 05:41:07 -0400 (EDT)*Sender*: owner-wri-mathgroup at wolfram.com

Hi Dan, > I have experimental data that is stored in a number of files. > I've been importing the files (as tables) into Mathematica > one-by-one, and working on them like so: > > rawdata=Import["C:\\filename.ASC", "Table"]; > > croppedcorrelation=Take[rawdata,{75,185}]; > > statistics=NonlinearRegress[croppedcorrelation, math_here...] > > then painstakingly picking out the relevant statistical > parameter from each file, collecting them all, and later > plotting them. What I'd really like to do is import an > entire directory's worth of these files (each file is a > different time point), do the same statistical thing to each > file, save the relevant parameters from each file in a table > form and then plot the whole shebang. > > One of the problems is that I don't know ahead of time how > many .ASC files are in the directory, so I need to set up the > For loop (or whatever structure) to account for that. I > tried using FileNames[] to get the list of files, assigning > them to an array, then using Import[] on each of those array > elements. It didn't work. Stating exactly how it didn't work (eg posting code) would've been useful. > Does anyone have any suggestions? You shouldn't need to use a For loop for this, and things are quite straightforward if you can use Import[] instead of some customised reading function. For example, we can set up some temporary data in a temporary directory using SetDirectory["c:/Tmpfiles"] outf = "test_"; outf = StringJoin[outf, ToString[PaddedForm[#, 3, NumberPadding -> {"0", "0"}, NumberSigns -> {"", ""}]], ".dat"] & /@ Range[5] ( test = {#, 10 * Exp[-#/10] + .5 * Random[]} & /@ Range[128]; Export[outf[[#]], test, "TABLE"] ) & /@ Range[Length@outf] Now, read them using ifil = FileNames["test*.dat"]; data = Import[ifil[[#]], "TABLE"] & /@ Range[Length@ifil]; We can plot them using ( ListPlot[ data[[#]], PlotRange -> {0, 10}, PlotLabel -> StringJoin["File : ", ifil[[#]]]]; ) & /@ Range[Length@ifil] Or Needs["Graphics`Graphics3D`"] iplt = ListPlot[#, DisplayFunction -> Identity] & /@ data; Show[StackGraphics[iplt], DisplayFunction -> $DisplayFunction]; And perform the regressions using stat = NonlinearRegress[data[[#]], a Exp[b x], x, {{a, 10}, {b, -1}}] & /@ Range[Length@ifil]; We can now pick the interesting bits from stat, for example best = BestFitParameters /. stat evar = EstimatedVariance /. stat Regards, Dave.