Re: Importing (and dealing with) multiple files
- To: mathgroup at smc.vnet.net
- Subject: [mg60799] Re: [mg60778] Importing (and dealing with) multiple files
- From: "David Annetts" <davidannetts at aapt.net.au>
- Date: Thu, 29 Sep 2005 05:41:07 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
Hi Dan,
> I have experimental data that is stored in a number of files.
> I've been importing the files (as tables) into Mathematica
> one-by-one, and working on them like so:
>
> rawdata=Import["C:\\filename.ASC", "Table"];
>
> croppedcorrelation=Take[rawdata,{75,185}];
>
> statistics=NonlinearRegress[croppedcorrelation, math_here...]
>
> then painstakingly picking out the relevant statistical
> parameter from each file, collecting them all, and later
> plotting them. What I'd really like to do is import an
> entire directory's worth of these files (each file is a
> different time point), do the same statistical thing to each
> file, save the relevant parameters from each file in a table
> form and then plot the whole shebang.
>
> One of the problems is that I don't know ahead of time how
> many .ASC files are in the directory, so I need to set up the
> For loop (or whatever structure) to account for that. I
> tried using FileNames[] to get the list of files, assigning
> them to an array, then using Import[] on each of those array
> elements. It didn't work.
Stating exactly how it didn't work (eg posting code) would've been useful.
> Does anyone have any suggestions?
You shouldn't need to use a For loop for this, and things are quite
straightforward if you can use Import[] instead of some customised reading
function. For example, we can set up some temporary data in a temporary
directory using
SetDirectory["c:/Tmpfiles"]
outf = "test_";
outf = StringJoin[outf, ToString[PaddedForm[#, 3, NumberPadding ->
{"0", "0"}, NumberSigns -> {"", ""}]], ".dat"] & /@ Range[5]
(
test = {#, 10 * Exp[-#/10] + .5 * Random[]} & /@ Range[128];
Export[outf[[#]], test, "TABLE"]
) & /@ Range[Length@outf]
Now, read them using
ifil = FileNames["test*.dat"];
data = Import[ifil[[#]], "TABLE"] & /@ Range[Length@ifil];
We can plot them using
(
ListPlot[
data[[#]], PlotRange -> {0, 10}, PlotLabel ->
StringJoin["File :
", ifil[[#]]]];
) & /@ Range[Length@ifil]
Or
Needs["Graphics`Graphics3D`"]
iplt = ListPlot[#, DisplayFunction -> Identity] & /@ data;
Show[StackGraphics[iplt], DisplayFunction -> $DisplayFunction];
And perform the regressions using
stat = NonlinearRegress[data[[#]], a Exp[b x], x, {{a, 10}, {b,
-1}}] & /@ Range[Length@ifil];
We can now pick the interesting bits from stat, for example
best = BestFitParameters /. stat
evar = EstimatedVariance /. stat
Regards,
Dave.