Re: CONDITION/DATA ANALYSIS/PLOTS
- To: mathgroup at smc.vnet.net
- Subject: [mg107918] Re: CONDITION/DATA ANALYSIS/PLOTS
- From: Luci Ellis <luci at verbeia.com>
- Date: Tue, 2 Mar 2010 07:56:16 -0500 (EST)
- References: <hmak7b$bje$1@smc.vnet.net> <hmimod$5rq$1@smc.vnet.net>
Dear jihane,
I am not Bob Hanlon, but let's see if I can also be of some help. It's
hard to be sure without seeing your data if this would work. I am
assuming that your data file is some sort of matrix of numbers, with
"End Of File" at the end of each column, as your code suggests.
On 2010-03-02 20:45:49 +1100, jihane said:
>
> << PlotLegends`
>
> (* Change Flight number in labelling *)
>
> SetDirectory[NotebookDirectory[]];
>
> cutOffFreq = 20;
> fileTime = 85;
>
>
> (**********************USER INPUT***********************************)
> \
>
> (*Specify column numbers within text file to be extracted*)
> fineTimeCol = 1;
>
> X0Col = 6;
> Y0Col = 7;
> Z0Col = 8;
>
> (*Total number of columns in the text file*)
> numberOfColumns = 12;
>
>
> (*For RMS plots*)
> (*Specify how often to calculate RMS values; i.e. every ___ seconds*)
> \
>
> RMSseconds = 1;
>
> (*For PSD plots*)
> (*Choose how often to integrate PSD plots to find RMS contribution*)
> \
>
> fileTime = 85;
> numSeconds = 60;
>
>
>
>
> RawDataPlots = True;
>
> (********************************************************************)
>
>
> (*The samples are collected at 32Hz for a85 minute file*)
>
> sampleFreq = 32;
>
> (*Read data from file*)
> data = Import["UofT_Nanowire_Flight1 - Copy.dat"];
>
> listOfZeroes = Table[0, {i, numberOfColumns}];
>
> (*Specify which columns to read*)
> td = ReplacePart[listOfZeroes, fineTimeCol -> 1];
>
> x0d = ReplacePart[listOfZeroes, X0Col -> 1];
> y0d = ReplacePart[listOfZeroes, Y0Col -> 1];
> z0d = ReplacePart[listOfZeroes, Z0Col -> 1];
>
> (*Split data into appropriate columns, drop "EndOfFile" at the end of \
> the list*)
>
>
> X0D = Drop[data.x0d, -1];
> Y0D = Drop[data.y0d, -1];
> Z0D = Drop[data.z0d, -1];
Here is the first thing I'd change. Instead of ReplacePart and
listOfZeroes and so on, why not just use:
X0D = Most[data[[All,X0Col]];
Y0D = Most[data[[All,X0Col]];
Z0D = Most[data[[All,X0Col]];
note that Most[] is equivalent to Drop[x,-1]
>
>
> (********************************************************************************************************************)
\
(*Start
>
> analysis for each group of X, Y, Z*)
> (********************************************************************************************************************)
(*Assign
>
> the X, Y, Z lists*)
>
> XD = X0D;
> YD = Y0D;
> ZD = Z0D;
>
> XDNew = {};
> YDNew = {};
> ZDNew = {};
>
> LZ = Length[ZD];
>
> For[j = 1, j <= LZ, j++;
> If[ZD[[j]] < 0.05, ZDNew = Append[ZDNew, ZD[[j]]];
> XDNew = Append[XDNew, XD[[j]]]; YDNew = Append[YDNew, YD[[j]]]];
> ];
This is where Bob Hanlon's post is relevant. He intended you to replace
the For[] loop above with something like this.
SplitBy[data /. x_?(# > .1 &) -> Null, NumericQ] /. {Null, ___} -> Sequence[]
But because you are really wanting to split a three-vector (X, Y, Z) by
the value of Z, it's a bit trickier.
You need the full syntax of Replace (/.) to specify the level of the
list you are replacing at.
plotData = SplitBy[Replace[Transpose[{X0D,Y0D,Z0D}], x_?(Last[#] > .005
&) -> Null, 1], Head] /. {Null} ->
Sequence[]
To see what is going on, let's break it up. Here are your triples
data = RandomReal[{0, .1}, {30, 3}]
{{0.0695089, 0.041404, 0.000962538}, {0.0542868, 0.0356201,
0.0172699}, {0.0126049, 0.0977735, 0.0267022}, {0.0320958, 0.0640111,
0.0670509}, {0.0248244, 0.0637462, 0.0426013}, {0.0539459, 0.0567951,
0.0899871}, {0.0837307, 0.0309503, 0.0575072}, {0.0844311, 0.035911,
0.0186672}, {0.0266531, 0.0824807, 0.078472}, {0.0546202, 0.0441246,
0.0317306}, {0.0935794, 0.0847413, 0.0922612}, {0.0984859, 0.0546285,
0.000364843}, {0.0189445, 0.00818944, 0.0883822}, {0.00642631, 0.0539467,
0.0780225}, {0.0626157, 0.0372797, 0.0154142}, {0.0317539, 0.0259473,
0.0125016}, {0.0580047, 0.0659501, 0.0684925}, {0.0333913, 0.0775575,
0.0620592}, {0.0303167, 0.0248388, 0.067798}, {0.00594988, 0.0886844,
0.0430903}, {0.0577544, 0.0335647, 0.00181778}, {0.0535661, 0.049339,
0.0987746}, {0.0832517, 0.00895941, 0.0507834}, {0.0284413, 0.0289574,
0.027981}, {0.0943071, 0.000771151, 0.0806029}, {0.0440539, 0.0653912,
0.0788237}, {0.0167852, 0.045078, 0.0358629}, {0.0972686, 0.0207125,
0.0452583}, {0.085462, 0.0508639, 0.0438656}, {0.0441169, 0.0377907,
0.021743}}
You want to get rid of the triples where the last element is greater
than the threshold.
In[42]:= Replace[data, x_?(Last[#] > .08 &) -> Null, 1]
Out[42]= {{0.0695089, 0.041404, 0.000962538}, {0.0542868, 0.0356201,
0.0172699}, {0.0126049, 0.0977735, 0.0267022}, {0.0320958, 0.0640111,
0.0670509}, {0.0248244, 0.0637462, 0.0426013}, Null, {0.0837307, 0.0309503,
0.0575072}, {0.0844311, 0.035911, 0.0186672}, {0.0266531, 0.0824807,
0.078472}, {0.0546202, 0.0441246, 0.0317306}, Null, {0.0984859, 0.0546285,
0.000364843}, Null, {0.00642631, 0.0539467, 0.0780225}, {0.0626157,
0.0372797, 0.0154142}, {0.0317539, 0.0259473, 0.0125016}, {0.0580047,
0.0659501, 0.0684925}, {0.0333913, 0.0775575, 0.0620592}, {0.0303167,
0.0248388, 0.067798}, {0.00594988, 0.0886844, 0.0430903}, {0.0577544,
0.0335647, 0.00181778}, Null, {0.0832517, 0.00895941,
0.0507834}, {0.0284413, 0.0289574, 0.027981}, Null, {0.0440539, 0.0653912,
0.0788237}, {0.0167852, 0.045078, 0.0358629}, {0.0972686, 0.0207125,
0.0452583}, {0.085462, 0.0508639, 0.0438656}, {0.0441169, 0.0377907,
0.021743}}
As you can see, what we have after this step is a list of elements,
some of which are triples (Head is List) and some of which are the
value Null (Head is Symbol). So we can split them according to each
element's Head.
In[60]:= SplitBy[Replace[data, x_?(Last[#] > .08 &) -> Null, 1], Head]
Out[60]= {{{0.0695089, 0.041404, 0.000962538}, {0.0542868, 0.0356201,
0.0172699}, {0.0126049, 0.0977735, 0.0267022}, {0.0320958, 0.0640111,
0.0670509}, {0.0248244, 0.0637462, 0.0426013}}, {Null}, {{0.0837307,
0.0309503, 0.0575072}, {0.0844311, 0.035911, 0.0186672}, {0.0266531,
0.0824807, 0.078472}, {0.0546202, 0.0441246,
0.0317306}}, {Null}, {{0.0984859, 0.0546285,
0.000364843}}, {Null}, {{0.00642631, 0.0539467, 0.0780225}, {0.0626157,
0.0372797, 0.0154142}, {0.0317539, 0.0259473, 0.0125016}, {0.0580047,
0.0659501, 0.0684925}, {0.0333913, 0.0775575, 0.0620592}, {0.0303167,
0.0248388, 0.067798}, {0.00594988, 0.0886844, 0.0430903}, {0.0577544,
0.0335647, 0.00181778}}, {Null}, {{0.0832517, 0.00895941,
0.0507834}, {0.0284413, 0.0289574, 0.027981}}, {Null}, {{0.0440539,
0.0653912, 0.0788237}, {0.0167852, 0.045078, 0.0358629}, {0.0972686,
0.0207125, 0.0452583}, {0.085462, 0.0508639, 0.0438656}, {0.0441169,
0.0377907, 0.021743}}}
And then use the final replacement rule {Null..} ->Sequence[] to "get
rid" of the {Null} lists. The double dot should handle cases where
there are successive Null elements in a sublist, which don't appear in
my test data.
You are left with lists of triples. You can plot each subset of triples
using something like this:
Grid[Partition[Graphics3D[{Blue, Thick, Line[#]}] & /@ plotData, 2]]
Or if that isn't what you want, you can split the triples up again into
columns and plot against time.
>
> (*xMean= mean of raw data rounded to 6 decimal places*)
Why do you need to round the data? And did you know you could do all
this in less code like this?
{XDemeaned, YDemeaned, ZDemeaned, XDemeanedNew, YDemeanedNew, ZDemeanedNew } =
Standardize[#, Mean,( 1 &)]& /@ {XD, YD, ZD, XDNew, YDNew, ZDNew}
> xMean = N[Mean[XD], 6];
> yMean = N[Mean[YD], 6];
> zMean = N[Mean[ZD], 6];
>
> xMeanNew = N[Mean[XDNew], 6];
> yMeanNew = N[Mean[YDNew], 6];
> zMeanNew = N[Mean[ZDNew], 6];
>
> (*Demean the raw data*)
> XDemeaned = XD - xMean;
> YDemeaned = YD - yMean;
> ZDemeaned = ZD - zMean;
>
> XDemeanedNew = XDNew - xMeanNew;
> YDemeanedNew = YDNew - yMeanNew;
> ZDemeanedNew = ZDNew - zMeanNew;
>
> (*Create list of course time based on number of data points and \
> sample frequency*)
> TD = (Range[Length[XD]] - 1)/sampleFreq;
> TDNew = (Range[Length[ZDNew]] - 1)/sampleFreq;
>
>
> (*Time interval*)
> dt = TD[[2]] - TD[[1]];
> dtNew = TDNew[[2]] - TDNew[[1]];
{extra code snipped}
I hope that helps.
Regards,
Luci