Re: Time series minima and maxima
- To: mathgroup at smc.vnet.net
- Subject: [mg115947] Re: Time series minima and maxima
- From: Jagra <jagra24891 at mypacks.net>
- Date: Thu, 27 Jan 2011 03:39:37 -0500 (EST)
- References: <ihm4nf$kgv$1@smc.vnet.net>
Some follow up that I hope spurs additional discussion.
Get and format some data:
data = FinancialData["SPY", {"May 1, 2006", "Jan. 21, 2011"}][[All,
2]];
data = data/First@data;
data = Transpose[{Range[Length@data], data}];
Define 2 functions:
First Ray's method (I think I've got the typos fixed this time):
findMinimaMaxima[data_, window_] := With[{k = window},
data[[k + Flatten@Position[Partition[data[[All, 2]], 2 k + 1, 1],
x_List /; x[[k + 1]] < Min[Delete[x, k + 1]] || x[[k + 1]] >
Max[Delete[x, k + 1]]]]]]
Now another approach, although not as flexible:
findMinimaMaxima2[data_] := data[[Accumulate@(Length[#] & /@
Split[Prepend[Sign[Rest@data[[All, 2]] - Most@data[[All, 2]]], 0]])]]
Look at what each the functions does. First findMinimaMaxima2[]:
minmax = findMinimaMaxima2[data];
{Length@data, Length@minmax}
ListLinePlot@minmax
This selects all minima and maxima and results (in this instance) in
about a 49% data compression, but it doesn't have the flexibility of
expanding the window.
Ray's method, with a window of 2, yields fewer and arguably more
important extrema:
minmax2 = findMinimaMaxima[data, 2];
{Length@data, Length@minmax2}
ListLinePlot@minmax2
But look at what happens when we expand the window to 60:
minmax2 = findMinimaMaxima[data, 60];
ListLinePlot[{data, minmax2}]
Some of the minima and maxima no longer alternate.
Applying findMinimaMaxima2[] to the output of findMinimaMaxima[] gives
a workaround...
minmax3 = findMinimaMaxima2[minmax2];
ListLinePlot[{data, minmax2, minmax3}]
But this seems like a clumsy way to address the problem.
So, the idea of using a fixed window to look left and right doesn't
quite do everything one would like. I began thinking about an
alternative that could use a range value R (e.g. a percent move up or
down) that the function would need to meet or exceed to set the next
minima or maxima. Here's my first try:
findMinimaMaxima3[data_, R_] := Module[{d, n, positions},
d = data[[All, 2]];
n = Transpose[{data[[All, 1]], Rest@FoldList[If[(#2 <= #1 + #1*R &&
#2 >= #1) || (#2 >= #1 - #1* R && #2 <= #1), #1, #2] &, d[[1]], d]}];
n = Sign[Rest@n[[All, 2]] - Most@n[[All, 2]]];
positions = Flatten@Rest[Most[Position[n, Except[0]]]];
data[[positions]]
]
minmax4 = findMinimaMaxima3[data, 0.1];
ListLinePlot[{data, minmax4}]
This too benefits from post processing with findMinimaMaxima2[]
ListLinePlot[{data, findMinimaMaxima2[minmax4]}]
But if you look closely, you see that it misses the extremes if they
go beyond the R value in several positions - including the chart's
absolute minimum and maximum as well as along the big moves up and
down. Changing the R value shows how it misses the top and bottoms
even more:
minmax4 = findMinimaMaxima3[data, 0.15];
ListLinePlot[{data, minmax4}]
So, I need to reconsider. Anyone can look at a plot of the data and
easily identify the important minima and maxima. It seems harder to
get an algorithm to do it. A window and/or an R value seem important
to the solution, but neither on their own seems enough (at least not
in the approaches above).
Can anyone extend any of the approaches shown or suggest an
alternative to identifying the important minima and maxima?
Happy to forward a notebook with all of this code and discussion in
it. Let me know if anyone needs it.
Thank you,
Jagra