Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: Re: Correlation function and data

• To: mathgroup at smc.vnet.net
• Subject: [mg24785] Re: [mg24748] Re: Correlation function and data
• From: Daniel Lichtblau <danl at wolfram.com>
• Date: Thu, 10 Aug 2000 00:32:26 -0400 (EDT)
• References: <200008090631.CAA00912@smc.vnet.net>
• Sender: owner-wri-mathgroup at wolfram.com

Deborah Leddon wrote:
> 
> Hello,
>  I am trying to construct an autocorrelation function that can be
> applied to a data set as follows;
> 
> n=Length[data];
> c[y_]:= Sum[(data[[i]] - Mean[data] )* ( data[[i+y]] - Mean[data])/
>                 Variance[data], {i,1,n - y}];
> corrfunction = Array[c, {n - 1}];
> 
> ListPlot[corrfunction, PlotJoined->True]
> 
> Problem is , this routine works well for data sets up to a length of
> 300 points, but gets unusually long for larger data sets , say
> around 1000-3000 points in length. I've tried  "Map" (using
> anonymous function rules), Table-Evaluate, etc..
> 
> Anyone got any ideas? I would much appreciate them!
> 
> Regards,
> Deb L.

You are doing alot of recomputation. If you precompute the mean and
variance, and translate the data before doing the correlations, you will
get a factor of n improvement.

<<Statistics`

data = Table[Random[], {1000}];
n = Length[data];
mean = Mean[data];
transdata = data - mean;
var = Variance[data];
c[y_]:= Sum[transdata[[i]]*transdata[[i+y]], {i,1,n-y}] / var;

In[41]:= Timing[corrfunction = Array[c, {n-1}];]
Out[41]= {20.64 Second, Null}

You can make this considerably faster using ListCorrelate instead of
Sum.

In[72]:= Timing[c2 = ListCorrelate[transdata, Drop[transdata,1], 1, 0] /
var;]
Out[72]= {0.01 Second, Null}

In[74]:= Max[Abs[c2-corrfunction]]
                   -13
Out[74]= 1.42109 10


Daniel Lichtblau
Wolfram Research

• References:
  • Re: Correlation function and data
    • From: "Deborah Leddon " <Dleddon@students.cas.unt.edu>