Re: Slow performance gathering property data from fitted objects
- To: mathgroup at smc.vnet.net
- Subject: [mg101023] Re: Slow performance gathering property data from fitted objects
- From: Bill Rowe <readnews at sbcglobal.net>
- Date: Sun, 21 Jun 2009 07:05:34 -0400 (EDT)
On 6/20/09 at 4:03 AM, genesplicer28 at yahoo.com (Todd Allen) wrote:
>During some recent research I asked Mathematica to calculate 70,000
>simple linear regression fits to 70,000 small datasets. This
>worked well and resulted in a list of 70,000
>fitted model objects being finished in under 5 minutes.
>My next step was to collect the RSquared property for each of the
>70,000 fits contained in the list, and Mathematica took over 3
>hours to accomplish this. This seems
>horrendously slow simply to gather values from a list. Has anyone
>ran into a similar problem? Any suggestions how I might be able to
>significantly speed the gathering of properties from large lists of
>fitted model objects?
>The snippet of code I was using to gather the RSquared values is
>below:
>piedlrRsquared = Table[piedlrmodels[[i]]["RSquared"], {i, 1, Length[pied=
lrmodels]}]
>// Timing;
Consider:
In[15]:= data = RandomReal[1, {1000, 5}];
In[16]:= Timing[models = LinearModelFit[#, {1, x}, x] & /@ data;]
Out[16]= {0.536635,Null}
In[17]:= Timing[#["RSquared"] & /@ models;]
Out[17]= {0.555774,Null}
In[18]:= Timing[Table[models[[n]]["RSquared"], {n, Length@models}];]
Out[18]= {1.10845,Null}
In[19]:= Timing[Correlation[#, Range[Length@#]]^2 & /@ data;]
Out[19]= {0.223803,Null}
Use of Table and Part involves some overhead that is avoided
using a different method.