Mathematica 9 is now available
Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2009

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Slow performance gathering property data from fitted objects

  • To: mathgroup at smc.vnet.net
  • Subject: [mg101023] Re: Slow performance gathering property data from fitted objects
  • From: Bill Rowe <readnews at sbcglobal.net>
  • Date: Sun, 21 Jun 2009 07:05:34 -0400 (EDT)

On 6/20/09 at 4:03 AM, genesplicer28 at yahoo.com (Todd Allen) wrote:

>During some recent research I asked Mathematica to calculate 70,000
>simple linear regression fits to 70,000 small datasets.  This
>worked well and resulted in a list of 70,000
>fitted model objects being finished in under 5 minutes.

>My next step was to collect the RSquared property for each of the
>70,000 fits contained in the list, and Mathematica took over 3
>hours to accomplish this.  This seems
>horrendously slow simply to gather values from a list.  Has anyone
>ran into a similar problem?  Any suggestions how I might be able to
>significantly speed the gathering of properties from large lists of
>fitted model objects?

>The snippet of code I was using to gather the RSquared values is
>below:

>piedlrRsquared = Table[piedlrmodels[[i]]["RSquared"], {i, 1, Length[pied=
lrmodels]}]
>// Timing;

Consider:

In[15]:= data = RandomReal[1, {1000, 5}];

In[16]:= Timing[models = LinearModelFit[#, {1, x}, x] & /@ data;]

Out[16]= {0.536635,Null}

In[17]:= Timing[#["RSquared"] & /@ models;]

Out[17]= {0.555774,Null}

In[18]:= Timing[Table[models[[n]]["RSquared"], {n, Length@models}];]

Out[18]= {1.10845,Null}

In[19]:= Timing[Correlation[#, Range[Length@#]]^2 & /@ data;]

Out[19]= {0.223803,Null}

Use of Table and Part involves some overhead that is avoided
using a different method.



  • Prev by Date: Re: compress lists with mean for equal elements
  • Next by Date: Re: Slow performance gathering property data from fitted
  • Previous by thread: Re: Slow performance gathering property data from fitted
  • Next by thread: Number format