Re: Slow performance gathering property data from fitted objects
- To: mathgroup at smc.vnet.net
- Subject: [mg101023] Re: Slow performance gathering property data from fitted objects
- From: Bill Rowe <readnews at sbcglobal.net>
- Date: Sun, 21 Jun 2009 07:05:34 -0400 (EDT)
On 6/20/09 at 4:03 AM, genesplicer28 at yahoo.com (Todd Allen) wrote: >During some recent research I asked Mathematica to calculate 70,000 >simple linear regression fits to 70,000 small datasets. This >worked well and resulted in a list of 70,000 >fitted model objects being finished in under 5 minutes. >My next step was to collect the RSquared property for each of the >70,000 fits contained in the list, and Mathematica took over 3 >hours to accomplish this. This seems >horrendously slow simply to gather values from a list. Has anyone >ran into a similar problem? Any suggestions how I might be able to >significantly speed the gathering of properties from large lists of >fitted model objects? >The snippet of code I was using to gather the RSquared values is >below: >piedlrRsquared = Table[piedlrmodels[[i]]["RSquared"], {i, 1, Length[pied= lrmodels]}] >// Timing; Consider: In[15]:= data = RandomReal[1, {1000, 5}]; In[16]:= Timing[models = LinearModelFit[#, {1, x}, x] & /@ data;] Out[16]= {0.536635,Null} In[17]:= Timing[#["RSquared"] & /@ models;] Out[17]= {0.555774,Null} In[18]:= Timing[Table[models[[n]]["RSquared"], {n, Length@models}];] Out[18]= {1.10845,Null} In[19]:= Timing[Correlation[#, Range[Length@#]]^2 & /@ data;] Out[19]= {0.223803,Null} Use of Table and Part involves some overhead that is avoided using a different method.