Re: Need speed partitioning WeatherData
- To: mathgroup at smc.vnet.net
- Subject: [mg112517] Re: Need speed partitioning WeatherData
- From: Bill Rowe <readnews at sbcglobal.net>
- Date: Sat, 18 Sep 2010 07:25:39 -0400 (EDT)
On 9/17/10 at 6:41 AM, public at fonseca.info (P. Fonseca) wrote:
>Meanwhile I figured out that the problem comes from the time
>DateDifference takes.
>One process of DateDifference costs 0.0045 seconds on my computer
>(ex. DateDifference[{1990, 10, 1, 0, 0, 0}, {1999, 10, 1, 0, 0,
>0}]).
<snip>
>Does someone has a simpler solution for a 100x speed up?
Try converting your dates to integers using AbsoluteTime then
computing the difference. That is on my system:
In[2]:= DateDifference[{1990, 10, 1, 0, 0, 0}, {1999, 10, 1, 0, 0,
0}] // Timing
Out[2]= {0.009792,3287}
In[3]:= Subtract @@ (AbsoluteTime /@ {{1990, 10, 1, 0, 0, 0}, {1999,
10, 1, 0, 0, 0}})/86400 // Timing
Out[3]= {0.000142,-3287}
which is close to the 100X speed up you are looking for. Note,
these times are too small to be considered very accurate. But if
the date formats were pre-processed to be in the form output by
AbsoluteTime, it is clear there would be a substantial speed improvement.