Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: runs slowly?

• To: mathgroup at smc.vnet.net
• Subject: [mg122356] Re: runs slowly?
• From: Armand Tamzarian <mike.honeychurch at gmail.com>
• Date: Wed, 26 Oct 2011 17:39:09 -0400 (EDT)
• Delivered-to: l-mathgroup@mail-archive0.wolfram.com
• References: <201110231025.GAA10587@smc.vnet.net> <j83a7d$k07$1@smc.vnet.net> <j862lo$5t7$1@smc.vnet.net>

On Oct 25, 9:22 pm, Armand Tamzarian <mike.honeychu... at gmail.com>
wrote:
> On Oct 24, 8:13 pm, Michael Stern <nycst... at gmail.com> wrote:
>
> > All the date functions in Mathematica are extremely slow. In benchmarks I sent to Wolfram a few months ago (and some of which can be seen athttp://www.wheels.org/monkeywrench/?p=453), the date functions in Mathematica ran about 1/82 the speed of the same functions in Visual Basic for Application s. If you know the format of the dates you are converting, you may be able to roll your own date functions much faster than Wolfram's. I have encourag ed them to improve this code, but they will probably pay closer attention if others complain as well.
>
> > Michael
>
> > Sent from my iPad
>
> I'd have to agree with you.
>
> If you're doing thousands or tens of thousands of calculations daily
> that require date manipulations these are often the rate determining
> step. It seems like an example where a "super function" is a
> significant drawback. What I mean is that if you are working with a
> constant format you can, as Michael says above, write your own
> function that is between one and two orders of magnitude faster than
> the built in function. For the volumes of calculations that you might
> find yourself doing if working in a financial entity, these efficiency
> gains can be the difference between Mathematica being a competitive software
> option or not.
>
> A competitor software might have a different function for each date
> format, analogous to the roll your own function that you write,
> whereas Mathematica might have one super function that handles all format s
> etc. etc. (but slowly). For prototyping this is not likely to be a
> problem. For large scale high volumes of calculations the competitor
> wins on efficiency.
>
> Mike
>
>
>
>
>
>
>
>
>
> > On Oct 23, 2011, at 6:25 AM, Robert McHugh <bob_mchugh_2... at yahoo.com> wrote:
>
> > > The following code takes over a minute to run on my machine.  Is this expected behavior?
>
> > > t = {"14-May-10 09:58:05", {"Day", "-", "MonthNameShort", "-", "YearShort", " ", "Hour", ":", "Minute", ":", "Second"}};
> > > tList = Table[t, {i, 100000}];
> > > a = Timing[AbsoluteTime[#] & /@ tList ;]
>
> > > For reference, an operation like the following takes less than a tenth of a second. (Of course this second example needs quite a bit of modification to provide a correct answer, but it does show how fast the program can operate on a large list.)
> > > t = {14, 5, 10, 9, 58, 05};
> > > tList = Table[t, {i, 100000}];
> > > a = Timing[( ((#[[3]] 0 + #[[2]] 30 + #[[1]]24) + #[[4]]) 60 + #[[5]]) 60 + #[[6]] & /@ tList;]
>
> > > Some background: am analyzing some historical data (about 500 000 records, one data point a minute for about a year) and am making a few utilities to retrieve the data for any given time interval.   My original plan was to change the time stamp to absolute time and then use a select statement.  This step in the above example, changing the time stamps to absolute time, is the rate limiting step in the code (everything else runs in about 5 seconds).
>
> > > Was wondering if someone could explain why AbsoluteTime[] is relatively slow operation and perhaps some faster operations for date and time comparisons.
>
> > > Thanks

On a related matter, one of the frustrating things I find is lack of
packing for date lists. It would be very useful if lists of dates
could be formed into packed arrays. Each element in a date list is a
known type (Integer for the first 5 and Real for the last) so given
that fixed type format of date lists it seems to me that packing could
be implemented.

Additionally if you had time series data in which the data were the
same type you basically have a list of the form {date/time (all fixed
types), value (fixed type)} so being able to pack time series data
would improve the efficiency of these operations immensely. Currently
these operations are slow relative to other list operations due (in
part) to the lack of packing.

Mike

• References:
  • AbsoluteTime[] runs slowly?
    • From: Robert McHugh <bob_mchugh_2000@yahoo.com>