Re: Speed of repeated matrix calculation
- To: mathgroup at smc.vnet.net
- Subject: [mg49784] Re: [mg49717] Speed of repeated matrix calculation
- From: DrBob <drbob at bigfoot.com>
- Date: Sat, 31 Jul 2004 03:14:03 -0400 (EDT)
- References: <200407291143.HAA10250@smc.vnet.net> <opsbxiicf5iz9bcq@monster.cox-internet.com> <A5D9160A-E24C-11D8-A615-000D9350C9C2@videotron.ca>
- Reply-to: drbob at bigfoot.com
- Sender: owner-wri-mathgroup at wolfram.com
You might actually send me the data (export the x and r matrices as .csv files, for instance), and I'll see what I can figure out. Other than that, just a few things occur to me.
One is that Inner was significantly faster than Table (for me, with my example data).
Secondly, if there are lots of exact numbers it could slow things down, or having ANY exact numbers may prevent Mathematica using a compiled code for the dot products. Try replacing xx=N@x, rr=N@r, and use xx and rr in the Table statement. My data was all symbols, so this may not be it, but... It's easy to try it.
Third, check to see if x and r are packed arrays, using Developer`PackedArrayQ[]. Especially if one of them is packed and the other isn't, experiment with ToPackedArray and FromPackedArray to see if packing or unpacking speeds things up. If unpacking speeds things up, the downside is more storage, of course. You'd use, for instance, xx=Developer`ToPackedArray[x].
Bobby
On Fri, 30 Jul 2004 13:19:43 -0400, Gregory Lypny <gregory.lypny at videotron.ca> wrote:
> Hello Dr. Bob,
>
> I'm using version 5.0.0.
>
> The repeated quadratic form (portfolio variance calculations) is quick
> at .43 seconds. My exact notation is
>
> varianceRp = Table[{x[[i, All]].V.x[[i, All]]}, {i, 1, totalObs}],
>
> where totalObs is 7,320. The problem is with that pesky repeated dot
> product (portfolio return calculations). I use
>
> Rp = Table[{x[[i, All]].r[[i, All]]}, {i, 1, totalObs}] or
>
> Rp2 = Table[{x[[i]].r[[i]]}, {i, 1, totalObs}], and, in either case, it
> takes 128.99 seconds!
>
> Do you think it's the format of my data. Here are the first five rows
> of x and r:
>
> x
> \!\({{3\/10, 3\/10, 2\/5}, {0.3183977497466464`, 0.3638831425675959`, \
> 0.3177190712974436`}, {0.15984884693034268`, 0.6393953877213707`, \
> 0.20075558631757812`}, {0.28134163384933736`, 0.422012450774006`, \
> 0.29664596320473446`}, {0.2333851448354868`, 0.33340734976498115`, \
> 0.4332074073777712`}}\)
>
> r
> {{0.0613669, 0.120612, 0.111651}, {0.0371958,
> 0.0875982, 0.297405}, {-0.0165332, 0.141302,
> 0.242225}, {0.0174065, -0.0236906, 0.201915}, {0.0281807,
> 0.00599887,
> 0.227316}}
>
> Any thoughts?
>
> Regards,
>
> Gregory
>
> On Jul 29, 2004, at 10:28 PM, DrBob wrote:
>
>> Here are three ways to do it, with timings for a smaller problem:
>>
>> n=1000;
>> x=Array[f,{n,3}];
>> v=Array[h,{3,3}];
>> Timing[a=Tr[x.v.Transpose[x],List];]
>> Timing[b=Table[x[[i]].v.x[[i]],{i,1,n}];]
>> Timing[c=(#.v.#&)/@x;]
>> a == b == c
>>
>> {5.594 Second,Null}
>>
>> {0.031 Second,Null}
>>
>> {0.032 Second,Null}
>>
>> True
>>
>> The first method clearly isn't competitive. Eliminating it and
>> increasing the problem size, I get:
>>
>> n=7000;
>> x=Array[f,{n,3}];
>> v=Array[h,{3,3}];
>> Timing[b=Table[x[[i]].v.x[[i]],{i,1,n}];]
>> Timing[c=(#.v.#&)/@x;]
>> b == c
>>
>> {0.234 Second,Null}
>>
>> {0.203 Second,Null}
>>
>> True
>>
>> For the other problem, the following methods are very fast.
>>
>> n=7000;
>> x=Array[f,{n,3}];
>> y=Array[g,{n,3}];
>> v=Array[h,{3,3}];
>> Timing[a=Table[x[[i]].y[[i]],{i,1,n}];]
>> Timing[b=Inner[Dot,dum@@x,dum@@y,List];]
>> a == b
>>
>> {0.093 Second,Null}
>>
>> {0.063 Second,Null}
>>
>> True
>>
>> But the first is really the same as your method; are you using an
>> older version of Mathematica, perhaps?
>>
>> Bobby
>>
>> On Thu, 29 Jul 2004 07:43:39 -0400 (EDT), Gregory Lypny
>> <gregory.lypny at videotron.ca> wrote:
>>
>>> Hello everyone,
>>>
>>> Just curious to know whether I'm doing this repeated matrix
>>> calculation
>>> efficiently by using the Table command.
>>>
>>> I've got matrices x and y, which are both 7000x3, and a matrix V which
>>> is 3x3.
>>>
>>> If I create a 7000x1 vector, q, whose elements are equal to each of
>>> the
>>> row quadratic forms x.V.x, I can do it by creating a table.
>>>
>>> q = Table[{x[[i, All]].V.x[[i, All]]}, {i, 1, 7000}];
>>>
>>> The calculation is done in a split second on a G4 iBook.
>>>
>>> However, if I use Table to create a 7000x1 vector, d, of the dot
>>> products of the rows of x and y, the calculation takes more than two
>>> minutes!
>>>
>>> d = Table[{x[[i, All]].y[[i, All]]}, {i, 1, 7000}];
>>>
>>> Why the big difference in time? Is there a better way?
>>>
>>> Greg
>>>
>>>
>>>
>>
>>
>>
>> --
>> DrBob at bigfoot.com
>> www.eclecticdreams.net
>>
>
>
>
--
DrBob at bigfoot.com
www.eclecticdreams.net
- References:
- Speed of repeated matrix calculation
- From: Gregory Lypny <gregory.lypny@videotron.ca>
- Speed of repeated matrix calculation