Re: More memory-efficient inner product for large last

• To: mathgroup at smc.vnet.net
• Subject: [mg106884] Re: [mg106870] More memory-efficient inner product for large last
• From: Leonid Shifrin <lshifr at gmail.com>
• Date: Tue, 26 Jan 2010 06:33:48 -0500 (EST)
• References: <201001251009.FAA09421@smc.vnet.net>

```Vince,

Actually I must apologize. The intended code was

Clear[dotLazy];
dotLazy[first_?ArrayQ, second_?ArrayQ] :=
With[{fdims = Most@Dimensions@first, sdims = Most@Dimensions@second},
Module[{a, b, plus},
With[{firstSymbolic = Array[a, fdims],
secondSymbolic = Array[b, sdims]},
plus[x_] := x;
plus[x__] /; Length[{x}] =!= 2 := Fold[plus, First@{x}, Rest@{x}];
plus[x_, y_] /; Head[x] =!= plus :=
Total[{x, y} /. {a[i_, j_] :> first[[i, j]],
a[i_] :> first[[i]], b[i_] :> second[[i]],
b[i_, j_] :> second[[i, j]]}];
firstSymbolic.secondSymbolic /. Plus -> plus]]];

which is different from the one I posted by plus[x_, y_] /; Head[x] =!= plus
plus[x_, y_] /; Head[x] =!= Plus (plus vs Plus). This was intended to avoid
infinite recursion
for cases like plus[plus[1,2],3], but actually, due to the way Fold wroks,
this is unnecessary
alltogether. Likewise, the rule plus[x_]:=x is an unnecessary garbage. Some
intermediate variables
can also be skipped. The following will work  just as well, while being a
bit more concise:

Clear[dotLazy];
dotLazy[first_?ArrayQ, second_?ArrayQ] :=
Module[{fdims, sdims, firstSymbolic, secondSymbolic, a, b, plus},
plus[x_, y_] :=   Total[{x, y} /. {z_a :> (first[[##]] & @@ z),
z_b :> (second[[##]] & @@ z)}];
plus[x__] := Fold[plus, First@{x}, Rest@{x}];
Array, {{a, b}, Most@Dimensions@# & /@ {first, second}}] /.
Plus :> plus]

Regards,
Leonid

On Mon, Jan 25, 2010 at 5:53 PM, Vincent N. Virgilio <virgilio at ieee.org>wrote:

>
>
> On Mon, Jan 25, 2010 at 9:16 AM, Leonid Shifrin <lshifr at gmail.com> wrote:
>
>> Hi Vince,
>>
>> I suggest that you use lazy matrix multiplication, which can be
>> implemented for example as follows:
>>
>>
>>
> SNIP
>
>
>> As can be seen, my version is less memory-efficient for list-to-list dot
>> product, but vastly
>> more efficient for other operations. I did not test on such huge lists as
>> your original ones since
>> I don't have so much memory at my disposal at the moment (running Eclipse
>> and SQLDeveloper),
>> but I would expect similar effect.
>>
>> Hope this helps.
>>
>> Regards,
>> Leonid
>>
>>
> Leonid,
>
> Phenomenal work! Yours saved ~ 1.5GB RAM over mine (for 1.2M elements).
>
> Thank you very much.
>
> Vince Virgilio
>
>
>

```

• Prev by Date: Re: Re: NotebookGet/Read/EvaluateSelection Issues
• Next by Date: Re: NotebookGet/Read/EvaluateSelection Issues
• Previous by thread: Re: More memory-efficient inner product for large last
• Next by thread: Re: More memory-efficient inner product for large last