Re: More memory-efficient inner product for large last

*To*: mathgroup at smc.vnet.net*Subject*: [mg106884] Re: [mg106870] More memory-efficient inner product for large last*From*: Leonid Shifrin <lshifr at gmail.com>*Date*: Tue, 26 Jan 2010 06:33:48 -0500 (EST)*References*: <201001251009.FAA09421@smc.vnet.net>

Vince, Actually I must apologize. The intended code was Clear[dotLazy]; dotLazy[first_?ArrayQ, second_?ArrayQ] := With[{fdims = Most@Dimensions@first, sdims = Most@Dimensions@second}, Module[{a, b, plus}, With[{firstSymbolic = Array[a, fdims], secondSymbolic = Array[b, sdims]}, plus[x_] := x; plus[x__] /; Length[{x}] =!= 2 := Fold[plus, First@{x}, Rest@{x}]; plus[x_, y_] /; Head[x] =!= plus := Total[{x, y} /. {a[i_, j_] :> first[[i, j]], a[i_] :> first[[i]], b[i_] :> second[[i]], b[i_, j_] :> second[[i, j]]}]; firstSymbolic.secondSymbolic /. Plus -> plus]]]; which is different from the one I posted by plus[x_, y_] /; Head[x] =!= plus instead of plus[x_, y_] /; Head[x] =!= Plus (plus vs Plus). This was intended to avoid infinite recursion for cases like plus[plus[1,2],3], but actually, due to the way Fold wroks, this is unnecessary alltogether. Likewise, the rule plus[x_]:=x is an unnecessary garbage. Some intermediate variables can also be skipped. The following will work just as well, while being a bit more concise: Clear[dotLazy]; dotLazy[first_?ArrayQ, second_?ArrayQ] := Module[{fdims, sdims, firstSymbolic, secondSymbolic, a, b, plus}, plus[x_, y_] := Total[{x, y} /. {z_a :> (first[[##]] & @@ z), z_b :> (second[[##]] & @@ z)}]; plus[x__] := Fold[plus, First@{x}, Rest@{x}]; Dot @@ MapThread[ Array, {{a, b}, Most@Dimensions@# & /@ {first, second}}] /. Plus :> plus] Regards, Leonid On Mon, Jan 25, 2010 at 5:53 PM, Vincent N. Virgilio <virgilio at ieee.org>wrote: > > > On Mon, Jan 25, 2010 at 9:16 AM, Leonid Shifrin <lshifr at gmail.com> wrote: > >> Hi Vince, >> >> I suggest that you use lazy matrix multiplication, which can be >> implemented for example as follows: >> >> >> > SNIP > > >> As can be seen, my version is less memory-efficient for list-to-list dot >> product, but vastly >> more efficient for other operations. I did not test on such huge lists as >> your original ones since >> I don't have so much memory at my disposal at the moment (running Eclipse >> and SQLDeveloper), >> but I would expect similar effect. >> >> Hope this helps. >> >> Regards, >> Leonid >> >> > Leonid, > > Phenomenal work! Yours saved ~ 1.5GB RAM over mine (for 1.2M elements). > > Thank you very much. > > Vince Virgilio > > >

**References**:**More memory-efficient inner product for large last dimension?***From:*Vince Virgilio <blueschi@gmail.com>