Re: Re: Map vs. Table
- To: mathgroup at smc.vnet.net
- Subject: [mg75455] Re: [mg75412] Re: Map vs. Table
- From: János <janos.lobb at yale.edu>
- Date: Tue, 1 May 2007 03:26:48 -0400 (EDT)
- References: <f0sfrs$ncb$1@smc.vnet.net> <200704281004.GAA09221@smc.vnet.net>
On Apr 28, 2007, at 6:04 AM, Szabolcs wrote:
> zac wrote:
>> Dear Group,
>>
>> consider a set of data (y values):
>>
>> data = Table[i*j, {i, 5}, {j, 10}]
>>
>> I want to label each Integer with an x value:
>>
>> label := Table[{i, #} & /@ data[[i]], {i, Length[data]}]
>>
>
> It is better to use Set instead of SetDelayed here, otherwise the
> Table
> will be re-evaluated every time you use label. (Use = instead of :=)=
>
>> I've understand that list creation is much faster with Map than with
>> Table.
>
> It is generally a good idea to avoid explicitly indexing lists when =
> not
> necessary. But here you *do* need the index i, as a label.
>
>> Is there an effective way to convert the Table to Map in the label
>> function (either with nested Map or to incorporate the Table function
>> into the Map)?
>>
>> Would it be really faster for very large datasets than with Table?
>>
>> thanks
>>
>> Istvan Zachar
>>
>
> There are many ways to do this calculation. If you want to avoid
> indexing data, then you either need to use MapIndexed or pre-generate
> the labels.
>
> I tried these:
>
> In[46]:=
> data=Table[i j, {i, 2000}, {j, 2000}];
>
> In[47]:=
> Timing[label=Table[{i,#}& /@ data[[i]], {i, Length[data]}];]
>
> Out[47]=
> {1.031 Second,Null}
>
> In[48]:=
> Timing[label=MapIndexed[First@Outer[List, #2, #1]&, data];]
>
> Out[48]=
> {1.407 Second,Null}
>
> In[49]:=
> Timing[label=MapIndexed[Thread[Append[#2, #1]]&, data];]
>
> Out[49]=
> {1.609 Second,Null}
>
> In[50]:=
> Timing[label=Thread/@Transpose[{Range@Length@data, data}];]
>
> Out[50]=
> {1.422 Second,Null}
>
> You must not take these timings too seriously (the times are too
> short,
> and inconsistent with every run), though they suggest that using Table
> is faster then the rest.
>
> The problem is that with larger data matrices only Table works, all =
> the
> others run out of memory! I guess that this is because data is
> stored as
> a packed array and the Table approach is the only one that doesn't
> break
> this internal representation.
>
> In[51]:=
> <<Developer`
> PackedArrayQ[data]
>
> Out[52]=
> True
>
> PackedArrayQ[label] is only true when label is generated by Table.
>
> So I tried to come up with an approach that is similarly simple to
> Table, and did this:
>
> label = Transpose@Inner[List, Range@Length@data, data, List];
>
> But to my greatest disappointment it turned out to be the slowest
> of all
> (and also breaks the packed array representation).
>
> So probably the Table solution is the best to use (because it needs =
> the
> least memory). It is also the most straightforward and most
> readable way
> of doing this. It seems that a strictly functional style is not always
> the best way do things in Mathematica.
>
> Can anyone come up with a good way of solving this problem without
> indexing data?
>
> Szabolcs
I do not have a concrete example here but my newbie instinct says
that using Dispatch might be of use here to catch the speed daemon :)
J=E1nos=