MathGroup Archive 2007

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Map vs. Table

  • To: mathgroup at smc.vnet.net
  • Subject: [mg75412] Re: Map vs. Table
  • From: Szabolcs <szhorvat at gmail.com>
  • Date: Sat, 28 Apr 2007 06:04:53 -0400 (EDT)
  • Organization: University of Bergen
  • References: <f0sfrs$ncb$1@smc.vnet.net>

zac wrote:
> Dear Group,
> 
> consider a set of data (y values):
> 
> data = Table[i*j, {i, 5}, {j, 10}]
> 
> I want to label each Integer with an x value:
> 
> label := Table[{i, #} & /@ data[[i]], {i, Length[data]}]
> 

It is better to use Set instead of SetDelayed here, otherwise the Table 
will be re-evaluated every time you use label. (Use = instead of :=)

> I've understand that list creation is much faster with Map than with
> Table.

It is generally a good idea to avoid explicitly indexing lists when not 
necessary. But here you *do* need the index i, as a label.

> Is there an effective way to convert the Table to Map in the label
> function (either with nested Map or to incorporate the Table function
> into the Map)?
 >
 > Would it be really faster for very large datasets than with Table?
 >
 > thanks
 >
 > Istvan Zachar
 >

There are many ways to do this calculation. If you want to avoid 
indexing data, then you either need to use MapIndexed or pre-generate 
the labels.

I tried these:

In[46]:=
data=Table[i j, {i, 2000}, {j, 2000}];

In[47]:=
Timing[label=Table[{i,#}& /@ data[[i]], {i, Length[data]}];]

Out[47]=
{1.031 Second,Null}

In[48]:=
Timing[label=MapIndexed[First@Outer[List, #2, #1]&, data];]

Out[48]=
{1.407 Second,Null}

In[49]:=
Timing[label=MapIndexed[Thread[Append[#2, #1]]&, data];]

Out[49]=
{1.609 Second,Null}

In[50]:=
Timing[label=Thread/@Transpose[{Range@Length@data, data}];]

Out[50]=
{1.422 Second,Null}

You must not take these timings too seriously (the times are too short, 
and inconsistent with every run), though they suggest that using Table 
is faster then the rest.

The problem is that with larger data matrices only Table works, all the 
others run out of memory! I guess that this is because data is stored as 
a packed array and the Table approach is the only one that doesn't break 
this internal representation.

In[51]:=
<<Developer`
PackedArrayQ[data]

Out[52]=
True

PackedArrayQ[label] is only true when label is generated by Table.

So I tried to come up with an approach that is similarly simple to 
Table, and did this:

label = Transpose@Inner[List, Range@Length@data, data, List];

But to my greatest disappointment it turned out to be the slowest of all 
(and also breaks the packed array representation).

So probably the Table solution is the best to use (because it needs the 
least memory). It is also the most straightforward and most readable way 
of doing this. It seems that a strictly functional style is not always 
the best way do things in Mathematica.

Can anyone come up with a good way of solving this problem without 
indexing data?

Szabolcs


  • Prev by Date: maximum entropy method for deconvolution
  • Next by Date: Re: Map vs. Table
  • Previous by thread: Re: Map vs. Table
  • Next by thread: Re: Map vs. Table