MathGroup Archive 2004

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: []Speeding Up Indexing and Joining

  • To: mathgroup at
  • Subject: [mg52490] Re: []Speeding Up Indexing and Joining
  • From: Bill Rowe <readnewsciv at>
  • Date: Mon, 29 Nov 2004 01:22:39 -0500 (EST)
  • Sender: owner-wri-mathgroup at

On 11/28/04 at 1:07 AM, bbongiorno at (Benedetto
Bongiorno) wrote:

>Here is a simple example.

>Again on large mattices, it takes many many hours







>Create Index from Intersection of the first columns of matrix a and
>= matrix b

>lastCol =1;
>aa = Transpose[Take[Transpose[a],{firstCol, lastCol}]];
>bb = Transpose[Take[Transpose[b],{firstCol, lastCol}]];

More efficient is

aa = a[[All,1]];
bb = b[[All,1]];
idx = Intersection[aa,bb];

>Select Records (Rows) from both matrix a and matrix b that equal
>the = idx. The idx consists of numerics

>Locate Position Of Each Record In each matrix using idx

>step1 = Position[aa,idx[[i]]];

>step1 = Position[bb,idx[[i]]];

>Extract a Records

>ans01 =Extract[a,ans01];
>ans02 =Extract[b,ans02];

More efficient is:

ans01 = Select[a, MemberQ[idx, First@#]&];
ans02 = Select[b, MemberQ[idx, First@#]&];

>Sort each matrix by Column 1 using a defined function

>ans01=matsort[ans01,1]; ans02=matsort[ans02,1];

You didn't say how you coded matsort. If matsort is coded using For it can be speeded up significantly by definiing it as:


And to show this yields the same solution

RowJoin[matsort[ans02, 1], matsort[ans01, 1]]
{{133, 1, 46, "go", 6, 133,  "harry", 45, "fo"}, 
 {165, 88, 45, "mo", 7, 165,  "pete", 44, "fo"}, 
 {6589, 77, 69, "xo", 11, 6589, "ben", 69, "fo"}} 
Although, I probably would have done

RowJoin[matsort[ans02, 1],  Rest/@matsort[ans01, 1]]

to eliminate the redundant column.


>Out[69]= =

To reply via email subtract one hundred and four

  • Prev by Date: pair sums applied to trignometry sums
  • Next by Date: Re: Integration of piecewise function
  • Previous by thread: Re: pair sums applied to trignometry sums
  • Next by thread: again Compile