Re: DeleteDuplicates is too slow?
- To: mathgroup at smc.vnet.net
- Subject: [mg107195] Re: [mg107150] DeleteDuplicates is too slow?
- From: Bob Hanlon <hanlonr at cox.net>
- Date: Fri, 5 Feb 2010 03:22:30 -0500 (EST)
- Reply-to: hanlonr at cox.net
n = 100000;
data = RandomInteger[{0, 9}, {n, 2}] // N;
Length[DeleteDuplicates[data, #1[[1]] === #2[[1]] &]] // Timing
{1.39164,10}
Length[Union[data, SameTest -> (#1[[1]] === #2[[1]] &)]] // Timing
{0.289784,10}
Bob Hanlon
---- "Zeringue wrote:
=============
Hello,
Suppose you have the following.
Data = RandomReal[1,{N,2}];
sameQ[_,_]=False;
sameQ[{x_,y_},{x_,z_}]=True;
Timing[DeleteDuplicates[data,sameQ]][[1]];
If N is a large number this takes an ungodly amount of time?
Is there a more efficient way to delete the duplicate entries of Data ?
ie.
Data = {{1.,2.},{1.,3.},{2.,3.}};
Would become:
{{1.,2.},{ 2.,3.}};
Thanks,
Clint Zeringue