RE: Eliminate duplicates using Union [] in a
- To: mathgroup at smc.vnet.net
- Subject: [mg48083] RE: [mg48063] Eliminate duplicates using Union [] in a
- From: "DrBob" <drbob at bigfoot.com>
- Date: Tue, 11 May 2004 05:20:06 -0400 (EDT)
- Sender: owner-wri-mathgroup at wolfram.com
Here's Union with "same" meaning identical first and third elements: test={{a,b,c,d},{d,e,f,g},{a,b,c,g}}; same=#1[[{1,3}]]===#2[[{1,3}]]&; Union[test,SameTest\[Rule]same] {{a,b,c,d},{d,e,f,g}} DrBob www.eclecticdreams.net -----Original Message----- From: Mark Coleman [mailto:mark at markscoleman.com] To: mathgroup at smc.vnet.net Subject: [mg48083] [mg48063] Eliminate duplicates using Union [] in a I know that if one applies the Union command to a list of values, it will return the list with duplicates removed. Can I use the Union command to eliminate duplicate when I have a list of data "records"? Specifically, I have a large data set that I read into a list on a row-oriented basis. Each element of the list consists of another list of about 30 different fields. Two of these fields are geographic indicators. If two or more records have identical values in these two fields, then the full records are considered duplicates and need to be identified and removed from the larger list (or at least marked for removal). Can Union do this? Thanks, -Mark