[Date Index]
[Thread Index]
[Author Index]
Re: Fast selection of lots of elements from a large list
*To*: mathgroup at smc.vnet.net
*Subject*: [mg128249] Re: Fast selection of lots of elements from a large list
*From*: Ray Koopman <koopman at sfu.ca>
*Date*: Sat, 29 Sep 2012 02:57:34 -0400 (EDT)
*Delivered-to*: l-mathgroup@mail-archive0.wolfram.com
*Delivered-to*: l-mathgroup@wolfram.com
*Delivered-to*: mathgroup-newout@smc.vnet.net
*Delivered-to*: mathgroup-newsend@smc.vnet.net
*References*: <k435e5$ngc$1@smc.vnet.net>
On Sep 27, 8:28 pm, Mark Coleman <markspcole... at gmail.com> wrote:
> Greetings,
>
> I've been using Mathematica to perform cluster analysis on a data set with about 600,000 rows and 60 columns. I've had the FindCluster procedure return a unique row identifier (12 character string) rather than the clustered data because I want to "join" these results to another data set for further analysis. To accomplish this I've been using the Position function to identify the element numbers in each cluster.
>
> To give a specific example, my cluster analysis identifiers twevle clusters on my original data set. The first of these clusters contains about 15,000 row identifiers. The extract the corresponding data from other data sets, I find the position of each identifier in my original data set using the simple code
>
> q=clusterResults[[1]]; (* row id's for first cluster *)
> p=Map[Position[rowIDs,#]&,q];
>
> where, "rowIDs" are the first column from the other dataset that contain the same string identifiers (rowIDs has about 600,000 sublists). I then Extract these elements ("rows") from the data set and continue my analysis.
>
> Unfortunately this is quite slow. Doing this on a sample of 1000 elements requires 340 seconds on my desktop computer. Some of my clusters have many tens of thousands of elements. I'm hoping someone can suggest a faster way of doing this.
>
> Thanks,
>
> Mark
See the thread "Extracting some elements, members of another list",
that ran Sep 14-17, 2010:
http://groups.google.com/group/comp.soft-sys.math.mathematica/browse_frm/thread/f8685f194db18175
Prev by Date:
**Re: Reduce command Mathematica**
Next by Date:
**How to lock down a Dynamic object in a report**
Previous by thread:
**Re: Fast selection of lots of elements from a large list**
Next by thread:
**Reduce command Mathematica**
| |