MathGroup Archive 2000

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Fast (compiled) routine for element testing and replacement in large matrices?

  • To: mathgroup at smc.vnet.net
  • Subject: [mg23956] Re: Fast (compiled) routine for element testing and replacement in large matrices?
  • From: Mark Fisher <me.fisher at atl.frb.org>
  • Date: Sun, 18 Jun 2000 03:00:40 -0400 (EDT)
  • Organization: Federal Reserve Bank of Atlanta
  • References: <8iccl1$9oj@smc.vnet.net>
  • Sender: owner-wri-mathgroup at wolfram.com

Gareth,

I'll show you a simple speed up, but first let me make two comments
about your use of "D" as a symbol. First, "D" is protected; it's a
built-in function for differentiation, so it's probably a bad idea to
use it in a program for something else. Second, you haven't given it a
value (of course you'd have to Unprotect it to do so). To see the
consequences, let's run your example:

d2wreal[w_,l_,u_] := If[(w>l)&&(D<=u),1,0]
SetAttributes[d2wreal,Listable]

a = Table[Random[], {i, 1, 100}, {j, 1, 100}];
Timing[b = d2wreal[a, 0.2, 0.4]][[1]]
0.5 Second (* my timing *)

Note that

Union[Flatten[b]] --> {0, If[D <= 0.4, 1, 0]}

Now for the speedup. Since I don't know what you "really meant", I'll
just make up an example for illustrative purposes.

This is supposed to be what you intended:

SetAttributes[fun, Listable]
fun[x_] := If[x > .5, 0, 1]

Timing[b = fun[a];][[1]]
0.297 Second

Here is the speedup (Map a pure function)

Timing[b = Map[If[# > .5, 0, 1] &, a, {2}];][[1]]
0.015 Second

Faster by a factor of 20. 

Suppose you need to set a parameter automatically:

makefun[p_] := If[#>p, 0, 1]&

Timing[b = Map[makefun[.5], a, {2}];][[1]]
0.016 Second

The speedup stays about 20 times faster for the larger matrix:

A = Table[Random[], {1000}, {1000}];

"your way"
Timing[b = fun[A];][[1]]
31.359 Second

"my way"
Timing[b = Map[makefun[.5], A, {2}];][[1]]
1.343 Second

--Mark

Mark Fisher
Economic Advisor
Research Department
Federal Reserve Bank of Atlanta
Atlanta, GA 30303
404-521-8757 (voice)
404-521-8810 (fax)
mark.fisher at atl.frb.org

personal web page: www.markfisher.net



"Gareth J. Russell" wrote:
> 
> Hello,
> 
> My first post. I am moving from another software package to Mathematica, and
> I often work with very large matrices in the context of simulations, so
> speed is important. Here is a problem for which I would like a serious speed
> increase.
> 
> Given a matrix of real numbers, I want to generate a new matrix where
> values from a certain range are replaced with 1, and the remainder with
> zero. Matrix size is often on the order of 1000 by 1000 (they are always
> square).
> 
> I defined a function that does the comparison using If[],
> and made it listable:
> 
> d2wreal[w_,l_,u_] := If[(w>l)&&(D<=u),1,0]
> SetAttributes[d2wreal,Listable]
> 
> These are typical times:
> 
> a = Table[Random[], {i, 1, 100}, {j, 1, 100}];
> t = Timing[b = d2wreal[a, 0.2, 0.4]][[1]]
> 0.46666666666666856 Second
> 
> a = Table[Random[], {i, 1, 1000}, {j, 1, 1000}];
> t = Timing[b = d2wreal[a, 0.2, 0.4]][[1]]
> 65.55 Second
> 
> I tried to generate some compiled alternatives myself, but I couldn't find
> one that was faster. I reckon a 10x speed-up should be possible, possibly
> more. My other software takes only 0.75 seconds for the 1000 by 1000 matrix
> function.
> 
> Can anyone suggest a way to speed this up?
> 
> Thanks,
> 
> Gareth
> 
> 
> ==================================================
> Dr. Gareth J. Russell
> 
> NEW ADDRESS and E-MAIL FROM 1ST SEPTEMBER, 1999
> 
> Center for Environmental Research and Conservation
>   MC 5556
>     Columbia University
>       1200 Amsterdam Avenue
>         New York, NY 10027, U.S.A.
> 
>  Phone: ++1 212 854 5094
>    Fax: ++1 212 854 8188
> E-mail: russell at cerc.columbia.edu
>    WWW: http://web.utk.edu/~grussell   (NO CHANGE)
> 
> OLD ADDRESS (AND STILL MY EMPLOYERS)
> 
> Department of Ecology and Evolutionary Biology
>   University of Tennessee
>     569 Dabney Hall
>       Knoxville, TN 37996-1610, USA
> ==================================================


  • Prev by Date: Re: Fast (compiled) routine for element testing and replacement in large matrices?
  • Next by Date: Re: RE:Working Precision
  • Previous by thread: Re : Fast (compiled) routine for element testing and replacement in large matrices? A more general question about Listable
  • Next by thread: RE: Re: Fast (compiled) routine for element testing and replacement in large matrices?