Re: Fast (compiled) routine for element testing and replacement in large matrices?
- To: mathgroup at smc.vnet.net
- Subject: [mg23956] Re: Fast (compiled) routine for element testing and replacement in large matrices?
- From: Mark Fisher <me.fisher at atl.frb.org>
- Date: Sun, 18 Jun 2000 03:00:40 -0400 (EDT)
- Organization: Federal Reserve Bank of Atlanta
- References: <8iccl1$9oj@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Gareth,
I'll show you a simple speed up, but first let me make two comments
about your use of "D" as a symbol. First, "D" is protected; it's a
built-in function for differentiation, so it's probably a bad idea to
use it in a program for something else. Second, you haven't given it a
value (of course you'd have to Unprotect it to do so). To see the
consequences, let's run your example:
d2wreal[w_,l_,u_] := If[(w>l)&&(D<=u),1,0]
SetAttributes[d2wreal,Listable]
a = Table[Random[], {i, 1, 100}, {j, 1, 100}];
Timing[b = d2wreal[a, 0.2, 0.4]][[1]]
0.5 Second (* my timing *)
Note that
Union[Flatten[b]] --> {0, If[D <= 0.4, 1, 0]}
Now for the speedup. Since I don't know what you "really meant", I'll
just make up an example for illustrative purposes.
This is supposed to be what you intended:
SetAttributes[fun, Listable]
fun[x_] := If[x > .5, 0, 1]
Timing[b = fun[a];][[1]]
0.297 Second
Here is the speedup (Map a pure function)
Timing[b = Map[If[# > .5, 0, 1] &, a, {2}];][[1]]
0.015 Second
Faster by a factor of 20.
Suppose you need to set a parameter automatically:
makefun[p_] := If[#>p, 0, 1]&
Timing[b = Map[makefun[.5], a, {2}];][[1]]
0.016 Second
The speedup stays about 20 times faster for the larger matrix:
A = Table[Random[], {1000}, {1000}];
"your way"
Timing[b = fun[A];][[1]]
31.359 Second
"my way"
Timing[b = Map[makefun[.5], A, {2}];][[1]]
1.343 Second
--Mark
Mark Fisher
Economic Advisor
Research Department
Federal Reserve Bank of Atlanta
Atlanta, GA 30303
404-521-8757 (voice)
404-521-8810 (fax)
mark.fisher at atl.frb.org
personal web page: www.markfisher.net
"Gareth J. Russell" wrote:
>
> Hello,
>
> My first post. I am moving from another software package to Mathematica, and
> I often work with very large matrices in the context of simulations, so
> speed is important. Here is a problem for which I would like a serious speed
> increase.
>
> Given a matrix of real numbers, I want to generate a new matrix where
> values from a certain range are replaced with 1, and the remainder with
> zero. Matrix size is often on the order of 1000 by 1000 (they are always
> square).
>
> I defined a function that does the comparison using If[],
> and made it listable:
>
> d2wreal[w_,l_,u_] := If[(w>l)&&(D<=u),1,0]
> SetAttributes[d2wreal,Listable]
>
> These are typical times:
>
> a = Table[Random[], {i, 1, 100}, {j, 1, 100}];
> t = Timing[b = d2wreal[a, 0.2, 0.4]][[1]]
> 0.46666666666666856 Second
>
> a = Table[Random[], {i, 1, 1000}, {j, 1, 1000}];
> t = Timing[b = d2wreal[a, 0.2, 0.4]][[1]]
> 65.55 Second
>
> I tried to generate some compiled alternatives myself, but I couldn't find
> one that was faster. I reckon a 10x speed-up should be possible, possibly
> more. My other software takes only 0.75 seconds for the 1000 by 1000 matrix
> function.
>
> Can anyone suggest a way to speed this up?
>
> Thanks,
>
> Gareth
>
>
> ==================================================
> Dr. Gareth J. Russell
>
> NEW ADDRESS and E-MAIL FROM 1ST SEPTEMBER, 1999
>
> Center for Environmental Research and Conservation
> MC 5556
> Columbia University
> 1200 Amsterdam Avenue
> New York, NY 10027, U.S.A.
>
> Phone: ++1 212 854 5094
> Fax: ++1 212 854 8188
> E-mail: russell at cerc.columbia.edu
> WWW: http://web.utk.edu/~grussell (NO CHANGE)
>
> OLD ADDRESS (AND STILL MY EMPLOYERS)
>
> Department of Ecology and Evolutionary Biology
> University of Tennessee
> 569 Dabney Hall
> Knoxville, TN 37996-1610, USA
> ==================================================