MathGroup Archive 2005

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Re: Weighted histogram

  • To: mathgroup at smc.vnet.net
  • Subject: [mg62321] Re: [mg62287] Re: Weighted histogram
  • From: ggroup at sarj.ca
  • Date: Sat, 19 Nov 2005 23:19:13 -0500 (EST)
  • References: <dliv00$g1b$1@smc.vnet.net><dlkcct$pr0$1@smc.vnet.net> <200511191054.FAA16469@smc.vnet.net> <1051079141.20051119132905@aggarwal.ca>
  • Reply-to: ggroup at sarj.ca
  • Sender: owner-wri-mathgroup at wolfram.com

On Saturday, November 19, 2005 at 1:29 PM, I wrote:

> On Saturday, November 19, 2005 at 05:54 GMT -0500, Kalymereau wrote:

>> It is possible to construct a binning and then to fill each bin by the
>> corresponding values in a ponderated by the weights in w. But the tests
>> I made in this direction were very slow for large lists, and I would
>> like something more natural.

>> It seems to me to be a very basic problem of data analysis, any other
>> idea ?

> Well, not really.  The following probably won't win any speed awards,
> but it isn't outrageous (about 0.75 minute for 10^6 elements here).

> Needs["Graphics`Graphics`"];
> n = 10;
> m = 10;
> a = Table[Random[Real, m], {n}];
> w = Table[Random[], {n}];
> aw = Transpose[{a, w}];
> bins = Table[{i, i + 1}, {i, 0, m - 1}];
> Function[x,
>       {
>         tmp = Select[aw, x[[1]] <= #[[1]] < x[[2]] &];
>         aw = Complement[aw, tmp];
>         Total[tmp[[All, 2]]],
>         Mean[x]
>         }
>       ] /@ bins // BarChart
> Clear[tmp];

The following, though based on the same idea, is significantly faster:

Needs["Graphics`Graphics`"];
Needs["Statistics`DataManipulation`"];
n = 10^6;
m = 10;
a = Table[Random[Real, m], {n}];
w = Table[Random[], {n}];
Module[{aw, dat, tmp, bins, dbin = 1},
    aw = Sort[Transpose[{a, w}]];
    bins = Table[i, {i, dbin, m, dbin}];
    dat = Function[x,
          tmp = TakeWhile[aw, #[[1]] < x &];
          aw = Drop[aw, Length[tmp]];
          Total[tmp[[All, 2]]]
          ] /@ bins;

    Histogram[
      dat,
      FrequencyData -> True, HistogramCategories -> Join[{0}, bins],
      HistogramScale -> 1]
    ] // Timing


  • Prev by Date: Re: Same scaling for plots/ charts
  • Next by Date: Newbie with simple questions (take 2)
  • Previous by thread: Re: Re: Weighted histogram
  • Next by thread: Re: Weighted histogram