MathGroup Archive 2006

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Conditions with Statistical Functions

  • To: mathgroup at smc.vnet.net
  • Subject: [mg66163] Re: Conditions with Statistical Functions
  • From: Gregory Lypny <gregory.lypny at videotron.ca>
  • Date: Tue, 2 May 2006 02:43:19 -0400 (EDT)
  • References: <16568608.1146308002406.JavaMail.root@eastrmwml01.mgt.cox.net> <e31vb8$hgd$1@smc.vnet.net> <4454EC45.8030009@gmail.com> <B242D500-B1BF-4293-BB70-3C9C562AFF4D@videotron.ca> <22d35c5a0605010846p7c08eb61y26e91da12bbeae7d@mail.gmail.com>
  • Sender: owner-wri-mathgroup at wolfram.com

Hi Jean-Marc,

This has been a big help!  Thank you.  It was very kind of you to  
take the time to write such a lucid and detailed explanation.

I'm embarrassed about the transposition question.  That had occurred  
to me when I started thinking about the problem initially, but I  
quickly forgot about it when I got bogged down in trying to construct  
the appropriate function (unsuccessfully, of course).

I think your explanation has helped me better understand the use of  
slots (#) and the ampersand (&).  If I understand correctly, the  
ampersand is part of the definition of a pure function and is  
therefore required to terminate it.  That means that its placement in  
nested functions is important.  So, at the risk of messing up, the  
first ampersand turns the greater-than condition into a pure  
function, and the second turns the Select part, along with the  
enclosing greater-than condition, into another pure function.  Each  
has its own slot (#1) or user-supplied variable, which in your  
example is the matrix "1st".

I will now apply it to my data.  I suspect I will be able to cut down  
processing time over using a Do loop.  If I'm feeling courageous, I  
will try to use your approach and Bob Hanlon's to create similar  
functions using Count.

Regards,

	Greg



On Mon, May 1, 2006, at 11:46 AM, Jean-Marc Gulliet wrote:

> On 5/1/06, Gregory Lypny <gregory.lypny at videotron.ca> wrote:
>> Thanks again, Jean-Marc.
>>
>> This is elegant.  As I mentioned before, I'm facing an uphill  
>> battle with
>> Mathematica's syntax, and in particular, the meaning and placement  
>> of things
>> like #, #1, or #[[1]], &, @, @@, @@@, /@, /. .  It's all a bit  
>> overwhelming!
>>
>> I think I understand your second version better, so I'll work with  
>> it first.
>>  If I'm not mistaken, /@ tells the Mean function to map over its  
>> argument
>> and #1 directs that mapping to the first argument, which for Mean  
>> applied to
>> a matrix is a vector.  What I never would have gotten on my own is  
>> the use
>> of the second ampersand in parenthesis.  Is that meant to connect  
>> Select
>> with Transpose?  I'm also not quite clear on why we need Transpose  
>> because
>> Mean operates on columns anyway.
>>
>> Regards,
>>
>>  Greg
>>
>>
>>
>> On Sun, Apr 30, 2006, at 12:56 PM, Jean-Marc Gulliet wrote:
>>
>>
>> In[3]:=
>>
>> Mean /@ (Transpose[lst /. x_ /; x <= 100 -> 0] /.
>>
>>    0 -> Sequence[])
>>
>>
>>
>>
>> Out[3]=
>>
>>       1311  1450  527
>>
>> {125, ----, ----, ---}
>>
>>        8     9     3
>>
>>
>>
>>
>> In[4]:=
>>
>> Mean /@ (Select[#1, #1 > 100 & ] & ) /@ Transpose[lst]
>>
>>
>>
>>
>> Out[4]=
>>
>>       1311  1450  527
>>
>> {125, ----, ----, ---}
>>
>>        8     9     3
>>
>>
>>
>>
>> Best regards,
>>
>> Jean-Marc
>>
> Hi Gregory,
>
> Let us try to follow gradually what is going on. Say that our data set
> is a 12 by 4 matrix of integer entries:
>
> lst={{109,168,173,109},{4,143,200,90},{181,162,85,196},{30,
>      108,86,34},{94,127,144,34},{199,109,195,188},{176,
>    34,46,110},{95,27,160,109},{43,
>    71,130,66},{56,148,109,163},{110,43,50,53},{32,34,16,95}}
>
> First, we will focus on the second expression, which use the  
> *Select* function.
>
> Mean/@(Select[#1,#1>100&]&)/@Transpose[lst]
>
> Out[3]=
>      965  1111  875
> {155, ---, ----, ---}
>       7    7     6
>
> If we look at the full form ? Mathematica internal representation ? of
> the above expression, we see that (Select[#1, #1 > 100 & ] & ) is an
> expression made of nested pure functions. Pure functions are
> equivalent to anonymous functions in, say, LISP. The #n's (or
> Slot[n]'s) are placeholders for variables and a pure function
> definition ends by an ampersand character '&'. In our function, it is
> really important to realize that the first #1 has nothing to do with
> the second #1 because they are located in different function
> definitions (this is clearer in the full form of the expression).
>
> In[6]:=
> FullForm[HoldForm[Mean /@ (Select[#1, #1 > 100 & ] & ) /@ Transpose 
> [lst]]]
>
> Out[6]//FullForm=
> HoldForm[Map[Mean,
>  Map[Function[Select[Slot[
>      1],Function[Greater[Slot[1],100]]]],Transpose[lst]]]]
>
> Note, we could have written the pure function with explicit ? local
> ?variable name such as in the line below. *Select* works on each row
> and applies the test to each element.
>
> In[10]:=
> Map[Mean,Map[Function[rowvec,Select[rowvec,Function[elem,
>      Greater[elem,100]]]],Transpose[lst]]]
>
> Out[10]=
>      965  1111  875
> {155, ---, ----, ---}
>       7    7     6
>
> Now, let us see why we need to transpose the matrix first. Below, we
> can see the original matrix. By visual inspection, we notice that the
> first column has five values that are greater than 100, and the
> second, third and fourth columns have seven,seven,and six values
> greater than 100, respectively.
>
> In[2]:=
> TableForm[lst]
>
> Out[2]//TableForm=
> 109   168   173   109
>
> 4     143   200   90
>
> 181   162   85    196
>
> 30    108   86    34
>
> 94    127   144   34
>
> 199   109   195   188
>
> 176   34    46    110
>
> 95    27    160   109
>
> 43    71    130   66
>
> 56    148   109   163
>
> 110   43    50    53
>
> 32    34    16    95
>
> Applying the select clause to the original matrix without
> transposition yields the following result: any values less than or
> equal to 100 have been discarded; however, the structure of the
> original matrix has been changed too. Not only do we have a collection
> of row vectors of unequal lengths, but also many elements have shifted
> to the left!
>
> In[13]:=
> (Select[#1,#1>100&]&)/@lst//TableForm
>
> Out[13]//TableForm=
> 109   168   173   109
>
> 143   200
>
> 181   162   196
>
> 108
>
> 127   144
>
> 199   109   195   188
>
> 176   110
>
> 160   109
>
> 130
>
> 148   109   163
>
> 110
> On the other hand, transposing first allows getting the correct values
> in each row.
>
> In[12]:=
> (Select[#1,#1>100&]&)/@Transpose[lst]//TableForm
>
> Out[12]//TableForm=
> 109   181   199   176   110
>
> 168   143   162   108   127   109   148
>
> 173   200   144   195   160   130   109
>
> 109   196   188   110   109   163
>
> Finally, mapping *Mean* to the resulting list of lists allows to
> compute the mean of each row vectors that correspond to our original
> columns without the unwanted values (so using *Map* allows us to
> change the behavior of Mean which is to compute column by column).
>
> In[15]:=
> Mean/@(Select[#1,#1>100&]&)/@Transpose[lst]//Trace
>
> Well, I hope that I have not been to long and to obscure in my
> explanation and that indeed I have successfully shred some light on
> Mathematica programming.
>
> Best regards,
> Jean-Marc


  • Prev by Date: Re: Conditions with Statistical Functions
  • Next by Date: Re: Limit bug?
  • Previous by thread: Re: Conditions with Statistical Functions
  • Next by thread: Re: Re: Conditions with Statistical Functions