Services & ResourcesWolfram ForumsMathGroup Archive

[Date Index] [Thread Index] [Author Index]

Re: Joint Entropy

• To: mathgroup at smc.vnet.net
• Subject: [mg65587] Re: Joint Entropy
• From: Jean-Marc Gulliet <jeanmarc.gulliet at gmail.com>
• Date: Mon, 10 Apr 2006 02:31:07 -0400 (EDT)
• Organization: The Open University, Milton Keynes, UK
• References: <e1agqs$ro$1@smc.vnet.net>
• Sender: owner-wri-mathgroup at wolfram.com

Sensei wrote:
> Hi! I'm writing some functions to analyze the informative content of  
> sequences, and I've stopped trying to produce the joint entropy.
> 
> These are my auxiliary functions:
> 
> (* Generates a sequence of random numbers *)
> In[2]:=
> RandomSequence[nsamples_,min_,max_]:=Table[
>      Random[Integer,{min,max}], {nsamples}
>      ]
> 
> (* Alphabet of a sequence *)
> In[3]:=
> SignalAlphabet[signal_]:=Union[signal]
> 
> (* Gives the probability of a symbol *)
> In[13]:=
> SymbolProbability[symbol_,signal_]:=Count[signal,symbol]/Length[signal]
> 
> (* Gives the list of all symbols and their probabilities *)
> In[20]:=
> SignalProbabilityList[signal_]:=Map[
>      {#,SymbolProbability[#,signal]}&,
>      SignalAlphabet[signal]]
> 
> (* Calculates the entropy *)
> In[24]:=
> SignalEntropy[signal_]:=-1*Fold[Plus, 0,
>        Map[Log[2,Last[#]]&,SignalProbability[signal]]]
> 
> 
> Now, my question is, how to produce the joint probability of two  
> sequences ``mathematica style''? So, given X and Y, I can produce the  
> alphabet of XY, that is the cartesian product of the two alphabets  
> (using CartesianProduct), but... well, I don't know how to make a  
> good code! As I said previously, I'm new to mathematica... How should  
> I proceed?

Hi Sensei,

Could anything like the following may be of any help (the  Cartesian 
product is implemented with the *Outer* built-in function)?

In[1]:=
X = {a, b, c, d};
Y = {d, e, f};
Outer[List, X, Y]

Out[3]=
{{{a, d}, {a, e}, {a, f}}, {{b, d}, {b, e}, {b, f}},
   {{c, d}, {c, e}, {c, f}}, {{d, d}, {d, e}, {d, f}}}

In[4]:=
Flatten[%]

Out[4]=
{a, d, a, e, a, f, b, d, b, e, b, f, c, d, c, e, c,
   f, d, d, d, e, d, f}

In[5]:=
Union[%]

Out[5]=
{a, b, c, d, e, f}

Best regards,
Jean-Marc

• Follow-Ups:
  • Re: Re: Joint Entropy
    • From: Andrzej Kozlowski <akoz@mimuw.edu.pl>