Re: Joint Entropy
- To: mathgroup at smc.vnet.net
- Subject: [mg65629] Re: [mg65575] Joint Entropy
- From: bsyehuda at gmail.com
- Date: Tue, 11 Apr 2006 04:04:45 -0400 (EDT)
- References: <200604090832.EAA00800@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
Hi Sensei,
I noticed that you detemined the dictionary from the appearences in th
eseries. Don't you know this in advance. Determining the dictionary from a
sample path is an approcimation, afor low probability events it is correct
only for LONG series.
Tuples , a new function in version 5.2 saved all the trouble with
flattening, and is faster then the implementation of Outer (or Distribute
which can also be used in this case).
Tuples[{{a,b,c},{1,2,3}}]
returns
{{a, 1}, {a, 2}, {a, 3}, {b, 1}, {b, 2}, {b, 3}, {c, 1}, {c, 2}, {c, 3}}
and this is the set of all pairs from both dictionaries
Now you can use the approach you presented for the pairs, etc.
regrads
yehuda
On 4/9/06, Sensei <senseiwa at mac.com> wrote:
>
> Hi! I'm writing some functions to analyze the informative content of
> sequences, and I've stopped trying to produce the joint entropy.
>
> These are my auxiliary functions:
>
> (* Generates a sequence of random numbers *)
> In[2]:=
> RandomSequence[nsamples_,min_,max_]:=Table[
> Random[Integer,{min,max}], {nsamples}
> ]
>
> (* Alphabet of a sequence *)
> In[3]:=
> SignalAlphabet[signal_]:=Union[signal]
>
> (* Gives the probability of a symbol *)
> In[13]:=
> SymbolProbability[symbol_,signal_]:=Count[signal,symbol]/Length[signal]
>
> (* Gives the list of all symbols and their probabilities *)
> In[20]:=
> SignalProbabilityList[signal_]:=Map[
> {#,SymbolProbability[#,signal]}&,
> SignalAlphabet[signal]]
>
> (* Calculates the entropy *)
> In[24]:=
> SignalEntropy[signal_]:=-1*Fold[Plus, 0,
> Map[Log[2,Last[#]]&,SignalProbability[signal]]]
>
>
> Now, my question is, how to produce the joint probability of two
> sequences ``mathematica style''? So, given X and Y, I can produce the
> alphabet of XY, that is the cartesian product of the two alphabets
> (using CartesianProduct), but... well, I don't know how to make a
> good code! As I said previously, I'm new to mathematica... How should
> I proceed?
>
> Thanks for any hints!
>
> PS. If the code above is not so good, please let me know! :)
>
> --
> Sensei <senseiwa at mac.com>
>
> The optimist thinks this is the best of all possible worlds.
> The pessimist fears it is true. [J. Robert Oppenheimer]
>
>
>
- References:
- Joint Entropy
- From: Sensei <senseiwa@mac.com>
- Joint Entropy