[Date Index] [Thread Index] [Author Index]

Re: IEEE single precision number.

• To: mathgroup at smc.vnet.net
• Subject: [mg51849] Re: [mg51782] IEEE single precision number.
• From: János <janos.lobb at yale.edu>
• Date: Wed, 3 Nov 2004 01:25:50 -0500 (EST)
• References: <200411020705.CAA21601@smc.vnet.net>
• Sender: owner-wri-mathgroup at wolfram.com

On Nov 2, 2004, at 2:05 AM, Vangelis Marinakis wrote:

> Dear everyone,
>
> Looking at the internet I found this email:
>
> *****************************************************************
>
> Hi,
>
> I want to convert a group/stream of Bytes to various internal
> Mathematica numerical types, particularly into Real (machine precision
> number).  The motivation is the fast reading of binary files.
>
> I don't want to deal with the Std Package
> Utilities`BinaryFiles`ReadBinary or Developer'BinaryImport, which do
> have the conversion utilities (e.g. into Real32, etc), since they are
> SLOW or otherwise clunky.
>
> When employing the suggested fast reading of binary files:
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> There are numerous means of reading data into Mathematica from binary
> files.  However, most of them are too slow.  Through experience and
> some newsgroup research, I've found the fast easy way:
>
> stream = OpenRead["Cals9", DOSTextFormat -> False];
>    (* Note:  this DOSTextFormat->False option is key to using
> ReadList[] in binary mode; otherwise, it wants to digest text.  Readin
> Byte chunks, then post-format.  Remarkably, the DosTextFormat option
> is not mentioned/discussed anywhere in the Mathematica book or Help
> Browser *)
>
> SetStreamPosition[stream, 78*131072];
>      (* Position to desired read starting location within file...count
> is in Bytes, and is index base = 0  *).
>
> data = ReadList[stream, Byte, 131072];
>       (* Reads 131072 Bytes into linear List "data" *)
>
> Close[stream];
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
> you end up with a list of Bytes.  Converting it to 16-bit integers is
> easy:
>
> data = Map[(#1[[1]] + 256*#1[[2]] & ), Partition[data, 2]];
>   (* takes original Byte List "data" and makes 16-bit "words", writing
> over List "data".  Result is half-as-long List of words.  *)
>
>
> ...but how would one convert it to real numbers (given that every 4
> bytes comprises an IEEE single-precision number)?
>
> Thanks.
>
> franki at aerodyne.com
>
> *****************************************************************
>
> Does anyone no the answer, because I have exactly the same problem!
>
> Thanks.
>
> Vangelis Marinakis.
>
>

Based upon http://www.wordiq.com/definition/IEEE_754

Let's say you have the bytes coming as normalized binary 
representations, and you have them as strings in a list  like 
{00110000,....,10100011}  then you can join four together with

fourbytestrings = Map[StringJoin, Partition[bytestrings, 4]]

Let's say one of them is :

In[146]:=
byte4 = "11000010111011010100\
000000000000"

The first digit gives the sign:

In[189]:=
If[StringTake[byte4,1 ]=="1",-1,1]

Out[189]=
-1

The digits between 2 and 9 gives the biased exponential.  That 
exponential will help to separate the integer part from the fraction

Here is that substring:
In[149]:=
StringTake[byte4, {2, 9}]
Out[149]=
"10000101"

Here is the exponential with the bias=127 removed:

In[203]:=
shift = FromDigits[
     IntegerDigits[
      ToExpression[StringTake[
        byte4, {2, 9}]]], 2] -
    127
Out[203]=
6

This method of mine works only if this shift above is  between 0 and 
23.  It will not work for a 4byte like 
"10001101010101001001101100111111" where the shift is -101.

Then you can slice the rest of byte4 into two:

In[208]:=
st1 = StringJoin["1",
    StringTake[byte4,
     {10, 10 + shift - 1}]]
Out[208]=
"1110110"

In[213]:=
st2 = StringTake[byte4,
    {10 + shift, 32}]
Out[213]=
"10100000000000000"

Then you can create two sets containing the digits of the above two 
variables and another two lists containing the appropriate powers of 
two:

In[169]:=
st1set = ToExpression[
    Characters[st1]]
Out[169]=
{1, 1, 1, 0, 1, 1, 0}

In[170]:=
st2set = ToExpression[
    Characters[st2]]
Out[170]=
{1, 0, 1, 0, 0, 0, 0, 0, 0,
   0, 0, 0, 0, 0, 0, 0, 0}

In[171]:=
twowhole = Table[2^i,
    {i, Length[st1set] - 1, 0,
     -1}]
Out[171]=
{64, 32, 16, 8, 4, 2, 1}

In[180]:=
twofraction = Table[2^(-i),
    {i, 1, Length[st2set]}]
Out[180]=
{1/2, 1/4, 1/8, 1/16, 1/32,
   1/64, 1/128, 1/256, 1/512,
   1/1024, 1/2048, 1/4096,
   1/8192, 1/16384, 1/32768,
   1/65536, 1/131072}

Then you "Inner"  them pairwise and add them together and multiply it 
with the right sign

In[246]:=
N[If[StringTake[byte4, 1] ==
      "1", -1, 1]*
    (Inner[Times, st1set,
      twowhole, Plus] +
     Inner[Times, st2set,
      twofraction, Plus])]
Out[246]=
-118.625


All in one:

In[263]:=
N[If[StringTake[byte4, 1] ==
      "1", -1, 1]*
    (Inner[Times, ToExpression[
       Characters[StringJoin[
         "1", StringTake[
          byte4, {10, 10 +
           FromDigits[
           IntegerDigits[
           ToExpression[
           StringTake[byte4, {
           2, 9}]]], 2] -
           127 - 1}]]]],
      Table[2^i, {i,
        Length[ToExpression[
           Characters[
           StringJoin["1",
           StringTake[byte4,
           {10, 10 +
           FromDigits[
           IntegerDigits[
           ToExpression[
           StringTake[byte4,
           {2, 9}]]], 2] -
           127 - 1}]]]]] - 1,
        0, -1}], Plus] +
     Inner[Times, ToExpression[
       Characters[StringTake[
         byte4,
         {10 + FromDigits[
           IntegerDigits[
           ToExpression[
           StringTake[byte4,
           {2, 9}]]], 2] -
           127, 32}]]],
      Table[2^(-i), {i, 1,
        Length[ToExpression[
          Characters[
           StringTake[byte4,
           {10 + FromDigits[
           IntegerDigits[
           ToExpression[
           StringTake[byte4,
           {2, 9}]]], 2] -
           127, 32}]]]]}],
      Plus])]
Out[263]=
-118.625

As I said it works just for particular cases, but it might gives you 
the insight how to do it for all.  If you have a list of fourbytes than 
you can replace all the byte4-s with #, put an & to the end of the 
expression and Map it to the list of fourbytes.

János

----------------------------------------------
Trying to argue with a politician is like lifting up the head of a 
corpse.
(S. Lem: His Master Voice)

• References:
  • IEEE single precision number.
    • From: Vangelis Marinakis <V.Marinakis@damtp.cam.ac.uk>