Re: IEEE single precision number.
- To: mathgroup at smc.vnet.net
- Subject: [mg51849] Re: [mg51782] IEEE single precision number.
- From: János <janos.lobb at yale.edu>
- Date: Wed, 3 Nov 2004 01:25:50 -0500 (EST)
- References: <200411020705.CAA21601@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
On Nov 2, 2004, at 2:05 AM, Vangelis Marinakis wrote: > Dear everyone, > > Looking at the internet I found this email: > > ***************************************************************** > > Hi, > > I want to convert a group/stream of Bytes to various internal > Mathematica numerical types, particularly into Real (machine precision > number). The motivation is the fast reading of binary files. > > I don't want to deal with the Std Package > Utilities`BinaryFiles`ReadBinary or Developer'BinaryImport, which do > have the conversion utilities (e.g. into Real32, etc), since they are > SLOW or otherwise clunky. > > When employing the suggested fast reading of binary files: > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > There are numerous means of reading data into Mathematica from binary > files. However, most of them are too slow. Through experience and > some newsgroup research, I've found the fast easy way: > > stream = OpenRead["Cals9", DOSTextFormat -> False]; > (* Note: this DOSTextFormat->False option is key to using > ReadList[] in binary mode; otherwise, it wants to digest text. Readin > Byte chunks, then post-format. Remarkably, the DosTextFormat option > is not mentioned/discussed anywhere in the Mathematica book or Help > Browser *) > > SetStreamPosition[stream, 78*131072]; > (* Position to desired read starting location within file...count > is in Bytes, and is index base = 0 *). > > data = ReadList[stream, Byte, 131072]; > (* Reads 131072 Bytes into linear List "data" *) > > Close[stream]; > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > you end up with a list of Bytes. Converting it to 16-bit integers is > easy: > > data = Map[(#1[[1]] + 256*#1[[2]] & ), Partition[data, 2]]; > (* takes original Byte List "data" and makes 16-bit "words", writing > over List "data". Result is half-as-long List of words. *) > > > ...but how would one convert it to real numbers (given that every 4 > bytes comprises an IEEE single-precision number)? > > Thanks. > > franki at aerodyne.com > > ***************************************************************** > > Does anyone no the answer, because I have exactly the same problem! > > Thanks. > > Vangelis Marinakis. > > Based upon http://www.wordiq.com/definition/IEEE_754 Let's say you have the bytes coming as normalized binary representations, and you have them as strings in a list like {00110000,....,10100011} then you can join four together with fourbytestrings = Map[StringJoin, Partition[bytestrings, 4]] Let's say one of them is : In[146]:= byte4 = "11000010111011010100\ 000000000000" The first digit gives the sign: In[189]:= If[StringTake[byte4,1 ]=="1",-1,1] Out[189]= -1 The digits between 2 and 9 gives the biased exponential. That exponential will help to separate the integer part from the fraction Here is that substring: In[149]:= StringTake[byte4, {2, 9}] Out[149]= "10000101" Here is the exponential with the bias=127 removed: In[203]:= shift = FromDigits[ IntegerDigits[ ToExpression[StringTake[ byte4, {2, 9}]]], 2] - 127 Out[203]= 6 This method of mine works only if this shift above is between 0 and 23. It will not work for a 4byte like "10001101010101001001101100111111" where the shift is -101. Then you can slice the rest of byte4 into two: In[208]:= st1 = StringJoin["1", StringTake[byte4, {10, 10 + shift - 1}]] Out[208]= "1110110" In[213]:= st2 = StringTake[byte4, {10 + shift, 32}] Out[213]= "10100000000000000" Then you can create two sets containing the digits of the above two variables and another two lists containing the appropriate powers of two: In[169]:= st1set = ToExpression[ Characters[st1]] Out[169]= {1, 1, 1, 0, 1, 1, 0} In[170]:= st2set = ToExpression[ Characters[st2]] Out[170]= {1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0} In[171]:= twowhole = Table[2^i, {i, Length[st1set] - 1, 0, -1}] Out[171]= {64, 32, 16, 8, 4, 2, 1} In[180]:= twofraction = Table[2^(-i), {i, 1, Length[st2set]}] Out[180]= {1/2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024, 1/2048, 1/4096, 1/8192, 1/16384, 1/32768, 1/65536, 1/131072} Then you "Inner" them pairwise and add them together and multiply it with the right sign In[246]:= N[If[StringTake[byte4, 1] == "1", -1, 1]* (Inner[Times, st1set, twowhole, Plus] + Inner[Times, st2set, twofraction, Plus])] Out[246]= -118.625 All in one: In[263]:= N[If[StringTake[byte4, 1] == "1", -1, 1]* (Inner[Times, ToExpression[ Characters[StringJoin[ "1", StringTake[ byte4, {10, 10 + FromDigits[ IntegerDigits[ ToExpression[ StringTake[byte4, { 2, 9}]]], 2] - 127 - 1}]]]], Table[2^i, {i, Length[ToExpression[ Characters[ StringJoin["1", StringTake[byte4, {10, 10 + FromDigits[ IntegerDigits[ ToExpression[ StringTake[byte4, {2, 9}]]], 2] - 127 - 1}]]]]] - 1, 0, -1}], Plus] + Inner[Times, ToExpression[ Characters[StringTake[ byte4, {10 + FromDigits[ IntegerDigits[ ToExpression[ StringTake[byte4, {2, 9}]]], 2] - 127, 32}]]], Table[2^(-i), {i, 1, Length[ToExpression[ Characters[ StringTake[byte4, {10 + FromDigits[ IntegerDigits[ ToExpression[ StringTake[byte4, {2, 9}]]], 2] - 127, 32}]]]]}], Plus])] Out[263]= -118.625 As I said it works just for particular cases, but it might gives you the insight how to do it for all. If you have a list of fourbytes than you can replace all the byte4-s with #, put an & to the end of the expression and Map it to the list of fourbytes. János ---------------------------------------------- Trying to argue with a politician is like lifting up the head of a corpse. (S. Lem: His Master Voice)
- References:
- IEEE single precision number.
- From: Vangelis Marinakis <V.Marinakis@damtp.cam.ac.uk>
- IEEE single precision number.