Binary Files
- To: mathgroup at smc.vnet.net
- Subject: [mg13904] Binary Files
- From: Jason Gill <jgill at vbimail.champlain.edu>
- Date: Sun, 6 Sep 1998 02:55:41 -0400
- Organization: IBM Microelectronics
- Sender: owner-wri-mathgroup at wolfram.com
I am using the following function to read binary files. I would not
consider the data files very large, and yet the process is extremely
slow. I am running Mathematica 3.0 for win95. I have installed the
MathLink program FastReadBinary Files, which help a ton, but it is
still to slow. The net is I use this function on multiple files to
create a three dimensional data matrix with Dimensions on the order of
[6,1000,300]. Mathematica handles the data O.K., once loaded but the
loading takes to long. For the aforementioned matrix, loading the
files can take upwards of 15 minutes. Which seems excessie to me.
Using the timming function, the largest consumer of time is the actual
binary read. Does anyone have any suggestions? I also have access to
Mathematica (3.0) running on an RS/6000 is there a Mathlink equivalent
to Fast Read Binary available for this platform? Any suggestions would
be greatly appreciated.
Options[ReadandConvertLMX]={SetDirPath->"d:\lmxdata\janussq"};
ReadandConvertLMX[filename_,opts___]:=
Module[{inline,setpath,convert6,waste,
},
directory=SetDirPath /. {opts} /. Options[ReadandConvertLMX];
SetDirectory[directory];
inline = OpenReadBinary[filename];
{modules,parameters,timestep,powerHour}=
ReadListBinary[inline,Int16,4,
ByteOrder->MostSignificantByteFirst];
waste=ReadListBinary[inline,Byte,120];
data=ReadListBinary[inline,Byte];
data=Take[Partition[data,4],(modules*parameters)]; Close[inline];
ResetDirectory[];
convert6 = Compile[{b1, b2, b3, b4}, If[b1 < 128,
(b2*256^2+b3*256+b4)/256^3 16^(b1-64),
-(b2*256^2+b3*256+b4)/256^3 16^(b1-192)]]; ResetDirectory[];
data=Transpose[Partition[Apply[convert6,data,2],parameters]];
Dimensions[data]];
--
Jason Gill
IBM Microelectronics
Essex Junction, VT 05452
Phone (802)769-3350
Fax (802)769-1220
email: jgill at vbimail.champlain.edu