Re: Re: Importing binary file is toooooo slow...
- To: mathgroup at smc.vnet.net
- Subject: [mg54220] Re: [mg54206] Re: [mg54159] Importing binary file is toooooo slow...
- From: "Yasvir A. Tesiram, PhD" <tesiramy at omrf.ouhsc.edu>
- Date: Sun, 13 Feb 2005 22:17:07 -0500 (EST)
- References: <200502130521.AAA03731@smc.vnet.net>
- Sender: owner-wri-mathgroup at wolfram.com
G'day, I left it too late to reply initially and so am jumping in the middle of the thread. My apologies if some of this is a repeat of what you may have already received. As of version 5.1 treatment of binary files has improved dramatically. As for 4.1 and later the functionality similar to what is now available in 5.1 can be found in the Experimental package. The syntax may be slightly confusing. The Experimental methods are by no means speedy. For sheer speed try the FastBinaryFiles package http://library.wolfram.com/infocenter/MathSource/354/ For Windows OS's the binary provided should work off the bat. There may be some issues compiling the source for newer systems, particularly MacOS, and *nix OS. Unfortunately I don't have an accessible download site and because I can't attach files using this list, contact me directly if the FastBinaryFiles don't work on your platform. Also, there is JLink and Java methods for file operations. I started doing this, had moderate success and then 5.1 came out. So I gave up, and just kept moving with 5.1. I understand that sometimes its just not possible to keep up to date with the latest advances, in which case you have to roll your own method. I think you'll find FastBinaryFiles suitable for what you are trying to do. Regards Yas On Sun, 13 Feb 2005, Clifford Martin wrote: > Actually I don't think you're doing anything wrong. > Mathematica just doesn't do binary file handling very > well ---but if you use the Experimental Binary Import > its much faster. > > <<Experimental`BinaryImport > > then commands like you used. It should be faster but > still slow by any other modern program standards. I > generally find Mathematica unsuitable for large binary > files. It is orders of magnitude slower than its major > competitor. Unfortunately! > > CAM > > --- "J. K. Smith" <rob at pi-overe.com> wrote: > > > I'm using v.4.2.1.0 -- > > > > I am importing (wrong word?) binary data from a > > file: > > > > << Utilities`BinaryFiles` > > > > y = ReadListBinary["filename", SignedInt16, > > ByteOrder->LeastSignificantByteFirst]; > > > > The file is only 34k bytes long but it takes 16 > > seconds to do it. This > > isn't a slow machine so clearly I'm doing something > > wrong again. > > > > I would appreciate any suggestions on how to make > > this thing sizzle as I > > feel sure it can. Right now I'm dead if I have to > > do a big file. > > > > Thanks > > > > remove the dash if emailing > > > > >
- References:
- Re: Importing binary file is toooooo slow...
- From: Clifford Martin <camartin@snet.net>
- Re: Importing binary file is toooooo slow...