MathGroup Archive 2005

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Re: Importing binary file is toooooo slow...

  • To: mathgroup at smc.vnet.net
  • Subject: [mg54220] Re: [mg54206] Re: [mg54159] Importing binary file is toooooo slow...
  • From: "Yasvir A. Tesiram, PhD" <tesiramy at omrf.ouhsc.edu>
  • Date: Sun, 13 Feb 2005 22:17:07 -0500 (EST)
  • References: <200502130521.AAA03731@smc.vnet.net>
  • Sender: owner-wri-mathgroup at wolfram.com

G'day,
I left it too late to reply initially and so am jumping in the middle of
the thread. My apologies if some of this is a repeat of what you may
have already received. As of version 5.1 treatment of binary files has
improved dramatically.
As for 4.1 and later the functionality similar to what is now available in
5.1 can be found in the Experimental package. The syntax may be slightly
confusing. The Experimental methods are by no means speedy.
For sheer speed try the FastBinaryFiles package

http://library.wolfram.com/infocenter/MathSource/354/

For Windows OS's the binary provided should work off the bat. There may be
some issues compiling the source for newer systems, particularly MacOS,
and *nix OS. Unfortunately I don't have an accessible download site and
because I can't attach files using this list, contact me directly if the
FastBinaryFiles don't work on your platform.

Also, there is JLink and Java methods for file operations. I started doing
this, had moderate success and then 5.1 came out. So I gave up, and just
kept moving with 5.1. I understand that sometimes its just not possible to
keep up to date with the latest advances, in which case you have to roll
your own method. I think you'll find FastBinaryFiles suitable for what
you are trying to do.
Regards
Yas



On Sun, 13 Feb 2005, Clifford Martin wrote:

> Actually I don't think you're doing anything wrong.
> Mathematica just doesn't do binary file handling very
> well ---but if you use the Experimental Binary Import
> its much faster.
>
> <<Experimental`BinaryImport
>
> then commands like you used. It should be faster but
> still slow by any other modern program standards. I
> generally find Mathematica unsuitable for large binary
> files. It is orders of magnitude slower than its major
> competitor. Unfortunately!
>
> CAM
>
> --- "J. K. Smith" <rob at pi-overe.com> wrote:
>
> > I'm using v.4.2.1.0 --
> >
> > I am importing (wrong word?) binary data from a
> > file:
> >
> > << Utilities`BinaryFiles`
> >
> > y = ReadListBinary["filename", SignedInt16,
> > ByteOrder->LeastSignificantByteFirst];
> >
> > The file is only 34k bytes long but it takes 16
> > seconds to do it. This
> > isn't a slow machine so clearly I'm doing something
> > wrong again.
> >
> > I would appreciate any suggestions on how to make
> > this thing sizzle as I
> > feel sure it can.  Right now I'm dead if I have to
> > do a big file.
> >
> > Thanks
> >
> > remove the dash if emailing
> >
> >
>


  • Prev by Date: Re: Probable answer to a mystery
  • Next by Date: Re: Quite Upset with NIntegrate
  • Previous by thread: Re: Importing binary file is toooooo slow...
  • Next by thread: Re: Importing binary file is toooooo slow...