MathGroup Archive 2004

[Date Index] [Thread Index] [Author Index]

Search the Archive

RE: Re: Excessive Mathematica memory use, revisited.

  • To: mathgroup at smc.vnet.net
  • Subject: [mg46940] RE: [mg46928] Re: Excessive Mathematica memory use, revisited.
  • From: "Virgilio, Vincent" <Vincent.Virgilio at itt.com>
  • Date: Tue, 16 Mar 2004 19:55:04 -0500 (EST)
  • Sender: owner-wri-mathgroup at wolfram.com

Frank, thanks, Binarica seems to hold the most promise, given my current resources. I've run across it before, but decided to go with BinaryImport[] simply because it is packaged with Mathematica. Clearly that decision needs to be reevaluated. Probably, eventually, I'll try to extend Binarica.

Mariusz, with your advice, I've seen MaxMemoryUsed[] during import of an ~ 5MB image climb to around 85MB. ByteCount[image] was on the order 5MB. MemoryInUse[] was higher than ByteCount[image], I think. Again, when I have time, I'll post something more definite.

Jens, if I understand correctly, it is possible that even if I do manage to efficiently import all of this data into Mathematica, I still won't be able to do much with it. That will mostly depend on how the DigitalImageProcessing package handles the data.

János, I think you're talking about distributed shared memory. Would that be easier for Wolfram to implement than a traditional data tile cache (as in ENVI/IDL, Gimp, etc.)? I realize this is highly conjectural.

Vince Virgilio

-----Original Message-----
From: Frank Iannarilli [mailto:frankeye at cox.net]
To: mathgroup at smc.vnet.net
Subject: [mg46940] [mg46928] Re: Excessive Mathematica memory use, revisited.


Virgilio,

Check out "binarica":
http://library.wolfram.com/infocenter/MathSource/4870/

The summary: 
The Binarica notebook contains cells for fast reading of a binary file
and conversion of its IEEE single-precision floating point byte stream
into Mathematica (machine-precision) Reals. The processing maintains
the (Mathematica 4.X) PackedArray representation so crucial to memory
and speed efficiency.

Although crafted only for PC-endian byte format, and merely for IEEE
single-precision, the methods provided are sufficiently generalizable
by an interested user.

I've found it quite tricky, but doable, to get near-optimal space-time
performance on binary import.

Regards,


************************************
This email and any files transmitted with it are proprietary and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email
in error please notify the sender. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of ITT Industries, Inc.
The recipient should check this email and any attachments for the presence of viruses. ITT Industries accepts no liability for any damage caused by any virus transmitted by this
email.
************************************


  • Prev by Date: Re: shade area between vertical lines
  • Next by Date: Exporting data and comments
  • Previous by thread: RE: Re: Excessive Mathematica memory use, revisited.
  • Next by thread: Mathematica programmer needed