MathGroup Archive 2011

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Memory when reding in data

  • To: mathgroup at
  • Subject: [mg121002] Re: Memory when reding in data
  • From: David Bailey <dave at>
  • Date: Sun, 21 Aug 2011 05:32:22 -0400 (EDT)
  • Delivered-to:
  • References: <j2lehm$ngv$>

On 19/08/2011 11:39, Peter Rodenbach wrote:
> Hi everyone.
> I am analysing image data and I have the problem that after about 50
> images or so Mathematica quits the calculation with the hint out of
> memory. The interesting part of the code i wrote is :
> (*Import the first image into Maxvals, which will be compared to all \
> other images and overwritten if any point is found to be larger*)
> SetDirectory[
>    "E:\\...];
> Initialise = OpenRead["s2_260_0000.edf", BinaryFormat ->  True];
> MaxValues =
>    BinaryRead["s2_260_0000.edf",
>     Table["UnsignedInteger16", {512}, {512}]];
> counter = Dimensions[MaxValues];
> (*Load all experimental images and perform maximum projection*)
> MaxProjection[filename_] := {OpenRead[filename, BinaryFormat ->  True],
>     Imagevalues =
>     BinaryRead[filename, Table["UnsignedInteger16", {512}, {512}]],
>    For[i = 1, i<= counter[[1]], i++,
>     	For [j = 1, j<= counter[[2]], j++,
>      If[MaxValues[[i, j]]>= Imagevalues[[i, j]],
>       MaxValues[[i, j]] = MaxValues[[i, j]],
>       MaxValues[[i, j]] = Imagevalues[[i, j]]]
>      	]
>     ], Close[Imagevalues]}
> MaxProjection[#]&  /@ FileNames@"*edf"
> So I guess Mathm saves all the data of all the images after each
> other. But once the comparision is done with the one MaxProjection
> Image I want to have, all the data can be dumped from the memory and
> then the next image can be compared with this MaxProjection image.
> Does anyone have an idea? Thx.
Your code looks wrong because you open each file with OpenRead and never 
close (or use) the stream it produces (I wonder if binary streams open a 
large buffer).
Try something like this:

strm=OpenRead[filename, BinaryFormat ->  True];
Imagevalues=BinaryRead[strm, Table["UnsignedInteger16", {512}, {512}]];

Notice that it is the stream that should be closed - not the data!

Also, I would use Module to define functions, rather than just using a 
list of expressions { .......}.

With a Module, you can specify certain  variables, such as strm, to be 
local to the module, and also, you won't return a list of garbage as the 
result of the call - which could cause trouble if one of those objects 
is big!

David Bailey

  • Prev by Date: Re: NDSolve with square root
  • Next by Date: Re: Getting 3d printable models out of 3d graphics in Mathematica
  • Previous by thread: Re: Memory when reding in data
  • Next by thread: Speeding Up Known Integrations