MathGroup Archive 2011

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Memory when reding in data

  • To: mathgroup at smc.vnet.net
  • Subject: [mg121002] Re: Memory when reding in data
  • From: David Bailey <dave at removedbailey.co.uk>
  • Date: Sun, 21 Aug 2011 05:32:22 -0400 (EDT)
  • Delivered-to: l-mathgroup@mail-archive0.wolfram.com
  • References: <j2lehm$ngv$1@smc.vnet.net>

On 19/08/2011 11:39, Peter Rodenbach wrote:
> Hi everyone.
>
> I am analysing image data and I have the problem that after about 50
> images or so Mathematica quits the calculation with the hint out of
> memory. The interesting part of the code i wrote is :
>
> (*Import the first image into Maxvals, which will be compared to all \
> other images and overwritten if any point is found to be larger*)
>
> SetDirectory[
>    "E:\\...];
>
> Initialise = OpenRead["s2_260_0000.edf", BinaryFormat ->  True];
> MaxValues =
>    BinaryRead["s2_260_0000.edf",
>     Table["UnsignedInteger16", {512}, {512}]];
> counter = Dimensions[MaxValues];
>
> (*Load all experimental images and perform maximum projection*)
>
> MaxProjection[filename_] := {OpenRead[filename, BinaryFormat ->  True],
>     Imagevalues =
>     BinaryRead[filename, Table["UnsignedInteger16", {512}, {512}]],
>    For[i = 1, i<= counter[[1]], i++,
>     	For [j = 1, j<= counter[[2]], j++,
>      If[MaxValues[[i, j]]>= Imagevalues[[i, j]],
>       MaxValues[[i, j]] = MaxValues[[i, j]],
>       MaxValues[[i, j]] = Imagevalues[[i, j]]]
>      	]
>     ], Close[Imagevalues]}
>
>
> MaxProjection[#]&  /@ FileNames@"*edf"
>
> So I guess Mathm saves all the data of all the images after each
> other. But once the comparision is done with the one MaxProjection
> Image I want to have, all the data can be dumped from the memory and
> then the next image can be compared with this MaxProjection image.
>
> Does anyone have an idea? Thx.
>
Your code looks wrong because you open each file with OpenRead and never 
close (or use) the stream it produces (I wonder if binary streams open a 
large buffer).
Try something like this:

strm=OpenRead[filename, BinaryFormat ->  True];
Imagevalues=BinaryRead[strm, Table["UnsignedInteger16", {512}, {512}]];
..........
Close[strm]


Notice that it is the stream that should be closed - not the data!

Also, I would use Module to define functions, rather than just using a 
list of expressions { .......}.

With a Module, you can specify certain  variables, such as strm, to be 
local to the module, and also, you won't return a list of garbage as the 
result of the call - which could cause trouble if one of those objects 
is big!

David Bailey
http://www.dbaileyconsultancy.co.uk







  • Prev by Date: Re: NDSolve with square root
  • Next by Date: Re: Getting 3d printable models out of 3d graphics in Mathematica
  • Previous by thread: Re: Memory when reding in data
  • Next by thread: Speeding Up Known Integrations