MathGroup Archive 2012

[Date Index] [Thread Index] [Author Index]

Search the Archive

Cautionary tale : Remember to close StringToStream...

I some code running extremely slow and finally traced the problem to a division by zero which should take very little time...

2/3/12 10:10:15 In[2]:= Timing[1.0/0.0]
2/3/12 10:10:15 During evaluation of In[2]:= Power::infy: Infinite expression 1/0. encountered. >>
2/3/12 10:10:15 Out[2]= {0.,ComplexInfinity}

However I was experiencing something like this...

2/3/12 10:11:04 In[4]:= Timing[1.0/0.0]
2/3/12 10:11:12 During evaluation of In[4]:= Power::infy: Infinite expression 1/0. encountered. >>
2/3/12 10:11:12 Out[4]= {7.75,ComplexInfinity}

As the Barman said to the Bear "Why the big paws?"...

I traced the problem to the way I was reading in my data.  I had loaded a big file as a string, then having split this into separate items ran a pattern search for everything which looked like a number and then used Read[StringToStream[x], Number] to convert those items into numbers.  You will note that I did not close the streams created by StringToStream.

It appears that the time taken to process Message scales approximately as the square of the number of open streams.  Not a problem with say 1000 streams open but with 30,000 streams left open, on my machine, results in a few seconds of delay.  With hundreds of thousands of open streams my code just stopped.

Turning the warning message off...
2/3/12 10:15:43 In[5]:= Off[Power::infy]; Timing[1.0/0.0]
2/3/12 10:15:43 Out[5]= {0.,ComplexInfinity}
Sidesteps the problem, so it appears that it is a stream issue which makes some kind of sense as Messages come down a stream of their own.

In the future I will be more careful but this amused me (once I'd stopped bashing my head into the desk) so I thought I'd share it.

ReadClose[str_String]:=Reap[Apply[(Sow[Read[#,Number]]; Close[#])&,{StringToStream[str]}]][[2,1,1]];

  • Prev by Date: Re: Funny Behavior of Module
  • Next by Date: Re: Problem using Solve or Nsolve
  • Previous by thread: How to insert an animated gif file in a Mathematica notebook (for the purpose of HTML exporting)
  • Next by thread: Mathematica 8 - kernel hangs