Services & Resources / Wolfram Forums
-----
 /
MathGroup Archive
2002
*January
*February
*March
*April
*May
*June
*July
*August
*September
*October
*November
*December
*Archive Index
*Ask about this page
*Print this page
*Give us feedback
*Sign up for the Wolfram Insider

MathGroup Archive 2002

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Large-scale enumerations and memory problems

  • To: mathgroup at smc.vnet.net
  • Subject: [mg35095] Re: Large-scale enumerations and memory problems
  • From: Jens-Peer Kuska <kuska at informatik.uni-leipzig.de>
  • Date: Tue, 25 Jun 2002 03:39:32 -0400 (EDT)
  • Organization: Universitaet Leipzig
  • References: <af6i3m$llh$1@smc.vnet.net>
  • Reply-to: kuska at informatik.uni-leipzig.de
  • Sender: owner-wri-mathgroup at wolfram.com

"Mark E. Berres" wrote:
Hi,

you need OpenAppen[] and must Write[] your results
sequential to the file. Export[] will always
use the complete expression ..

Regards
  Jens

> 
> Hi,
> 
>     I am performing some enumerations (v4.1 under Win-XP with 768 MB
> physical memory and
> 4GB swap space) with the following simple code:
> 
> Enum[l_List, (k_Integer)?Positive] := Block[{minusone = Enum[l, k - 1], i,
>      n = Length[l]}, Join @@ Table[(Prepend[#1, l[[i]]] & ) /@ minusone, {i,
> n}]].
> 
> What this does is to take a supplied list of objects, l, and generate all
> possible combinations of
> these objects in a string of length k. Thus, even for small l and moderate
> values of k, the number
> of combinations becomes very large.
> 
> I am exporting the results to a file by
> 
> Export["file.txt", Enum[{G , A , T, C}, 6], "List"] (I want one entry per
> line of text)
> 
> but when k becomes "too large", for example if l=4
> and k=14, 4^14 = ~268 million, the kernal eventually crashes complaining
> about memory. I estimated
> that approx. 1.61 GB is required to store all strings in memory before
> writing to disk - plenty of room
> available on my machine, but still it crashes. It appears that the
> calculation first does its thing and then
> when initiating a write to disk, crashes. Is there a way to write the
> results one-at-a-time, rather than
> the whole-thing-at-once? Or does my problem lie elsewhere?
> 
>                 Sincerely,
> 
>                 Mark


  • Prev by Date: RE: Large-scale enumerations and memory problems
  • Next by Date: Re: friendly challenge 3
  • Previous by thread: RE: Large-scale enumerations and memory problems
  • Next by thread: RE: Large-scale enumerations and memory problems