MathGroup Archive 2011

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: random line in a very large file

  • To: mathgroup at
  • Subject: [mg117002] Re: random line in a very large file
  • From: David Bailey <dave at>
  • Date: Mon, 7 Mar 2011 05:48:08 -0500 (EST)
  • References: <ikvoit$f62$>

On 06/03/2011 10:44, Ramiro wrote:
> Hi everyone,
> I have very large files (close to 1gb).  I want to find a random line
> on it, I wanted to compare the Mathematica native commands, versus
> calling a unix command such as sed.  For example:
> file = "example";
> n = 1000000;
> Export[file, Range[n], "List"];
> i = RandomInteger[{1, n}];
> str = OpenRead[file];
> Skip[str, "String", i];
> sample1 = Read[str, Expression];
> Print[sample1];
> Close[str];

Are the lines of equal length? If they are, you could calculate where 
each line starts, and open the file in binary mode, and position using 
SetStreamPosition, then read in the correct number of bytes. Remember 
that some systems (Windows) use two bytes at the end of each line!

David Bailey

  • Prev by Date: FindFit power law problem
  • Next by Date: results from Array and ParallelArray differ
  • Previous by thread: random line in a very large file
  • Next by thread: Re: random line in a very large file