Re: random line in a very large file
- To: mathgroup at smc.vnet.net
- Subject: [mg117002] Re: random line in a very large file
- From: David Bailey <dave at removedbailey.co.uk>
- Date: Mon, 7 Mar 2011 05:48:08 -0500 (EST)
- References: <ikvoit$f62$1@smc.vnet.net>
On 06/03/2011 10:44, Ramiro wrote:
> Hi everyone,
>
> I have very large files (close to 1gb). I want to find a random line
> on it, I wanted to compare the Mathematica native commands, versus
> calling a unix command such as sed. For example:
>
> file = "example";
> n = 1000000;
> Export[file, Range[n], "List"];
> i = RandomInteger[{1, n}];
>
> str = OpenRead[file];
> Skip[str, "String", i];
> sample1 = Read[str, Expression];
> Print[sample1];
> Close[str];
>
Are the lines of equal length? If they are, you could calculate where
each line starts, and open the file in binary mode, and position using
SetStreamPosition, then read in the correct number of bytes. Remember
that some systems (Windows) use two bytes at the end of each line!
David Bailey
http://www.dbaileyconsultancy.co.uk