Re: random line in a very large file
- To: mathgroup at smc.vnet.net
- Subject: [mg117063] Re: random line in a very large file
- From: Ramiro <ramiro.barrantes at gmail.com>
- Date: Wed, 9 Mar 2011 06:56:36 -0500 (EST)
- References: <ikvoit$f62$1@smc.vnet.net> <il2dft$68b$1@smc.vnet.net>
On Mar 7, 5:53 am, David Bailey <d... at removedbailey.co.uk> wrote:
> On 06/03/2011 10:44, Ramiro wrote:
>
>
>
> > Hi everyone,
>
> > I have very large files (close to 1gb). I want to find a random line
> > on it, I wanted to compare the Mathematica native commands, versus
> > calling a unix command such as sed. For example:
>
> > file = "example";
> > n = 1000000;
> > Export[file, Range[n], "List"];
> > i = RandomInteger[{1, n}];
>
> > str = OpenRead[file];
> > Skip[str, "String", i];
> > sample1 = Read[str, Expression];
> > Print[sample1];
> > Close[str];
>
> Are the lines of equal length? If they are, you could calculate where
> each line starts, and open the file in binary mode, and position using
> SetStreamPosition, then read in the correct number of bytes. Remember
> that some systems (Windows) use two bytes at the end of each line!
>
> David Baileyhttp://www.dbaileyconsultancy.co.uk
Thanks so much. Readline did the trick.