Re: How can I do a "grep -v" equivalent in Import[]?
- To: mathgroup at smc.vnet.net
- Subject: [mg91819] Re: How can I do a "grep -v" equivalent in Import[]?
- From: Szabolcs Horvát <szhorvat at gmail.com>
- Date: Tue, 9 Sep 2008 06:56:10 -0400 (EDT)
- Organization: University of Bergen
- References: <g9t6mq$j8s$1@smc.vnet.net>
Jason Ledbetter wrote:
> Folk,
> I have some data that is column formatted with a dual line header every few
> lines; there may also be a variable number of lines between each header
> output. The first line is readily identifiable with say "grep ^FOO", but the
> second line in the header has white space at the beginning.
>
> So what I need to do is the equivalent of "grep -v" to get rid of the
> headers. My ultimate goal is to create a named List[] variable for each
> "column" in the data, but I need to get rid of the headers.
>
> Alternatively, I could regex match on the lines I DO want I think.. just
> trying to wrap my head around how to do this in Mathematica.
> For example, "egrep '^[0-9]+' foo.txt" gets me what I need in most shells...
> I'd just like mathematica to do the same on import.
>
> Do I need to process the import procedurally and dump the data I want into a
> new Table[]?
>
> I've been able to figure out how to get columnar data out of the import
> using "foo=data[[All,{1}]" and so on (e.g., first column in the data)... I
> just need to pre-filter the headers out.
>
I would like to note that it is possible to read from pipes with
Mathematica. For example, the following works for dropping lines
starting with "foo":
Import["!grep -v ^foo mytable.txt", "Table"]
The "!" character tells Mathematica to run the command and read its
output. Just make sure to always specify the data type (e.g. "Table"
above) when you use this syntax with Import[].
I hope this helps,
Szabolcs