Re: Problems reading text files with embedded delimiters
- To: mathgroup at smc.vnet.net
- Subject: [mg125614] Re: Problems reading text files with embedded delimiters
- From: David Reiss <dbreiss at gmail.com>
- Date: Thu, 22 Mar 2012 05:48:00 -0500 (EST)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
- References: <jk9bfb$bo1$1@smc.vnet.net>
One possibility for this is to read the file as Lines and, given that you know that there is only one column with the unstructured text field, to parse the other columns from the left and the right up to that column's data since each of those columns are delimited uniquely by commas. Best, David On Mar 20, 3:26 am, Mark Coleman <markspcole... at gmail.com> wrote: > Hi, > > I'm struggling with importing a large flat file into Mathematica. The file, > which was pulled off one of my company's legacy mainframe systems, > contains about 100 fields and several million rows, including a mix of > dates, reals, and text. The fields are delimited by commas. > > One of the text fields, however, is quite large and contains commas, > line feeds, carriage returns, etc -- lots of characters commonly used > as delimiters. It is also a variable length field. There does not seem > to be anything characters within the field that would allow the > specification of a simple delimiter. I've tried the default Import[] > command as well as various permutations of ReadList[] using Record, > Word, and String specifications. > > I'm wondering if anyone could suggest an approach that would allow me > to read this file. More precisely, is there a way to avoid reading > characters embedded in the large text field that Mathematica thinks are field > delimiters? > > Thanks, > > Mark