Re: Import[] - limited from a pipe?
- To: mathgroup at smc.vnet.net
- Subject: [mg103973] Re: Import[] - limited from a pipe?
- From: Albert Retey <awnl at gmx-topmail.de>
- Date: Wed, 14 Oct 2009 07:55:54 -0400 (EDT)
- References: <hb3g2j$m04$1@smc.vnet.net>
Hi,
> I'm looking into various methods of importing data into mathematica from a
> piece of perl middleware (that talks to a database). Right now, the perl
> script is outputting CSV that I'm pulling in with Import[....,"CSV"] or
> "List" in some cases.
>
> In investigating other options (like Get) it seems that there are some
> limitations on Import[] that I don't quite understand.
>
> As a test, I'm looking at two components:
>
> 1 - a perl script that outputs "List[1,2,3,4]" (no newlines) and then
> exits.
> 2 - the output from that perl script redirected to a file "foo"
>
> In the first case:
>
> In[17]:= Import["!/full/path/test.pl", "Elements"]
> Import::fmtnosup: $Failed is not a supported Import format.
> Out[17]= $Failed
>
> In the second case:
>
> In[24]:= Import["/full/path/to/foo", "Elements"]
> Out[24]= {Comments, ExpressionList, Get, HeldExpressions}
>
> (* changing Elements to Get in the file version results in an appropriate
> List[1,2,3,4,5] being imported *)
>
> Is this a bug or expected behavior?
I would call it an expected bug: you need to understand that Import is
just a wrapper for numerous import filter routines, most of them being
implemented as seperate programs. So it doesn't come as a surprise that
not all of them support every feature.
> If it is expected, where can I find/generate more information on the
> limitations of Import with a pipe?
In this case it looks like Import[..,"Elements"] tries to reopen/seek
the input file which will not work for a pipe. I have no insight in what
the code really does but it doesn't come as a big surprise that this
doesn't work, although a more careful implementation probably could
avoid the problem.
Knowing that the above probably is not much help I have another
suggestion: Have you considered to interact with the database directly?
If it is a relational database that can be accessed with DatabaseLink
that probably would be a better way to access the data and completley
avoid the problems you have with Import. If DatabaseLink can't access
the db, maybe there are programming APIs that you can directly use from
Mathematica, e.g. for Java, .NET or C? Or are you really using something
that is only easily achieved with perl?
hth,
albert