Re: Import from web addresses fails
- To: mathgroup at smc.vnet.net
- Subject: [mg105639] Re: Import from web addresses fails
- From: jwinter <john.winterflood at gmail.com>
- Date: Sat, 12 Dec 2009 07:01:32 -0500 (EST)
- References: <200910081149.HAA13464@smc.vnet.net> <han6ab$pie$1@smc.vnet.net>
I have discovered much the same problem trying to download some astronomy data from NED (Nasa Extragalactic Database). If I run the following line it works fine and gives the data as an html web page: Import["http://nedwww.ipac.caltech.edu/cgi-bin/nph-allsky? ra_constraint=Between&ra_1=0&ra_2=0.0002&name_prefix3=2MASX&of=pre_text"] But if I change the command slightly so that it responds with the data in TSV (Tab Separated Values) file format then it says "the connection cannot be opened": Import["http://nedwww.ipac.caltech.edu/cgi-bin/nph-allsky? ra_constraint=Between&ra_1=0&ra_2=0.0002&name_prefix3=2MASX&of=ascii_tab"] FetchURL::conopen: The connection to URL http://nedwww.ipac.caltech.edu/cgi-bin/nph-allsky?ra_constraint=Between&ra_1=0&ra_2=0.0002&name_prefix3=2MASX&of=ascii_tab cannot be opened. If the URL is correct, you might need to configure your firewall program, or you might need to set a proxy in the Internet connectivity tab of the Preferences dialog (or by calling SetInternetProxy). For HTTPS connections, you might need to inspect the authenticity of the server's SSL certificate and choose to accept it. However both of these requests to NED work fine from a browser. So I assume there is a bug in Mathematica in which it ignores responses to an http-type url request which do not look like html (or something)! Here are the above URLs as tiny urls: http://tinyurl.com/ydjfxd5 http://tinyurl.com/ybjyx2x Maybe someone who got Bobs Import[] to work can check to see if this one wants to work for them (it is a very small amount of data that is "Import[]"ed)? I cannot "download" the "file" and run Import[] locally as the only reason that I am running Import[] is in order to download the file - automatically, one small segment at a time, and concatenate them into a complete data collection - since Nasa only allow a few thousand records per request and I want 1.5million. The HTML format Import[] which does work is not as useful as its data contains significantly reduced precision compared to the TSV version. I have now reported this as a bug and it has been given a tech support reference number of [TS48601]. It will be interesting to see what Wolfram's reply might be.