MathGroup Archive 2014

[Date Index] [Thread Index] [Author Index]

Search the Archive

Re: Downloading Multiple Files Via Import and FTP

  • To: mathgroup at smc.vnet.net
  • Subject: [mg132198] Re: Downloading Multiple Files Via Import and FTP
  • From: Roland Franzius <roland.franzius at uos.de>
  • Date: Wed, 15 Jan 2014 04:16:16 -0500 (EST)
  • Delivered-to: l-mathgroup@mail-archive0.wolfram.com
  • Delivered-to: l-mathgroup@wolfram.com
  • Delivered-to: mathgroup-outx@smc.vnet.net
  • Delivered-to: mathgroup-newsendx@smc.vnet.net
  • References: <lb02o5$dn7$1@smc.vnet.net>

Am 13.01.2014 07:57, schrieb Gregory Lypny:
> Hello everyone,
>
> Mathematica's Import function works well for downloading single files from a remote computer using ftp, but it gets stuck when downloading anything more than about a 20 files in succession, even when those files are just few bytes.
>
> If I define the path to a folder on my remote mac as
>
> 	ftpPath="ftp://user:password@domainName//filePath/=93
>
> I can easily download a single file, myFile, using
>
> 	Import[ftpPath<>myFile, "CSV=94]
>
> But if I try to download, say, 60 small files whose names are contained in myFileNames by mapping across the names (or by using Table)
>
> 	Import[ftpPath<> #, "CSV=94] &/@ myFileNames
>
> I get two errors: (1) URLFetch::invhttp : Failure when receiving data from the peer; and (2) FetchURL::conopen.
>
> I thought that maybe Mathematica was sending in the requests too quickly, so I tried slowing things down by inserting a one-second pause between each call to Import but that didn't help. Maybe I have to close a socket between each import but I don't know how to do that.
>
> Anyone know of a fix?

Uploading and downloading many files or a very large file using  a 
ftp-browser like fileZilla, in the protocoll window you observe that the 
server rather often breaks the connection.

Its the browsers duty to reconnect and and resume the broken transfer 
session.

The rather short connection timeout seems to be a security feature of 
ftp servers against automated data suction. The ftp password security is 
an illusion, it applies to the naive users only.

-- 

Roland Franzius



  • Prev by Date: Re: Questions Regarding Mathematica Kernel Supplied
  • Next by Date: Vertices of a polyhedron
  • Previous by thread: Downloading Multiple Files Via Import and FTP
  • Next by thread: Re: Questions Regarding Mathematica Kernel Supplied