Re: Downloading Multiple Files Via Import and FTP
- To: mathgroup at smc.vnet.net
- Subject: [mg132198] Re: Downloading Multiple Files Via Import and FTP
- From: Roland Franzius <roland.franzius at uos.de>
- Date: Wed, 15 Jan 2014 04:16:16 -0500 (EST)
- Delivered-to: l-mathgroup@mail-archive0.wolfram.com
- Delivered-to: l-mathgroup@wolfram.com
- Delivered-to: mathgroup-outx@smc.vnet.net
- Delivered-to: mathgroup-newsendx@smc.vnet.net
- References: <lb02o5$dn7$1@smc.vnet.net>
Am 13.01.2014 07:57, schrieb Gregory Lypny: > Hello everyone, > > Mathematica's Import function works well for downloading single files from a remote computer using ftp, but it gets stuck when downloading anything more than about a 20 files in succession, even when those files are just few bytes. > > If I define the path to a folder on my remote mac as > > ftpPath="ftp://user:password@domainName//filePath/=93 > > I can easily download a single file, myFile, using > > Import[ftpPath<>myFile, "CSV=94] > > But if I try to download, say, 60 small files whose names are contained in myFileNames by mapping across the names (or by using Table) > > Import[ftpPath<> #, "CSV=94] &/@ myFileNames > > I get two errors: (1) URLFetch::invhttp : Failure when receiving data from the peer; and (2) FetchURL::conopen. > > I thought that maybe Mathematica was sending in the requests too quickly, so I tried slowing things down by inserting a one-second pause between each call to Import but that didn't help. Maybe I have to close a socket between each import but I don't know how to do that. > > Anyone know of a fix? Uploading and downloading many files or a very large file using a ftp-browser like fileZilla, in the protocoll window you observe that the server rather often breaks the connection. Its the browsers duty to reconnect and and resume the broken transfer session. The rather short connection timeout seems to be a security feature of ftp servers against automated data suction. The ftp password security is an illusion, it applies to the naive users only. -- Roland Franzius