Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2012

[Date Index] [Thread Index] [Author Index]

Search the Archive

Import HTTP data in asynchronous/parallel way

  • To: mathgroup at smc.vnet.net
  • Subject: [mg125923] Import HTTP data in asynchronous/parallel way
  • From: Rodrigo Murta <murta at rodrigomurta.com>
  • Date: Sat, 7 Apr 2012 05:57:00 -0400 (EDT)
  • Delivered-to: l-mathgroup@mail-archive0.wolfram.com

Hi All

    I'm testing some web scraping using Mathematica and would like to
know how to work with Import in an asynchronous/parallel way.
    Now I'm using Parallelize to do that. It works but it doesn't look
like the best way to do that, due to the kernel number limitation.

   My code is like that:
   result = Parallelize[Import/@urlList]

  How can I do it in another asynchronous way? Something like
backgroud process using & in bash?
  I can do it inside Mathematica? I know that I could speed up a lot
my scrap with that.

tks in advance
Murta



  • Prev by Date: complex triple sum
  • Next by Date: Re: Graphics3D and transparent background
  • Previous by thread: Re: complex triple sum
  • Next by thread: Re: Import HTTP data in asynchronous/parallel way