Re: Tracking progress in ParallelDo

*To*: mathgroup at smc.vnet.net*Subject*: [mg132599] Re: Tracking progress in ParallelDo*From*: Szabolcs HorvÃt <szhorvat at gmail.com>*Date*: Thu, 17 Apr 2014 05:10:15 -0400 (EDT)*Delivered-to*: l-mathgroup@mail-archive0.wolfram.com*Delivered-to*: l-mathgroup@wolfram.com*Delivered-to*: mathgroup-outx@smc.vnet.net*Delivered-to*: mathgroup-newsendx@smc.vnet.net*References*: <lig9n7$cfl$1@smc.vnet.net>

There was a very similar question asked on StackExchange. You might be interested in the answers there. http://mathematica.stackexchange.com/questions/1548/monitor-doesnt-work-with-paralleltable Here's a slightly modified version of Leonid's solution: SetSharedVariable[progress] progress = 0; Monitor[ ParallelTable[progress++; Pause[0.5]; i, {i, 100}], progress ] The caveat here is that SetSharedVariable causes 'progress' to be always evaluated on the main kernel. Accessing the main kernel is expensive (it takes a long time). So this only makes sense if: - it is reasonable to break the parallel evaluation into the smallest possible pieces (equivalent of Method -> "FinestGrained") - each single evaluation in your ParallelDo takes considerably longer than the overhead due to the callback to the main kernel Otherwise doing this will kill performance. Personally I do not recommend doing this unless you are experienced with parallelization in Mathematica and understand what factors affect performance. Do check my link for more complex but also more efficient variations on the same idea (Eli Lansey's answer). Szabolcs On 2014-4-14, 5:28 , Brian Beckage wrote: > I'm applying a function repeatedly using ParallelDo. I would like to keep track of its progress. I've tried using both my own counter variable as well as using the increments within the ParallelDo function (below). Neither provides a way of sequentially tracking how many of the iterations have been spawned (not completed--but just kicked off). > > How could I track that iteration 1, 2, 3, 4, 5 etc. have been started? Is there a better way of monitoring progress of the ParalleDo? > > Thanks for your help! > > Brian > > > USING MY OWN COUNTER: myCounter: > > parallelBootstrapFireMultiplier[nBoot_, fireMultiplierRange_, parmList1_, > nYears_, initLandscapeProbs_] := Module[{outIter, myCounter}, > > ParallelEvaluate[Clear[myCounter]]; > myCounter = 0; > SetSharedVariable[myCounter]; > Reap[ > ParallelDo[ > myCounter++; > Print["Starting parallelBootstrapMultiplier step ", myCounter, " out of ", > nBoot]; > outIter = > iterFireMultiplier[fireMultiplierRange, parmList1, nYears, > initLandscapeProbs]; > ParallelSow[outIter], > {nBoot}] > ] > ] > > RESULTS in: > > (kernel 2) Starting parallelBootstrapMultiplier step 2 out of 6 > (kernel 1) Starting parallelBootstrapMultiplier step 2 out of 6 > (kernel 1) Starting parallelBootstrapMultiplier step 4 out of 6 > (kernel 2) Starting parallelBootstrapMultiplier step 4 out of 6 > (kernel 2) Starting parallelBootstrapMultiplier step 5 out of 6 > (kernel 1) Starting parallelBootstrapMultiplier step 6 out of 6 > > USING THE INCREMENT i FROM ParallelDo: > > parallelBootstrapFireMultiplier[nBoot_,fireMultiplierRange_,parmList1_,nYears_,initLandscapeProbs_]:= Module[{outIter}, > Reap[ > ParallelDo[ > Print["Starting parallelBootstrapMultiplier step ",i, " out of ",nBoot]; > outIter=iterFireMultiplier[fireMultiplierRange,parmList1,nYears,initLandscapeProbs]; (*This returns a list*) > ParallelSow[outIter], > {i,nBoot}] > ] > ] > > RESULTS in: > Starting parallelBootstrapMultiplier step 1 out of 6 > Starting parallelBootstrapMultiplier step 4 out of 6 > Starting parallelBootstrapMultiplier step 5 out of 6 > Starting parallelBootstrapMultiplier step 2 out of 6 > Starting parallelBootstrapMultiplier step 3 out of 6 > Starting parallelBootstrapMultiplier step 6 out of 6 >