Services & Resources / Wolfram Forums / MathGroup Archive
-----

MathGroup Archive 2012

[Date Index] [Thread Index] [Author Index]

Search the Archive

Dynamically launching more kernels *during* a parallel calculation -- is it safe?

  • To: mathgroup at smc.vnet.net
  • Subject: [mg124727] Dynamically launching more kernels *during* a parallel calculation -- is it safe?
  • From: Szabolcs HorvÃt <szhorvat at gmail.com>
  • Date: Thu, 2 Feb 2012 04:58:42 -0500 (EST)
  • Delivered-to: l-mathgroup@mail-archive0.wolfram.com

It appears that it is possible to launch more kernels *during* a
parallel calculation, and they will be utilized for the rest of the
calculation.

Here's a simple way to test and illustrate (in a fresh kernel):

LaunchKernels[2]

Parallelize[
 Table[Pause@RandomInteger[20]; $KernelID, {i, 8}],
 Method -> "FinestGrained"
]

Now use Evaluation -> Interrupt Evaluation... to interrupt the
evaluation and go into a subsession.  In the subsession evaluate

$KernelCount (* check that we only have 2 kernels *)

LaunchKernels[4] (* launch 4 more *)

Return (* return from the dialog *)

Notice that when the Table[] finally finishes, we get results between
1 and 6.  This means that all 6 kernels have been used for the rest of
the evaluation, even though some kernels were only launched in the
middle of the calculation.

Question:

Is it safe to do this?  Can this break anything?

Consider some code like this:

manageKernels[] :=(* uses LaunchKernels[] to launch more is compute
resources become available *)

SetSharedFunction[manageKernels] (* ensure that manageKernels[] is
always evaluated on the master kernel *)

Parallelize[
 Table[manageKernels[]; compute[i], {i, 100}],
 Method -> "FinestGrained"
]

Is this going to be safe?  What if manageKernels[] uses CloseKernels[] as well?



  • Prev by Date: Mathematical Statistics with Mathematica - special discounted student pricing
  • Next by Date: Re: Input cells destroy Grids?
  • Previous by thread: Mathematical Statistics with Mathematica - special discounted student pricing
  • Next by thread: Plots for terms of polynomials. Slider initial values?