Re: How to local files on ParallelKernels
- To: mathgroup at smc.vnet.net
- Subject: [mg116765] Re: How to local files on ParallelKernels
- From: Albert Retey <awnl at gmx-topmail.de>
- Date: Sat, 26 Feb 2011 06:09:17 -0500 (EST)
- References: <ik2n9v$9kb$1@smc.vnet.net> <ik8449$mte$1@smc.vnet.net>
Hi,
> Nobody has any idea how to make them write the result from remote
> kernels to a local file? any help is appreciated.
If the remote kernels need to write to the same file, it must be on a
shared file system. You should look up how to set that up for the OS you
are using. If you have that working, you would need some locking
mechanism to avoid that the parallel write operations will mess up your
file. Within Mathematica, the easiest way to set up such a locking is
via the CriticalSection function, which you can look up in the
documentation.
However, what I would rather do is to transfer the results back to the
master kernel and let the master write the results to the file
(especially if there is no shared file system available yet). Here are
some simple approaches that you can build on:
1) use a share variable for the result, it will not be written to file
whenever a new result is available, though:
SetSharedVariable[results]; results = Table[0, {10}];
ParallelTable[results[[i]] = i -> $KernelID, {i, 1, 10}];
results
2) If you really want the results to be written to a file as soon as
they are available you might want to use a combination of ParallelSubmit
and a loop of calls to WaitNext and PutAppend. See the documentation of
WaitNext for a starting point.
Note that with either version some extra work might be needed to avoid
the overhead to eat up all the speedup from parallelization.
hth,
albert