Parallel Evaluation

Evaluating Elements of an Expression in Parallel

Sends the e to available links, then collects results.

We developed the basic idea in a previous example.

SetAttributes[ParallelEvaluate, HoldAll]

ParallelEvaluate[h_[exprs___]] :=
      h @@ Wait[ List @@ Queue /@ Hold[exprs] ]

This can be simplified somewhat to

ParallelEvaluate[h_[exprs___]] :=
      h @@ Wait[ Queue /@ Unevaluated[{exprs}] ]

Parallel Mapping

Evaluates in parallel.

We could reduce it to ParallelEvaluate, but it's so simple to express directly:

ParallelMap[f_, h_[exprs___]] :=
      h @@ Wait[ Composition[Queue,f] /@ {exprs} ]

Parallel Tables

Parallelizing the outermost dimension, preserving dynamic scoping of the iterator variable

ParallelTable[e_, {i_Symbol, range__}, rest___List] :=
    ParallelMap[ Block[{i=#}, Table[e, rest]]&, Range[range] ]

Example and Traces

Initializing Remote Kernels

Move definitions of global functions and variables to the slaves:

Collect definitions (downvalues, etc.) and turn them into held programs

send them to all slaves for evaluation

ExportEnvironment["context`"]

Mathematica code is portable: no need to have machine-specific code on slaves!
Use existing MathLink connection to transport code: no distribution problems, no common FS needed. Even the client code for the toolkit itself is transported over this link.

Example: Eigenvalues of Matrices

A parameter and two functions.

The parameter prec gives the desired precision for the computation of the eigenvalues of a random n by n matrix.

tf measures the time it takes to find the eigenvalues.

Make them known to all slaves.

Check (we can now clear the local value of prec; it is no longer needed).

A sample run

Or you can perform the same computation on all slaves to measure their relative speed.

Example: an Infinite Calculation

These polynomials are conjectured to be irreducible.

This computation benefits from latency hiding.

Explicit scheduling; try to keep all processors busy, but avoid long input queues.

d=150; ids={};
While[True,
  While[$QueueLength&#xF4310, (* queue one new job *)
    AppendTo[ids,Queue[{d},{d,Length[FactorList[poly[d,x]]]}]];
    d++
  ];
  If[Length[ids]>0, (* collect one result *)
    {res,id,ids} = WaitOne[ids];
    Print[res]
  ];
]

Collect orphans after an interrupt:

Document converted by Mathematica of Wolfram Research