Virtual Shared Memory

How can processes communicate with each other?

Message Passing

Shared Memory

Virtual Shared Memory: faking it on distributed memory machines.

When a process needs the value of a global (shared) variable or wants to set its value, it sends a request back to the scheduler. The scheduler maintains the variables, udpates their values, and sends back the requested values.

Declaring Shared Variables/Functions

A declaration sets up the slave-side definitions and transports them to all slaves, which, therefore, need no separate init code.

Examples

We should have at least two remote kernels here.

Declare shared variables.

Basic tests.

Atomic update:

No special syntax is required to use shared variables. Just do things the normal way. If a variable is shared, all accesses are transparently sent to the master kernel.

The prototypical synchronization problem: between reading and writing a shared variable y, another process can change the value or read an obsolete value.

A sequential execution (using Map instead of parallelMap) would ensure that the final value of y were 5. With concurrent access, it will usually be smaller.

Synchronization

Exclusive access to variables can be achieved with one new primitive operation, a conditional atomic write. In machine code, it is often called test and set or read-modify-write.

Set s to e (just like s=e), but only if s is currently unset or its value is . Return the (old or new) value of s. The read-modify-write is atomic.

This attempt fails:

Release exclusive access.

Now r2 succeees.

Critical Sections

Exclusive access to a variable or other resource can be controlled by a lock. Here is the typical code added to the previous example about asynchronous access to a variable.

The Dining Philosophers

The classic example for access to shared resources, in this case forks placed between philosophers sitting at a round table. A philosopher needs the two forks to his left and right to eat.

The variables forks[1],...,forks[n] serve to reserve forks by conditionally setting them to a unique value.

Each dining philosopher is one process.

Here's the behaviour of a philosopher: either thinking or eating.

We want essentially ParallelMap, but need a way to stop them cleanly.

After an interrupt, clean up:

Document converted by Mathematica of Wolfram Research