Lectures

Work in progress

Lecture 1 - Course Intro and Why Parallel Functional Programming?

This is the lecture that sets the scene for the course. It introduces what we mean by parallelism, and distinguishes that from concurrency. It explains why functional languages are well suited for parallel programming, and this despite the fact that they really failed to deliver on the promise of solving parallel programming in the past. So why are they interesting now?

Slides

Code

Competition

randomInts = 
  take 200000 (randoms (mkStdGen 211570155)) 
    :: [Integer]

You should submit your code using the Fire system. This is for fun, so you may form any group you like. We suggest that a single person submits the code (with comments about who has worked on it).

The submitted entries and John's slides giving the results are available in this zipped file.

Reading

• Note that we distinguish between parallel and concurrent programming. Our undergraduate curriculum at Chalmers has arguably overemphasized the latter (locks, semaphores, synchronisation mechanisms and the like) and underemphasized the business of making programs run faster by using many cores (see this discussion by Simon Marlow from Microsoft Research, who makes distinctions that exactly mirror our view). Similar opinions were later expressed by Bob Harper, who has radically reshaped the introductory computing curriculum at CMU to place parallelism at the centre — a major source of inspiration for this course.

  On Simon Marlow's home page, you can find notes and slides from his CEFP course on Parallel and Concurrent Programming in Haskell. The notes give a good explanation of why the topics of this course are interesting. They also make the same distinction between concurrency and parallelism as that made in this course. Later in the course, Simon himself will come and tell us about his worhttp://community.haskell.org/~simonmar/par-tutorial.pdf"> on the Par Monad.

• Be inspired by this video of Simon Peyton Jones lecturing on parallel programming in Haskell
• Chapter 24 of Real World Haskell covers concurrent and multicore programming.

Lecture 2 - from par and pseq to Strategies

This lecture considers par and pseq more critically, and concludes that it might be a good idea to separate the control of behaviour relating to parallelisation from the description of the algorithm itself. The idea of Strategies is described in a well-known paper called Algorithms + Strategies = Parallelism by Trinder, Hammond, Loidl and Peyton Jones. More recently, Marlow and some of the original authors have updated the idea, in Seq no more: Better Strategies for Parallel Haskell. We expect you to read both of these papers. The lecture is based on the newer paper.

Slides

Code

Reading

See above for papers

The documentation of the Strategies Library is very helpful.

Lecture 3 - Threadscope and GHC events (Andres Löh, Well-Typed LLP)

Andres expanded on some of the topics from the first two lectures, emphasising the need to think about lazy evaluation (and degrees of evaluation) when parallelising programs. He showed some pitfalls of parallel programming through small examples, but based on his experience of parallelising larger programs, gained through his work on the Parallel GHC project. He went more deeply into how to use Threadscope to debug parallel Haskell programs, and into the associated GHC event system. (Well-Typed have made significant improvements to Threadscope and the event system recently.) He ran his examples on an 8 core machine in Leipzig and we looking on enviously.

One thing we hope to do at the end of the course is to compile a list of suggestions for useful improvements to Threadscope. So keep a note of your ideas along these lines.

Slides etc.

You would be well advised to study the code and to try some of the exercises.

Lecture 4 - The Par Monad (Simon Marlow, Microsoft Research, Cambridge)

The lecture was an example of what we call "hearing it from the horse's mouth". It was about a new programming model for deterministic parallelism. Simon introduced the Par Monad, a monad for deterministic parallelism, and showed how I-structures are used to exchange information between parallel tasks (or "blobs"), see his Haskell'11 paper with Ryan Newton and Simon PJ. Simon showed some nice examples of how to program in this style. He showed an example of pipeline parallelism (using streams) where the dynamic behaviour is simply not possble to express using Strategies. (This example is very new and is not in the lecture notes mentioned below.) At the end of the lecture, he briefly showed the kmeans problem that is described in his lecture notes. Students began work on parallelising it, but someone else had booked the room for 17.00, so we all had to leave. Students on the course are advised to complete the parallelisation of the kmeans example (and Simon has provided a solution, which we may well return to in the next lecture).

Slides

Code

The stream example is in code/euler35

Lecture 5 - More on the Par Monad

Relating par, strategies and the Par Monad. Shows the k-means example in detail. Also proposes some possible topics for student presentations. For reading and code, see the previous lecture. The lecture also contains another programming challenge.

Slides

Lecture 6 - Skeletons (Jost Berthold, DIKU, Copenhagen University)

Jost discussed skeletons as a means to structure parallel computations -- viewing skeletons as higher order functions. He distinguished three types of skeletons: small scale skeletons (like parMap), process communication topology skeletons, and proper algorithmic skeletons (like divide and conquer). He briefly discussed the Eden language. The standard way of using Eden requires a modified GHC runtime. Jost promises that it will soon be possible to download the necessary Eden modules and skeleton library from the Eden home page. It is also possible to use the Eden skeletons on top of a simulation of the modified runtime, built on Concurrent Haskell -- thus using Eden as a library with an unmodified GHC runtime. This also provides decent speed-ups. More details of this shortly.

Slides

Lecture 7 - Data Parallelism I

This lecture is all about Guy Blelloch's seminal work on the NESL programming language and on parallel functional algorithms and associated cost models. The best introduction is to watch the video of his marvellous invited talk at ICFP 2010, which John and Mary had the pleasure to attend. There are pages about NESL and about his publications in general. For the notions of work and depth, see this part of the 1996 CACM paper, and also this page, which considers work and depth for three algorithms.

Slides

Lecture 8 - Data Parallelism II

Data parallel programming using the Repa library, which gives flat data parallelism. The main source is the Repa paper from ICFP 2010. Thanks to Ben Lippmeier for letting me borrow some slides.

Slides

Lecture 9 - Parallel Programming in Erlang

This lecture introduced Erlang for Haskell programmers, taking parallelising quicksort as an example, both within one Erlang VM and distributed across a network. The latest version of the Erlang system can be downloaded from here. There is a Windows installer. Many linux versions have an Erlang packagage available, but not necessarily a package suitable for development of Erlang code, and not necessarily the latest version. On Ubuntu, try

sudo apt-get install erlang-dev

If that doesn't work or you can't find an appropriate package, build the VM from source.

Slides