What is a "spark" in Haskell

MultithreadingHaskellParallel ProcessingMulticore

Multithreading Problem Overview


I'm confused about the notion of "spark"

Is it a thread in Haskell? Or is the action of spawning a new thread ?

Thanks everybody:

So to summarize, sparks are not thread but more of unit of computation (tasks to put it in C#/Java terms). So it's the Haskell way of implementing the task parallelism.

Multithreading Solutions


Solution 1 - Multithreading

Sparks are not threads. forkIO introduces Haskell threads (which map down onto fewer real OS threads). Sparks create entries in the work queues for each thread, from which they'll take tasks to execute if the thread becomes idle.

As a result sparks are very cheap (you might have billions of them in a program, while you probably won't have more than a million Haskell threads, and less than a dozen OS threads on half a dozen cores).

Think of it like this:

spark model

Solution 2 - Multithreading

See A Gentle Introduction to Glasgow Parallel Haskell.

> Parallelism is introduced in GPH by the par combinator, which takes two arguments that are to be evaluated in parallel. The expression p `par` e (here we use Haskell's infix operator notation) has the same value as e, and is not strict in its first argument, i.e. bottom `par` e has the value of e. (bottom denotes a non-terminating or failing computation.) Its dynamic behaviour is to indicate that p could be evaluated by a new parallel thread, with the parent thread continuing evaluation of e. We say that p has been sparked, and a thread may subsequently be created to evaluate it if a processor becomes idle. Since the thread is not necessarily created, p is similar to a lazy future.

[Emphasis in original]

Solution 3 - Multithreading

If I understand it correctly, a spark is an entry in a queue of jobs requiring work. A pool of threads take entries from this queue and runs them. Typically there is one thread per physical processor, so this scheme maximises throughput and minimizes thread context switching.

Solution 4 - Multithreading

It looks like it is similar to a "task" in Intel Threading Building Blocks.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionCheickView Question on Stackoverflow
Solution 1 - MultithreadingDon StewartView Answer on Stackoverflow
Solution 2 - MultithreadingApocalispView Answer on Stackoverflow
Solution 3 - MultithreadingPaul JohnsonView Answer on Stackoverflow
Solution 4 - MultithreadingAftershockView Answer on Stackoverflow