Create multiple threads and wait for all of them to complete

How can I create multiple threads and wait for all of them to complete?

It depends which version of the .NET Framework you are using. .NET 4.0 made thread management a whole lot easier using Tasks:

class Program
{
	static void Main(string[] args)
	{
		Task task1 = Task.Factory.StartNew(() => doStuff());
		Task task2 = Task.Factory.StartNew(() => doStuff());
		Task task3 = Task.Factory.StartNew(() => doStuff());

		Task.WaitAll(task1, task2, task3);
                Console.WriteLine("All threads complete");
	}

	static void doStuff()
	{
		//do stuff here
	}
}

In previous versions of .NET you could use the BackgroundWorker object, use ThreadPool.QueueUserWorkItem(), or create your threads manually and use Thread.Join() to wait for them to complete:

static void Main(string[] args)
{
	Thread t1 = new Thread(doStuff);
	t1.Start();

	Thread t2 = new Thread(doStuff);
	t2.Start();

	Thread t3 = new Thread(doStuff);
	t3.Start();

	t1.Join();
	t2.Join();
	t3.Join();

	Console.WriteLine("All threads complete");
}

I think you need WaitHandler.WaitAll. Here is an example:

public static void Main(string[] args)
{
    int numOfThreads = 10;
    WaitHandle[] waitHandles = new WaitHandle[numOfThreads];

    for (int i = 0; i < numOfThreads; i++)
    {
        var j = i;
        // Or you can use AutoResetEvent/ManualResetEvent
        var handle = new EventWaitHandle(false, EventResetMode.ManualReset);
        var thread = new Thread(() =>
                                {
                                    Thread.Sleep(j * 1000);
                                    Console.WriteLine("Thread{0} exits", j);
                                    handle.Set();
                                });
        waitHandles[j] = handle;
        thread.Start();
    }
    WaitHandle.WaitAll(waitHandles);
    Console.WriteLine("Main thread exits");
    Console.Read();
}

FCL has a few more convenient functions.

(1) Task.WaitAll, as well as its overloads, when you want to do some tasks in parallel (and with no return values).

var tasks = new[]
{
    Task.Factory.StartNew(() => DoSomething1()),
    Task.Factory.StartNew(() => DoSomething2()),
    Task.Factory.StartNew(() => DoSomething3())
};
Task.WaitAll(tasks);

(2) Task.WhenAll when you want to do some tasks with return values. It performs the operations and puts the results in an array. It's thread-safe, and you don't need to using a thread-safe container and implement the add operation yourself.

var tasks = new[]
{
    Task.Factory.StartNew(() => GetSomething1()),
    Task.Factory.StartNew(() => GetSomething2()),
    Task.Factory.StartNew(() => GetSomething3())
};
var things = Task.WhenAll(tasks);

I've made a very simple extension method to wait for all threads of a collection:

using System.Collections.Generic;
using System.Threading;

namespace Extensions {
    public static class ThreadExtension {
        public static void WaitAll (this IEnumerable<Thread> threads) {
            if (threads != null) {
                foreach (Thread thread in threads) {
                    thread.Join();
                }
            }
        }
    }
}

Then you simply call:

List<Thread> threads = new List<Thread>();
// Add your threads to this collection
threads.WaitAll();

In .NET 4.0, you can use the Task Parallel Library.

In earlier versions, you can create a list of Thread objects in a loop, calling Start on each one, and then make another loop and call Join on each one.

If you don't want to use the Task class (for instance, in .NET 3.5), you can just start all your threads, and then add them to the list and join them in a foreach loop.

Example:

List<Thread> threads = new List<Thread>();

// Start threads
for (int i = 0; i < 10; i++) {
    int tmp = i; // Copy value for closure
    Thread t = new Thread(() => Console.WriteLine(tmp));
    t.Start();
    threads.Add(t);
}

// Join threads (wait threads)
foreach (Thread thread in threads) {
    thread.Join();
}

I don't know if there is a better way, but the following describes how I did it with a counter and background worker thread.

private object _lock = new object();
private int _runningThreads = 0;

private int Counter{
    get{
        lock(_lock)
            return _runningThreads;
    }
    set{
        lock(_lock)
            _runningThreads = value;
    }
}

Now whenever you create a worker thread, increment the counter:

var t = new BackgroundWorker();
// Add RunWorkerCompleted handler

// Start thread
Counter++;

In work completed, decrement the counter:

private void RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
    Counter--;
}

Now you can check for the counter anytime to see if any thread is running:

if(Couonter>0){
    // Some thread is yet to finish.
}

Most proposed answers don't take into account a time-out interval, which is very important to prevent a possible deadlock. Next is my sample code. (Note that I'm primarily a Win32 developer, and that's how I'd do it there.)

//'arrRunningThreads' = List<Thread>

//Wait for all threads
const int knmsMaxWait = 3 * 1000;			//3 sec timeout
int nmsBeginTicks = Environment.TickCount;
foreach(Thread thrd in arrRunningThreads)
{
	//See time left
	int nmsElapsed = Environment.TickCount - nmsBeginTicks;
	int nmsRemain = knmsMaxWait - nmsElapsed;
	if(nmsRemain < 0)
		nmsRemain = 0;

	//Then wait for thread to exit
	if(!thrd.Join(nmsRemain))
	{
		//It didn't exit in time, terminate it
		thrd.Abort();

		//Issue a debugger warning
		Debug.Assert(false, "Terminated thread");
	}
}

In my case, I could not instantiate my objects on the the thread pool with Task.Run() or Task.Factory.StartNew(). They would not synchronize my long running delegates correctly.

I needed the delegates to run asynchronously, pausing my main thread for their collective completion. The Thread.Join() would not work since I wanted to wait for collective completion in the middle of the parent thread, not at the end.

With the Task.Run() or Task.Factory.StartNew(), either all the child threads blocked each other or the parent thread would not be blocked, ... I couldn't figure out how to go with async delegates because of the re-serialization of the await syntax.

Here is my solution using Threads instead of Tasks:

using (EventWaitHandle wh = new EventWaitHandle(false, EventResetMode.ManualReset))
{
  int outdex = mediaServerMinConnections - 1;
  for (int i = 0; i < mediaServerMinConnections; i++)
  {
    new Thread(() =>
    {
      sshPool.Enqueue(new SshHandler());
      if (Interlocked.Decrement(ref outdex) < 1)
        wh.Set();
    }).Start();
  }
  wh.WaitOne();
}