Awaiting multiple Tasks with different results

C#.Net Async Await Task Parallel-Library .Net 4.5

C# Problem Overview

I have 3 tasks:

private async Task<Cat> FeedCat() {}
private async Task<House> SellHouse() {}
private async Task<Tesla> BuyCar() {}

They all need to run before my code can continue and I need the results from each as well. None of the results have anything in common with each other

How do I call and await for the 3 tasks to complete and then get the results?

C# Solutions

Solution 1 - C#

After you use WhenAll, you can pull the results out individually with await:

var catTask = FeedCat();
var houseTask = SellHouse();
var carTask = BuyCar();

await Task.WhenAll(catTask, houseTask, carTask);

var cat = await catTask;
var house = await houseTask;
var car = await carTask;

You can also use Task.Result (since you know by this point they have all completed successfully). However, I recommend using await because it's clearly correct, while Result can cause problems in other scenarios.

Solution 2 - C#

Just await the three tasks separately, after starting them all.

var catTask = FeedCat();
var houseTask = SellHouse();
var carTask = BuyCar();

var cat = await catTask;
var house = await houseTask;
var car = await carTask;

Solution 3 - C#

If you're using C# 7, you can use a handy wrapper method like this...

public static class TaskEx
{
    public static async Task<(T1, T2)> WhenAll<T1, T2>(Task<T1> task1, Task<T2> task2)
    {
        return (await task1, await task2);
    }
}

...to enable convenient syntax like this when you want to wait on multiple tasks with different return types. You'd have to make multiple overloads for different numbers of tasks to await, of course.

var (someInt, someString) = await TaskEx.WhenAll(GetIntAsync(), GetStringAsync());

However, see Marc Gravell's answer for some optimizations around ValueTask and already-completed tasks if you intend to turn this example into something real.

Solution 4 - C#

Given three tasks - FeedCat(), SellHouse() and BuyCar(), there are two interesting cases: either they all complete synchronously (for some reason, perhaps caching or an error), or they don't.

Let's say we have, from the question:

Task<string> DoTheThings() {
    Task<Cat> x = FeedCat();
    Task<House> y = SellHouse();
    Task<Tesla> z = BuyCar();
    // what here?
}

Now, a simple approach would be:

Task.WhenAll(x, y, z);

but ... that isn't convenient for processing the results; we'd typically want to await that:

async Task<string> DoTheThings() {
    Task<Cat> x = FeedCat();
    Task<House> y = SellHouse();
    Task<Tesla> z = BuyCar();

    await Task.WhenAll(x, y, z);
    // presumably we want to do something with the results...
    return DoWhatever(x.Result, y.Result, z.Result);
}

but this does lots of overhead and allocates various arrays (including the params Task[] array) and lists (internally). It works, but it isn't great IMO. In many ways it is simpler to use an async operation and just await each in turn:

async Task<string> DoTheThings() {
    Task<Cat> x = FeedCat();
    Task<House> y = SellHouse();
    Task<Tesla> z = BuyCar();
    
    // do something with the results...
    return DoWhatever(await x, await y, await z);
}

Contrary to some of the comments above, using await instead of Task.WhenAll makes no difference to how the tasks run (concurrently, sequentially, etc). At the highest level, Task.WhenAll predates good compiler support for async/await, and was useful when those things didn't exist. It is also useful when you have an arbitrary array of tasks, rather than 3 discreet tasks.

But: we still have the problem that async/await generates a lot of compiler noise for the continuation. If it is likely that the tasks might actually complete synchronously, then we can optimize this by building in a synchronous path with an asynchronous fallback:

Task<string> DoTheThings() {
    Task<Cat> x = FeedCat();
    Task<House> y = SellHouse();
    Task<Tesla> z = BuyCar();

    if(x.Status == TaskStatus.RanToCompletion &&
       y.Status == TaskStatus.RanToCompletion &&
       z.Status == TaskStatus.RanToCompletion)
        return Task.FromResult(
          DoWhatever(a.Result, b.Result, c.Result));
       // we can safely access .Result, as they are known
       // to be ran-to-completion

    return Awaited(x, y, z);
}

async Task Awaited(Task<Cat> a, Task<House> b, Task<Tesla> c) {
    return DoWhatever(await x, await y, await z);
}

This "sync path with async fallback" approach is increasingly common especially in high performance code where synchronous completions are relatively frequent. Note it won't help at all if the completion is always genuinely asynchronous.

Additional things that apply here:

with recent C#, a common pattern is for the async fallback method is commonly implemented as a local function:

 Task<string> DoTheThings() {
     async Task<string> Awaited(Task<Cat> a, Task<House> b, Task<Tesla> c) {
         return DoWhatever(await a, await b, await c);
     }
     Task<Cat> x = FeedCat();
     Task<House> y = SellHouse();
     Task<Tesla> z = BuyCar();
 
     if(x.Status == TaskStatus.RanToCompletion &&
        y.Status == TaskStatus.RanToCompletion &&
        z.Status == TaskStatus.RanToCompletion)
         return Task.FromResult(
           DoWhatever(a.Result, b.Result, c.Result));
        // we can safely access .Result, as they are known
        // to be ran-to-completion
 
     return Awaited(x, y, z);
 }

prefer ValueTask<T> to Task<T> if there is a good chance of things ever completely synchronously with many different return values:

 ValueTask<string> DoTheThings() {
     async ValueTask<string> Awaited(ValueTask<Cat> a, Task<House> b, Task<Tesla> c) {
         return DoWhatever(await a, await b, await c);
     }
     ValueTask<Cat> x = FeedCat();
     ValueTask<House> y = SellHouse();
     ValueTask<Tesla> z = BuyCar();
 
     if(x.IsCompletedSuccessfully &&
        y.IsCompletedSuccessfully &&
        z.IsCompletedSuccessfully)
         return new ValueTask<string>(
           DoWhatever(a.Result, b.Result, c.Result));
        // we can safely access .Result, as they are known
        // to be ran-to-completion
 
     return Awaited(x, y, z);
 }

if possible, prefer IsCompletedSuccessfully to Status == TaskStatus.RanToCompletion; this now exists in .NET Core for Task, and everywhere for ValueTask<T>

Solution 5 - C#

You can store them in tasks, then await them all:

var catTask = FeedCat();
var houseTask = SellHouse();
var carTask = BuyCar();

await Task.WhenAll(catTask, houseTask, carTask);

Cat cat = await catTask;
House house = await houseTask;
Car car = await carTask;

Solution 6 - C#

In case you are trying to log all errors make sure you keep Task.WhenAll line in your code, lot of comments suggest that you can remove it and wait for individual tasks. Task.WhenAll is really important for error handling. Without this line you potentially leaving your code open for unobserved exceptions.

var catTask = FeedCat();
var houseTask = SellHouse();
var carTask = BuyCar();

await Task.WhenAll(catTask, houseTask, carTask);

var cat = await catTask;
var house = await houseTask;
var car = await carTask;

Imagine FeedCat throws exception in the following code:

var catTask = FeedCat();
var houseTask = SellHouse();
var carTask = BuyCar();

var cat = await catTask;
var house = await houseTask;
var car = await carTask;

In that case you will never await on houseTask nor carTask. There are 3 possible scenarios here:

SellHouse is already completed successfully when FeedCat failed. In this case you are fine.
SellHouse is not complete and fails with exception at some point. Exception is not observed and will be rethrown on finalizer thread.
SellHouse is not complete and contains awaits inside it. In case your code runs in ASP.NET SellHouse will fail as soon as some of the awaits will completed inside it. This happens because you basically made fire & forget call and synchronization context was lost as soon as FeedCat failed.

Here is error that you will get for case (3):

System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Web.ThreadContext.AssociateWithCurrentThread(Boolean setImpersonationContext)
   at System.Web.HttpApplication.OnThreadEnterPrivate(Boolean setImpersonationContext)
   at System.Web.HttpApplication.System.Web.Util.ISyncContext.Enter()
   at System.Web.Util.SynchronizationHelper.SafeWrapCallback(Action action)
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Web.ThreadContext.AssociateWithCurrentThread(Boolean setImpersonationContext)
   at System.Web.HttpApplication.OnThreadEnterPrivate(Boolean setImpersonationContext)
   at System.Web.HttpApplication.System.Web.Util.ISyncContext.Enter()
   at System.Web.Util.SynchronizationHelper.SafeWrapCallback(Action action)
   at System.Threading.Tasks.Task.Execute()<---

For case (2) you will get similar error but with original exception stack trace.

For .NET 4.0 and later you can catch unobserved exceptions using TaskScheduler.UnobservedTaskException. For .NET 4.5 and later unobserved exceptions are swallowed by default for .NET 4.0 unobserved exception will crash your process.

More details here: Task Exception Handling in .NET 4.5

Solution 7 - C#

You can use Task.WhenAll as mentioned, or Task.WaitAll, depending on whether you want the thread to wait. Take a look at the link for an explanation of both.

https://stackoverflow.com/questions/6123406/waitall-vs-whenall

Solution 8 - C#

Forward Warning

Just a quick headsup to those visiting this and other similar threads looking for a way to parallelize EntityFramework using async+await+task tool-set: The pattern shown here is sound, however, when it comes to the special snowflake of EF you will not achieve parallel execution unless and until you use a separate (new) db-context-instance inside each and every *Async() call involved.

This sort of thing is necessary due to inherent design limitations of ef-db-contexts which forbid running multiple queries in parallel in the same ef-db-context instance.

Capitalizing on the answers already given, this is the way to make sure that you collect all values even in the case that one or more of the tasks results in an exception:

  public async Task<string> Foobar() {
	async Task<string> Awaited(Task<Cat> a, Task<House> b, Task<Tesla> c) {
		return DoSomething(await a, await b, await c);
	}
  
    using (var carTask = BuyCarAsync())
    using (var catTask = FeedCatAsync())
    using (var houseTask = SellHouseAsync())
    {
		if (carTask.Status == TaskStatus.RanToCompletion //triple
			&& catTask.Status == TaskStatus.RanToCompletion //cache
			&& houseTask.Status == TaskStatus.RanToCompletion) { //hits
			return Task.FromResult(DoSomething(catTask.Result, carTask.Result, houseTask.Result)); //fast-track
		}

		cat = await catTask;
        car = await carTask;
		house = await houseTask;
		//or Task.AwaitAll(carTask, catTask, houseTask);
		//or await Task.WhenAll(carTask, catTask, houseTask);
        //it depends on how you like exception handling better
			
		return Awaited(catTask, carTask, houseTask);
   }
 }

An alternative implementation that has more or less the same performance characteristics could be:

 public async Task<string> Foobar() {
    using (var carTask = BuyCarAsync())
    using (var catTask = FeedCatAsync())
    using (var houseTask = SellHouseAsync())
    {
		cat = catTask.Status == TaskStatus.RanToCompletion ? catTask.Result : (await catTask);
        car = carTask.Status == TaskStatus.RanToCompletion ? carTask.Result : (await carTask);
		house = houseTask.Status == TaskStatus.RanToCompletion ? houseTask.Result : (await houseTask);
		
        return DoSomething(cat, car, house);
     }
 }

Solution 9 - C#

Use Task.WhenAll and then await the results:

var tCat = FeedCat();
var tHouse = SellHouse();
var tCar = BuyCar();
await Task.WhenAll(tCat, tHouse, tCar);
Cat cat = await tCat;
House house = await tHouse;
Tesla car = await tCar; 
//as they have all definitely finished, you could also use Task.Value.

Solution 10 - C#

var dn = await Task.WhenAll<dynamic>(FeedCat(),SellHouse(),BuyCar());

if you want to access Cat, you do this:

var ct = (Cat)dn[0];

This is very simple to do and very useful to use, there is no need to go after a complex solution.

Solution 11 - C#

isnt the await statment making the code to run in sequential order? consider the following code

class Program
{
    static Stopwatch _stopwatch = new();

    static async Task Main(string[] args)
    {
        Console.WriteLine($"fire hot");
        _stopwatch.Start();
        var carTask = BuyCar();
        var catTask = FeedCat();
        var houseTask = SellHouse();
        await carTask;
        await catTask;
        await houseTask;
        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} done!");

        Console.WriteLine($"using await");
        _stopwatch.Restart();
        await BuyCar();
        await FeedCat();
        await SellHouse();            

        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} done!");
    }

    static async Task BuyCar()
    {
        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} buy car started");
        await Task.Delay(2000);
        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} buy car done");
    }

    static async Task FeedCat()
    {
        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} feed cat started");
        await Task.Delay(1000);
        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} feed cat done");
    }

    static async Task SellHouse()
    {
        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} sell house started");
        await Task.Delay(10);
        Console.WriteLine($"{_stopwatch.ElapsedMilliseconds} sell house done");
    }
}

fire hot
0 buy car started
3 feed cat started
4 sell house started
18 sell house done
1004 feed cat done
2013 buy car done
2014 done!
using await
0 buy car started
2012 buy car done
2012 feed cat started
3018 feed cat done
3018 sell house started
3033 sell house done
3034 done!

Content Type	Original Author	Original Content on Stackoverflow
Question	Ian Vink	View Question on Stackoverflow
Solution 1 - C#	Stephen Cleary	View Answer on Stackoverflow
Solution 2 - C#	Servy	View Answer on Stackoverflow
Solution 3 - C#	Joel Mueller	View Answer on Stackoverflow
Solution 4 - C#	Marc Gravell	View Answer on Stackoverflow
Solution 5 - C#	Reed Copsey	View Answer on Stackoverflow
Solution 6 - C#	samfromlv	View Answer on Stackoverflow
Solution 7 - C#	Christian Phillips	View Answer on Stackoverflow
Solution 8 - C#	XDS	View Answer on Stackoverflow
Solution 9 - C#	It'sNotALie.	View Answer on Stackoverflow
Solution 10 - C#	taurius litvinavicius	View Answer on Stackoverflow
Solution 11 - C#	Mathias Nohall	View Answer on Stackoverflow