How to await a list of tasks asynchronously using LINQ?

C#LinqAsync Await

C# Problem Overview


I have a list of tasks that I created like this:

public async Task<IList<Foo>> GetFoosAndDoSomethingAsync()
{
    var foos = await GetFoosAsync();

    var tasks = foos.Select(async foo => await DoSomethingAsync(foo)).ToList();

    ...
}

By using .ToList(), the tasks should all start. Now I want to await their completion and return the results.

This works in the above ... block:

var list = new List<Foo>();
foreach (var task in tasks)
    list.Add(await task);
return list;

It does what I want, but this seems rather clumsy. I'd much rather write something simpler like this:

return tasks.Select(async task => await task).ToList();

... but this doesn't compile. What am I missing? Or is it just not possible to express things this way?

C# Solutions


Solution 1 - C#

LINQ doesn't work perfectly with async code, but you can do this:

var tasks = foos.Select(DoSomethingAsync).ToList();
await Task.WhenAll(tasks);

If your tasks all return the same type of value, then you can even do this:

var results = await Task.WhenAll(tasks);

which is quite nice. WhenAll returns an array, so I believe your method can return the results directly:

return await Task.WhenAll(tasks);

Solution 2 - C#

To expand on Stephen's answer, I've created the following extension method to keep the fluent style of LINQ. You can then do

await someTasks.WhenAll()

namespace System.Linq
{
    public static class IEnumerableExtensions
    {
        public static Task<T[]> WhenAll<T>(this IEnumerable<Task<T>> source)
        {
            return Task.WhenAll(source);
        }
    }
}

Solution 3 - C#

One issue with Task.WhenAll is that it would create a parallelism. In most of the cases it might be even better, but sometimes you want to avoid it. For example, reading data in batches from DB and sending data to some remote web service. You don't want to load all the batches to the memory but hit the DB once the previous batch has been processed. So, you have to break the asynchronicity. Here is an example:

var events = Enumerable.Range(0, totalCount/ batchSize)
   .Select(x => x*batchSize)
   .Select(x => dbRepository.GetEventsBatch(x, batchSize).GetAwaiter().GetResult())
   .SelectMany(x => x);
foreach (var carEvent in events)
{
}

Note .GetAwaiter().GetResult() converting it to synchronous. DB would be hit lazily only once batchSize of events have been processed.

Solution 4 - C#

Use Task.WaitAll or Task.WhenAll whichever is approriate.

Solution 5 - C#

Task.WhenAll should do the trick here.

Solution 6 - C#

Expanding on Stephen's answer it can also be expressed without .ToList() as follows:

var tasks = foos.Select(aFoo => aFoo.DoSomething());
await Task.WhenAll(tasks).ConfigureAwait(true);

Background: In some scenarios calling .ToList() can result in side effects executed at that time because the enumerable is then enumerated. If the enumerable is a call to a set of APIs or a set of queries, this may not be the desired behavior at that time. Without .ToList() the enumerable will be enumerated when the task is await.

More specifically: With (Fluent) NHibernate you'd typically avoid using .ToList() on queries as otherwise you may end up reading the entire result set. This may be way more data that you want.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionMatt Johnson-PintView Question on Stackoverflow
Solution 1 - C#Stephen ClearyView Answer on Stackoverflow
Solution 2 - C#ClementView Answer on Stackoverflow
Solution 3 - C#Boris LipschitzView Answer on Stackoverflow
Solution 4 - C#L.BView Answer on Stackoverflow
Solution 5 - C#AmeenView Answer on Stackoverflow
Solution 6 - C#ManfredView Answer on Stackoverflow