How to await a list of tasks asynchronously using LINQ?
C#LinqAsync AwaitC# Problem Overview
I have a list of tasks that I created like this:
public async Task<IList<Foo>> GetFoosAndDoSomethingAsync()
{
var foos = await GetFoosAsync();
var tasks = foos.Select(async foo => await DoSomethingAsync(foo)).ToList();
...
}
By using .ToList()
, the tasks should all start. Now I want to await their completion and return the results.
This works in the above ...
block:
var list = new List<Foo>();
foreach (var task in tasks)
list.Add(await task);
return list;
It does what I want, but this seems rather clumsy. I'd much rather write something simpler like this:
return tasks.Select(async task => await task).ToList();
... but this doesn't compile. What am I missing? Or is it just not possible to express things this way?
C# Solutions
Solution 1 - C#
LINQ doesn't work perfectly with async
code, but you can do this:
var tasks = foos.Select(DoSomethingAsync).ToList();
await Task.WhenAll(tasks);
If your tasks all return the same type of value, then you can even do this:
var results = await Task.WhenAll(tasks);
which is quite nice. WhenAll
returns an array, so I believe your method can return the results directly:
return await Task.WhenAll(tasks);
Solution 2 - C#
To expand on Stephen's answer, I've created the following extension method to keep the fluent style of LINQ. You can then do
await someTasks.WhenAll()
namespace System.Linq
{
public static class IEnumerableExtensions
{
public static Task<T[]> WhenAll<T>(this IEnumerable<Task<T>> source)
{
return Task.WhenAll(source);
}
}
}
Solution 3 - C#
One issue with Task.WhenAll
is that it would create a parallelism. In most of the cases it might be even better, but sometimes you want to avoid it. For example, reading data in batches from DB and sending data to some remote web service. You don't want to load all the batches to the memory but hit the DB once the previous batch has been processed. So, you have to break the asynchronicity. Here is an example:
var events = Enumerable.Range(0, totalCount/ batchSize)
.Select(x => x*batchSize)
.Select(x => dbRepository.GetEventsBatch(x, batchSize).GetAwaiter().GetResult())
.SelectMany(x => x);
foreach (var carEvent in events)
{
}
Note .GetAwaiter().GetResult()
converting it to synchronous. DB would be hit lazily only once batchSize
of events have been processed.
Solution 4 - C#
Use Task.WaitAll
or Task.WhenAll
whichever is approriate.
Solution 5 - C#
Task.WhenAll should do the trick here.
Solution 6 - C#
Expanding on Stephen's answer it can also be expressed without .ToList()
as follows:
var tasks = foos.Select(aFoo => aFoo.DoSomething());
await Task.WhenAll(tasks).ConfigureAwait(true);
Background: In some scenarios calling .ToList()
can result in side effects executed at that time because the enumerable is then enumerated. If the enumerable is a call to a set of APIs or a set of queries, this may not be the desired behavior at that time. Without .ToList()
the enumerable will be enumerated when the task is await.
More specifically: With (Fluent) NHibernate you'd typically avoid using .ToList()
on queries as otherwise you may end up reading the entire result set. This may be way more data that you want.