How can I fetch an array of URLs with Promise.all?
JavascriptPromiseEs6 PromiseFetch ApiJavascript Problem Overview
If I have an array of urls:
var urls = ['1.txt', '2.txt', '3.txt']; // these text files contain "one", "two", "three", respectively.
And I want to build an object that looks like this:
var text = ['one', 'two', 'three'];
I’ve been trying to learn to do this with fetch
, which of course returns Promise
s.
Some things I’ve tried that don’t work:
var promises = urls.map(url => fetch(url));
var texts = [];
Promise.all(promises)
.then(results => {
results.forEach(result => result.text()).then(t => texts.push(t))
})
This doesn’t look right, and in any case it doesn’t work — I don’t end up with an array ['one', 'two', 'three'].
Is using Promise.all
the right approach here?
Javascript Solutions
Solution 1 - Javascript
Yes, Promise.all
is the right approach, but you actually need it twice if you want to first fetch
all urls and then get all text
s from them (which again are promises for the body of the response). So you'd need to do
Promise.all(urls.map(u=>fetch(u))).then(responses =>
Promise.all(responses.map(res => res.text()))
).then(texts => {
…
})
Your current code is not working because forEach
returns nothing (neither an array nor a promise).
Of course you can simplify that and start with getting the body from each response right after the respective fetch promise fulfilled:
Promise.all(urls.map(url =>
fetch(url).then(resp => resp.text())
)).then(texts => {
…
})
or the same thing with await
:
const texts = await Promise.all(urls.map(async url => {
const resp = await fetch(url);
return resp.text();
}));
Solution 2 - Javascript
For some reason neither of Bergi's examples worked for me. It would simply give me empty results. After some debugging it seemes like the promise would return before the fetch had finished, hence the empty results.
However, Benjamin Gruenbaum had an answer here earlier, but deleted it. His method did work for me, so I'll just copy-paste it here, as an alternative in case anyone else runs into any problems with the first solution here.
var promises = urls.map(url => fetch(url).then(y => y.text()));
Promise.all(promises).then(results => {
// do something with results.
});
Solution 3 - Javascript
You should use map
instead of forEach
:
Promise.all(urls.map(url => fetch(url)))
.then(resp => Promise.all( resp.map(r => r.text()) ))
.then(result => {
// ...
});
Solution 4 - Javascript
The suggested array urls = ['1.txt', '2.txt', '3.txt']
does not make much
sense to me, so I will instead use:
urls = ['https://jsonplaceholder.typicode.com/todos/2',
'https://jsonplaceholder.typicode.com/todos/3']
The JSONs of the two URLs:
{"userId":1,"id":2,"title":"quis ut nam facilis et officia qui",
"completed":false}
{"userId":1,"id":3,"title":"fugiat veniam minus","completed":false}
The goal is to get an array of objects, where each object contains the title
value from the corresponding URL.
To make it a little more interesting, I will assume that there is already an array of names that I want the array of URL results (the titles) to be merged with:
namesonly = ['two', 'three']
The desired output is an array of objects:
[{"name":"two","loremipsum":"quis ut nam facilis et officia qui"},{"name":"three","loremipsum":"fugiat veniam minus"}]
where I have changed the attribute name title
to loremipsum
.
const namesonly = ['two', 'three']; const urls = ['https://jsonplaceholder.typicode.com/todos/2', 'https://jsonplaceholder.typicode.com/todos/3'];
Promise.all(urls.map(url => fetch(url)
.then(response => response.json())
.then(responseBody => responseBody.title)))
.then(titles => {
const names = namesonly.map(value => ({ name: value }));
console.log('names: ' + JSON.stringify(names));
const fakeLatins = titles.map(value => ({ loremipsum: value }));
console.log('fakeLatins:\n' + JSON.stringify(fakeLatins));
const result =
names.map((item, i) => Object.assign({}, item, fakeLatins[i]));
console.log('result:\n' + JSON.stringify(result));
})
.catch(err => {
console.error('Failed to fetch one or more of these URLs:');
console.log(urls);
console.error(err);
});
.as-console-wrapper { max-height: 100% !important; top: 0; }
Reference
Solution 5 - Javascript
In case, if you are using axios. We can achieve this like:
const apiCall = (endpoint:string)=> axios.get(${baseUrl}/${endpoint}
)
axios.all([apiCall('https://first-endpoint'),apiCall('https://second-endpoint')]).then(response => {
response.forEach(values => values)
}).catch(error => {})
Solution 6 - Javascript
Here is a clean way to do it.
const requests = urls.map((url) => fetch(url));
const responses = await Promise.all(requests);
const promises = responses.map((response) => response.text());
return await Promise.all(promises);