Google Firestore - How to get several documents by multiple ids in one round-trip?

Javascriptnode.jsFirebaseGoogle Cloud-PlatformGoogle Cloud-Firestore

Javascript Problem Overview


I am wondering if it's possible to get multiple documents by a list of ids in one round trip (network call) to the Firestore database.

Javascript Solutions


Solution 1 - Javascript

if you're within Node:

https://github.com/googleapis/nodejs-firestore/blob/master/dev/src/index.ts#L978

/**
* Retrieves multiple documents from Firestore.
*
* @param {...DocumentReference} documents - The document references
* to receive.
* @returns {Promise<Array.<DocumentSnapshot>>} A Promise that
* contains an array with the resulting document snapshots.
*
* @example
* let documentRef1 = firestore.doc('col/doc1');
* let documentRef2 = firestore.doc('col/doc2');
*
* firestore.getAll(documentRef1, documentRef2).then(docs => {
*   console.log(`First document: ${JSON.stringify(docs[0])}`);
*   console.log(`Second document: ${JSON.stringify(docs[1])}`);
* });
*/

This is specifically for the server SDK

UPDATE: "Cloud Firestore [client-side sdk] Now Supports IN Queries!"

https://firebase.googleblog.com/2019/11/cloud-firestore-now-supports-in-queries.html

myCollection.where(firestore.FieldPath.documentId(), 'in', ["123","456","789"])

Solution 2 - Javascript

They have just announced this functionality, https://firebase.googleblog.com/2019/11/cloud-firestore-now-supports-in-queries.html .

Now you can use queries like, but mind that the input size can't be greater than 10.

userCollection.where('uid', 'in', ["1231","222","2131"])

Solution 3 - Javascript

In practise you would use firestore.getAll like this

async getUsers({userIds}) {
    const refs = userIds.map(id => this.firestore.doc(`users/${id}`))
    const users = await this.firestore.getAll(...refs)
    console.log(users.map(doc => doc.data()))
}

or with promise syntax

getUsers({userIds}) {
    const refs = userIds.map(id => this.firestore.doc(`users/${id}`))
    this.firestore.getAll(...refs).then(users => console.log(users.map(doc => doc.data())))
}

Solution 4 - Javascript

You could use a function like this:

function getById (path, ids) {
  return firestore.getAll(
    [].concat(ids).map(id => firestore.doc(`${path}/${id}`))
  )
}

It can be called with a single ID:

getById('collection', 'some_id')

or an array of IDs:

getById('collection', ['some_id', 'some_other_id'])

Solution 5 - Javascript

No, right now there is no way to batch multiple read requests using the Cloud Firestore SDK and therefore no way to guarantee that you can read all of the data at once.

However as Frank van Puffelen has said in the comments above this does not mean that fetching 3 documents will be 3x as slow as fetching one document. It is best to perform your own measurements before reaching a conclusion here.

Solution 6 - Javascript

If you are using flutter, you can do the following:

Firestore.instance.collection('your collection name').where(FieldPath.documentId, whereIn:[list containing multiple document IDs]).getDocuments();

This will return a Future containing List<DocumentSnapshot> which you can iterate as you feel fit.

Solution 7 - Javascript

With Firebase Version 9 (Dec, 2021 Update):

You can get multiple documents by multiple ids in one round-trip using "documentId()" and "in" with "where" clause:

import {
  query,
  collection,
  where,
  documentId,
  getDocs
} from "firebase/firestore";

const q = query(
  collection(db, "products"),
  where(documentId(), "in", 
    [
      "8AVJvG81kDtb9l6BwfCa", 
      "XOHS5e3KY9XOSV7YYMw2", 
      "Y2gkHe86tmR4nC5PTzAx"
    ]
  ),
);

const productsDocsSnap = await getDocs(q);

productsDocsSnap.forEach((doc) => {
  console.log(doc.data()); // "doc1", "doc2" and "doc3"
});

Solution 8 - Javascript

Surely the best way to do this is by implementing the actual query of Firestore in a Cloud Function? There would then only be a single round trip call from the client to Firebase, which seems to be what you're asking for.

You really want to be keeping all of your data access logic like this server side anyway.

Internally there will likely be the same number of calls to Firebase itself, but they would all be across Google's super-fast interconnects, rather than the external network, and combined with the pipelining which Frank van Puffelen has explained, you should get excellent performance from this approach.

Solution 9 - Javascript

You can perform an IN query with the document IDs (up to ten):

import {
    query,
    collection,
    where,
    getDocs,
    documentId,
} from 'firebase/firestore';

export async function fetchAccounts(
    ids: string[]
) {
    // use lodash _.chunk, for example
    const result = await Promise.all(
        chunk(ids, 10).map(async (chunkIds) => {
            const accounts = await getDocs(
                query(
                    collection(firestore, 'accounts'),
                    where(documentId(), 'in', chunkIds)
                ));
            return accounts.docs.filter(doc => doc.exists()).map(doc => doc.data());
        })
    );
    return result.flat(1);
}

Solution 10 - Javascript

For some who are stucked in same problem here is a sample code:

List<String> documentsIds = {your document ids};

FirebaseFirestore.getInstance().collection("collection_name")
.whereIn(FieldPath.documentId(), documentsIds).get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
			@Override
			public void onComplete(@NonNull Task<QuerySnapshot> task) {
				if (task.isSuccessful()) {
					 for (DocumentSnapshot document : Objects.requireNonNull(task.getResult())) {
						YourClass object = document.toObject(YourClass.class);
						// add to your custom list
					}   
				}
				
			}
		}).addOnFailureListener(new OnFailureListener() {
			@Override
			public void onFailure(@NonNull Exception e) {
				e.printStackTrace();
			}
		});

Solution 11 - Javascript

Here's how you would do something like this in Kotlin with the Android SDK.
May not necessarily be in one round trip, but it does effectively group the result and avoid many nested callbacks.

val userIds = listOf("123", "456")
val userTasks = userIds.map { firestore.document("users/${it!!}").get() }

Tasks.whenAllSuccess<DocumentSnapshot>(userTasks).addOnSuccessListener { documentList ->
    //Do what you need to with the document list
}

Note that fetching specific documents is much better than fetching all documents and filtering the result. This is because Firestore charges you for the query result set.

Solution 12 - Javascript

For the ones who want to do it using Angular, here is an example:

First some library imports are needed: (must be preinstalled)

import * as firebase from 'firebase/app'
import { AngularFirestore, AngularFirestoreCollection } from '@angular/fire/firestore'

Some configuration for the collection:

yourCollection: AngularFirestoreCollection;

constructor(
    private _db : AngularFirestore,
) { 
    // this is your firestore collection
    this.yourCollection = this._db.collection('collectionName');
}

Here is the method to do the query: ('products_id' is an Array of ids)

getProducts(products_ids) {
    var queryId = firebase.firestore.FieldPath.documentId();
    this.yourCollection.ref.where(queryId, 'in', products_ids).get()
        .then(({ docs }) => {
            console.log(docs.map(doc => doc.data()))
        })
}

Solution 13 - Javascript

I hope this helps you, it works for me.

getCartGoodsData(id) {

    const goodsIDs: string[] = [];

    return new Promise((resolve) => {
      this.fs.firestore.collection(`users/${id}/cart`).get()
        .then(querySnapshot => {
          querySnapshot.forEach(doc => {
            goodsIDs.push(doc.id);
          });

          const getDocs = goodsIDs.map((id: string) => {
            return this.fs.firestore.collection('goods').doc(id).get()
              .then((docData) => {
                return docData.data();
              });
          });

          Promise.all(getDocs).then((goods: Goods[]) => {
            resolve(goods);
          });
        });
    });
  }

Solution 14 - Javascript

This doesn't seem to be possible in Firestore at the moment. I don't understand why Alexander's answer is accepted, the solution he proposes just returns all the documents in the "users" collection.

Depending on what you need to do, you should look into duplicating the relevant data you need to display and only request a full document when needed.

Solution 15 - Javascript

Yes, it is possible. Sample in .NET SDK for Firestore:

/*List of document references, for example:
    FirestoreDb.Collection(ROOT_LEVEL_COLLECTION).Document(DOCUMENT_ID);*/
    List<DocumentReference> docRefList = YOUR_DOCUMENT_REFERENCE_LIST;
    
    // Required fields of documents, not necessary while fetching entire documents
    FieldMask fieldMask = new FieldMask(FIELD-1, FIELD-2, ...);
    
    // With field mask
    List<DocumentSnapshot> documentSnapshotsMasked = await FirestoreDb.GetAllSnapshotsAsync(docRefList, fieldMask);
    
    // Without field mask
    List<DocumentSnapshot>documentSnapshots = await FirestoreDb.GetAllSnapshotsAsync(docRefList);

Documentation in .NET:

  1. Get all snapshots

  2. Field mask

Solution 16 - Javascript

The best you can do is not use Promise.all as your client then must wait for .all the reads before proceeding.

Iterate the reads and let them resolve independently. On the client side, this probably boils down to the UI having several progress loader images resolve to values independently. However, this is better than freezing the whole client until .all the reads resolve.

Therefore, dump all the synchronous results to the view immediately, then let the asynchronous results come in as they resolve, individually. This may seem like petty distinction, but if your client has poor Internet connectivity (like I currently have at this coffee shop), freezing the whole client experience for several seconds will likely result in a 'this app sucks' experience.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionJoonView Question on Stackoverflow
Solution 1 - JavascriptNick FranceschinaView Answer on Stackoverflow
Solution 2 - JavascriptjeadonaraView Answer on Stackoverflow
Solution 3 - JavascriptSebastianView Answer on Stackoverflow
Solution 4 - JavascriptJP de la TorreView Answer on Stackoverflow
Solution 5 - JavascriptSam SternView Answer on Stackoverflow
Solution 6 - JavascriptMonisView Answer on Stackoverflow
Solution 7 - JavascriptKai - Kazuya ItoView Answer on Stackoverflow
Solution 8 - JavascriptChris WilsonView Answer on Stackoverflow
Solution 9 - JavascriptNacho ColomaView Answer on Stackoverflow
Solution 10 - JavascriptRehanView Answer on Stackoverflow
Solution 11 - JavascriptMarkymarkView Answer on Stackoverflow
Solution 12 - JavascriptsantillanixView Answer on Stackoverflow
Solution 13 - JavascriptMuhammad MabroukView Answer on Stackoverflow
Solution 14 - JavascriptHoreaView Answer on Stackoverflow
Solution 15 - JavascriptOm PalsanawalaView Answer on Stackoverflow
Solution 16 - JavascriptRonnie RoystonView Answer on Stackoverflow