Are nested promises normal in node.js?

Javascript Problem Overview

The problem I have been struggling with for two weeks now while learning node.js is how to do synchronous programming using node. I found that no matter how I try to do things sequentially I always end up with nested promises. I have found that there are modules such as Q to help with promise chaining as far as maintainability.

What I don't understand while doing research is Promise.all(), Promise.resolve() and Promise.reject(). Promise.reject is pretty much self explanatory by the name but when writing an application I am confused on how to include any of these in functions or objects without breaking the behavior of the application.

There is definitely a learning curve to node.js when coming from a programming language such as Java or C#. The question that still resides is if promise chaining is normal (best practice) in node.js.

Example:

driver.get('https://website.com/login').then(function () {
    loginPage.login('company.admin', 'password').then(function () {
        var employeePage = new EmployeePage(driver.getDriver());

        employeePage.clickAddEmployee().then(function() {
            setTimeout(function() {
                var addEmployeeForm = new AddEmployeeForm(driver.getDriver());

                addEmployeeForm.insertUserName(employee.username).then(function() {
                    addEmployeeForm.insertFirstName(employee.firstName).then(function() {
                        addEmployeeForm.insertLastName(employee.lastName).then(function() {
                            addEmployeeForm.clickCreateEmployee().then(function() {
                                employeePage.searchEmployee(employee);
                            });
                        });
                    });
                });
            }, 750);
        });
    });
});

Javascript Solutions

Solution 1 - Javascript

No, one of the great advantages of Promises is that you you can keep your async code linear rather than nested (callback hell from continuation passing style).

Promises give you return statements and error throwing, which you lose with continuation passing style.

You need to return the promise from your async functions so you can chain on the returned value.

Here's an example:

driver.get('https://website.com/login')
  .then(function() {
    return loginPage.login('company.admin', 'password')
  })
  .then(function() {
    var employeePage = new EmployeePage(driver.getDriver());
    return employeePage.clickAddEmployee();
  })
  .then(function() {
    setTimeout(function() {
      var addEmployeeForm = new AddEmployeeForm(driver.getDriver());

      addEmployeeForm.insertUserName(employee.username)
        .then(function() {
          return addEmployeeForm.insertFirstName(employee.firstName)
        })
        .then(function() {
          return addEmployeeForm.insertLastName(employee.lastName)
        })
        .then(function() {
          return addEmployeeForm.clickCreateEmployee()
        })
        .then(function() {
          return employeePage.searchEmployee(employee)
        });
    }, 750);
});

Promise.all takes an array of promises and resolves once all promises resolve, if any are rejected, the array is rejected. This allows you to execute async code concurrently rather than serially, and still wait for the result of all concurrent functions. If you're comfortable with a threaded model, think spawning threads and then joining.

Example:

addEmployeeForm.insertUserName(employee.username)
    .then(function() {
        // these two functions will be invoked immediately and resolve concurrently
        return Promise.all([
            addEmployeeForm.insertFirstName(employee.firstName),
            addEmployeeForm.insertLastName(employee.lastName)
        ])
    })
    // this will be invoked after both insertFirstName and insertLastName have succeeded
    .then(function() {
        return addEmployeeForm.clickCreateEmployee()
    })
    .then(function() {
        return employeePage.searchEmployee(employee)
    })
    // if an error arises anywhere in the chain this function will be invoked
    .catch(function(err){
        console.log(err)
    });

Promise.resolve() and Promise.reject() are methods used when creating a Promise. They're used to wrap an async function using callbacks so that you can work with Promises instead of callbacks.

Resolve will resolve/fulfill the promise (this means a chained then method will be called with the resulting value).
Reject will reject the promise (this means any chained then method(s) will not be called, but the first chained catch method will be called with the error that arose).

I left your setTimeout in place to preserve your programs behavior, but it's likely unnecessary.

Solution 2 - Javascript

Use async library and use async.series instead of nested chainings which looks really ugly and hard to debug/understand.

async.series([
    methodOne,
    methodTwo
], function (err, results) {
    // Here, results is the value from each function
    console.log(results);
});

The Promise.all(iterable) method returns a promise that resolves when all of the promises in the iterable argument have resolved, or rejects with the reason of the first passed promise that rejects.

var p1 = Promise.resolve(3);
var p2 = 1337;
var p3 = new Promise(function(resolve, reject) {
  setTimeout(resolve, 100, "foo");
}); 

Promise.all([p1, p2, p3]).then(function(values) { 
  console.log(values); // [3, 1337, "foo"] 
});

The Promise.resolve(value) method returns a Promise object that is resolved with the given value. If the value is a thenable (i.e. has a then method), the returned promise will "follow" that thenable, adopting its eventual state; otherwise the returned promise will be fulfilled with the value.

var p = Promise.resolve([1,2,3]);
p.then(function(v) {
  console.log(v[0]); // 1
});

https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

Solution 3 - Javascript

I just answered a similar question where I explained a technique that uses generators to flatten Promise chains in a nice way. The technique gets its inspiration from coroutines.

Take this bit of code

Promise.prototype.bind = Promise.prototype.then;

const coro = g => {
  const next = x => {
    let {done, value} = g.next(x);
    return done ? value : value.bind(next);
  }
  return next();
};

Using it, you can transform your deeply-nested Promise chain into this

coro(function* () {
  yield driver.get('https://website.com/login')
  yield loginPage.login('company.admin', 'password');
  var employeePage = new EmployeePage(driver.getDriver());
  yield employeePage.clickAddEmployee();
  setTimeout(() => {
    coro(function* () {
      var addEmployeeForm = new AddEmployeeForm(driver.getDriver());
      yield addEmployeeForm.insertUserName(employee.username);
      yield addEmployeeForm.insertFirstName(employee.firstName);
      yield addEmployeeForm.insertLastName(employee.lastName);
      yield addEmployeeForm.clickCreateEmployee();
      yield employeePage.searchEmployee(employee);
    }());
  }, 750);
}());

Using named generators, we can make that even more legible

// don't forget to assign your free variables
// var driver = ...
// var loginPage = ...
// var employeePage = new EmployeePage(driver.getDriver());
// var addEmployeeForm = new AddEmployeeForm(driver.getDriver());
// var employee = ...

function* createEmployee () {
  yield addEmployeeForm.insertUserName(employee.username);
  yield addEmployeeForm.insertFirstName(employee.firstName);
  yield addEmployeeForm.insertLastName(employee.lastName);
  yield addEmployeeForm.clickCreateEmployee();
  yield employeePage.searchEmployee(employee);
}

function* login () {
  yield driver.get('https://website.com/login')
  yield loginPage.login('company.admin', 'password');
  yield employeePage.clickAddEmployee();
  setTimeout(() => coro(createEmployee()), 750);
}

coro(login());

However, this only scratches the surface of what's possible using coroutines to control the flow of promises. Read the answer I linked above that demonstrates some of the other advantages and capabilities of this technique.

If you do intend to use coroutines for this purpose, I encourage you to check out the co library.

Hope this helps.

PS not sure why you're using setTimeout in this fashion. What is the point of waiting for 750 ms specifically ?

Solution 4 - Javascript

I removed the unnecessary nesting. Ill use syntax from 'bluebird'(my preferred Promise library) http://bluebirdjs.com/docs/api-reference.html

var employeePage;

driver.get('https://website.com/login').then(function() {
    return loginPage.login('company.admin', 'password');
}).then(function() {
    employeePage = new EmployeePage(driver.getDriver());    
    return employeePage.clickAddEmployee();
}).then(function () {
    var deferred = Promise.pending();
    setTimeout(deferred.resolve,750);
    return deferred.promise;
}).then(function() {
    var addEmployeeForm = new AddEmployeeForm(driver.getDriver());
    return Promise.all([addEmployeeForm.insertUserName(employee.username),
                        addEmployeeForm.insertFirstName(employee.firstName),
                        addEmployeeForm.insertLastName(employee.lastName)]);
}).then(function() {
    return addEmployeeForm.clickCreateEmployee();
}).then(function() {
    return employeePage.searchEmployee(employee);
}).catch(console.log);

I modified your code to include examples for all you questions.

There is no need to use the async library when working with promises. Promises are a very powerful by themselves and I think its an anti-pattern to mix promises and libraries like async.
Normally you should avoid using the var deferred = Promise.pending() style...unless

> 'when wrapping a callback API that doesn't follow the standard > convention. Like setTimeout:'

https://github.com/petkaantonov/bluebird/wiki/Promise-anti-patterns

For the setTimeout example..create a 'deferred' promise...resolve the promise inside setTimeout and then return the promise outside setTimeout. This might seem a little unintuitive. Look at this example, I answered another question. https://stackoverflow.com/questions/35646401/q-js-promise-with-node-missing-error-handler-on-socket-typeerror-cannot-cal/35647151#35647151

Normally, you can get away with using Promise.promisify(someFunction) to convert a callback type function into a Promise returning function.

Promise.all Lets say your are making multiple calls to an service that return asynchronously. If they don't depend on each other, you can make the calls simultaneously.

Just pass the function calls as an array. Promise.all([promiseReturningCall1, promiseReturningCall2, promiseReturningCall3]);

Finally add a catch block to the very end..to make sure you catch any error. This will catch any exception anywhere in the chain.

Solution 5 - Javascript

Since this post is a top result for "nested promises" on Google, and having struggled with promises in my early days of learning node.js from a C# background, I thought I'd post something that would help others making a similar transition/evolution.

The voted-up answer by Tate is totally correct in that it does force a sequence, but the issue for most .NET or Java developers is that we're just not used to so many things being async operations in a synchronous language. You have to be super-aware of what's async, because outer blocks continue & complete before any async action would.

To illustrate, here's some code (complete with nesting & two bugs!) I struggled with while learning promises with 'pg-promise':

            exports.create = async function createMeet(thingJson, res, next) {
    let conn;
    if (helpers.isDate(helpers.isStringDate(thingJson.ThingDate))){
        db.connect()
            .then(obj => {
                conn = obj;
                conn.proc('spCreateThing',[
                    thingJson.ThingName,
                    thingJson.ThingDescription,
                    thingJson.ThingDate])
                    .then(data => {
                        res.status(201).json(data);
                        res.send();
                    })
                    .catch(error =>{
                        console.error("Error creating a Thing via spCreateThing(" + thingJson + "): " + error);
                        next(createError(500, "Failed to create a Thing!"));
                    })
                    .finally(()  => {
                        conn.done(); //appropriate time to close the connection
                    });
                })
            .catch(error =>{
                console.error("Error establishing postgres database connection: " + error);
                next(createError(500, "Error establishing postgres database connection: " + error));
            })
            .finally(()  => { //this finally block will execute before the async actions fired in first .then() complete/start
                    conn.done(); //so this would close the connection before conn.proc() has completed/started
            });
        res.send(); //this will execute immediately following .connect() BEFORE any of the chained promise results,
        // thus sending a response before we've even figured out if the connection was successful and started the proc 
    } else {
        console.error("Attempt to create a Thing without valid date: " + thingJson.ThingDate);
        next(createError(400, "Must specify a valid date: " + thingJson.ThingDate));
    }

On top of that, the code that calls this function (i.e. a route handler) will complete before the db connection process even starts.

So, the net of it is that outer functions define the promise structure and initiate the async calls, but then immediately complete their block since JS is first a synchronous language; so be aware and assume all async calls don't even start until after the block that called it is complete.

I know this is obvious to career JS developers (and is to me now), but I hope this really helps others new to these concepts.

Solution 6 - Javascript

Your next step is to go from nesting to chaining. You need to realize that each promise is an isolated promise that can be chained in a parent promise. In other words, you can flatten the promises to a chain. Each promise result can be passed to the next one.

Here is a great blog post about it: Flattening Promise Chains. It uses Angular but you can ignore that and look at how a deep nesting of promises turns into a chain.

Another good answer is right here on StackOverflow: Understanding javascript promises; stacks and chaining.

Solution 7 - Javascript

You can chain promises like this:

driver.get('https://website.com/login').then(function () {
    return loginPage.login('company.admin', 'password')
)}.then(function () {
    var employeePage = new EmployeePage(driver.getDriver());

    return employeePage.clickAddEmployee().then(function() {
        setTimeout(function() {
            var addEmployeeForm = new AddEmployeeForm(driver.getDriver());
        return addEmployeeForm.insertUserName(employee.username).then(function() {
                retun addEmployeeForm.insertFirstName(employee.firstName)
         }).then(function() {
                return addEmployeeForm.insertLastName(employee.lastName)
         }).then(function() {
             return addEmployeeForm.clickCreateEmployee()
         }).then(function () {
             retrun employeePage.searchEmployee(employee);
        })}, 750);
        });
    });
});

Solution 8 - Javascript

Yeah like @TateThurston said, we chain them. It's even more aesthetically pleasing when you use es6 arrow functions 