node.js async libs

There are a ton of libraries that help with fixing the layers of callback syndrome.

In fact, there's too many, which one do i use?

I use Async.js.

> Async is a utility module which > provides straight-forward, powerful > functions for working with > asynchronous JavaScript. Although > originally designed for use with > node.js, it can also be used directly > in the browser.

Examples

async.map(['file1','file2','file3'], fs.stat, function(err, results){
    // results is now an array of stats for each file
});

async.filter(['file1','file2','file3'], path.exists, function(results){
    // results now equals an array of the existing files
});

async.parallel([
    function(){ ... },
    function(){ ... }
], callback);

async.series([
    function(){ ... },
    function(){ ... }
]);

pick one you like :P. I like async for example. But Step is also pretty famous. I think it is a very good thing that there are so many modules. The node.js community is putting out some really good modules. Installing them with NPM will not cost you any effort at all.

I like to use promises from Q:

If a function cannot return a value or throw an exception without blocking, it can return a promise instead. A promise is an object that represents the return value or the thrown exception that the function may eventually provide. A promise can also be used as a proxy for a remote object to overcome latency.

On the first pass, promises can mitigate the “Pyramid of Doom”: the situation where code marches to the right faster than it marches forward.

step1(function (value1) {
    step2(value1, function(value2) {
        step3(value2, function(value3) {
            step4(value3, function(value4) {
                // Do something with value4
            });
        });
    });
});

With a promise library, you can flatten the pyramid.

Q.fcall(step1)
.then(step2)
.then(step3)
.then(step4)
.then(function (value4) {
    // Do something with value4
}, function (error) {
    // Handle any error from step1 through step4
})
.done();

With this approach, you also get implicit error propagation, just like try, catch, and finally. An error in step1 will flow all the way to step5, where it’s caught and handled.

The callback approach is called an “inversion of control”. A function that accepts a callback instead of a return value is saying, “Don’t call me, I’ll call you.”. Promises un-invert the inversion, cleanly separating the input arguments from control flow arguments. This simplifies the use and creation of API’s, particularly variadic, rest and spread arguments.

The problem I have with other sync libraries is they often required me to define all my tasks at the start, and did not offer a clean intuitive API I found. I simply just wanted to push tasks to a group, whenever and whereever, and then execute that group in either parallel or serial fashion.

I love the flow functionality inside TaskGroup, and used it for a lot of big projects including DocPad and BugHerd. Examples are in the README.