Using NodeJS for a big project

Javascriptnode.js

Javascript Problem Overview


Is NodeJS a good framework/codebase for a large server-side application? What I am looking to develop is a large application that will require HTTP transactions (states) and large amounts of concurrent users.

From what I've read online, NodeJS is not the best tool to use when it comes to large programs. What I've come across is as follows:

  • NodeJS runs on JavaScript which runs on event loops which are not very efficient when used in bulk.
  • NodeJS may be non-blocking, but all the requests are handled within a single thread so this can cause a bit of a bottleneck when many requests are handled.
  • NodeJS is built atop its own HTTP server so future maintenance will require its own sysadmin/developer hybrid to take care of the application.
  • There isn't as much well-tested and diverse software available for NodeJS that helps you build a bigger application.

Is there something I'm missing? Is NodeJS really as powerful as it can be?

Javascript Solutions


Solution 1 - Javascript

> Is NodeJS a good framework/codebase for a large server-side application?

That question is a bit subjective but I'm including actual objective points which solve real problems when working with node in a large project.

Update after working on project for awhile:

It is best as a front end / API server which is I/O bound (most front end/api servers are). If you have backend compute needs (processing etc...) it can be paired which other technologies (C# net core, go, Java etc... worker nodes)

I created this project as a sample illustrating most points - Sane Node Development: https://github.com/bryanmacfarlane/sanenode

NodeJS is not built atop its own http server. It's built atop the V8 chrome javascript engine and doesn't assume an http server. There is a built in http module as well as the popular express web server but there's also socket modules (as well as socket.io). It's not just an http server.

The single thread does not cause a bottleneck because all I/O is evented and asynchronous. This link explains it well: http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/

As far as the software module, you can search at the npm registry. Always look at how many other folks use it (downloads) and visit the github repo to see if it's actively being maintained (or is there a bunch of issue never getting attention).

Regarding "large project" what I've found critical for sane development is:

  1. Compile time support (and intellisense): Find issues when you compile. If you didn't think you needed this like I did when I started, you will change your mind after that first big refactor.

  2. Eliminate Callback Hell: Keep performance which is critical (noted above) but eliminate callback code. Use async / await to write linear code and keep async perf. Integrates with promises but much better than solely using promises.

  3. Tooling: Lots of options but I've found best is Typescript (ES6/7 today), VS Code (intellisense), Mocha (unit testing).

  4. Instrumentation/Logging: Getting insights into your app with tracing and instrumentation is critical.

  5. Build on well vetted frameworks: I use express as an example but that's a preference and there's others.

Solution 2 - Javascript

Node.js is a very good tool to build distributed network services. What is your large scale application design is more than a question 'which tools to use'. A lot of people use node.js in a very heterogeneous way together with ruby, php, erlang, apache & nginx & HAproxy. If you don't know why you need node you probably don't need it. Possible reasons to consider node:

  • you want to share common Javascript code between server and client

  • you expect highly concurrent load (thousands to hundreds of thousands simultaneous connections per server)

  • you (or your team) is much more proficient with JavaScript than with any other available language/framework

  • if one of 7600+ modules is implementing large portion of required functionality

Solution 3 - Javascript

One "issue" with NodeJS, is that building a large application requires discipline on the part of the developer / team.

This is particularly true with multiple teams within the same company. Existing frameworks are a bit loose, and different teams will come up with different approaches to solving an issue.

KrakenJS is a framework, built on top of express. It adds a layer of convention and configuration that should make it easy(er) to build large projects, involving multiple teams.

Solution 4 - Javascript

Really NodeJs is powerful in its own way, Some more information,

  1. You can run multiple instance of your app under load balance to handle massive request.
  2. Choose NodeJs to read 2000 files instead calculating 20th prime number.
  3. Put NodeJs busy with reading/writing in files or ports.
  4. Very useful when you need to broadcast your response to multiple client.
  5. Don't care about dead lock in NodeJs, But care about how frequent you are doing same operation.
  6. Most important thing is, the values live in V8 engine until the process is terminated. Be sure how much lines of code, you are going to feed in NodeJs.

Solution 5 - Javascript

I find the most important thing is to use CPU time as least as possible. If your application needs to use CPU intensively, event loop latency would increase and the application would fail to respond any other requests.

Solution 6 - Javascript

So far as I have learned, It's the raw speed and async await that makes the biggest difference. For those who are moderately skilled in Javascript and particularly understand REST as well as the above mentioned aspects, node.js is one of the best solutions for big enterprise applications. Correct me if I am wrong, but even using raw express (without those frameworks built on top of it ) is actually good enough if teams agree on certain conventions and standards to follow.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionmatskoView Question on Stackoverflow
Solution 1 - JavascriptbryanmacView Answer on Stackoverflow
Solution 2 - JavascriptAndrey SidorovView Answer on Stackoverflow
Solution 3 - JavascriptLenny MarkusView Answer on Stackoverflow
Solution 4 - JavascriptHILARUDEEN S ALLAUDEENView Answer on Stackoverflow
Solution 5 - JavascriptDeniz OzgerView Answer on Stackoverflow
Solution 6 - Javascriptkk_hackerView Answer on Stackoverflow