How to deploy Node.js application with deep node_modules structure on Windows?

I've run into a curious issue - apparently some Node.js module have so deep folder hierarchies that Windows copy command (or PowerShell's Copy-Item which is what we're actually using) hits the infamous "path too long" error when path is over 250 chars long.

For example, this is a folder hierarchy that a single Node module can create:

node_modules\nodemailer\node_modules\simplesmtp\node_modules\
xoauth2\node_modules\request\node_modules\form-data\node_modules\
combined-stream\node_modules\delayed-stream\...

It seems insane but is a reality with Node modules.

We need to use copy-paste during deployment (we're not using a "clever" target platform like Heroku where Git deployment would be an option) and this is a serious limitation on Windows.

Isn't there a npm command or something that would compact the node_modules folder or maybe include only what's actually necessary at runtime? (Node modules usually contain test folders etc. which we don't need to deploy.) Any other ideas how to work around it? Not using Windows is unfortunately not an option :)

just to add to this... another thing that helped me was listing out all installed modules with npm ls.

which will give you a tree of modules and versions... from there it's pretty easy to identify which ones are duplicates... npm dedupe didn't do anything for me. I'm not sure if that's a bug or what (Node v 10.16)

So once you identify a duplicate module install it to the root node_module directory by using npm install [email protected] --save-dev. The version is important.

after that, I wiped out my node_modules directory and did a fresh npm install.

Short version

1. npm ls to get a list of all installed modules.
2. look through those modules and identify duplicate modules (version is important)
3. npm install module@version --save-dev to install those modules in the root node_modules directory and update package.json.
4. rmdir node_modules to delete the node_modules directory.
5. npm install to pull down a fresh copy of your dependencies.

Once I did that, everything was much cleaner.

I also recommend commenting your package.json file to show which ones were brought down to flatten the node_modules tree.

I don't think there's any great solution given your constraints, but here are some things that may help.

• Try using npm dedupe to optimize your directory hierarchy which may shorten some paths
• Use npm install --production to install without the development tools
• Take some of those deeply nested dependencies (just enough to avoid the problem, I suggest) and move them to the top-level node_modules directory. Just keep track of them so you know which are your true dependencies and which are workarounds for this problem.
• OR move some of those deep dependencies to the highest node_modules directory under your_project/node_modules/pkg_with_deep_deps that will allow them to have short enough paths but still work. So this would be your_project/node_modules/pkg_with_deep_deps/node_modules.
  • I think require should be able to find those properly at run time. You'll just need to clearly document what you have manually changed, why you have done it, and keep your own true dependencies accurately represented in package.json

Here is a github issue discussion that elaborates on this problem in detail.

npm v3(released recently) solves this issue by flattening out the dependencies.. Check the release notes here in https://github.com/npm/npm/releases/tag/v3.0.0 under flat flat section.

And the last comment on this issue https://github.com/npm/npm/issues/3697

I wrote a node module called "npm-flatten" that flattens your dependencies for you here: https://www.npmjs.org/package/npm-flatten

If you are looking for a distrubtion, I also wrote a NuGet package that will integrate a complete node.js environment with your .NET project here: http://www.nuget.org/packages/NodeEnv/

Feedback would be welcome.

Something that helped me was to map a local drive to my Node.js folder:

net use n: \computername\c$\users\myname\documents\node.js /persistent:yes

Before: c:\users\myname\documents\node.js\projectname (45 characters) After: n:\projectname (14 characters which is 31 chars less)

In many cases this allowed some modules to be installed.

I will say that I just re-discovered this problem today when I was attempting to backup all my code to a USB drive.

"C:\Users\myname\Documents\Node.js\angular-phonecat\node_modules\karma\node_modules\chokidar\node_modules\anymatch\node_modules\micromatch\node_modules\regex-cache\node_modules\benchmarked\node_modules\file-reader\node_modules\extend-shallow\benchmark\fixtures is too long."

Even when I tried to back them up using the N: drive letter it still failed in some cases due to path lengths but it was just enough to fix the one above.

1. During release build, You can prevent Visual studio scanning these files / folder by setting the folder properties as a Hidden folder (JUST set it to node_modules). Reference: http://issues.umbraco.org/issue/U4-6219#comment=67-19103

2. You can exclude files or folder that are published during packaging by including following XML node in the CsProject file.

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  ...
  <OutputPath>bin\</OutputPath>
   <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
  <ExcludeFilesFromDeployment>File1.aspx;File2.aspx</ExcludeFilesFromDeployment>
  <ExcludeFoldersFromDeployment>Folder1;Folder2</ExcludeFoldersFromDeployment>
</PropertyGroup>

I found one solution from Microsoft Node.js Guidelines.

• Start in a short path (e.g. c:\src)
• > npm install -g rimraf delete files that exceed max_path
• > npm dedupe moves duplicate packages to top-level
• > npm install -g flatten-packages moves all packages to top-level, but can cause versioning issues
• Upgrade to npm@3 which attempts to the make the node_modules folder heirarchy maximally flat.
  • Ships with Node v5
  • Or… > npm install –g npm-windows-upgrade

This is a not a proper solution, rather a work around when you are in a hurry, but you can use 7-Zip to zip your folder, move the zipped file and unzip it without any issue.

We used that solution to deploy a Node.js application where it was not possible to do a clean npm install.