Node npm windows file paths are too long to install packages

Situation

I want to use gulp and related front-end tool chains in Windows-hosted development environments. I'm hitting a wall trying to use gulp plug-ins like Browser-Sync, because the node_modules folder graph fans out making the windows file paths too long to copy the files. I'd like a pragmatic approach for handling this problem right now on Windows, irrespective of what the Node community may or may not provide to improve npm usability on the Windows in the future.

2 Questions

1. Is there an npm workflow for Windows that just works the way it was intended? "run the command and the files install" (e.g. comparable to npm on OSX, npm on Linux, ruby gems or even nuget) I don't want to fiddle with a bunch of manual file edits, symlinks, etc. every time I use npm on Windows.

2. Is there a well-documented, stable Cygwin workflow for npm and node execution to workaround the Windows API file path limits?

Gory details listed below...

General Problem

• Running npm install from a standard Windows command prompt fails on deeply nested node_modules hierarchies.
• Per Joyent's github repo thread, this is an acknowledged issue with no palatable workarounds for developers in Windows-centric environments. (Really?)
• NT Kernel supports file path lengths up to 32,767 characters.
• Windows API's MAXPATH is limited to 260 characters.
• Windows API handles file operations for all major Windows shells and whatnot including: Explorer, CMD, Powershell,MYSgit bash, etc. (MS really? How long has NTFS been around?)
• Cygwin supports long file paths, but npm.cmd doesn't work out-of-box due to crlf formatting. I tried the DOS2Unix transform on npm to get it working with Cygwin, but there seem to be other issues with this.

My Current Hack

• Create an "n" folder as a staging area on the root of C:, because this shortens my folder path.
• Run npm inside the "n" folder to install modules for whatever I need.
• Fire up Cygwin and use cp to copy the node_modules folder into a destination project.
• Rinse and repeat when dependencies change or when I need to spin up a new project.

Other Unpalatable Workarounds

Symbolic Links can be used to shorten file paths, but these are kludgy hacks. As the npm ecosystem grows, nested dependency chains will become too long and this workaround become unusable.

Adding ALL dependences to the root folder's package.json file was mentioned in one thread I came across. Although this approach will flatten the folder structure and prevent loading of duplicate modules, this workaround feels unnatural. It also kills the usability, durability, and productivity of npm, because you have to fiddle with files and folders post-install either manually or with some hacky scripts. The approach is also vulnerable to the same fate that Symbolic Links approach may eventually suffer.

The problem with deeply nested folders on Windows has been mostly solved starting with npm version 3.x.

According to npm:

>.npm@3 makes the install "maximally flat" by hoisting everything it can to the top level node_modules. This means nesting only occurs on conflicts and as such, trees should never get very deep. As such, the windows path length limitation shouldn't be run into.

I have just installed npm 3.1.0 and tried it out on a package that was throwing the dreaded The specified path, file name, or both are too long error.

The problem went away.

You can get the latest npm builds from here : npm releases

Windows 8.1 and 10 have an option to increase the Win32 path limit:

• Open Group Policy Editor (Press Windows+R and type gpedit.msc and hit Enter)
• Navigate to the following directory: Local Computer Policy\Computer Configuration\Administrative Templates\System\Filesystem
• Doubleclick on Enable Win32 long paths option and enable it.

This is a work around solution.

There are some node modules that flattens your dependencies for you.
Links are here:

• npm-flatten
• npm-dedupe

What these modules are doing can be done manually as well. This is the only real solution exists as of now, i.e to have all your modules at a single level, requiring each other, instead of all having private copies of their dependencies nested deeply.

Allan -

From the github issue you linked,

> npm will add dedupe-at-install-time by default. This is significantly more feasible than Node's module system changing, but it is still not exactly trivial, and involves a lot of reworking of some long-entrenched patterns.

This is (finally) currently in the works at npm, going by the name multi-stage-install, and is targeted for npm@3. npm development lead Forrest Norvell is going to spend some time running on Windows in the new year, so please do create windows-related issues on the npm issue tracker < https://github.com/npm/npm/issues >

I have the same issue. Flattening the dependencies isn't a complete solution, since you might be using modules that depend on different versions of the same dependent module. I discovered the gulp-run module stopped working after flattening (related to module assumptions about bin/.bin directories, I suspect). Drat!

There's lots of discussion about the problem, but no solution in sight: https://github.com/joyent/node/issues/6960

https://github.com/npm/npm/issues/3697

A workaround that's working for me is to manually add dependencies my project doesn't explicitly need.

If you want to identify which packages are giving you problems, I found PathLengthChecker quite useful. Just extract the EXE and run the GUI or command line app. The other way I've uncovered the problem is to try to build in Visual Studio, but it fails without telling you which directory name is too long.

Here's a command line example of my workaround:

mkdir c:\reallylongdirectorywillbreakinwindows
cd c:\reallylongdirectorywillbreakinwindows
npm init
npm install --save-dev grunt-bower-task
PathLengthChecker.exe RootDirectory="C:\reallylongdirectorywillbreakinwindows" MinLength=260

I got back:

> 261: C:\reallylongdirectorywillbreakinwindows\node_modules\grunt-bower-task\node_modules\bower\node_modules\update-notifier\node_modules\latest-version\node_modules\package-json\no de_modules\registry-url\node_modules\npmconf\node_modules\config-chain\readme.markdown

> [snip - there were 12 of them]

According to the npm ls command:

└─┬ grunt-bower-task@0.4.0
  ├── async@0.1.22
  ├─┬ bower@1.3.12
  │ ├─┬ update-notifier@0.2.0
  │ │ ├─┬ latest-version@0.2.0
  │ │ │ └─┬ package-json@0.2.0
  │ │ │   └─┬ registry-url@0.1.1
  │ │ │     └─┬ npmconf@2.1.1
  │ │ │       ├─┬ once@1.3.1
  │ │ │       │ └── wrappy@1.0.1

Let's go with npmconf - it's the container for all the over-length files that are causing issues. We need npmconf 2.1.1.

npm install --save-dev npmconf@2.1.1
(now delete the node_modules directory - you may have to use Windows Explorer if you can't do it with rmdir /s)
npm install
PathLengthChecker.exe RootDirectory="C:\reallylongdirectorywillbreakinwindows" MinLength=260

No results - all files are within limits!

The obvious caveat here is that it only works once per package - dependencies on different versions of the same module can't be installed at the root node_modules level because node doesn't account for versions in the directory structure.

This workaround isn't perfect, but it solves my main goals of having node work on Windows, and since the resolution is right in package.json, the workaround works for other developers and build servers without any manual or global fussing.

If you are OK with installing it globally this could be a work-around:

You can adjust the path where npm is installing the global modules to something very short (usually it is: c:\users\\{username}\AppData\Roaming\npm\npm_modules) which already takes a lot of characters.

To adjust it see here: https://stackoverflow.com/questions/19874582/change-default-global-installation-directory-for-node-js-modules-in-windows/19876059#19876059

If you adjust it to, e.g., c:\n\ in some cases it might solve the issue.

This is what finally fixed it for me...

After installing gulp and receiving errors, run... gulp

When you see a package failing, install it manually with --no-bin-link.

sudo npm install {package} --no-bin-link

Where {package} is the package that is having problems.

After all of this I was receiving an Error in plugin 'gulp-notify' Message: not found: notify-send.

This was due to a plugin issue with Vagrant. You can either turn off notifications..

export DISABLE_NOTIFIER=true;

Or install the plugin with Vagrant.

Best of luck.. I spent a long time on this, even after following a lot of people's recommendations.

Brandon

In windows:

1. Using your windows explorer, Navigate to your vagrant shared folder (I am using scotchbox by the way) e.g C:\scotchbox/public/gulpProject
2. In the address bar of the folder, type cmd and press Enter
3. Do your gulp installation npm install

npm install --no-bin-link. You will have a entire flattened node_modules