How to manage client-side JavaScript dependencies?

Although there are great solutions to manage dependencies on the server side, I could not find any that satisfies all my needs to have a coherent client side JavaScript dependency management workflow. I want to satisfy these 5 requirements:

1. Manage my client side dependencies in a format similar to npm's package.json or bower's bower.json

2. It should have the flexibility to point to git repo or actual js files (either on web or locally) in my dependency.json file for lesser known libraries (npm let you point to git repos)

3. It should minify and namespace all libraries into a single file like ender - that's the only js file I would need to put in my <script> tag in the client side

4. It should have out of box support for CoffeeScript like BoxJS4 (now dead)

5. In the browser, I should be able to use either require style:

    var $ = require('jquery');
    var _ = require('underscore');
   

   Or better yet, do headjs style:

    head.js(['jquery', 'underscore', 'mylib'], function($, _, mylib) {
      // executed when all libraries are loaded
    });
   

If no one such single tool exists, what is the best combination of tools i.e. a tool-chain that I can combine using something like volo (or grunt)?

I have already researched all the tools I have linked to in here and they satisfy only upto 3 of my requirements at best individually. So, please don't post again about these tools. I would only accept an answer that provides a single tool that satisfies all 5 of my requirements or if someone posts a concrete workflow/script/working example of a toolchain of multiple such tools that also satisfies all my requirements. Thank you.

require.js does everything you need.

My answer to this question may help you

Example:

Client app project hierarchy:

sampleapp
    |___ main.js
    |___ cs.js
    |___ require.js

main.js is where you initialize your client application and configure require.js:

require.config({
    baseUrl: "/sampleapp",
    paths: {
        jquery: "libs/jquery", // Local
        underscore: "http://underscorejs.org/underscore-min.js", // Remote
        backbone: "https://github.com/documentcloud/backbone/blob/master/backbone-min.js" // Remote on github
    },
    shim: {
        backbone: {
            deps: ["underscore", "jquery"] // Backbone depends on jquery and underscore
        }
    }
});

require(["cs!someCoffeescriptFile", "jquery", "backbone", "underscore"], function (SomeCoffeescriptFile, $, Backbone, _) {
    // Dependencies are loaded...
    // Execute code
});

Dependencies will use the cs plugin when prepended by "cs!". The cs plugin compiles the coffeescript file.

When you go in prod, you can pre-compile your whole project with r.js.

node ./node_modules/requirejs/bin/r.js -o buildclientconfig.js

Here are your requirements:

• Manage my client side dependencies in a format similar to npm's package.json or bower's component.json. Different but AS GOOD!

• I should have the flexibility to point to git repo or actual js files (either on web or locally) in my dependency.json file for lesser known libraries (npm let's you point to git repos). YES

• It should minify and namespace all libraries into a single file like ender - that's the only js file I would need to put in my script-tag in the client side. YES with r.js.

• It should have out of box support for coffeescript like Box. YES

• In the browser I can use either require style or headjs. YES

http://requirejs.org/ is the one you are looking for i believe

As @Guillaume86 I think hem will get you the closest to where you want to be.

In hem dependencies are managed using a combination of npm and hem. Use npm to explicitly install all of your projects external dependencies. Use hem to specify which dependencies (both external and local) should be stitched together for you client side operations.

I created a skeleton project of this so you can see how this would work - you can see it at https://github.com/dsummersl/clientsidehem

Adding dependencies

Use npm to search for a specific dependency and then modify the package.json file to ensure that the dependency is tracked in the future. Then specify the dependency for your application in slug.json.

For example, suppose you wanted to add the coffee-script dependency. Just use npm to install the dependency and save it to your package.json file:

1. npm --save install coffee-script
2. Manually edit the slug.json file. Add "coffee-script" to "dependencies".

Suppose you wanted to include your own module 'bloomfilters' and it wasn't in the npm registry. You could add it to your project in the following way:

1. npm --save install https://github.com/dsummersl/bloomfilters/tarball/master
2. Manually edit the slug.json file. Add "bloomfilters" to "dependencies".

Local modules

If you want to include your own coffee or javascript you can do so by adding those files to the app/ folder. Note that in order to expose your script via the 'require' method you must make it a CommonJS module. It is very simple - see the hem docs.

Local files

If you want to include non-CommonJS non 'require' code you can also stitch that by referencing your custom javascript or coffeescript via the 'libs' list in slug.json.

CSS

Hem will stitch together your CSS too, if you want. See the hem docs.

Building

Once you have your dependencies listed, you can use hem to stitch them all together.

# make sure all dependencies are present:
npm install .
# make public/application.js
hem build
# see your minified js in public/application.js

Notes

Hem was meant for the spinejs project - but you don't have to use it for that. Ignore any docs mentioning spine as you wish...

Well, I'm surprised that no one mentionned Browserify yet.

1. supports package.json format
2. uses npm underneath which can use a github (or any git) repo as package source
3. minifies and concatenates all dependencies into a single file.
4. supports coffeescript if you include it in your dependencies
5. require style all the way.
6. supports source maps

I'm pretty sure Hem meet your requirements (I use a personnal fork with additional compilers - jade and stylus - it's easy to customize to your needs). It uses npm to manage depedencies.

You might want to take a look at Yeoman, which uses several techniques to help you with your requirements.

> Our workflow is comprised of three tools for improving your > productivity and satisfaction when building a web app: yo (the > scaffolding tool), grunt (the build tool) and bower (for package > management).

Build-in support for CoffeeScript, Compass and more. Works with r.js (RequireJS), unittesting etc.

As for your requirements:

1. Bower is used for dependency management
2. Bower can work with local files, git://, http:// and more
3. Build-in support for minification and concatenation (even for your images)
4. Build-in support to automatically compile CoffeeScript & Compass (with LiveReload)
5. As stated in the build process: if you're using AMD, I will pass those modules through r.js so you don't have to.

All features:

> Lightning-fast scaffolding — Easily scaffold new projects with > customizable templates (e.g HTML5 Boilerplate, Twitter Bootstrap), > RequireJS and more. > > Great build process — Not only do you get > minification and concatenation; I also optimize all your image files, > HTML, compile your CoffeeScript and Compass files, if you're using > AMD, I will pass those modules through r.js so you don't have to. > > Automatically compile CoffeeScript & Compass — Our LiveReload watch > process automatically compiles source files and refreshes your browser > whenever a change is made so you don't have to. > > Automatically lint your scripts — All your scripts are automatically run against JSHint to ensure they're following language best-practices. > > Built-in preview server — No more having to fire up your own HTTP Server. My built-in > one can be fired with just one command. > > Awesome Image Optimization — I optimize all your images using OptiPNG and JPEGTran so your users can spend less time downloading assets and more time using your app. > > Killer package management — Need a dependency? It's just a keystroke > away. I allow you to easily search for new packages via the > command-line (e.g. `bower search jquery), install them and keep them > updated without needing to open your browser. > > PhantomJS Unit Testing — Easily run your unit tests in headless WebKit via PhantomJS. When > you create a new application, I also include some test scaffolding for > your app.

Bower may suits your needs (1) and (2) for the rest you have requirejs. From the readme :

Bower is a package manager for the web. Bower lets you easily install assets such as images, CSS and JavaScript, and manages dependencies for you.

To install a package:

bower install jquery
bower install git://github.com/maccman/package-jquery.git
bower install http://code.jquery.com/jquery-1.7.2.js
bower install ./repos/jquery

Look at Jam package manager. Following is the description from its homepage

> For front-end developers who crave maintainable assets, Jam is a package manager for JavaScript. Unlike other repositories, we put the browser first.

It seems a lot similar to npm in how it works.

Install package like below

jam install backbone

keeping packages up to date by executing

jam upgrade
jam upgrade {package} 

Optimize packages for production

jam compile compiled.min.js

Jam dependencies can be added in package.json file.

For complete documentation read Jam Documentation

I just came across inject.js

Some of the features, from the project site:

> Inject (Apache Software License 2.0) is a revolutionary way to manage your dependencies in a Library Agnostic way. Some of its major features include: >

• CommonJS Compliance in the Browser (exports.*)

• View the full CommonJS Support Matrix

• Cross domain retrieval of files (via easyXDM)

• localStorage (load a module once)

There are a couple of options:

• http://browserify.org/ which allows you to import modules
• RequireJS addresses the same problem
• One that seems to be in active development is JoinJS

Component might also be of interest, it does not manage dependencies per se but allows you to use chopped up versions of otherwise large libraries.

I use hem with npm, and I wanted to add some additional benefits that I think weren't covered so far.

• Hem has a self-contained web server (strata) so you can develop your code without even needing to recompile. I never use hem build unless I am publishing an app.
• You don't need to use Spine.js to use hem, you can use it to compile arbitrary coffeescript packages if you set up slug.json correctly. Here's one of my packages that is auto-compiled with cakefile: https://github.com/HarvardEconCS/TurkServer/tree/master/turkserver-js-client
• Speaking of the above, hem allows you to link other dependencies on your local system in with npm link and combines them seamlessly even when you are using the strata server. In fact, you needn't even use the cake method above, you can just link directly to coffeescript from dependent projects.
• Hem supports eco (embedded Coffeescript) for views and Stylus for CSS, and compiles all that, along with your Coffeescript, into one JS and one CSS file.

Here's a basic list for getting set up with a Spine, hem, coffeescript app. Feel free to ignore the Spine parts. In fact, sometimes I use spine app to set up a directory structure for a non-Spine app, then edit slug.json to change to a different compilation structure.

1. Install NPM: curl http://npmjs.org/install.sh | sh on a *nix system. I'll assume it's available from the command line.

2. Install hem globally (npm install -g hem). Development has branched as of late so you might want to get it straight out of github (https://github.com/spine/hem), checkout a branch, and npm install -g . in that folder.

3. npm install -g spine.app will make spine available as a global command

4. spine app folder will make a Spine project called app in folder, generating the right directory structure and a bunch of skeleton files to get started.

5. cd to folder and edit dependencies.json for the libraries you need. Add them to slug.json so that hem knows where to find them as well.

6. Optional: npm link any of your local packages in development to node_modules, and you can add them to slug.json for hem (either an index.js to include directly or an index.coffee if you want hem to compile it.)

7. npm install . to download all the dependencies you just entered in.

8. If you take a look at the default spine config, there is a app/lib/setup.coffee where you require all the libraries you need from your dependencies. Examples:

    # Spine.app had these as dependencies by default
    require('json2ify')
    require('es5-shimify')
    require('jqueryify')
   
    require('spine')
    require('spine/lib/local')
    require('spine/lib/ajax')
    require('spine/lib/manager')
    require('spine/lib/route')
   
    # d3 was installed via dependencies.json
    require 'd3/d3.v2'
   

9. In index.coffee, you just require lib/setup and load the main controller for your app. In addition, you need to require any other classes in those other controllers. You can use spine controller something or spine model something to generate templates for controllers and models. Typical Spine controller looks like the following, using node's require:

    Spine = require('spine')
    # Require other controllers
    Payment = require('controllers/payment')
   
    class Header extends Spine.Controller
      constructor: ->
        # initialize the class
        
      active: ->
        super
        @render()
        
      render: ->
        # Pull down some eco files
        @html require('views/header')   
    
    # Makes this visible to other controllers    
    module.exports = Header
   

10. The default generated index.html will usually be fine for loading your app, but modify as necessary. Per your requirements, it only pulls in one js and one css file, which you never need to modify.

11. Edit your stylus files as necessary in the css folder. It's a lot more flexible than CSS :)

12. From folder, run hem server to start a hem server, and navigate to localhost:9294 to see your app. (If you installed hem globally.) It has some hidden arguments, for example --host 0.0.0.0 listens on all ports.

13. Build the rest of your app using proper MVC techniques, and use stylus for CSS and eco for views. Or don't use Spine at all, and hem will still work great with Coffeescript and npm. There are many examples of projects using both models.

One more thing: normally, hem server will update automatically as you update your code and save files, which makes it a cinch to debug. Running hem build will compile your app into two files, application.js, which is minified and application.css. If you run hem server after this, it will use those files and no longer update automatically. So don't hem build until you actually need a minified version of your app for deployment.

Additional references: https://stackoverflow.com/questions/11561014/spine-js-hem-getting-started

Here's a solution that takes a very different approach: package up all the modules into a JSON object and require modules by reading and executing the file content without additional requests.

Pure clientside demo implementation: http://strd6.github.io/editor/

https://github.com/STRd6/require/blob/master/main.coffee.md

STRd6/require depends on having a JSON package available at runtime. The require function is generated for that package. The package contains all the files your app could require. No further http requests are made because the package bundles all dependencies. This is as close as one can get to the Node.js style require on the client.

The structure of the package is as follows:

entryPoint: "main"
distribution:
  main: 
    content: "alert(\"It worked!\")"
  ...
dependencies:
  <name>: <a package>

Unlike Node a package doesn't know it's external name. It is up to the pacakge including the dependency to name it. This provides complete encapsulation.

Given all that setup here's a function that loads a file from within a package:

loadModule = (pkg, path) ->
  unless (file = pkg.distribution[path])
    throw "Could not find file at #{path} in #{pkg.name}" 

  program = file.content
  dirname = path.split(fileSeparator)[0...-1].join(fileSeparator)

  module =
    path: dirname
    exports: {}

  context =
    require: generateRequireFn(pkg, module)        
    global: global
    module: module
    exports: module.exports
    PACKAGE: pkg
    __filename: path
    __dirname: dirname
  
  args = Object.keys(context)
  values = args.map (name) -> context[name]

  Function(args..., program).apply(module, values)

  return module

This external context provides some variable that modules have access to.

A require function is exposed to modules so they may require other modules.

Additional properties such as a reference to the global object and some metadata are also exposed.

Finally we execute the program within the module and given context.

This answer will be most helpful to those who wish to have a synchronous node.js style require statement in the browser and are not interested in remote script loading solutions.

Check out cartero if you using node/express on the backend.

I'd suggest to check out the dojo toolkit which seems to meet most of your requirements. The one I'm not sure about is CoffeeScript.

dojo works with modules written in the Asynchronous Module Definition (AMD) format. It has a build system with packages and you can aggregate them in one or several files (called layers). Apparently it accepts git type repositories, more details on the build system here:

http://dojotoolkit.org/documentation/tutorials/1.8/build/

For the record, v1.9 beta is expected next month.

Another framework that satisfies all my criterion released recently: http://duojs.org/ (and also supports treating other resources like CSS as dependencies).

dependency injection with asynchronous loading + browserify will be another good choice, compares to requirejs

asynchronous-frontend-dependency-management-without-AMD