Solution: load independently compiled Webpack 2 bundles dynamically

I would like to share how to bundle an application that acts as a plugin host and how it can load installed plugins dynamically.

1. Both the application and the plugins are bundled with Webpack
2. The application and plugins are compiled and distributed independently.

There are several people on the net who are looking for a solution to this problem:

• https://stackoverflow.com/questions/38502035/multi-project-build-and-dynamically-loading-modules-with-webpack
• https://stackoverflow.com/questions/42780950/loading-prebuilt-webpack-bundles-at-runtime
• https://stackoverflow.com/questions/40035445/how-to-expose-objects-from-webpack-bundle-and-inject-external-libs-into-compiled-bundle
• Dynamic Requires
  https://github.com/webpack/webpack/issues/118

The solution described here is based on @sokra's Apr 17, 2014 comment on Webpack issue #118 and is slightly adapted in order to work with Webpack 2. https://github.com/webpack/webpack/issues/118

Main points:

• A plugin needs an ID (or "URI") by which it registers at the backend server, and which is unique to the application.

• In order to avoid chunk/module ID collisions for every plugin, individual JSONP loader functions will be used for loading the plugin's chunks.

• Loading a plugin is initiated by dynamically created <script> elements (instead of require()) and let the main application eventually consume the plugin's exports through a JSONP callback.

Note: You may find Webpack's "JSONP" wording misleading as actually no JSON is transferred but the plugin's Javascript wrapped in a "loader function". No padding takes place at server-side.

Building a plugin

A plugin's build configuration uses Webpack's output.library and output.libraryTarget options.

Example plugin configuration:

module.exports = {
  entry: './src/main.js',
  output: {
    path: path.resolve(__dirname, './dist'),
    publicPath: '/' + pluginUri + '/',
    filename: 'js/[name].js',
    library: pluginIdent,
    libraryTarget: 'jsonp'
  },
  ...
}

It's up to the plugin developer to choose an unique ID (or "URI") for the plugin and make it available in the plugin configuration. Here I use the variable pluginURI:

// unique plugin ID (using dots for namespacing)
var pluginUri = 'com.companyX.pluginY'

For the library option you also have to specify an unique name for the plugin. Webpack will use this name when generating the JSONP loader functions. I derive the function name from the plugin URI:

// transform plugin URI into a valid function name
var pluginIdent = "_" + pluginUri.replace(/\./g, '_')

Note that when the library option is set Webpack derives a value for the output.jsonpFunction option automatically.

When building the plugin Webpack generates 3 distribution files:

dist/js/manifest.js
dist/js/vendor.js
dist/js/main.js

Note that vendor.js and main.js are wrapped in JSONP loader functions whose names are taken from output.jsonpFunction and output.library respectively.

Your backend server must serve the distribution files of each installed plugin. For example, my backend server serves the content of a plugin's dist/ directory under the plugin's URI as the 1st path component:

/com.companyX.pluginY/js/manifest.js
/com.companyX.pluginY/js/vendor.js
/com.companyX.pluginY/js/main.js

That's why publicPath is set to '/' + pluginUri + '/' in the example plugin config.

Note: The distribution files can be served as static resources. The backend server is not required to do any padding (the "P" in JSONP). The distribution files are "padded" by Webpack already at build time.

Loading plugins

The main application is supposed to retrieve the list of the installed plugin (URI)s from the backend server.

// retrieved from server
var pluginUris = [  'com.companyX.pluginX',  'com.companyX.pluginY',  'org.organizationX.pluginX',]

Then load the plugins:

loadPlugins () {
  pluginUris.forEach(pluginUri => loadPlugin(pluginUri, function (exports) {
    // the exports of the plugin's main file are available in `exports`
  }))
}

Now the application has access to the plugin's exports. At this point, the original problem of loading an independently compiled plugin is basically solved :-)

A plugin is loaded by loading its 3 chunks (manifest.js, vendor.js, main.js) in sequence. Once main.js is loaded the callback will be invoked.

function loadPlugin (pluginUri, mainCallback) {
  installMainCallback(pluginUri, mainCallback)
  loadPluginChunk(pluginUri, 'manifest', () =>
    loadPluginChunk(pluginUri, 'vendor', () =>
      loadPluginChunk(pluginUri, 'main')
    )
  )
}

Callback invocation works by defining a global function whose name equals output.library as in the plugin config. The application derives that name from the pluginUri (just like we did in the plugin config already).

function installMainCallback (pluginUri, mainCallback) {
  var _pluginIdent = pluginIdent(pluginUri)
  window[_pluginIdent] = function (exports) {
    delete window[_pluginIdent]
    mainCallback(exports)
  }
}

A chunk is loaded by dynamically creating a <script> element:

function loadPluginChunk (pluginUri, name, callback) {
  return loadScript(pluginChunk(pluginUri, name), callback)
}

function loadScript (url, callback) {
  var script = document.createElement('script')
  script.src = url
  script.onload = function () {
    document.head.removeChild(script)
    callback && callback()
  }
  document.head.appendChild(script)
}

Helper:

function pluginIdent (pluginUri) {
  return '_' + pluginUri.replace(/\./g, '_')
}

function pluginChunk (pluginUri, name) {
  return '/' + pluginUri + '/js/' + name + '.js'
}