Web workers without a separate Javascript file?
JavascriptWeb WorkerJavascript Problem Overview
As far as I can tell, web workers need to be written in a separate JavaScript file, and called like this:
new Worker('longrunning.js')
I'm using the closure compiler to combine and minify all my JavaScript source code, and I'd rather not have to have my workers in separate files for distribution. Is there some way to do this?
new Worker(function() {
//Long-running work here
});
Given that first-class functions are so crucial to JavaScript, why does the standard way to do background work have to load a whole other JavaScript file from the web server?
Javascript Solutions
Solution 1 - Javascript
http://www.html5rocks.com/en/tutorials/workers/basics/#toc-inlineworkers
> What if you want to create your worker script on the fly, or create a self-contained page without having to create separate worker files? With Blob(), you can "inline" your worker in the same HTML file as your main logic by creating a URL handle to the worker code as a string
Full example of BLOB inline worker:
<!DOCTYPE html>
<script id="worker1" type="javascript/worker">
// This script won't be parsed by JS engines because its type is javascript/worker.
self.onmessage = function(e) {
self.postMessage('msg from worker');
};
// Rest of your worker code goes here.
</script>
<script>
var blob = new Blob([
document.querySelector('#worker1').textContent
], { type: "text/javascript" })
// Note: window.webkitURL.createObjectURL() in Chrome 10+.
var worker = new Worker(window.URL.createObjectURL(blob));
worker.onmessage = function(e) {
console.log("Received: " + e.data);
}
worker.postMessage("hello"); // Start the worker.
</script>
Solution 2 - Javascript
The html5rocks solution of embedding the web worker code in HTML is fairly horrible.
And a blob of escaped JavaScript-as-a-string is no better, not least because it complicates work-flow (Closure compiler can't operate on strings).
Personally I really like the toString methods, but @dan-man THAT regex!
My preferred approach:
// Build a worker from an anonymous function body
var blobURL = URL.createObjectURL( new Blob([ '(',
function(){
//Long-running work here
}.toString(),
')()' ], { type: 'application/javascript' } ) ),
worker = new Worker( blobURL );
// Won't be needing this anymore
URL.revokeObjectURL( blobURL );
Support is the intersection of these three tables:
- http://caniuse.com/#feat=webworkers
- http://caniuse.com/#feat=blobbuilder
- http://caniuse.com/#feat=bloburls
This won't work for a SharedWorker however, because the URL must be an exact match, even if the optional 'name' parameter matches. For a SharedWorker, you'll need a separate JavaScript file.
2015 update - The ServiceWorker singularity arrives
Now there's an even more powerful way of solving this problem. Again, store the worker code as a function, (rather than a static string) and convert using .toString(), then insert the code into CacheStorage under a static URL of your choice.
// Post code from window to ServiceWorker...
navigator.serviceWorker.controller.postMessage(
[ '/my_workers/worker1.js', '(' + workerFunction1.toString() + ')()' ]
);
// Insert via ServiceWorker.onmessage. Or directly once window.caches is exposed
caches.open( 'myCache' ).then( function( cache )
{
cache.put( '/my_workers/worker1.js',
new Response( workerScript, { headers: {'content-type':'application/javascript'}})
);
});
There are two possible fall-backs. ObjectURL as above, or more seamlessly, put a real JavaScript file at /my_workers/worker1.js
Advantages of this approach are:
- SharedWorkers can also be supported.
- Tabs can share a single cached copy at a fixed address. The blob approach proliferates random objectURLs for every tab.
Solution 3 - Javascript
You can create a single JavaScript file that is aware of its execution context and can act as both a parent script and a worker. Let's start off with a basic structure for a file like this:
(function(global) {
var is_worker = !this.document;
var script_path = is_worker ? null : (function() {
// append random number and time to ID
var id = (Math.random()+''+(+new Date)).substring(2);
document.write('<script id="wts' + id + '"></script>');
return document.getElementById('wts' + id).
previousSibling.src;
})();
function msg_parent(e) {
// event handler for parent -> worker messages
}
function msg_worker(e) {
// event handler for worker -> parent messages
}
function new_worker() {
var w = new Worker(script_path);
w.addEventListener('message', msg_worker, false);
return w;
}
if (is_worker)
global.addEventListener('message', msg_parent, false);
// put the rest of your library here
// to spawn a worker, use new_worker()
})(this);
As you can see, the script contains all code for both the parent's and the worker's point of view, checking if its own individual instance is a worker with !document. The somewhat unwieldy script_path computation is used to accurately calculate the script's path relative to the parent page, as the path supplied to new Worker is relative to the parent page, not the script.
Solution 4 - Javascript
Using the Blob method, how about this for a worker factory:
var BuildWorker = function(foo){
var str = foo.toString()
.match(/^\s*function\s*\(\s*\)\s*\{(([\s\S](?!\}$))*[\s\S])/)[1];
return new Worker(window.URL.createObjectURL(
new Blob([str],{type:'text/javascript'})));
}
So you could use it like this...
var myWorker = BuildWorker(function(){
//first line of worker
self.onmessage(){....};
//last line of worker
});
EDIT:
I've just extended this idea further to make it easier to do cross-thread communication: bridged-worker.js.
EDIT 2:
The above link is to a gist I created. Someone else later turned it into an actual repo.
Solution 5 - Javascript
Web workers operate in entirely separate contexts as individual Program's.
This means that code cannot be moved from one context to another in object form, as they would then be able to reference objects via closures belonging to the other context.
This is especially crucial as ECMAScript is designed to be a single threaded language, and since web workers operate in separate threads, you would then have the risk of non-thread-safe operations being performed.
This again means that web workers need to be initialized with code in source form.
The spec from WHATWG says
> If the origin of the resulting > absolute URL is not the same as the > origin of the entry script, then throw > a SECURITY_ERR exception. > > Thus, scripts must be external files > with the same scheme as the original > page: you can't load a script from a > data: URL or javascript: URL, and an > https: page couldn't start workers > using scripts with http: URLs.
but unfortunately it doesn't really explain why one couldn't have allowed passing a string with source code to the constructor.
Solution 6 - Javascript
Recent answer (2018)
You can use Greenlet:
> Move an async function into its own thread. A simplified single-function version of Workerize.
Example:
import greenlet from 'greenlet'
const getName = greenlet(async username => {
const url = `https://api.github.com/users/${username}`
const res = await fetch(url)
const profile = await res.json()
return profile.name
})
console.log(await getName('developit'))
Solution 7 - Javascript
a better to read way for a inline worker..
var worker_fn = function(e)
{
self.postMessage('msg from worker');
};
var blob = new Blob(["onmessage ="+worker_fn.toString()], { type: "text/javascript" });
var worker = new Worker(window.URL.createObjectURL(blob));
worker.onmessage = function(e)
{
alert(e.data);
};
worker.postMessage("start");
Solution 8 - Javascript
Taking Adria's response and putting it in a copy-pastable function which works with current Chrome and FF but not IE10 (worker from blob causes a security error).
var newWorker = function (funcObj) {
// Build a worker from an anonymous function body
var blobURL = URL.createObjectURL(new Blob(
['(', funcObj.toString(), ')()'],
{type: 'application/javascript'}
));
var worker = new Worker(blobURL);
// Won't be needing this anymore
URL.revokeObjectURL(blobURL);
return worker;
}
And here's a working example http://jsfiddle.net/ubershmekel/YYzvr/
Solution 9 - Javascript
Depending on your use case you can use something like
>task.js Simplified interface for getting CPU intensive code to run on all cores (node.js, and web)
A example would be
function blocking (exampleArgument) {
// block thread
}
// turn blocking pure function into a worker task
const blockingAsync = task.wrap(blocking);
// run task on a autoscaling worker pool
blockingAsync('exampleArgumentValue').then(result => {
// do something with result
});
Solution 10 - Javascript
A simple promisified version, Function#callAsWorker, that takes a thisArg and arguments (just like call), and returns a promise:
Function.prototype.callAsWorker = function (...args) {
return new Promise( (resolve, reject) => {
const code = `self.onmessage = e => self.postMessage((${this.toString()}).call(...e.data));`,
blob = new Blob([code], { type: "text/javascript" }),
worker = new Worker(window.URL.createObjectURL(blob));
worker.onmessage = e => (resolve(e.data), worker.terminate());
worker.onerror = e => (reject(e.message), worker.terminate());
worker.postMessage(args);
});
}
// Demo
function add(...nums) {
return nums.reduce( (a,b) => a+b );
}
// Let the worker execute the above function, with the specified arguments
add.callAsWorker(null, 1, 2, 3).then(function (result) {
console.log('result: ', result);
});
Solution 11 - Javascript
Take a look at the vkThread plugin. With htis plugin you can take any function in your main code and execute it in a thread (web worker). So, you don't need to create a special "web-worker file".
http://www.eslinstructor.net/vkthread/
--Vadim
Solution 12 - Javascript
So I think we have another cool option for this now, thanks to template literals in ES6. That allows us to dispense with the extra worker function (and its weird scope) and just write the code that's intended for the worker as multiline text, much like the case where we were using
Solution 13 - Javascript
You can use web workers in same javascript file using inline webworkers.
The below article will address you to easily understand the webworkers and their limitations and debugging of webworkers.
Solution 14 - Javascript
I think the better way to do this is using a Blob object, below you can see a simple example.
// create a Blob object with a worker code
var blob = new Blob(["onmessage = function(e) { postMessage('msg from worker'); }"]);
// Obtain a blob URL reference to our worker 'file'.
var blobURL = window.URL.createObjectURL(blob);
// create a Worker
var worker = new Worker(blobURL);
worker.onmessage = function(e) {
console.log(e.data);
};
worker.postMessage("Send some Data");
Solution 15 - Javascript
Try to use jThread. https://github.com/cheprasov/jThread
// You can use simple calling like this
jThread(
function(arr){
//... some code for Worker
return arr;
}
,function(arr){
//... done code
}
)( [1,2,3,4,5,6,7] ); // some params
Solution 16 - Javascript
here console:
var worker=new Worker(window.URL.createObjectURL(new Blob([function(){
//Long-running work here
postMessage('done');
}.toString().split('\n').slice(1,-1).join('\n')],{type:'text/javascript'})));
worker.addEventListener('message',function(event){
console.log(event.data);
});
Solution 17 - Javascript
https://developer.mozilla.org/es/docs/Web/Guide/Performance/Using_web_workers
// Syntax: asyncEval(code[, listener])
var asyncEval = (function () {
var aListeners = [], oParser = new Worker("data:text/javascript;charset=US-ASCII,onmessage%20%3D%20function%20%28oEvent%29%20%7B%0A%09postMessage%28%7B%0A%09%09%22id%22%3A%20oEvent.data.id%2C%0A%09%09%22evaluated%22%3A%20eval%28oEvent.data.code%29%0A%09%7D%29%3B%0A%7D");
oParser.onmessage = function (oEvent) {
if (aListeners[oEvent.data.id]) { aListeners[oEvent.data.id](oEvent.data.evaluated); }
delete aListeners[oEvent.data.id];
};
return function (sCode, fListener) {
aListeners.push(fListener || null);
oParser.postMessage({
"id": aListeners.length - 1,
"code": sCode
});
};
})();
Solution 18 - Javascript
Use my tiny plugin https://github.com/zevero/worker-create
var worker_url = Worker.createURL(function(e){
self.postMessage('Example post from Worker'); //your code here
});
var worker = new Worker(worker_url);
Solution 19 - Javascript
This is just an addition to above - I have a nice templates for testing web workers in jsFiddle. Rather than Blob it uses jsFiddles ?js api:
function workerFN() {
self.onmessage = function(e) {
switch(e.data.name) {
case "" :
break;
default:
console.error("Unknown message:", e.data.name);
}
}
}
// This is a trick to generate real worker script that is loaded from server
var url = "/echo/js/?js="+encodeURIComponent("("+workerFN.toString()+")()");
var worker = new Worker(url);
worker.addEventListener("message", function(e) {
switch(e.data.name) {
case "" :
break;
default:
console.error("Unknown message:", e.data.name);
}
})
Normal web worker and shared worker templates are available.
Solution 20 - Javascript
I discovered that CodePen currently does not syntax-highlight inline <script> tags that are not type="text/javascript" (or which have no type attribute).
So I devised a similar but slightly different solution using labeled blocks with break, which is the only way you can bail from a <script> tag without creating a wrapper function (which is unnecessary).
<!DOCTYPE html>
<script id="worker1">
worker: { // Labeled block wrapper
if (typeof window === 'object') break worker; // Bail if we're not a Worker
self.onmessage = function(e) {
self.postMessage('msg from worker');
};
// Rest of your worker code goes here.
}
</script>
<script>
var blob = new Blob([
document.querySelector('#worker1').textContent
], { type: "text/javascript" })
// Note: window.webkitURL.createObjectURL() in Chrome 10+.
var worker = new Worker(window.URL.createObjectURL(blob));
worker.onmessage = function(e) {
console.log("Received: " + e.data);
}
worker.postMessage("hello"); // Start the worker.
</script>
Solution 21 - Javascript
my take on it:
function BuildWorker(fn){
var str = fn.toString().match(/^[^{]+{([\s\S]+)}\s*$/m)[1];
return new Worker(window.URL.createObjectURL(
new Blob([str],{type:'text/javascript'})));
}
function createAsyncWorker(fn){
// asyncworker=createAsyncWorker(function(){
// importScripts('my_otherscript.js');
// self.onmessage = function([arg1,arg2]) {
// self.postMessage('msg from worker');
// };
// })
// await asyncworker.postMessage('arg1','value')
// await asyncworker.postMessage('arg1','value')
// asyncworker.worker.terminate()
var worker = BuildWorker(fn);
function postMessage(...message){
let external={}, promise= new Promise((resolve,reject)=>{external.resolve=resolve;external.reject=reject;})
worker.onmessage = function(message){ external.resolve(message.data)};
worker.postMessage(message); // Start the worker.
return promise;
}
return {worker,postMessage};
}
usage example:
autoarima = createAsyncWorker(function(){
importScripts("https://127.0.0.1:11000/arima.js")
self.onmessage=(message)=>{
let [action,arg1,arg2]=message.data
if(action=='load')
{
ARIMAPromise.then(ARIMA1 => {
ARIMA=ARIMA1
autoarima = new ARIMA({ auto: true });
// const ts = Array(10).fill(0).map((_, i) => i + Math.random() / 5)
// const arima = new ARIMA({ p: 2, d: 1, q: 2, P: 0, D: 0, Q: 0, S: 0, verbose: false }).train(ts)
// const [pred, errors] = arima.predict(10)
postMessage('ok')
});
}
if(action=='fit')
{
autoarima.fit(arg1)
postMessage('ok')
}
if(action=='predict')
{
postMessage(autoarima.predict(arg1,arg2))
}
};
})
autoarima.terminate=function(){ this.worker.terminate(); }
autoarima.load=async function(...args){return await this.postMessage('load',...args)}
autoarima.fit=async function(...args){return await this.postMessage('fit',...args)}
autoarima.predict=async function(...args){return await this.postMessage('predict',...args)}
await autoarima.load()
await autoarima.fit(b_values)
await autoarima.predict(1)
Solution 22 - Javascript
I use code like this, you can define your onmessage as a function other than plain text, so the editor can highlight your code and jshint works.
const worker = createWorker();
createWorker() {
const scriptContent = getWorkerScript();
const blob = new Blob([
scriptContent,
], {
type: "text/javascipt"
});
const worker = new Worker(window.URL.createObjectURL(blob));
return worker;
}
getWorkerScript() {
const script = {
onmessage: function (e) {
console.log(e);
let result = "Hello " + e.data
postMessage(result);
}
};
let content = "";
for (let prop in script){
content += `${prop}=${script[prop].toString()}`;
}
return content;
}
Solution 23 - Javascript
Yes, it is possible, I did it using Blob files and passing a callback
I'll show you what a class I wrote does and how it manages the execution of callbacks in the background.
First you instantiate the GenericWebWorker with whatever data you'd like to pass to callback that'll be executing in the Web Worker, that includes functions you want to use, in this case a number, a date and a function called blocker
var worker = new GenericWebWorker(100, new Date(), blocker)
This blocker function will execute an infinite while for n miliseconds
function blocker (ms) {
var now = new Date().getTime();
while(true) {
if (new Date().getTime() > now +ms)
return;
}
}
and then you use it like this
worker.exec((num, date, fnBlocker) => {
/*Everithing here does not block the main thread
and this callback has access to the number, date and the blocker */
fnBlocker(10000) //All of this run in backgrownd
return num*10
}).then(d => console.log(d)) //Print 1000
> Now, time to see the magic in the example below
/*https://github.com/fercarvo/GenericWebWorker*/
class GenericWebWorker {
constructor(...ags) {
this.args = ags.map(a => (typeof a == 'function') ? {type:'fn', fn:a.toString()} : a)
}
async exec(cb) {
var wk_string = this.worker.toString();
wk_string = wk_string.substring(wk_string.indexOf('{') + 1, wk_string.lastIndexOf('}'));
var wk_link = window.URL.createObjectURL( new Blob([ wk_string ]) );
var wk = new Worker(wk_link);
wk.postMessage({ callback: cb.toString(), args: this.args });
var resultado = await new Promise((next, error) => {
wk.onmessage = e => (e.data && e.data.error) ? error(e.data.error) : next(e.data);
wk.onerror = e => error(e.message);
})
wk.terminate(); window.URL.revokeObjectURL(wk_link);
return resultado
}
async parallel(arr, cb) {
var res = [...arr].map(it => new GenericWebWorker(it, ...this.args).exec(cb))
var all = await Promise.all(res)
return all
}
worker() {
onmessage = async function (e) {
try {
var cb = new Function(`return ${e.data.callback}`)();
var args = e.data.args.map(p => (p.type == 'fn') ? new Function(`return ${p.fn}`)() : p);
try {
var result = await cb.apply(this, args); //If it is a promise or async function
return postMessage(result)
} catch (e) { throw new Error(`CallbackError: ${e}`) }
} catch (e) { postMessage({error: e.message}) }
}
}
}
function blocker (ms) {
var now = new Date().getTime();
while(true) {
if (new Date().getTime() > now +ms)
return;
}
}
setInterval(()=> console.log("Not blocked " + Math.random()), 1000)
console.log("\n\nstarting blocking code in Worker\n\n")
var worker = new GenericWebWorker(100, new Date(), blocker)
worker.exec((num, date, fnBlocker) => {
fnBlocker(7000) //All of this run in backgrownd
return num*10
})
.then(d => console.log(`\n\nEnd of blocking code: result ${d}\n\n`)) //Print 1000
Solution 24 - Javascript
You can place the contents of your worker.js file inside backticks (which allows a multiline string constant) and create the worker from a blob like this:
var workerScript = `
self.onmessage = function(e) {
self.postMessage('message from worker');
};
// rest of worker code goes here
`;
var worker =
new Worker(createObjectURL(new Blob([workerScript], { type: "text/javascript" })));
This is handy if for whatever reason you don't want to have separate script tags for the worker.
Solution 25 - Javascript
Another solution is just to wrap the Worker in a function, then creating a blob invoking the function like so:
function workerCode() {
self.onmessage = function (e) {
console.log("Got message from parent", e.data);
};
setTimeout(() => {
self.postMessage("Message From Worker");
}, 2000);
}
let blob = new Blob([
"(" + workerCode.toString() + ")()"
], {type: "text/javascript"});
// Note: window.webkitURL.createObjectURL() in Chrome 10+.
let worker = new Worker(window.URL.createObjectURL(blob));
worker.onmessage = function (e) {
console.log("Received: " + e.data);
};
worker.postMessage("hello"); // Start the worker.
Solution 26 - Javascript
One-liner for running functions in workers:
const FunctionalWorker = fn => new Worker(window.URL.createObjectURL(new Blob(["(" + fn.toString() + ")()"], {type: "text/javascript"})));
Example usage:
let fn = FunctionalWorker(() => {
self.postMessage("hi");
});
fn.onmessage = msg => {
console.log(msg);
};
Solution 27 - Javascript
there have been a few answers, but here is another inline version.
note: "self" argument is purely cosmetic for linting purposes, actual worker code starts after first brace, self is as normal
inlineWorker (
"hello world",// initial message to send to worker
function(self){
// inline worker code.
self.onmessage = function (e) {
self.postMessage("thinking...");
for (var i=0;i<100000000;i++) {
var r = Math.random();
}
self.postMessage(e.data.toUpperCase());
}
},function(e){
// optional message handler
document.getElementById("log").innerHTML= "from worker:"+e.data;
});
function inlineWorker (msg,fn,onMsg) {
var
w=window,
U=!!w.webkitURL?w.webkitURL:w.URL,
src=fn.toString(),
s=src.indexOf('{'),
e=src.lastIndexOf('}'),
worker = new Worker(U.createObjectURL(
new Blob([ src.substring(s+1,e-1) ], { type: "text/javascript" })
));
if (typeof onMsg==="function") {
worker.addEventListener("message",onMsg);
}
if (msg) {
worker.postMessage(msg);
}
return worker;
}
<div id="log"></div>
Solution 28 - Javascript
I liked the answer that ifbamoq gave but was not able to comment because of stack overflow's stupid points policy. Therefor I'll give an example that shows some intensive work being done - and how it does not lock the main thread.
All without running into CORS problems with the null origin - if you're like me and like double-clicking the html files and treating them like little programs. :-)
<!DOCTYPE html>
<html>
<head>
<title>Worker example: One-core computation</title>
</head>
<body>
<p>The highest prime number discovered so far is: <div id="result"></div></p>
</body>
<script>
// let worker = new Worker('WebWorker.js'); // lets skip this to avoid null origin issues
let WorkerFn = (event) =>
{
let isPrime = false;
for (let n = 2; n <= 1_000_000; n++)
{
isPrime = true;
for(let i = 2; i <= Math.sqrt(n); i++)
if (n % i == 0)
isPrime = false; // If you can get thru all this shit and survive, ur prime!
if (isPrime)
postMessage(n);
}
}
let worker = new Worker(window.URL.createObjectURL(new Blob(["(" + WorkerFn.toString() + ")()"], {type: "text/javascript"})));
worker.onmessage = (event) =>
{
result.innerHTML = event.data;
}
</script>
</html>
Solution 29 - Javascript
This is a bit off topic answer, but ... you may not require the use of web workers to deal with long jobs on browsers.
Let's suppose you want to run several time an heavy computing (like you would do over an array) :
const heavyFct = () => {let i = 0; while(i<1e8) {++i}}
for (let i = 0; i < 100; ++i) {
heavyFct();
}
This should freeze your browser.
To avoid this we can rely on setTimeout this way:
const desync = (i = 0) => {
if (i >= 100) {return}
heavyFct();
setTimeout(() => desync(i + 1), 0);
}
desync();
Now you have your heavy computing running without freezing you computer