Promise Retry Design Patterns
Javascriptnode.jsPromiseJavascript Problem Overview
Edit
- Pattern that keep on retrying until the promise resolves (with
delayandmaxRetries). - Pattern that keeps on retrying until the condition
meets on the result (with
delayandmaxRetries). - A memory efficient dynamic Pattern with unlimited retries (
delayprovided).
Code for #1. Keeps on retrying until promise resolves (any improvements community for the language etc?)
Promise.retry = function(fn, times, delay) {
return new Promise(function(resolve, reject){
var error;
var attempt = function() {
if (times == 0) {
reject(error);
} else {
fn().then(resolve)
.catch(function(e){
times--;
error = e;
setTimeout(function(){attempt()}, delay);
});
}
};
attempt();
});
};
Use
work.getStatus()
.then(function(result){ //retry, some glitch in the system
return Promise.retry(work.unpublish.bind(work, result), 10, 2000);
})
.then(function(){console.log('done')})
.catch(console.error);
Code for #2 keep on retrying until a condition meets on the then result in a reusable way (condition is what will vary).
work.publish()
.then(function(result){
return new Promise(function(resolve, reject){
var intervalId = setInterval(function(){
work.requestStatus(result).then(function(result2){
switch(result2.status) {
case "progress": break; //do nothing
case "success": clearInterval(intervalId); resolve(result2); break;
case "failure": clearInterval(intervalId); reject(result2); break;
}
}).catch(function(error){clearInterval(intervalId); reject(error)});
}, 1000);
});
})
.then(function(){console.log('done')})
.catch(console.error);
Javascript Solutions
Solution 1 - Javascript
Something a bit different ...
Async retries can be achieved by building a .catch() chain, as opposed to the more usual .then() chain.
This approach is :
- only possible with a specified maximum number of attempts. (The chain must be of finite length),
- only advisable with a low maximum. (Promise chains consume memory roughly proportional to their length).
Otherwise, use a recursive solution.
First, a utility function to be used as a .catch() callback.
var t = 500;
function rejectDelay(reason) {
return new Promise(function(resolve, reject) {
setTimeout(reject.bind(null, reason), t);
});
}
Now you can build .catch chains very concisely :
1. Retry until the promise resolves, with delay
var max = 5;
var p = Promise.reject();
for(var i=0; i<max; i++) {
p = p.catch(attempt).catch(rejectDelay);
}
p = p.then(processResult).catch(errorHandler);
DEMO: https://jsfiddle.net/duL0qjqe/
2. Retry until result meets some condition, without delay
var max = 5;
var p = Promise.reject();
for(var i=0; i<max; i++) {
p = p.catch(attempt).then(test);
}
p = p.then(processResult).catch(errorHandler);
DEMO: https://jsfiddle.net/duL0qjqe/1/
3. Retry until result meets some condition, with delay
Having got your mind round (1) and (2), a combined test+delay is equally trivial.
var max = 5;
var p = Promise.reject();
for(var i=0; i<max; i++) {
p = p.catch(attempt).then(test).catch(rejectDelay);
// Don't be tempted to simplify this to `p.catch(attempt).then(test, rejectDelay)`. Test failures would not be caught.
}
p = p.then(processResult).catch(errorHandler);
test() can be synchronous or asynchronous.
It would also be trivial to add further tests. Simply sandwich a chain of thens between the two catches.
p = p.catch(attempt).then(test1).then(test2).then(test3).catch(rejectDelay);
DEMO: https://jsfiddle.net/duL0qjqe/3/
All versions are designed for attempt to be a promise-returning async function. It could also conceivably return a value, in which case the chain would follow its success path to the next/terminal .then().
Solution 2 - Javascript
2. Pattern that keeps on retrying until the condition meets on the result (with delay and maxRetries)
This is an nice way to do this with native promises in a recursive way:
const wait = ms => new Promise(r => setTimeout(r, ms));
const retryOperation = (operation, delay, retries) => new Promise((resolve, reject) => {
return operation()
.then(resolve)
.catch((reason) => {
if (retries > 0) {
return wait(delay)
.then(retryOperation.bind(null, operation, delay, retries - 1))
.then(resolve)
.catch(reject);
}
return reject(reason);
});
});
This is how you call it, assuming that func sometimes succeeds and sometimes fails, always returning a string that we can log:
retryOperation(func, 1000, 5)
.then(console.log)
.catch(console.log);
Here we're calling retryOperation asking it to retry every second and with max retries = 5.
If you want something simpler without promises, RxJs would help with that: https://github.com/Reactive-Extensions/RxJS/blob/master/doc/api/core/operators/retrywhen.md
Solution 3 - Javascript
There are many good solutions mentioned and now with async/await these problems can be solved without much effort.
If you don't mind a recursive approach then this is my solution.
function retry(fn, retries=3, err=null) {
if (!retries) {
return Promise.reject(err);
}
return fn().catch(err => {
return retry(fn, (retries - 1), err);
});
}
Solution 4 - Javascript
You can chain a new promise onto the prior one, thus delaying its eventual resolution until you know the final answer. If the next answer still isn't known, then chain another promise on it and keep chaining checkStatus() to itself until eventually you know the answer and can return the final resolution. That could work like this:
function delay(t) {
return new Promise(function(resolve) {
setTimeout(resolve, t);
});
}
function checkStatus() {
return work.requestStatus().then(function(result) {
switch(result.status) {
case "success":
return result; // resolve
case "failure":
throw result; // reject
case default:
case "inProgress": //check every second
return delay(1000).then(checkStatus);
}
});
}
work.create()
.then(work.publish) //remote work submission
.then(checkStatus)
.then(function(){console.log("work published"})
.catch(console.error);
Note, I also avoided creating the promise around your switch statement. Since you're already in a .then() handler, just returning a value is resolve, throwing an exception is reject and returning a promise is chaining a new promise onto the prior one. That covers the three branches of your switch statement without creating a new promise in there. For convenience, I do use a delay() function that is promise based.
FYI, this assumes the work.requestStatus() doesn't need any arguments. If it does need some specific arguments, you can pass those at the point of the function call.
It might also be a good idea to implement some sort of timeout value for how long you will loop waiting for completion so this never goes on forever. You could add the timeout functionality like this:
function delay(t) {
return new Promise(function(resolve) {
setTimeout(resolve, t);
});
}
function checkStatus(timeout) {
var start = Date.now();
function check() {
var now = Date.now();
if (now - start > timeout) {
return Promise.reject(new Error("checkStatus() timeout"));
}
return work.requestStatus().then(function(result) {
switch(result.status) {
case "success":
return result; // resolve
case "failure":
throw result; // reject
case default:
case "inProgress": //check every second
return delay(1000).then(check);
}
});
}
return check;
}
work.create()
.then(work.publish) //remote work submission
.then(checkStatus(120 * 1000))
.then(function(){console.log("work published"})
.catch(console.error);
I'm not sure exactly what "design pattern" you're looking for. Since you seem to object to the externally declared checkStatus() function, here's an inline version:
work.create()
.then(work.publish) //remote work submission
.then(work.requestStatus)
.then(function() {
// retry until done
var timeout = 10 * 1000;
var start = Date.now();
function check() {
var now = Date.now();
if (now - start > timeout) {
return Promise.reject(new Error("checkStatus() timeout"));
}
return work.requestStatus().then(function(result) {
switch(result.status) {
case "success":
return result; // resolve
case "failure":
throw result; // reject
case default:
case "inProgress": //check every second
return delay(1000).then(check);
}
});
}
return check();
}).then(function(){console.log("work published"})
.catch(console.error);
A more reusable retry scheme that could be used in many circumstances would define some reusable external code, but you seem to object to that so I haven't made that version.
Here's one other approach that uses a .retryUntil() method on the Promise.prototype per your request. If you want to tweak implementation details of this, you should be able to modify this general approach:
// fn returns a promise that must be fulfilled with an object
// with a .status property that is "success" if done. Any
// other value for that status means to continue retrying
// Rejecting the returned promise means to abort processing
// and propagate the rejection
// delay is the number of ms to delay before trying again
// no delay before the first call to the callback
// tries is the max number of times to call the callback before rejecting
Promise.prototype.retryUntil = function(fn, delay, tries) {
var numTries = 0;
function check() {
if (numTries >= tries) {
throw new Error("retryUntil exceeded max tries");
}
++numTries;
return fn().then(function(result) {
if (result.status === "success") {
return result; // resolve
} else {
return Promise.delay(delay).then(check);
}
});
}
return this.then(check);
}
if (!Promise.delay) {
Promise.delay = function(t) {
return new Promise(function(resolve) {
setTimeout(resolve, t);
});
}
}
work.create()
.then(work.publish) //remote work submission
.retryUntil(function() {
return work.requestStatus().then(function(result) {
// make this promise reject for failure
if (result.status === "failure") {
throw result;
}
return result;
})
}, 2000, 10).then(function() {
console.log("work published");
}).catch(console.error);
I still can't really tell what you want or what about all these approaches is not solving your issue. Since your approaches seem to all be all inline code and not using a resuable helper, here's one of those:
work.create()
.then(work.publish) //remote work submission
.then(function() {
var tries = 0, maxTries = 20;
function next() {
if (tries > maxTries) {
throw new Error("Too many retries in work.requestStatus");
}
++tries;
return work.requestStatus().then(function(result) {
switch(result.status) {
case "success":
return result;
case "failure":
// if it failed, make this promise reject
throw result;
default:
// for anything else, try again after short delay
// chain to the previous promise
return Promise.delay(2000).then(next);
}
});
}
return next();
}).then(function(){
console.log("work published")
}).catch(console.error);
Solution 5 - Javascript
Here is an "exponential backoff" retry implementation using async/await that can wrap any promise API.
note: for demonstration reasons snippet simulates a flaky endpoint with Math.random, so try a few times to see both success and failure cases.
/**
* Wrap a promise API with a function that will attempt the promise over and over again
* with exponential backoff until it resolves or reaches the maximum number of retries.
* - First retry: 500 ms + <random> ms
* - Second retry: 1000 ms + <random> ms
* - Third retry: 2000 ms + <random> ms
* and so forth until maximum retries are met, or the promise resolves.
*/
const withRetries = ({ attempt, maxRetries }) => async (...args) => {
const slotTime = 500;
let retryCount = 0;
do {
try {
console.log('Attempting...', Date.now());
return await attempt(...args);
} catch (error) {
const isLastAttempt = retryCount === maxRetries;
if (isLastAttempt) {
// Stack Overflow console doesn't show unhandled
// promise rejections so lets log the error.
console.error(error);
return Promise.reject(error);
}
}
const randomTime = Math.floor(Math.random() * slotTime);
const delay = 2 ** retryCount * slotTime + randomTime;
// Wait for the exponentially increasing delay period before retrying again.
await new Promise(resolve => setTimeout(resolve, delay));
} while (retryCount++ < maxRetries);
}
const fakeAPI = (arg1, arg2) => Math.random() < 0.25 ? Promise.resolve(arg1) : Promise.reject(new Error(arg2))
const fakeAPIWithRetries = withRetries({ attempt: fakeAPI, maxRetries: 3 });
fakeAPIWithRetries('arg1', 'arg2').then(results => console.log(results))
Solution 6 - Javascript
Check @jsier/retrier. Tested, documented, lightweight, easy-to-use, without external dependencies and already in production for quite some time now.
Supports:
- First attempt delay
- Delay between attempts
- Limiting number of attempts
- Callback to stop retrying if some condition is met (e.g. specific error is encountered)
- Callback to keep retrying if some condition is met (e.g. resolved value is unsatisfactory)
Installation:
npm install @jsier/retrier
Usage:
import { Retrier } from '@jsier/retrier';
const options = { limit: 5, delay: 2000 };
const retrier = new Retrier(options);
retrier
.resolve(attempt => new Promise((resolve, reject) => reject('Dummy reject!')))
.then(
result => console.log(result),
error => console.error(error) // After 5 attempts logs: "Dummy reject!"
);
The package has no external dependencies.
Solution 7 - Javascript
Building on the solution by holmberd with a little cleaner code and a delay
// Retry code
const wait = ms => new Promise((resolve) => {
setTimeout(() => resolve(), ms)
})
const retryWithDelay = async (
fn, retries = 3, interval = 50,
finalErr = Error('Retry failed')
) => {
try {
await fn()
} catch (err) {
if (retries <= 0) {
return Promise.reject(finalErr);
}
await wait(interval)
return retryWithDelay(fn, (retries - 1), interval, finalErr);
}
}
// Test
const getTestFunc = () => {
let callCounter = 0
return async () => {
callCounter += 1
if (callCounter < 5) {
throw new Error('Not yet')
}
}
}
const test = async () => {
await retryWithDelay(getTestFunc(), 10)
console.log('success')
await retryWithDelay(getTestFunc(), 3)
console.log('will fail before getting here')
}
test().catch(console.error)
Solution 8 - Javascript
If your code is placed in a class you could use a decorator for that. You have such decorator in the utils-decorators (npm install --save utils-decorators) lib:
import {retry} from 'utils-decorators';
class SomeService {
@retry(3)
doSomeAsync(): Promise<any> {
....
}
}
or you could use a wrapper function:
import {retryfy} from 'utils-decorators';
const withRetry = retryfy(originalFunc, 3);
Note: this lib is tree shakable so you won't pay extra bytes for the rest of the available decorators in this lib.
Solution 9 - Javascript
There are plenty answers here, but after some research i decided to go with a recursive approach. Im leaving my solution here for any one interested
function retry(fn, retriesLeft = 2, interval = 1000) {
return new Promise((resolve, reject) => {
fn()
.then(resolve)
.catch((error) => {
if (retriesLeft === 0) {
reject(error);
return;
}
setTimeout(() => {
console.log('retrying...')
retry(fn, retriesLeft - 1, interval).then(resolve).catch(reject);
}, interval);
});
});
}
Here is a stackblitz with a nice playground where you can get the feel on how it works. Just play around the intent variable to see the promise resolve/reject
Solution 10 - Javascript
Here's my attempt. I tried to take what I liked from all of the above answers. No external dependencies. Typescript + async / await (ES2017)
export async function retryOperation<T>(
operation: () => (Promise<T> | T), delay: number, times: number): Promise<T> {
try {
return await operation();
} catch (ex) {
if (times > 1) {
await new Promise((resolve) => setTimeout(resolve, delay));
return retryOperation(operation, delay, times - 1);
} else {
throw ex;
}
}
}
Usage:
function doSomething() {
return Promise.resolve('I did something!');
}
const retryDelay = 1000; // 1 second
const retryAttempts = 10;
retryOperation(doSomething(), retryDelay, retryAttempts)
.then((something) => console.log('I DID SOMETHING'))
.catch((err) => console.error(err));
Solution 11 - Javascript
Not sure why all the solutions proposed are recursive. An iterative solution with TypeScript that waits until the method returns something that is not undefined:
function DelayPromise(delayTime): Promise<void> {
return new Promise<void>((resolve) => setTimeout(resolve, delayTime));
}
interface RetryOptions {
attempts?: number;
delayMs?: number;
}
export async function retryOperation<T>(
operation: (attempt: number) => Promise<T>,
options: RetryOptions = {}
): Promise<T> {
const { attempts = 6, delayMs = 10000 } = options;
for (let i = 0; i < attempts; i++) {
const result = await operation(i);
if (typeof result !== 'undefined') {
return result;
}
await DelayPromise(delayMs);
}
throw new Error('Timeout');
}
Solution 12 - Javascript
async-retry.ts is trying to implement the pattern, I'm using it in production for some projects.
Installation:
npm install async-retry.ts --save
Usage:
import Action from 'async-retry.ts'
const action = async()=>{}
const handlers = [{
error: 'error1',
handler: async yourHandler1()=>{}
}, {
error: 'error2',
handler: async yourHandler2()=>{}
}]
await Action.retryAsync(action, 3, handlers)
This package is quite new but it is derived from a long lived package co-retry which implemented the retry pattern in generator function fashion.
Solution 13 - Javascript
function TryToSuccess(fun, reties) {
let attempt = 0;
let doTry = (...args) => {
attempt++;
return fun(...args)
.catch((err) => {
console.log("fail ", attempt);
if(attempt <= reties){
return doTry(...args);
} else {
return Promise.reject(err);
}
});
}
return doTry;
}
function asyncFunction(){
return new Promise((resolve, reject) => {
setTimeout(() => {
(window.findResult === true) ? resolve("Done") : reject("fail");
}, 2000);
});
}
var cloneFunc = TryToSuccess(asyncFunction, 3);
cloneFunc()
.then(res => {
console.log("Got Success. ", res)
})
.catch(err => {
console.log("Rejected with err ", err);
});
setTimeout(() => {
window.findResult = true;
}, 4000);
Solution 14 - Javascript
work.create()
.then(work.publish) //remote work submission
.then(function(result){
var maxAttempts = 10;
var handleResult = function(result){
if(result.status === 'success'){
return result;
}
else if(maxAttempts <= 0 || result.status === 'failure') {
return Promise.reject(result);
}
else {
maxAttempts -= 1;
return (new Promise( function(resolve) {
setTimeout( function() {
resolve(_result);
}, 1000);
})).then(function(){
return work.requestStatus().then(handleResult);
});
}
};
return work.requestStatus().then(handleResult);
})
.then(function(){console.log("work published"})
.catch(console.error);
Solution 15 - Javascript
One library can do this easily : promise-retry.
Here are some examples to test it :
const promiseRetry = require('promise-retry');
Expect second attempt to be successful :
it('should retry one time after error', (done) => {
const options = {
minTimeout: 10,
maxTimeout: 100
};
promiseRetry((retry, number) => {
console.log('test2 attempt number', number);
return new Promise((resolve, reject) => {
if (number === 1) throw new Error('first attempt fails');
else resolve('second attempt success');
}).catch(retry);
}, options).then(res => {
expect(res).toBe('second attempt success');
done();
}).catch(err => {
fail(err);
});
});
Expect only one retry :
it('should not retry a second time', (done) => {
const options = {
retries: 1,
minTimeout: 10,
maxTimeout: 100
};
promiseRetry((retry, number) => {
console.log('test4 attempt number', number);
return new Promise((resolve, reject) => {
if (number <= 2) throw new Error('attempt ' + number + ' fails');
else resolve('third attempt success');
}).catch(retry);
}, options).then(res => {
fail('Should never success');
}).catch(err => {
expect(err.toString()).toBe('Error: attempt 2 fails');
done();
});
});
Solution 16 - Javascript
My solution for TypeScript:
export const wait = (milliseconds: number): Promise<void> =>
new Promise(resolve => {
setTimeout(() => resolve(), milliseconds);
});
export const retryWithDelay = async (
fn,
retries = 3,
interval = 300
): Promise<void> =>
fn().catch(async error => {
if (retries <= 0) {
return Promise.reject(error);
}
await wait(interval);
return retryWithDelay(fn, retries - 1, interval);
});
Based on solutions above, fixed milliseconds for wait since it would default to 50 seconds instead of ms and now throws the error that caused the failure instead of a hardcoded mesasge.
Solution 17 - Javascript
I give you an async/await solution, have fun with it :)
async function scope() {
/* Performs an operation repeatedly at a given frequency until
it succeeds or a timeout is reached and returns its results. */
async function tryUntil(op, freq, tout) {
let timeLeft = tout;
while (timeLeft > 0) {
try {
return op();
} catch (e) {
console.log(timeLeft + " milliseconds left");
timeLeft -= freq;
}
await new Promise((resolve) => setTimeout(() => resolve(), freq));
}
throw new Error("failed to perform operation");
}
function triesToGiveBig() {
const num = Math.random();
if (num > 0.95) return num;
throw new Error();
}
try {
console.log(await tryUntil(triesToGiveBig, 100, 1000));
} catch (e) {
console.log("too small :(");
}
}
scope();
Solution 18 - Javascript
Just in case somebody is looking for a more generic solution. Here are my two cents:
Helper Function:
/**
* Allows to repeatedly call
* an async code block
*
* @callback callback
* @callback [filterError] Allows to differentiate beween different type of errors
* @param {number} [maxRetries=Infinity]
*/
function asyncRetry(
callback,
{ filterError = (error) => true, maxRetries = Infinity } = {}
) {
// Initialize a new counter:
let tryCount = 0;
// Next return an async IIFY that is able to
// call itself recursively:
return (async function retry() {
// Increment out tryCount by one:
tryCount++;
try {
// Try to execute our callback:
return await callback();
} catch (error) {
// If our callback throws any error lets check it:
if (filterError(error) && tryCount <= maxRetries) {
// Recursively call this IIFY to repeat
return retry();
}
// Otherwise rethrow the error:
throw error;
}
})();
}
Demo
Try 2 times:
await asyncRetry(async () => {
// Put your async code here
}, { maxRetries = 2 })
Try 2 times & only retry on DOMErrors:
await asyncRetry(async () => {
// Put your async code here
}, {
maxRetries = 2,
filterError: (error) => error instance of DOMError
})
Infine Retry: (Don't do this!)
await asyncRetry(async () => {
// Put your async code here
})
Solution 19 - Javascript
Simple Promise Retry :
function keepTrying(otherArgs, promise) {
promise = promise||new Promise();
// try doing the important thing
if(success) {
promise.resolve(result);
} else {
setTimeout(function() {
keepTrying(otherArgs, promise);
}, retryInterval);
}
}
Solution 20 - Javascript
Here is my solution:
- Preserve function type using Typescript.
- Accept a function with any parameters.
- Customize number of
maxRetries. - Customize delay behavior
type AnyFn = (...any: any[]) => any;
type Awaited<T> = T extends PromiseLike<infer U> ? U : T;
type DelayFn = (retry: number) => number;
const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
export function retry<Fn extends AnyFn>(
fn: Fn,
maxRetries: number,
getDelay: DelayFn = () => 5000
) {
let retries = 0;
return async function wrapped(
...args: Parameters<Fn>
): Promise<Awaited<ReturnType<Fn>>> {
try {
return await fn(...args);
} catch (e) {
if (++retries > maxRetries) throw e;
const delayTime = getDelay(retries);
console.error(e);
console.log(`Retry ${fn.name} ${retries} times after delaying ${delayTime}ms`);
await delay(delayTime);
return await wrapped(...args);
}
};
}
Usage
const badFn = () => new Promise((resolve, reject) => reject('Something is wrong');
const fn = retry(badFn, 5, (retry) => 2 ** retry * 1000);
fn();
// Something is wrong
// Retry badFn 1 times after delaying 2000ms
// Something is wrong
// Retry badFn 2 times after delaying 4000ms
// Something is wrong
// Retry badFn 3 times after delaying 8000ms
// Something is wrong
// Retry badFn 4 times after delaying 16000ms
// Something is wrong
// Retry badFn 5 times after delaying 32000ms
const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
function retry(fn, maxRetries, getDelay = () => 5000) {
let retries = 0;
return async function wrapped(...args) {
try {
return await fn(...args);
} catch (e) {
if (++retries > maxRetries) throw e;
const delayTime = getDelay(retries);
console.error(e);
console.log(`Retry ${fn.name} ${retries} times after delaying ${delayTime}ms`);
await delay(delayTime);
return await wrapped(...args);
}
};
}
const badFn = () => new Promise((resolve, reject) => reject('Something is wrong'));
const fn = retry(badFn, 5, (retry) => 2 ** retry * 1000);
fn();