technical, work, javascript, promises
21:41 -0400

The nice thing about Node.js is its asynchronous execution model, which means that it can handle many requests very quickly. The flip side of this is that it can also generate many requests very quickly, which is fine if they can then be handled quickly, and not so good when they can't. For one application that I'm working on, some of the work gets offloaded to an external process; a new process is created for each request. (Unfortunately, that's the architecture that I'm stuck with for now.) And when doing a batch operation, Node.js will happily spawn hundreds of processes at once, without caring that doing so will cause everything on my workstation to slow to a crawl.

Limiting concurrency in Node.js has beenwritten about elsewhere, but I'd like to share my promise-based version of this solution. In particular, this was built for thebluebird flavour of promises.

Suppose that we have a function f that performs some task, and returns a promise that is fulfilled when that task in completed. We want to ensure that we don't have too many instances of f running at the same time.

We need to keep track of how many instances are currently running, we need a queue of instances when we've reached our limit, and of course we need to define what our limit is.

var queue = [];
var numRunning = 0;
var max = 10

Our queue will just contain functions that, when called, will call f with the appropriate arguments, as well as perform the record keeping necessary for calling f. So to process the queue, we just check whether we are below our run limit, check whether the queue is non-empty, and run the function at the front of the queue.

function runnext()
{
numRunning--;
if (numRunning <= max & queue.length)
{
queue.shift()();
}
}

Now we create a wrapper function f1 that will limit the concurrency off. We will call f with the same arguments that f1 is called with. If we have already reached our limit, we queue the request; otherwile, we runf immediately. When we run f, whether it is immediately, or in the future, we must first increment our counter. After f is done, we process the next element in the queue. We must process the queue whether fsucceeds or not, and we don't want to change the resolution of f's promise, so we tack a finally onto the promise returned by f.

function f1 ()
{
var args = Array.prototype.slice.call(arguments);
return new Promise(function (resolve, reject) {
function run() {
numRunning++;
resolve(f.apply(undefined, args)
.finally(runnext));
}
if (numRunning > max)
{
queue.push(run);
}
else
{
run();
}
});
}

Of course, if you need to do this a lot, you may want to wrap this all up in a higher-order function. For example:

function limit(f, max)
{
var queue = [];
var numRunning = 0;

function runnext()
{
numRunning--;
if (numRunning <= max & queue.length)
{
queue.shift()();
}
}

return function ()
{
var args = Array.prototype.slice.call(arguments);
return new Promise(function (resolve, reject) {
function run() {
numRunning++;
resolve(f.apply(undefined, args)
.finally(runnext));
}
if (numRunning > max)
{
queue.push(run);
}
else
{
run();
}
});
};
}

This would be used as:

f = limit(f, 10);

which would replace f with a new function that is equivalent to f, except that only 10 instances will be running at a time.

0 Comments