2

I am wanting to learn how to run functions in parallel/concurrently in JavaScript. Over the past few days I've read several pages about promises, async functions, call backs and await, but I find most of the articles poorly written, unclear or the examples when run do no appear to be running in parallel, but I suspect just that the example used for example setTimout to defer a function call so the functions calls are not carried out in the order they appear in the code.

Here is what I mean when I say I want parallel processing. Lets say I had two functions that did a lot of work, in this example lets use counting, I will use small numbers but imagine if these numbers were very big.

var funcA = function(){ 
    for(var x = 0; x < 5;x++) {console.log("A:" + x}; 
    console.log("A done.");
}

var funcB = function(){ 
      for(var x = 0; x < 10;x++) {
      console.log("B:" +x}; console.log("B done.");
}

What I would hope to see if I were able to run these in parallel is somthing such as:

A:1
A:2
B:1
A:3
B:2
B:3
A:4
B:4
A:5
A Done.
B:5
B:6
B:7
B:8
B:9
B Done.

So I have been following an example at medium.com, and as is typical with these countless examples, they never show the processes doing any real work, and I think time outs are used to simulate it. The trouble is, its hard to see what timeouts are causing/assisting in parallel processing and which ones simulate work being done. With the example mentioned, if you make the program do some real work, you will see in fact that it is sequential (many examples I have looked at seem to be this way). The example below is based on the one from the medium.com article, but instead of using a timeout to simulate a long running task I am using a loop to count from 0 to a random number (I've left the timeout in but commented it out and instead replaced it with a for loop and call to the resolve function.This makes each call to makeRequest different in size and therefore the order of completion should change each time.

function makeRequest(letter) {
        return new Promise((resolve, reject) => {
            var rand = Math.random()* 999;
            for(var i = 0; i < rand; i++){
                   console.log(i + " " + letter);
            }
            resolve({'status':'done ' + letter});
            //setTimeout(() => resolve({ 'status': 'done '+letter }), 2000);
        });
}

async function process(arrayOfPromises) {
        console.time(`process`);    
        let responses = await Promise.all(arrayOfPromises);

        for(let r of responses) {console.log(r);}    
        console.timeEnd(`process`);    
        return;
}

async function handler() {
        let arrayOfPromises = [
           makeRequest('a'),
           makeRequest('b'),
           makeRequest('c'),
           makeRequest('d'),
           makeRequest('e'),
            ];
                                                        
       await process(arrayOfPromises);    
       console.log(`processing is complete`);
}

handler();

However, I can see that if I run the code, its not parallel/concurrent but sequential as in.

A:1
A:2
A:3
A:4
B:1
B:2
B:3
C:1
C:2
C:3
D:1
D:2
D:3
D:4
E:1
E:2
{status: 'done a'}
{status: 'done b'}
{status: 'done c'}
{status: 'done e'}
{status: 'done f'}

Questions:

  • a) What am I not getting about this?
  • b)What needs to be done to the above code so that JavaScript executes the method calls to makeRequest consurrently?
  • c) Is what I want even possible in JavaScript (given the single threaded execution of JavaScript and the event loop). It seems to me that really despite all these bloggers calling it parallel and concurrent (I know they are slightly different terms) it is in fact not!.
9
  • 2
    If you want true parallel execution, you'll need a separate thread - probably a web worker, eg stackoverflow.com/a/58262471 Commented Aug 22, 2020 at 19:27
  • Makerequest never gives up the execution context until its done. You need some settimeout or async generator to cause it to await and let other things execute instead. Commented Aug 22, 2020 at 19:38
  • @CertainPerformance That would be good if the procees you are running was insular, but what about something such as carrying out concurrent animation in two separate on screen objects such componentA and componentB which were not written by the same team and where each one takes a couple of seconds to do its thing? Commented Aug 22, 2020 at 19:40
  • @zero298 Can you give an example? Commented Aug 22, 2020 at 19:43
  • 1
    Amazing how well structured questions can get the best answers. Needed these ones, thanks Commented Jul 14, 2021 at 22:25

2 Answers 2

4

Trying to do real paralellism in a language that is synchronous and single-threaded in its very nature is ... ambitious. But not impossible ;)

All the tools you've found are the wrong ones. Promises, async/await (on top of Promises), and callbacks before that, they all solve the problem of "It'll take a while untill I receive an answer with this value, and since I'm single threaded, I don't want to stand around watch the clock and scratch my head, head, definitely head!"

None of them deal with "My boss wants me to split in two and be in two places at the same time/do two things at once."

Here come web worker into play. Worker allow you to execute some code on a different thread. Truth be told, I have little experience with them and can't just put you an example together, but at least I can point you in the right direction.

And eventually the worker will have finished its work and produce some kind of result, that's where we come back to Promises and co. To deal with the fact that on the main thread I'm waiting for some value that is computed "somewhere else" and that I'll eventually recieve. But I have no clue when this will be.

Sign up to request clarification or add additional context in comments.

2 Comments

Thanks for the thoughtful response. I wish all these bloggers would just quit these misleading posts. Its cost me a whole afternoon to figure out there is no real provision for concurrent method invocation in the classical and most useful sense other than the Worker which is only useful for self contained tasks. Its all just pseudo concurrency.
It is possible to run JavaScript functions on parallel hardware using compilers like gpu.js.
2

Like zero298 said, using Iterables/AsyncIterables to generate those messages might be the best way to go about it. A generator function, in my examples async* [Symbol.asyncIterator](); and * [Symbol.iterator]();, will allow for you to work with a only a small part of the total data that's being produced.

I've published an npm package that does something similar to what you're trying to do using AsyncIterables. This would be even more effective when you work with asynchronous tasks instead of just printing strings. I use this package for handling data from multiple filesystem crawlers in a single loop.

https://www.npmjs.com/package/iterable-joiner

https://github.com/cbroad/iterable-joiner

Here's some code that I believe is doing to what you were trying to do.

const { IterableJoiner } = require( "iterable-joiner" );

const printInHandler = false; // false means we will behave how the implementation in the question was written.

function makeRequest(letter) {
    return {
        async* [Symbol.asyncIterator]() {
            var rand = Math.random() * 999;
            for(var i = 0; i < rand; i++){

                // Original question printed to console here, I prefer yielding the value and printing it in handler().
                if( printInHandler ) {
                    yield i + " " + letter;
                } else {
                    yield;
                    console.log( i + " " + letter );
                }


            }
        }
    };
}


( async function handler() {
    let arrayOfIterables = [
       makeRequest('a'),
       makeRequest('b'),
       makeRequest('c'),
       makeRequest('d'),
       makeRequest('e'),
    ];
                
    const iterator = IterableJoiner.Async.Equitable.join( ...arrayOfIterables );

    for await ( const message of iterator ) {
        if( printInHandler ) {
            console.log( message );  // Use this if printing in handler()
        }
    }
   console.log(`processing is complete`);
} )();

In the output, you'll see that values for all five generators are being printed, but by the end, due to the random size, only one of them remained.

0 a
0 b
0 c
0 d
0 e
1 a
1 b
1 c
1 d
1 e
2 a
2 b
2 c
2 d
2 e
3 a
3 b
3 c

...

671 e
672 e
673 e
674 e
675 e
676 e
677 e
processing is complete

Here's the same thing with synchronous iterators. For my purposes, the ability to work with multiple asynchronous iterators is much more useful.

const { IterableJoiner } = require( "iterable-joiner" );

const printInHandler = false; // false means we will behave how the implementation in the question was written.

function makeRequest(letter) {
    return {
        * [Symbol.iterator]() {
            var rand = Math.random() * 999;
            for(var i = 0; i < rand; i++){

                // Original question printed to console here, I prefer yielding the value and printing it in handler().
                if( printInHandler ) {
                    yield i + " " + letter;
                } else {
                    yield;
                    console.log( i + " " + letter );
                }


            }
        }
    };
}


( async function handler() {
    let arrayOfIterables = [
       makeRequest('a'),
       makeRequest('b'),
       makeRequest('c'),
       makeRequest('d'),
       makeRequest('e'),
    ];
                
    const iterator = IterableJoiner.Sync.Equitable.join( ...arrayOfIterables );

    for( const message of iterator ) {
        if( printInHandler ) {
            console.log( message );  // Use this if printing in handler()
        }
    }
   console.log(`processing is complete`);
} )();

2 Comments

I just wanted to say this is awsome. :)
I'm glad to hear it. The iterable/async iterable patterns are really useful for handling things in a sane, predictable manner.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.