How to loop in node js, wait until each iteration

Question

I have a function:

var third = function(classes){
  for (var i = 0; i <= (classes.length-1); i++) {
       var Myurl = classes[i];

       return function(Myurl){
          request(Myurl,  function(err, resp, body) { 
             if (!err && resp.statusCode == 200) {
                var $ = cheerio.load(body);
                $("#item_details dl").each(function() {
                   var Values = [];
                   var New = [];

                   Values=$(this).find("dd strong").text();

                   New = Values.replace(/[\n\t\r]/g,"");
                   AllLinks.push(New);
                });

                console.log(AllLinks);
             };
          })
       }(MyUrl);

  };
};

The problem is when I do the above I only get the result of first loop element (i=0) in the console.log(AllLinks). How do I properly loop in node? I am new to node so any comments are much appreciated!

EDIT: If I define AllLinks in request then it seems to work, but not in correct order...

var third = function(classes){
      for (var i = 0; i <= (classes.length-1); i++) {
           var Myurl = classes[i];

            (function(Myurl){
              request(Myurl,  function(err, resp, body) { 
                 if (!err && resp.statusCode == 200) {
                    var $ = cheerio.load(body);
                    AllLinks=[];
                    $("#item_details dl").each(function() {
                       var Values = [];
                       var New = [];

                       Values=$(this).find("dd strong").text();

                       New = Values.replace(/[\n\t\r]/g,"");
                       AllLinks.push(New);
                    });

                    console.log(AllLinks);
                 }(Myurl);
              })
           };

      };
    };

@CallumLinington Thanks, I am a node beginner, so you maybe can provide an example about how the above function should be changed? best regards — user1665355
– user1665355, Commented May 26, 2015 at 10:02
Why do you want to return call result return function(Myurl){...}(MyUrl);? In certain scenarios it might be enough. Otherwise you might want to try promises or async library — Igor Popov
– Igor Popov, Commented May 26, 2015 at 10:03
I'm not sure how you implement request but if you drop 'return' you'll most likely be able to call all your urls from classes. If you want to wait for all calls to succeed async.js sounds like your choice. — Igor Popov
– Igor Popov, Commented May 26, 2015 at 10:09

Community · Accepted Answer · 2017-05-23 12:16:03Z

4

The major problem (apart from 'return') is that assuming request performs asyncronous operation your function returns when request is not completed and thus log contains no update.

You have two strategies in general:

call your code without 'return' as in the example above. This case all your requests will be eventually done but you have no control over when. It works well when you don't need them all to proceed, e.g. process AllLinks afterwards
Use any technology that supports waiting until all your calls are completed (async.js or promises). For example here Simplest way to wait some asynchronous tasks complete, in Javascript?

Thus you need:

function appendResultToItems(url, callback) {
 request(url,  function(err, resp, body) { 
   if (!err && resp.statusCode == 200) {
     var $ = cheerio.load(body);
     $("#item_details dl").each(function() {
       var Values = [];
       var New = [];

       Values=$(this).find("dd strong").text();

       New = Values.replace(/[\n\t\r]/g,"");
       AllLinks.push({result:New, url: url});
       callback();
     });
 });
}

var calls = [];

classes.forEach(function(Myurl){
  calls.push(function(callback) {
    appendResultToItems(Myurl, callback);
  });
});

async.parallel(calls, function() {
  console.log(AllLinks);
});

edited May 23, 2017 at 12:16

CommunityBot

11 silver badge

answered May 26, 2015 at 10:36

Igor Popov

2,16417 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

user1665355 Over a year ago

Great answer, please see my edit. Is that what you mean, the edit code works but AllLinks are not in correct order...

Igor Popov Over a year ago

You can't have them in order this case. What you can do is to write results by url or classes index and then reorder them

user1665355 Over a year ago

Btw, what is "callback" function in the example above?

Igor Popov Over a year ago

It's the most simple way of notifying the function has completed. Normally it is called with result and/or error as parameters

user1665355 Over a year ago

So the callback is asyncsCallback you mean?

|

Sergey Grigoriev · Accepted Answer · 2015-05-26 10:03:34Z

1

Use async.js#eachSeries to apply an asynchronous function to each element of a collection

answered May 26, 2015 at 10:03

Sergey Grigoriev

7197 silver badges15 bronze badges

2 Comments

user1665355 Over a year ago

thanks, but do you have an example of how the above function should be changed? I am a node beginner...

vijay Over a year ago

@user1665355 on async.js you will find more than examples

AmmarCSE · Accepted Answer · 2015-05-26 10:13:32Z

1

Your problem is you are using return within the loop. If you simply use an IIFE

var third = function(classes){
  for (var i = 0; i <= (classes.length-1); i++) {
       var Myurl = classes[i];

       (function(Myurl){
          request(Myurl,  function(err, resp, body) { 
             if (!err && resp.statusCode == 200) {
                var $ = cheerio.load(body);
                $("#item_details dl").each(function() {
                   var Values = [];
                   var New = [];

                   Values=$(this).find("dd strong").text();

                   New = Values.replace(/[\n\t\r]/g,"");
                   AllLinks.push(New);
                });

                console.log(AllLinks);
             };
          })
       })(MyUrl);

  };
};

AllLinks will be correctly generated.

answered May 26, 2015 at 10:13

AmmarCSE

30.7k6 gold badges48 silver badges53 bronze badges

1 Comment

user1665355 Over a year ago

that does not give the correct results unfortunately! AllLinks are not correct...

Collectives™ on Stack Overflow

How to loop in node js, wait until each iteration

3 Answers 3

8 Comments

2 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

8 Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related