NodeJS foreach loop with setInterval

Question

I'm trying to fetch data from array every two seconds and then send the request to the twitter API to follow the user. The problem is that after I run the script it instantly throws me all 400 users and then after two seconds another one is being thrown. How can I fix this so I see every user followed after two seconds and not see all those 400 thrown at the script start?

const params = {screen_name: 'user'};
client.get('friends/ids', params, function(error, tweets, response) {
  if (!error) {
      const body = JSON.parse(response.body);
      body.ids.slice(0, 400).forEach((element) => 
             setInterval( () => {
                client.post('friendships/create', {user_id: element}, function(error, tweets, response){
                    console.log(`User ${element} followed.`);
                })
            }, 2000)
        );
  }
});

you can check it with response.code under if condition alongside !error — Ashish
– Ashish, Commented Feb 4, 2020 at 15:40
You're setting up 400 intervals and starting them at the same time, not an interval that generates a new POST after the previous POST has completed. Perhaps I misunderstood your question. If you want 400 intervals, but don't want the initial 400 logged then you could have a first-get flag of similar. — JDunken
– JDunken, Commented Feb 4, 2020 at 15:53
Are you trying to POST or fetch data every 2 seconds? If you, as you wrote, need to fetch data every two seconds, the interval should be around the client.get() function. if you are looking to post one user_id at a time, you can accomplish it by making an interval loop let i = 0; let arr = body.ids.slice(0,400); setInterval(()=> { console.log(arr[i]); i = i+1; }, 2000) Of course you need to assing the interval to be able to stop it once you've iterated through all the ids — Frederik Sohnis
– Frederik Sohnis, Commented Feb 4, 2020 at 16:02

Frederik Sohnis · Accepted Answer · 2020-02-04 16:08:26Z

If you want to avoid using any libraries:

setInterval will wait 2 seconds to handle the script it contains but the rest of the code will keep going. This means the forEach loop will execute 400 times and set 400 different intervals of 2 seconds, which will end around the same time.

What you could use instead is something like:

const delayLoop(arr, delay, incr = 0) {
    if (incr < arr.length) {
        incr = incr + 1;
        setTimeout( () => {
            client.post('friendships/create', {user_id: arr[incr]}, function(error, tweets, response){
                console.log(`User ${arr[incr]} followed.`);
            })
            delayLoop(arr, delay, incr);
        }, delay)
    }
}

client.get('friends/ids', params, function(error, tweets, response) {
  if (!error) {
      const body = JSON.parse(response.body);
      let idArray = body.ids.slice(0, 400);
      delayLoop(idArray, 2000, 0);
  }
});

Note: I just wrote this off the top of my head so it may have some errors if you try and run it but you get the gist of it. You essentially create a recursive loop that iterates through your data. It can be made even more generic than this, or less so if you want the delay to be fixed.

This code will fetch all 400 users instantly an then 2 seconds to post each user, 800 seconds in total. If what you want is to "get" the users every 2 seconds, you should place the get request in a setTimeout() or setInterval()

Denis Tambovchanin · Accepted Answer · 2020-02-04 15:57:16Z

0

You can use libraries for this like Async or Bluebird

Or can do it like this:

const params = {
  screen_name: 'user'
};

function createFriendship(user_id, callback) {
  client.post('friendships/create', { user_id }, callback);
}

function processUsers(ids) {
  const userId = ids.shift();

  createFriendship(userId, (error, tweets, response) => {
    if (error) {
      /// Handle error here
    }

    console.log(`User ${userId} followed.`);

    if (ids.length) setTimeout(() => processUsers(ids), 2000);
  })
}

client.get('friends/ids', params, function (error, tweets, response) {
  if (!error) {
    const body = JSON.parse(response.body);

    processUsers(body.ids.slice(0, 400))
  }
});

answered Feb 4, 2020 at 15:57

Denis Tambovchanin

5683 silver badges12 bronze badges

Comments

Felix Lemke · Accepted Answer · 2020-02-05 07:41:25Z

The problem here is, that you register all async-tasks with the setInterval function on the macro-thread at the same time. Because they all have a delay of 2 seconds you'll see them all execute after this time.

id1: x-----------t-----------t----->
id2: x-----------t-----------t----->
...
400
     ^           ^           ^
     |           |           here all tasks are executed again
     |           here all tasks are executed
     here you register all intervals

In order to emit users one by one with a 2 seconds delay you could make use of RxJS.

import { of, interval } from 'rxjs';
import { delay, tap, takeUntil } from 'rxjs/operators';

client.get('friends/ids', params, (error, tweets, response) => {
  if (error) {
    return;
  }
  const body = JSON.parse(response.body);
  const interval$ = interval(2000);
  const maxReached$ = interval$.pipe(filter(index => index > 400));

  zip(interval$, of(...body.ids)).pipe(
    takeUntil(maxReached$),
    tap(([_, id]) => {
      client.post('friendships/create', {user_id: element}, () => console.log('err'))
    })
  ).subscribe();
});

You can even go further and create your own observable from the client.get function. This makes your code very clean. See the stackblitz.

Collectives™ on Stack Overflow

NodeJS foreach loop with setInterval

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related