Best way to loop through an array and parse each element into several data structures

Question

I have an array of ~11,000 JavaScript dictionaries, each representing 1 row in an Excel file.

I want to loop through this array and parse each element into a new datastructure. For example, I might have a function that will count for {"foo": true} or something.

As I have multiple of these functions, my question is would it be better to loop through this array for each function, or have one single loop with functions that parse each element and store it in a global variable?

Ex. I'm currently doing one single loop, and parsing each element into a global variable

const arr = [...];  // array of ~11,000 dictionaries

// example parsing function
let count = 0;
function countFoos(el) {
  if (el["foo"] === true) count++;
}

let count2 = 0;
function countBars(el) {
  if (el["bar"] === false) count2++;
}

arr.forEach(el => {
  countFoos(el);
  countBars(el);
});

But would it be better to do it this way?

class Parse {
  constructor(arr) {
    this.arr = arr;
    this.count = 0;
    this.count2 = 0;
  }

  countFoos() {
    this.arr.forEach((el) => {
      if (el["foo"] === true) this.count++;
    });
  }

  countBars() {
    this.arr.forEach((el) => {
      if (el["bar"] === false) this.count2++;
    });
  }
}

const arr = [...]; // array of ~11,000 dictionaries
let x = Parse();
x.countFoos();
x.countBars();

EDIT: I should've clarified early, the examples shown above are just very simplified examples of the production code. Approximately 20 'parsing functions' are being run on for each element, with each of its corresponding global variables being large dictionaries or arrays.

You could use reduce for this, which would only require one pass — Dan
– Dan, Commented Aug 13, 2020 at 19:03

Barmar · Accepted Answer · 2020-08-13 18:57:35Z

1

You should generally do just one iteration that calls both functions.

Iterating takes time, so doing two iterations will double the time taken to perform the iterations. How significant this is to the entire application depends on how much work is done in the body of the iteration. If the bodies are very expensive, the iteration time might fall into the noise. But if it's really simple, as in your examples of a simple test and variable increment, the iteration time will probably be significant.

answered Aug 13, 2020 at 18:57

Barmar

789k57 gold badges555 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Unmitigated · Accepted Answer · 2020-08-13 19:00:20Z

0

If you are worried about performance, the first method is better as it only involves one iteration over the entire array while the second approach requires two.

If think using classes is more readable, you could simply put write that as one method in the class.

class Parse {
  constructor(arr) {
    this.arr = arr;
    this.count = 0;
    this.count2 = 0;
  }

  count() {
    this.arr.forEach((el) => {
      countFoos(el), countBars(el);
    });
  }
  countFoos(el){
    if(el.foo === true) this.count1++;
  }
  countBars() {
    if(el.bar === false) this.count2++;
  }
}

edited Aug 13, 2020 at 19:00

answered Aug 13, 2020 at 18:55

Unmitigated

91.5k12 gold badges103 silver badges109 bronze badges

2 Comments

urvianoob Over a year ago

Yup I'm worried about performance because it's scaling outwards as we add more data to the array / functions to parse that data. However I'm worried that the current implementation is not very readable.

Unmitigated Over a year ago

@urvianoob Updated my answer.

Dan · Accepted Answer · 2020-08-13 19:07:07Z

I would approach this by using the Array.prototype.reduce function, which would only require a single pass over the given array. I also would not use a class here as it would not really make sense, but you can if you really want!

function count(arr) {
    return arr.reduce(([fooCount, barCount], next) => {
        if (next.foo === true) {
            fooCount = fooCount + 1
        }

        if (next.bar === false) {
            barCount = barCount + 1
        }

        return [fooCount, barCount]
    }, [0, 0]);
}

const [fooCount, barCount] = count(...);

You can also use generators to accomplish this, which is even better because it doesn't require that you to iterate the entire set of words in the dictionary, but it's a little more unwieldy to use.

This is actually easier to use than other examples that require if statements, because you could quite easily run a battery of functions over each result and add it to the accumulator.

Just remember though that you don't want to optimize before you prove something is a problem. Iterating 22000 objects is obviously more than iterating 11000, but it is still going to be quite fast!

imvain2 · Accepted Answer · 2020-08-13 19:23:19Z

0

Restricting the number of loops is your best option as it requires less overhead.

Also here is an idea, using the foreach to do the processing with if statements and using a single counter object to hold all of the values so they mean something and can be easily referenced later on.

const arr = [
   {"foo" : true,"bar" : false},{"bar" : true,"foo" : false}
];

let counters = {};

function inc(field) {
  if (counters[field] == undefined) {
    counters[field] = 0;
  }
  counters[field]++;
}

arr.forEach(el => {
  if (el["foo"] === true) {
    inc("foo");
  } 
if (el["bar"] === true) {
    inc("bar");
  }
});

console.log(counters);

edited Aug 13, 2020 at 19:23

answered Aug 13, 2020 at 19:02

imvain2

15.9k1 gold badge19 silver badges21 bronze badges

Collectives™ on Stack Overflow

Best way to loop through an array and parse each element into several data structures

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related