Return array with fast-csv in Node

Question

I am attempting to parse a large file using the fast-csv library and return its values as an array to a config.js file. Please help as the value of countries in the config's model.exports section ends up being undefined.

Parser:

import csv from 'fast-csv';

export function getCountries() {
let countries = [];
 csv.fromPath('./src/config/csv_configs/_country.csv')
    .on('data',
        function(data) {
            countries.push(data);
        })
    .on('end', function () {
         return countries;
     });
}

Config:

import {getCountries} from '../tools/csv_parser';

let countryList = [];

module.exports = {
  port: process.env.PORT || 8000,
  token: '',
  countries: getCountryList()
};

function getCountryList() {
  if (countryList.length === 0) {
    countryList = getCountries();
  }
  return countryList;
}

I understand this is due to me attempting to return a value from the anonymous function on(), however I do not know the proper approach.

RyanZim · Accepted Answer · 2018-12-13 15:01:46Z

4

You're correct that returning values from the callback in .on('end' is the source of your problem.

Streams are asynchronous. If you want to use this fast-csv library, you're going to need to return a promise from getCountries(). However, I'm assuming that's not what you want, since you're using the result in a config file, which is synchronous.

Either you need to read your csv synchronously, or you need to refactor the way your application works to be able to have your config be asynchronous. I'm assuming the second option isn't possible.

You probably want to look into using another CSV library that doesn't use streams, and is synchronous. Two examples from a quick Google search are:

I haven't used either of these libraries personally, but it looks like they'd support what you're trying to do. I'm assuming your CSV file is small enough to all be stored in memory at once, if not, you're going to have to explore more complicated options.

As a side note, is there any specific reason that the data has to be in CSV format? It would seem to be much easier to store it in JSON format. JSON can be imported to your config file directly with require; no external libraries needed.

answered Dec 13, 2018 at 15:01

RyanZim

7,0351 gold badge32 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Andrey Belenkiy Over a year ago

Thank you. I used the second synchronous library you suggested. The csv files are provided by an API I am integrating with and are relatively lengthy. Being able to parse them directly is preferable.

Collectives™ on Stack Overflow

Return array with fast-csv in Node

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related