Cannot push arrays to array - fast-csv, nodejs

Question

I'm using fast-csv npm package to parse incoming csv file and catching event data and displaying row is absolutely OK. I got what I want here.

What I've done:

fs.createReadStream(req.file.path)
  .pipe(csv.parse({ ignoreEmpty: true, trim: true }))
  .on("error", (error) => console.error(error))
  .on("data", (row) => console.log(row))
  .on("end", (rowCount) => console.log(rowCount));

What I got in data event:

['Data Entry Form']
['no','name','email','phone','address']
['1','John','[email protected]','09111111111','abc']
['2','Mary','[email protected]','09111111111','abc']

What I need:

let data = [];

  fs.createReadStream(req.file.path)
    .pipe(csv.parse({ ignoreEmpty: true, trim: true }))
    .on("error", (error) => console.error(error))
    .on("data", (row) => data.push(_.compact(row)))
    .on("end", (rowCount) => console.log(rowCount));

  return res.status(201).send({
    status: "success",
    data,
  });

NOT OK:

data is empty and I got no error.

Any Suggestions? Thanks.

Duc Nguyen · Accepted Answer · 2020-07-30 16:50:00Z

2

You should respond only when the read stream is ended, like this:

let data = [];

fs.createReadStream(req.file.path)
    .pipe(csv.parse({ ignoreEmpty: true, trim: true }))
    .on("error", (error) => console.error(error))
    .on("data", (row) => data.push(_.compact(row)))
    .on("end", (rowCount) => {
        console.log(rowCount);
        res.status(201).send({
            status: "success",
            data,
        });
    });

answered Jul 30, 2020 at 16:50

Duc Nguyen

8536 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

khukho Over a year ago

Can't I access from after read stream is ended because I have so much extra work to do with the data. The csv file will contain 2 types of data which will be inserted into 2 different tables like students and parents and some information like state will need to replace with real state_id instead of text because it is a foreign key in student table.

Duc Nguyen Over a year ago

You missed the point. You're reading from a read stream here, which is asynchronous. After all the data is read, fs will emit an end event and you can start doing things with your data. If you put that code outside the function registered for the end event, it will be executed before the read stream is ended, hence empty data is what you get.

khukho Over a year ago

Understood. Thank you so much.

Collectives™ on Stack Overflow

Cannot push arrays to array - fast-csv, nodejs

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related