split csv file into multiple csv files based on a column value using fast-csv in nodejs

Question

I have a csv file say like this:

ID   Name   TNumber
123  John   123456
123  Joe    789012
124  Tim    896578
124  Tom    403796

I would like to split it into 2 separate csv files based on the ID column.

I am using fast-csv to parse and other modifications on the csv. I need to split the file into two (in the above case) and then do the other operations. How can I achieve this?

Nino Filiu · Accepted Answer · 2019-01-14 00:17:21Z

3

This works:

const fs = require('fs');
const fastCsv = require('fast-csv');

const datas = {}; // data['123'] = CSV data filtered for id = 123
const options = {headers: true, delimiters: '\t'}; // relative to your CSV usage

fastCsv
    .fromPath('./data.csv', options)
    .on('data', d => {
        if (!datas[d.id]) datas[d.id] = [];
        datas[d.id].push(d)
    })
    .on('end', () => {
        Object.keys(datas).forEach(id => {
            // For each ID, write a new CSV file
            fastCsv
                .write(datas[id], options)
                .pipe(fs.createWriteStream(`./data-id-${id}.csv`));
        })
    });

Additionally this should be quite fast.

answered Jan 14, 2019 at 0:17

Nino Filiu

19k11 gold badges65 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

NewJackSwing4Ever Over a year ago

This looks beautifully useful. Unfortunately, when I tried it it output a csv named "data-id-undefined.csv" equivalent to the same tab del file from above. Also tried with comma separated. In FastCsv, aside from fromPath changing to parseFile, what else may keep it from working under Node 12.16.1 from 2020?

Collectives™ on Stack Overflow

split csv file into multiple csv files based on a column value using fast-csv in nodejs

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related