How can I write a buffer data to a file from readable.stream in Nodejs?

Question

How can I write a buffer data to a file from readable.stream in Nodejs? I know there are already npm package, I am asking this question for learning purpose only. I am also wondering why there is no such method available in npm 'fs' where user can pass readablestream and create a file directly?

I tried to write a stream.readableBuffer to a file using fs.write by passing the buffer directly, but somehow a small portion of file, is corrupt, after writing, I can see image but a small portion look black in it, my guess buffer has not written completely.

I pass formdata from ajax XMLHttpRequest to serverside controller (node js router in this case). and I used npm 'parse-formdata' to parse the request. below is the code:

  parseFormdata(req, function (err, data) {
    if (err) {
      logger.error(err);
      throw err
    }
    console.log('fields:', data.fields); // I have data here but how to write this data to a file?
     /** perhaps a bad way to write the data to a file, looking for a better way **/ 
    var chunk = data.parts[0].stream.readableBuffer.head.chunk;
    fs.writeFile(data.parts[0].filename, chunk, function(err) {
      if(err) {
          console.log(err);
      } else {
          console.log("The file was saved!");
      }
    });

could some body tell me a better approach to write the data (that I got from parsing of FormData) ?

grodzi · Accepted Answer · 2020-06-27 22:30:56Z

1

According to parseFormData

You may use the provided sample:

var pump = require('pump')
var concat = require('concat-stream')
pump(stream, concat(function (buf) {
  assert.equal(String(buf), String(file), 'file received')
  // then write to your file
  res.end()
}))

But you may do shorter:

const ws = fs.createWriteStream('out.txt')
data.parts[0].stream.pipe(ws)

Finally, note that library has not been updated since 2017, so there may be some vulnerabilities or so..

answered Jun 27, 2020 at 22:30

grodzi

5,7111 gold badge17 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Brainwash Over a year ago

thanks for your answer, please correct me, if i understood correctly i still need fs to write the buffer codeline-pump(stream, concat(function (buf) ?

grodzi Over a year ago

In any case, you need fs since you want to write to a file. Either: you get the string (via the pump, concat stuff) and fs.writeFile('out.txt', theString). Either you get the stream (via data.parts[0].stream) and you pipe the stream via the writableStream (that you get with fs.createWriteStream). There are more details about how stream work online, but tldr, using stream.pipe(ws) is like: for every chunk of the readable stream, output them to ws (which means append the chunk to the file)

Collectives™ on Stack Overflow

How can I write a buffer data to a file from readable.stream in Nodejs?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related