2

I have a varbinary type column jsonBin in db. I want to store json (string) which is compressed using zlib. zlib.gzipSync(data) or zlib.deflateSync(data) returns buffer object.

If I use base64 encoding to compress, it is working fine. But if I do without base64 encoding then I'm getting this error: incorrect header check (stack trace details below)

I want to transport this compressed data over n/w & want it to be of minimum length. Hence, using base64 is not a good idea. Also I have verified that the data is getting store correctly.

Code that doesn't work:

const zip = zlib.gzipSync(JSON.stringify(input));

// code to store zip in db
// code to retrieve zip from db which returns zipBuffer

 const originalObj = JSON.parse(zlib.unzipSync(Buffer.from(zipBuffer)));

This however works:

const zip = zlib.gzipSync(JSON.stringify(input), 'base64');

// code to store zip in db
// code to retrieve zip from db which returns zipBuffer

 const originalObj = JSON.parse(zlib.unzipSync(Buffer.from(zipBuffer)),'base64');

Can someone enlighten me what is going wrong?

PS: All other answers are about how to inflate/deflate but doesn't handle bytes array where I'm facing issue

Stack trace -

Error: incorrect header check\n at genericNodeError (node:internal/errors:983:15)\n at wrappedFn (node:internal/errors:537:14)\n at Zlib.zlibOnError [as onerror] (node:zlib:191:17)\n at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)\n at processChunkSync (node:zlib:459:12)\n at zlibBufferSync (node:zlib:180:12)\n at Object.syncBufferWrapper [as unzipSync] (node:zlib:811:14)\n at deCompressString (D:\workdir\gitlab\service\dist\routes\database\zlib-manager.js:61:42)\n at Object. (D:\workdir\gitlab\service\dist\routes\consumers.js:181:72)\n at Generator.next () {\n errno: -3,\n code: 'Z_DATA_ERROR'\n} Error: incorrect header check\n at genericNodeError (node:internal/errors:983:15)\n at wrappedFn (node:internal/errors:537:14)\n at Zlib.zlibOnError [as onerror] (node:zlib:191:17)\n at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)\n at processChunkSync (node:zlib:459:12)\n at zlibBufferSync (node:zlib:180:12)\n at Object.syncBufferWrapper [as unzipSync] (node:zlib:811:14)\n at deCompressString (D:\workdir\gitlab\service\dist\routes\database\zlib-manager.js:61:42)\n at Object. (D:\workdir\gitlab\service\dist\routes\consumers.js:181:72)\n at Generator.next ()","timestamp":"2024-07-24T09:34:56.325Z"}

Things I have tried:

  1. I tried storing this buffer directly in jsonBin column & also by converting it to bytes array
  2. Retreived this value - This value returns buffer even if I store data as bytesArray
  3. Tried to inflate it using zlib.inflateSync & also zlib.unzipSync respectively
3
  • "Also I have verified that the data is getting store correctly." How have you verified this? It is clear from the error and your success with Base64 that putting the binary zlib data and getting it back out of your jsonBin is corrupting the data. You have provided no code in your question for the things you have tried. You can't get help on a coding site if you provide no code to look at. Commented Jul 24, 2024 at 16:07
  • I currently, cannot put code to store the data as its bulky and cannot be put in few lines. I have provided the code how we are inflating & deflating data. Regarding zip, it is being stored directly as is. Commented Jul 25, 2024 at 4:04
  • What matters here is "// code to store zip in db" and "// code to retrieve zip from db which returns zipBuffer", which you're not showing. That's where the zlib data is getting corrupted. Commented Jul 25, 2024 at 5:26

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.