I have a varbinary type column jsonBin in db. I want to store json (string) which is compressed using zlib. zlib.gzipSync(data) or zlib.deflateSync(data) returns buffer object.
If I use base64 encoding to compress, it is working fine. But if I do without base64 encoding then I'm getting this error: incorrect header check (stack trace details below)
I want to transport this compressed data over n/w & want it to be of minimum length. Hence, using base64 is not a good idea. Also I have verified that the data is getting store correctly.
Code that doesn't work:
const zip = zlib.gzipSync(JSON.stringify(input));
// code to store zip in db
// code to retrieve zip from db which returns zipBuffer
const originalObj = JSON.parse(zlib.unzipSync(Buffer.from(zipBuffer)));
This however works:
const zip = zlib.gzipSync(JSON.stringify(input), 'base64');
// code to store zip in db
// code to retrieve zip from db which returns zipBuffer
const originalObj = JSON.parse(zlib.unzipSync(Buffer.from(zipBuffer)),'base64');
Can someone enlighten me what is going wrong?
PS: All other answers are about how to inflate/deflate but doesn't handle bytes array where I'm facing issue
Stack trace -
Error: incorrect header check\n at genericNodeError (node:internal/errors:983:15)\n at wrappedFn (node:internal/errors:537:14)\n at Zlib.zlibOnError [as onerror] (node:zlib:191:17)\n at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)\n at processChunkSync (node:zlib:459:12)\n at zlibBufferSync (node:zlib:180:12)\n at Object.syncBufferWrapper [as unzipSync] (node:zlib:811:14)\n at deCompressString (D:\workdir\gitlab\service\dist\routes\database\zlib-manager.js:61:42)\n at Object. (D:\workdir\gitlab\service\dist\routes\consumers.js:181:72)\n at Generator.next () {\n errno: -3,\n code: 'Z_DATA_ERROR'\n} Error: incorrect header check\n at genericNodeError (node:internal/errors:983:15)\n at wrappedFn (node:internal/errors:537:14)\n at Zlib.zlibOnError [as onerror] (node:zlib:191:17)\n at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)\n at processChunkSync (node:zlib:459:12)\n at zlibBufferSync (node:zlib:180:12)\n at Object.syncBufferWrapper [as unzipSync] (node:zlib:811:14)\n at deCompressString (D:\workdir\gitlab\service\dist\routes\database\zlib-manager.js:61:42)\n at Object. (D:\workdir\gitlab\service\dist\routes\consumers.js:181:72)\n at Generator.next ()","timestamp":"2024-07-24T09:34:56.325Z"}
Things I have tried:
- I tried storing this buffer directly in jsonBin column & also by converting it to bytes array
- Retreived this value - This value returns buffer even if I store data as bytesArray
- Tried to inflate it using zlib.inflateSync & also zlib.unzipSync respectively