How to append bytes, multi-bytes and buffer to ArrayBuffer in javascript?

Question

Javascript ArrayBuffer or TypedArrays dont have any kind of appendByte(), appendBytes(), or appendBuffer() methods. So if I want to fill an ArrayBuffer one value at a time, how do I do it?

var firstVal = 0xAB;              // 1 byte
var secondVal = 0x3D7F            // 2 bytes
var anotherUint8Array = someArr;

var buffer = new ArrayBuffer();   // I don't know the length yet
var bufferArr = new UInt8Array(buffer);

// following methods do not exist. What are the alternatives for each??
bufferArr.appendByte(firstVal);
bufferArr.appendBytes(secondVal);
bufferArr.appendBuffer(anotherUint8Array);

it's an array, use array syntax r[i]=x ex: github.com/rndme/download/blob/master/download.js#L69 also review the syntax for the Uint8Array constructor: developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… - you need a size on that array buffer to be able to use it like that.... — dandavis
– dandavis, Commented Nov 13, 2015 at 22:44

Paul S. · Accepted Answer · 2015-11-13 23:11:10Z

39

You can create a new TypedArray with a new ArrayBuffer, but you can't change the size of an existing buffer

function concatTypedArrays(a, b) { // a, b TypedArray of same type
    var c = new (a.constructor)(a.length + b.length);
    c.set(a, 0);
    c.set(b, a.length);
    return c;
}

Now can do

var a = new Uint8Array(2),
    b = new Uint8Array(3);
a[0] = 1; a[1] = 2;
b[0] = 3; b[1] = 4;
concatTypedArrays(a, b); // [1, 2, 3, 4, 0] Uint8Array length 5

If you want to use different types, go via Uint8Array as the smallest unit is a byte, i.e.

function concatBuffers(a, b) {
    return concatTypedArrays(
        new Uint8Array(a.buffer || a), 
        new Uint8Array(b.buffer || b)
    ).buffer;
}

This means .length will work as expected, you could now convert this to your typed array of choice (make sure it's a type that would accept the .byteLength of the buffer though)

From here, you could now implement any method you like for concatenating your data, e.g.

function concatBytes(ui8a, byte) {
    var b = new Uint8Array(1);
    b[0] = byte;
    return concatTypedArrays(ui8a, b);
}

var u8 = new Uint8Array(0);
u8 = concatBytes(u8, 0x80); // [128]

edited Nov 13, 2015 at 23:11

answered Nov 13, 2015 at 22:59

Paul S.

66.6k9 gold badges130 silver badges143 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

codneto Over a year ago

Thanks @Paul S. It would have been nice for such basic functions to be part of the spec and implemented natively, but your solution is pretty good. One question I still have how would you append a multi-byte, lets say a single 4-byte value to a typedarray. We can loop and call your suggested concatBytes function multiple times for each single byte, but can it be accomplished in better way?

Paul S. Over a year ago

@codneto how are your multi-bytes stored? e.g. if you're using ints, how do you know the difference between 0x3D7F and 0x00003D7F? And yes, I agree there should be some native concat, but I don't think there should be a native push or length changing - that's not how typed arrays work

codneto Over a year ago

S, I have some functions which return 4-byte value which I need to add to buffer. I also am reading some bytes from a stream and appending to buffer, and sometimes based on some flags I have to read 1, 2 or 4 byte value, all of which need to be appended to buffer. How should I accomplish appending these multi-byte values to buffer?

Paul S. Over a year ago

@codneto write it as a loop. Best place to loop would be creating a n long Uint8Array for the new data and using the loop to set the vales on it before concatenating with your previous data.

codneto Over a year ago

Based on your suggestion, I changed concatBytes as following. I think it will work. function concatBytes(ui8a, bytes) { var b = new Uint8Array(bytes.length); bytes.forEach(function (byte, index) { b[index] = byte; }); return concatTypedArrays(ui8a, b); } var u8 = new Uint8Array(0); // to append 0x80C83B u8 = concatBytes(u8, [0x80, 0xC8, 0x3B]); // 128, 200, 59]

|

newguy · Accepted Answer · 2017-05-12 09:06:56Z

Paul's answer allows you to concatenate one TypedArray to an existing TypedArray. In ES6, you can use the following function to concatenate multiple TypedArrays:

function concatenate(resultConstructor, ...arrays) {
    let totalLength = 0;
    for (const arr of arrays) {
        totalLength += arr.length;
    }
    const result = new resultConstructor(totalLength);
    let offset = 0;
    for (const arr of arrays) {
        result.set(arr, offset);
        offset += arr.length;
    }
    return result;
}

const ta = concatenate(Uint8Array,
    Uint8Array.of(1, 2), Uint8Array.of(3, 4));
console.log(ta); // Uint8Array [1, 2, 3, 4]
console.log(ta.buffer.byteLength); // 4

To append a new byte is:

const byte = 3;
concatenate(Uint8Array, Uint8Array.of(1, 2), Uint8Array.of(byte));

This method is found in ExploringJS.

Collectives™ on Stack Overflow

How to append bytes, multi-bytes and buffer to ArrayBuffer in javascript?

2 Answers 2

6 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related