How to store a byte array in Javascript

JavascriptArraysHtmlBytearrayByte

Javascript Problem Overview


I'm going to be storing a large array of byte values (most likely over a million) in Javascript. If I use a normal array with normal numbers, that will take 8 MB, because numbers are stored as IEEE doubles, but if I can store it as bytes, it will be only 1 MB.

I'd like to avoid wasting that much space for obvious reasons. Is there a way to store bytes as bytes instead of doubles? Browser compatibility isn't an issue for me, as long as it works in Chrome. This is in HTML5, if that makes a difference.

Javascript Solutions


Solution 1 - Javascript

By using typed arrays, you can store arrays of these types:

Type Value Range Size(bytes)
Int8Array -128 to 127 1
Uint8Array 0 to 255 1
Uint8ClampedArray 0 to 255 1
Int16Array -32768 to 32767 2
Uint16Array 0 to 65535 2
Int32Array -2147483648 to 2147483647 4
Uint32Array 0 to 4294967295 4
Float32Array -3.4E38 to 3.4E38 4
Float64Array -1.8E308 to 1.8E308 8
BigInt64Array -2^63 to 2^63 - 1 8
BigUint64Array 0 to 2^64 - 1 8
Demo in Stack Snippets & JSFiddle

var array = new Uint8Array(100);
array[42] = 10;
console.log(array[42]);

Solution 2 - Javascript

var array = new Uint8Array(100);    
array[10] = 256;
array[10] === 0 // true

I verified in firefox and chrome, its really an array of bytes :

var array = new Uint8Array(1024*1024*50);  // allocates 50MBytes

Solution 3 - Javascript

You could store the data in an array of strings of some large fixed size. It should be efficient to access any particular character in that array of strings, and to treat that character as a byte.

It would be interesting to see the operations you want to support, perhaps expressed as an interface, to make the question more concrete.

Solution 4 - Javascript

I wanted a more exact and useful answer to this question. Here's the real answer (adjust accordingly if you want a byte array specifically; obviously the math will be off by a factor of 8 bits : 1 byte):

class BitArray {
  constructor(bits = 0) {
    this.uints = new Uint32Array(~~(bits / 32));
  }

  getBit(bit) {
    return (this.uints[~~(bit / 32)] & (1 << (bit % 32))) != 0 ? 1 : 0;
  }

  assignBit(bit, value) {
    if (value) {
      this.uints[~~(bit / 32)] |= (1 << (bit % 32));
    } else {
      this.uints[~~(bit / 32)] &= ~(1 << (bit % 32));
    }
  }

  get size() {
    return this.uints.length * 32;
  }

  static bitsToUints(bits) {
    return ~~(bits / 32);
  }
}

Usage:

let bits = new BitArray(500);
for (let uint = 0; uint < bits.uints.length; ++uint) {
  bits.uints[uint] = 457345834;
}
for (let bit = 0; bit < 50; ++bit) {
  bits.assignBit(bit, 1);
}
str = '';
for (let bit = bits.size - 1; bit >= 0; --bit) {
  str += bits.getBit(bit);
}
str;

Output:

"00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000101000101100101010
 00011011010000111111111111111111
 11111111111111111111111111111111"

Note: This class is really slow to e.g. assign bits (i.e. ~2s per 10 million assignments) if it's created as a global variable, at least in the Firefox 76.0 Console on Linux... If, on the other hand, it's created as a variable (i.e. let bits = new BitArray(1e7);), then it's blazingly fast (i.e. ~300ms per 10 million assignments)!


For more info, see here:

Note that I used Uint32Array because there's no way to directly have a bit/byte array (that you can interact with directly) and because even though there's a BigUint64Array, JS only supports 32 bits:

> Bitwise operators treat their operands as a sequence of 32 bits > > ... > > The operands of all bitwise operators are converted to...32-bit integers

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionKendall FreyView Question on Stackoverflow
Solution 1 - JavascriptKendall FreyView Answer on Stackoverflow
Solution 2 - JavascriptreunsView Answer on Stackoverflow
Solution 3 - JavascriptHew WolffView Answer on Stackoverflow
Solution 4 - JavascriptAndrewView Answer on Stackoverflow