Size of json object? (in KBs/MBs)

JavascriptJson

Javascript Problem Overview


How can I know that how much data is transferred over wire in terms of size in kilobytes, megabytes?

Take for example

{
  'a': 1,
  'b': 2
}

How do I know what is the size of this payload is and not the length or items in the object

UPDATE

content-encoding:gzip
content-type:application/json
Transfer-Encoding:chunked
vary:Accept-Encoding

Javascript Solutions


Solution 1 - Javascript

An answer to the actual question should include the bytes spent on the headers and should include taking gzip compression into account, but I will ignore those things.

You have a few options. They all output the same answer when run:

If Using a Browser or Node (Not IE)

const size = new TextEncoder().encode(JSON.stringify(obj)).length
const kiloBytes = size / 1024;
const megaBytes = kiloBytes / 1024;

If you need it to work on IE, you can use a pollyfill

If Using Node

const size = Buffer.byteLength(JSON.stringify(obj))

(which is the same as Buffer.byteLength(JSON.stringify(obj), "utf8")).

Shortcut That Works in IE, Modern Browsers, and Node

const size = encodeURI(JSON.stringify(obj)).split(/%..|./).length - 1;

That last solution will work in almost every case, but that last solution will throw a URIError: URI malformed exception if you feed it input containing a string that should not exist, like let obj = { partOfAnEmoji: "👍🏽"[1] }. The other two solutions I provided will not have that weakness.

(Credits: Credit for the first solution goes here. Credit for the second solution goes to the utf8-byte-length package (which is good, you could use that instead). Most of the credit for that last solution goes to here, but I simplified it a bit. I found the test suite of the utf8-byte-length package super helpful, when researching this.)

Solution 2 - Javascript

For ascii, you can count the characters if you do...

JSON.stringify({
  'a': 1,
  'b': 2
}).length

If you have special characters too, you can pass through a function for calculating length of UTF-8 characters...

function lengthInUtf8Bytes(str) {
  // Matches only the 10.. bytes that are non-initial characters in a multi-byte sequence.
  var m = encodeURIComponent(str).match(/%[89ABab]/g);
  return str.length + (m ? m.length : 0);
}

Should be accurate...

var myJson = JSON.stringify({
  'a': 1,
  'b': 2,
  'c': 'Máybë itß nºt that sîmple, though.'
})

// simply measuring character length of string is not enough...
console.log("Inaccurate for non ascii chars: "+myJson.length)

// pass it through UTF-8 length function...
console.log("Accurate for non ascii chars: "+ lengthInUtf8Bytes(myJson))

/* Should echo...

Inaccurate for non ascii chars: 54
Accurate for non ascii chars: 59

*/

Working demo

Solution 3 - Javascript

Here is a function which does the job.

function memorySizeOf(obj) {
    var bytes = 0;

    function sizeOf(obj) {
        if(obj !== null && obj !== undefined) {
            switch(typeof obj) {
            case 'number':
                bytes += 8;
                break;
            case 'string':
                bytes += obj.length * 2;
                break;
            case 'boolean':
                bytes += 4;
                break;
            case 'object':
                var objClass = Object.prototype.toString.call(obj).slice(8, -1);
                if(objClass === 'Object' || objClass === 'Array') {
                    for(var key in obj) {
                        if(!obj.hasOwnProperty(key)) continue;
                        sizeOf(obj[key]);
                    }
                } else bytes += obj.toString().length * 2;
                break;
            }
        }
        return bytes;
    };

    function formatByteSize(bytes) {
        if(bytes < 1024) return bytes + " bytes";
        else if(bytes < 1048576) return(bytes / 1024).toFixed(3) + " KiB";
        else if(bytes < 1073741824) return(bytes / 1048576).toFixed(3) + " MiB";
        else return(bytes / 1073741824).toFixed(3) + " GiB";
    };

    return formatByteSize(sizeOf(obj));
};

console.log(memorySizeOf({"name": "john"}));

I got the snippet from following URL

> https://gist.github.com/zensh/4975495

Solution 4 - Javascript

Buffer.from(JSON.stringify(obj)).length will give you the number of bytes.

Solution 5 - Javascript

JSON-objects are Javascript-objects, this SO question already shows a way to calculate the rough size. <- See Comments/Edits

EDIT: Your actual question is quite difficult as even if you could access the headers, content-size doesn't show the gzip'ed value as far as I know.

If you're lucky enough to have a good html5 browser, this contains a nice piece of code:

var xhr = new window.XMLHttpRequest();
   //Upload progress
   xhr.upload.addEventListener("progress", function(evt){
   if (evt.lengthComputable) {
     var percentComplete = evt.loaded / evt.total;
     //Do something with upload progress
     console.log(percentComplete);
     }
   }

Therefore the following should work (can't test right now):

var xhr = new window.XMLHttpRequest();
if (xhr.lengthComputable) {
  alert(xhr.total);
}

Note that XMLHttpRequest is native and doesn't require jQuery.

Next EDIT:

I found a kind-of duplicate (slightly diffrent question, pretty much same answer). The important part for this question is

content_length = jqXHR.getResponseHeader("X-Content-Length");

Where jqXHR is your XmlHttpRequest.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestiondaydreamerView Question on Stackoverflow
Solution 1 - JavascriptdavidbludlowView Answer on Stackoverflow
Solution 2 - JavascriptBilly MoonView Answer on Stackoverflow
Solution 3 - JavascriptHari DasView Answer on Stackoverflow
Solution 4 - JavascriptshawnrushefskyView Answer on Stackoverflow
Solution 5 - JavascriptSebbView Answer on Stackoverflow