How to use curl to compare the size of the page with deflate enabled and without using it

CurlSizeGzip

Curl Problem Overview


I have apache with mod_deflate enabled. I would like to find out the size of the page with mod_deflate enabled and without, and compare how much performance is achieved in size. In curl, I seem to ask server for gzipped content using --compressed and to send the normal, but can't seem to find the size of that page. Any idea how to do that?

curl --head http://site

HTTP/1.1 200 OK
Date: Wed, 08 Feb 2012 08:48:04 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.7a mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
X-Powered-By: PHP/5.2.12
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=ce39b051a9cd493cbe4a86056e11d61f; path=/
Vary: Accept-Encoding
Content-Type: text/html

curl --head --compressed http://site

HTTP/1.1 200 OK
Date: Wed, 08 Feb 2012 08:48:19 GMT
Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.7a mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635
X-Powered-By: PHP/5.2.12
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=513b8ac5818fd043471c8aac44355898; path=/
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 20
Content-Type: text/html

Curl Solutions


Solution 1 - Curl

I think the only reliable way to get the size, is to actually download the file. However, curl offers a very convenient option for only outputting data of interest

-w/--write-out <format>
    Defines what to display on stdout after a completed and successful operation.

[...]

size_download  The total amount of bytes that were downloaded.

which means you can do something like this:

curl -so /dev/null http://www.whatsmyip.org/http-compression-test/ -w '%{size_download}'

Output:

8437

And to get the compressed size:

curl --compressed -so /dev/null http://www.whatsmyip.org/http-compression-test/ -w '%{size_download}'

Output:

3225

After that your comparison should be trivial.

Solution 2 - Curl

Copy/paste ready and human readable

Based on @flesk answer and on this here is a human readable version of the script:

#!/usr/bin/env bash
set -e

bytesToHuman() {
    b=${1:-0}; d=''; s=0; S=(Bytes {K,M,G,T,E,P,Y,Z}iB)
    while ((b > 1024)); do
        d="$(printf ".%02d" $((b % 1024 * 100 / 1024)))"
        b=$((b / 1024))
        let s++
    done
    echo "$b$d ${S[$s]}"
}

compare() {
  echo "URI: ${1}"

  SIZE=$(curl -so /dev/null "${1}" -w '%{size_download}')
  SIZE_HUMAN=$(bytesToHuman "$SIZE")
  echo "Uncompressed size : $SIZE_HUMAN"

  SIZE=$(curl --compressed -so /dev/null "${1}" -w '%{size_download}')
  SIZE_HUMAN=$(bytesToHuman "$SIZE")
  echo "Compressed size   : $SIZE_HUMAN"
}

compare https://stackoverflow.com/q/9190190/1480391

Output:

URI: https://stackoverflow.com/q/9190190/1480391
Uncompressed size : 106.69 KiB
Compressed size   : 24.47 KiB

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionAnjeshView Question on Stackoverflow
Solution 1 - CurlfleskView Answer on Stackoverflow
Solution 2 - CurlYves M.View Answer on Stackoverflow