Difference between $HOME and '~' (tilde)?

I had always thought that $HOME and ~ were exactly the same and thus could be used interchangeably. Today, when I tried to install pylibmc, a python binding to memcached, on my shared server the use of ~ gave me error but not $HOME. I would like to reason out why.

libmemcached is a requirement for pylibmc. I have libmemcached installed under my home directory because I have no root on the server. As a result, to install pylibmc, I need to make sure the installation script knows where to find libmemcached.

When executing python setup.py install --with-libmemcached=~, the installation script runs

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall 

-Wstrict-prototypes -fPIC -DUSE_ZLIB -I~/include 

-I/usr/local/include/python2.7 -c _pylibmcmodule.c 

-o build/temp.linux-i686-2.7/_pylibmcmodule.o -fno-strict-aliasing

which gives the errors that libmemcached can't be found.

If I change to --with-libmemcached=$HOME, the script runs

gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall 

-Wstrict-prototypes -fPIC -DUSE_ZLIB -I/home/waterbotte/include 

-I/usr/local/include/python2.7 -c _pylibmcmodule.c 

-o build/temp.linux-i686-2.7/_pylibmcmodule.o -fno-strict-aliasing

without any problem. It looks like the problem is that tilde doesn't get resolved. But why?

The tilde is part of a shell expansion (like in bash, csh, zsh, etc). The $HOME variable is exportable and can be used independent of a specific shell.

The shell replaces ~ with the user's home directory (update: or perhaps by the home directory of some other user, if ~ is followed by something other than a /), but only if it's the first character of a word.

--with-libmemcached=~ has ~ not in the beginning, so the shell leaves it alone.

~ is expanded ONLY if it is the first character of a word AND it is unquoted

$ echo "~"
~
$ echo foo~
foo~
$ echo ~
/home/guest
$ echo ~/foo
/home/guest/foo

~username is expanded to the HOME of the username.

$ echo ~root
/root
$ echo ~invaliduser
~invaliduser

To quote filenames, you should use $HOME or quote the suffix

$ echo "$HOME/foo bar"
/home/guest/foo bar
$ echo ~/"foo bar"
/home/guest/foo bar
$ echo ~root/"foo bar"
/root/foo bar

Note the following from "POSIX Tilde Expansion"

> The pathname resulting from tilde expansion shall be treated as if quoted to prevent it being altered by field splitting and pathname expansion.

The main difference is:

cd /tmp
ls "$HOME" #works
ls "~" #nope

So, shell expand the ~ only in few situations. In your case, the python script simple got ~ inside the script - not the expaded value.

Run the following script:

#!/bin/bash

sudo -H -u root bash<<EOF
echo $HOME
echo ~
EOF

Output:

/home/my_current_user
/root

You can see that ~ gets expanded later, by the target shell (run by root) while $HOME gets substituted by the source shell (run by my_current_user)