How do I git rebase the first commit?

I used git init to create a fresh repo, then made three commits. Now I want to rebase to go back and amend my first commit, but if I do git rebase -i HEAD~3 it complains! If I try the same with HEAD~2 then it kinda works but only lets me rearrange the last two commits.

How do I refer to the 'commit before there were any commits' or go back and insert an empty commit?

The easy way, with a recent-enough git (this has been out for a long time now so you should have this):

git rebase -i --root

The other easy way, as twalberg noted in a comment, is to use git checkout --orphan to set up to make a new root commit, which you can copy the old commits on top of. (This is what rebase -i --root ends up doing internally anyway.)

torek's answer is fine if you want to make changes to files that are already in the commit, edit the author/message, etc. But if you want to split the commit or anything like that, then chances are you're going to run into trouble because you can't do git reset HEAD~ from the initial commit.

To make this possible, you can insert an empty initial commit like so:

git checkout --orphan empty-initial-commit
git commit --allow-empty -m 'Empty initial commit'
git checkout <branch>
git rebase empty-initial-commit
git branch -d empty-initial-commit

then you can do git rebase -i, edit the commit (the first non-empty commit), and do git reset HEAD~ like normal.