id_rsa.pub file SSH Error: invalid format

My co worker and I have been trying to get my macbook to ssh into our AWS production server with no luck, this did work several months ago, but when we went to do it yesterday - we kept getting the following error message.

(I have replaced my username and hostname with a placeholder on all below sections)

$ ssh titan-data
Load key "/Users/aronlilland/.ssh/id_rsa.pub": invalid format
user@my_ip_address: Permission denied (publickey).

###Relavant config file data

<!-- config file -->
Host titan-data
    HostName my_ip_address
    User user
    IdentityFile ~/.ssh/id_rsa.pub
    AddKeysToAgent yes

I have gone through every step imaginable to get it working again, these are the following steps I have done in order:

###Step 1) ensured that the .pub file was added on the server

this step was repeated several times, I regenerated my .pub file 4 times through out this process of elimination. The error seems to be something local though, not something from the server, someone will need to correct me if I am wrong.

###Step 2) checked that the following folders and files have the correct permissions

drwx------    6 aronlilland  staff   192B Jan 18 09:41 .ssh/

drwx------    6 aronlilland  staff   192B Jan 18 09:41 ./
drwxr-xr-x+ 104 aronlilland  staff   3.3K Jan 16 16:09 ../
-rw-r--r--    1 aronlilland  staff   114B Jan 18 09:37 config
-rw-------    1 aronlilland  staff   1.7K Jan 18 09:41 id_rsa
-r--------@   1 aronlilland  staff   414B Jan 18 09:41 id_rsa.pub
-rw-r--r--    1 aronlilland  staff   4.6K Jan 18 09:48 known_hosts

this did not have any effect on fixing the error message.

###Step 3) deleted my id_rsa file and id_rsa.pub file and regenerate a new one, having my coworker add the new .pub file to the server

1. first way i generated my keys ssh-keygen -t rsa -b 4096 -C "[email protected]" as suggested in the github documentation for generating a new SSH key
   • added the new pub file to the server through my co worker, did not work. still getting the same error.
2. Second way I generated my .pub file after deleting the old one, was to run ssh-keygen -t rsa as suggested in the joyent manually generating your ssh key on OSX documentation
   • still got the same error

###Step 4) Upgraded ssh with homebrew this was done following the documentation listed here, by Tim Hilliard

###Step 5) deleted the ip address from my "known hosts" file (reaching for straws) did not do anything, just had to confirm the IP address again.

##Edit 1) One thing I did do recently, and I remember seeing error messages during the installation, was I installed another version of python the other night, so there are two versions of python installed on my computer. Im not sure if this would cause any issues with ssh.

$ python -V
Python 2.7.10
$ python3 -V
Python 3.6.4
$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. 
Thanks!

Warning: Python is installed at /Library/Frameworks/Python.framework

Homebrew only supports building against the System-provided Python or a
brewed Python. In particular, Pythons installed to /Library can interfere
with other software installs.

Edit 2)

added -v to ssh command, suggested by Alfabravo in comments

aronlilland:.ssh$ ssh titan-data -v
OpenSSH_7.6p1, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /Users/aronlilland/.ssh/config
debug1: /Users/aronlilland/.ssh/config line 1: Applying options for titan-data
debug1: Reading configuration data /usr/local/etc/ssh/ssh_config
debug1: Connecting to my_ip_address [my_ip_address] port 22.
debug1: Connection established.
debug1: identity file /Users/aronlilland/.ssh/id_rsa.pub type 0
debug1: key_load_public: No such file or directory
debug1: identity file /Users/aronlilland/.ssh/id_rsa.pub-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.2 pat OpenSSH* compat 0x04000000
debug1: Authenticating to my_ip_address:22 as 'user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:2l8QDc7x/fyzshKaU32jj9MeFtMAccu25MZBFNQuStQ
debug1: Host 'my_ip_address' is known and matches the ECDSA host key.
debug1: Found key in /Users/aronlilland/.ssh/known_hosts:19
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:MuOjm+6gAzaxGr/n/If4LVjk/0H5/VT6zvib9/9C7c8 /Users/aronlilland/.ssh/id_rsa.pub
debug1: Server accepts key: pkalg rsa-sha2-512 blen 279
Load key "/Users/aronlilland/.ssh/id_rsa.pub": invalid format
debug1: No more authentication methods to try.
user@my_ip_address: Permission denied (publickey).
aronlilland:.ssh$ 

##Edit 3) Checked to see if the .pub file is a valid format

aronlilland:.ssh$ ssh-keygen -l -f id_rsa.pub
2048 SHA256:MuOjm+6gAzaxGr/n/If4LVjk/0H5/VT6zvib9/9C7c8 aronlilland@arons-mbp.cacc.local (RSA)
aronlilland:.ssh$ touch foo.txt
aronlilland:.ssh$ ls
config       foo.txt      id_rsa       id_rsa.pub   known_hosts
aronlilland:.ssh$ ssh-keygen -l -f foo.txt
foo.txt is not a public key file.

##Edit 4)

Confirming that my id_rsa key and id_rsa.pub validate (I chopped off the end of my ssh-rsa key, there's no reason to include it on the web)

aronlilland:.ssh$ ssh-keygen -y -f id_rsa
Enter passphrase: 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQA .......

Confirming permissions that updating file permissions did not change access (I tried 0444, 444, 0644, 644 just to be sure)

aronlilland:.ssh$ chmod 0444 id_rsa.pub
aronlilland:.ssh$ ssh titan-data
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0444 for '/Users/aronlilland/.ssh/id_rsa.pub' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/Users/aronlilland/.ssh/id_rsa.pub": bad permissions
user@my_ip_address: Permission denied (publickey).

and the other

aronlilland:.ssh$ chmod 0644 id_rsa.pub
aronlilland:.ssh$ ssh titan-data
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/Users/aronlilland/.ssh/id_rsa.pub' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/Users/aronlilland/.ssh/id_rsa.pub": bad permissions
user@my_ip_address: Permission denied (publickey).

setting it back to chmod 400 id_rsa.pub

##Edit 5)

running -vv

aronlilland:.ssh$ chmod 400 id_rsa.pub
aronlilland:.ssh$ ssh titan-data -vv
OpenSSH_7.6p1, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /Users/aronlilland/.ssh/config
debug1: /Users/aronlilland/.ssh/config line 1: Applying options for titan-data
debug1: Reading configuration data /usr/local/etc/ssh/ssh_config
debug2: resolving "my_ip_address" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to my_ip_address [my_ip_address] port 22.
debug1: Connection established.
debug1: identity file /Users/aronlilland/.ssh/id_rsa.pub type 0
debug1: key_load_public: No such file or directory
debug1: identity file /Users/aronlilland/.ssh/id_rsa.pub-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.2 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to my_ip_address:22 as 'user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:2l8QDc7x/fyzshKaU32jj9MeFtMAccu25MZBFNQuStQ
debug1: Host 'my_ip_address' is known and matches the ECDSA host key.
debug1: Found key in /Users/aronlilland/.ssh/known_hosts:19
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /Users/aronlilland/.ssh/id_rsa.pub (0x7fe9ff41c510), explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:MuOjm+6gAzaxGr/n/If4LVjk/0H5/VT6zvib9/9C7c8 /Users/aronlilland/.ssh/id_rsa.pub
debug2: we sent a publickey packet, wait for reply
debug1: Server accepts key: pkalg rsa-sha2-512 blen 279
debug2: input_userauth_pk_ok: fp SHA256:MuOjm+6gAzaxGr/n/If4LVjk/0H5/VT6zvib9/9C7c8
Load key "/Users/aronlilland/.ssh/id_rsa.pub": invalid format
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
user@my_ip_address: Permission denied (publickey).

The IdentityFile configuration parameter should be pointed at the private key which the SSH client uses to prove its identity to the remote server. (The remote server, then, should have the contents of id_rsa.pub installed in its authorized_keys file, or an equivalent location).

You should be putting the path to id_rsa, not id_rsa.pub, as an argument to IdentityFile in your ~/.ssh/config.

your error is in having used the -f option with the public key name... you have to generate a new key, as you have sent (and compromised) your private key, by storing it in -f .ssh/id_rsa.pub

The private key is stored by default in .ssh/id_rsa and the public key is stored in .ssh/id_rsa.pub. By specifying .ssh/id_rsa.pub as the place of your private key, probably you'll have the public in .ssh/id_rsa.pub.pub (CHECK THIS) but anyway, you have sent your private key over the network, so it is compromised and you should have better generating a new key pair.

Next time, you can trust the default settings proposed for the key filenames or use the proper (without the .pub extension) for the key file.

Here's what worked for me.

Trying to connect to AWS server, I was getting this error: Load key "yourKey.pem": invalid format and there was nothing wrong with the key file content.

But generating fingerprint would produce : unable to load key <...> :Expecting: ANY PRIVATE KEY...

What worked for me was to use terminal with nano and manually create a brand new file with the same name

$ sudo nano yourKey.pem

paste the text content from the original key file and save it.

Then the new key's fingerprint matched and ssh access was restored.

Hope this help.

The following permissions should be set:

Private keys should get rw for owner, none for group, none for others:

chmod 600 id_rsa

Public keys should get rw for owner, r for group, r for others:

chmod 644 id_rsa.pub

The main error of the question is said to be about mixing up public and private key, but that same error can also occur for other reasons. For those who searched only for the invalid format error, this might help as well, which deals only with the private key instead:

I had a similar error Load key "/root/.ssh/id_rsa": invalid format when I tried

RUN echo "$ssh_prv_key" > /root/.ssh/id_rsa &&     chmod 600 /root/.ssh/id_rsa

This led to errors like identity file /root/.ssh/id_rsa type -1 invalid format and read_passphrase: can't open /dev/tty.

The right way would be to use

COPY id_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa

The solution explained: my private key was wrongly formatted - instead of many lines, it was passed as a one-liner, and you might have any other format issue like a forgotten "-" at the start or end, or something wrong at the end of the lines, like a missing newline format or an additional letter at the end of a line.

See Dockerfile: clone repo with passwordless private key. Errors: “authentication agent” or “read_passphrase: can't open /dev/tty” for more details, with the main idea from Add private key to ssh-agent in docker file, which again had the idea from Gitlab CI/Docker: ssh-add keeps asking for passphrase.

In my case, I was attempting to use the -i flag to point to the default key: ssh -i ~/.ssh/id_rsa.pub [email protected]

Removing the -i key, and simply calling ssh [email protected] resolved the error