Github accidentally committed their SSH RSA private key to a repository, causing infrastructure to break for many users who needed to update their keys.

This issue could have been avoided if SSH used certificates like TLS does, but SSH uses a Trust on First Use model which can lead to users blindly accepting new keys.

The solution is to deploy the private key to every front-end server and have it call out to an HSM-backed service to request new SSH host certificates signed by a private key in the HSM. Clients configured to trust the signing key will trust the certificate without manual intervention.

The main obstacle is that client tooling does not support this well, and so the most important improvement to the SSH ecosystem would be to make it easier to use certificates.