Firstly, there are several steps in step 3. What actually happens in the third step:
- the client verifies the certificate
- the client and server send some messages back and forth, whereby they discuss which secret key encryption they both can use
- the client creates a secret key for this encryption algorithm, encrypts the secret key with the server’s public key and sends the encrypted secret key to the server.
- the server can now decrypt the private key using its private key.
- the client and server now agree on the private key algorithm, and they share the private key. All communications are now encrypted and decrypted using a shared secret key.
- the client encrypts its credentials with a secret key ...
The reason for this is twofold. First, math for public-key cryptography is much more expensive than math for a private key. The idea is to do this expensive math only once to share a common secret. Secondly, the client knows the public key of the server, but the client may not even have a public key, since the server can send him a secret message? That is why they must agree to a shared secret key.
In any case, going to your questions:
Where does the service store its private key?
On Windows, this is in the certificate store provided by the operating system. In other operating systems, I do not know.
Why do I need a certificate?
Suppose you want to buy my lawn mower with a credit card, but you are worried that I can really be a scammer. You would like to know your real name so you can pursue me if I start charging your trip to Vegas on your map.
So, when we complete the transaction, I show you a piece of paper that says: "Eric Lippert claims that the owner of this sheet of paper is Eric Lippert, signed by Eric Lippert." Do you believe me? If you already trust Eric Lippert, you do not need a piece of paper, and if you do not trust Eric Lippert, then the paper does not establish trust. This is a "self-signed certificate."
Now, if I show you a piece of paper that says: “VeriSign stated that the owner of this piece of paper is Eric Lippert, signed by VeriSign,” then the question arises: do you trust VeriSign to verify my identity? If so, then this indicates that the person you are talking to is what they call themselves.
This is the purpose of the certificate. It establishes that a particular public key is indeed associated with a particular organization, as it is signed by a third party, a trusted third party called a certification body.
When you buy something with your credit card online, you probably trust the website to make only authorized payments to this card. The certificate verifies that the public key that you are going to use for encryption is indeed the public key of this website, and not some kind of evil hacker public key.
But the public key certificate is public, so can someone grab the certificate and pretend to be a server?
Yes, but that will not help them unless they steal a private key that is private. A person with a different certificate cannot decrypt messages encrypted with the public key, so they will not be able to exchange the secret key with the client! Even if they manage to trick the client, all they are going to get is a bit stream encoded by the client’s secret key, a secret key that an attacker cannot get without the private half of the certificate.
This means: if I allow access to my private key, someone can impersonate me, right?
Yes. The security of the entire system depends on the private key remaining private. That is why it is called a private key. If an attacker gains access to your private key, they can impersonate you at their discretion, so keep it secret, keep it safe. If your key is cracked, there is a mechanism by which the certificate can be revoked, but the damage will usually be done.