Wireshark can decrypt SSL traffic as long as you have the server private key. This can be extremely useful, if you have to debug HTTPS traffic and cannot use HTTP instead or put a MITM in the front (e.g. Windows CardSpace applications).

Unfortunately, the documentation on this feature is at this time rather thin. the wireshark wiki has one page dedicated to it (along with some sample traces - great to get started!!), but there is some information missing. This is what I did:

1. Make sure that the server private keys are in unencrypted PKCS#8 PEM format (RSA)

If in doubt, take a look at your key file. If it is binary, chances are that it is in a DER format which cannot be used with wireshark. Assuming that you have at least an PKCS#8 DER file, you can instruct openssl to convert this file for you:

openssl pkcs8 -nocrypt -in derfile.key -informat DER -out key.pem -outformat PEM

If your DER file is encrypted, you need decrypt the key with the right passphrase first. After you are done, you first line in the key.pem file should look like this:

-----BEGIN RSA PRIVATE KEY-----

2. Configure Wireshark to use this key

You have to go into the Preferences for SSL and configure the RSA key list. Check the wireshark wiki on how to do this. Make sure to specify the debug file - you really need this!

3. Capture you traffic and debug

If you now start to capture your traffic, you *should* be good to go. Make sure that you find a line like

ssl_init private key file c:\temp\key.pem successfully loaded

in you ssl debug line (at the top).

One particular issue that I had was that I got in the debug file for the first application packet the following debug output:

ssl_restore_session can't find stored session

This happens if your client talked to the server before you started the trace (or during an earlier trace) and some key exchange messages are missing. Restart your client (e.g. CardSpace or the browser) and the server, and you should be good to go. 

tag: ssl, network sniffing, wireshark