vCenter 5.0 & View 5.1 Certificate extravaganza - the OpenSSL way

With the release of View 5.1, VMware got strict on forcing people to be more secure. When installing 5.1, you have to make the view server fully trust in vCenter's own certificate, let it be a self signed or a real one. Now, getting a real certificate shouldn't be much trouble, or, if you are (or have) a demanding AD administrator, you should already have an own root certificate pushed by domain policies. In this case, you can quit reading this article.

If you don't have one, you could still make View to believe in vCenter's own certificate, accepting it's thumbprint.

For a more demanding solution, you could generate a root certificate (own CA), distribute it with an AD policy and chain vCenter's and View's certificates after it. That's more serious business.

We'll do this with OpenSSL now, of course MS CA services could be used as well.

1. First, download OpenSSL (apt-get or yum it, or here it is for windows).
2. Generate a new key file, this will be used everywhere: openssl genrsa -out rui.key 1024 - we don't encrypt it on purpose.
3. Generate a new root certificate (CA): openssl req -new -x509 -extensions v3_ca -key rui.key -out myroot.crt -days 3650 -config openssl.cnf



How do you make these things larger?

4. Bind the generated root certificate (myroot.crt in the above example) to a Domain Policy. Put it in the  Trusted Root Certification Authorities container, to have it pushed to all domain member computers (see this MS TechNet article, this is called gpmc.msc). Use gpupdate after that for a policy refresh on Windows machines (even on the DC!). This makes our root CA to be trusted on all domain computers. You can use certmgr.msc on domain members to check if the push was successful.

After this step, you shall have a trusted, domain wide root CA for yourself. Now we should start making special certificates for vCenter, View & any other host that needs one. For vCenter (on Windows platform):

1. Generate a new certificate request (there is a whole lot of material about vCenter & certificates on the web, see this kb, or this, even this excellent blog post collection): openssl req -new -key rui.key -out rui.csr -config openssl.cnf - Remember, Common Name wants to have the FQDN of the machine the certificate will be used on (i.e. vcenter1.yourdomain.com).
2. (Optional) If you want to make it really choosy, you can create a new file (ext.txt) that tells OpenSSL,that this key will only be used for Server Authentication, has a CRL and can serve multiple domain names. The file shall include these lines:
   extendedKeyUsage=serverAuth - limit cert usage
   crlDistributionPoints=URI:<http://someserver/mycrl> - link to CRL
   subjectAltName=DNS:<FQDN1>,DNS:<FQDN2>,DNS:<etc> - aliases
   Please note that, that I did not create a CRL file, so if you want to use one, you'll have to dig yourself into the <openssl ca> commands.
3. Sign the request with the Root CA: openssl x509 -req -in rui.csr -CA myroot.crt -CAkey rui.key -CAcreateserial -out rui.crt -days 3650 <-extfile ext.txt> - The last parameter is only required if step 2 was done.
4. Export the signed vCenter certificate into PKCS12: openssl pkcs12 -export -in rui.crt -inkey rui.key -name rui -passout pass:testpassword -out rui.pfx - testpassword must be used, if you don't want to fall into the vpxd -p trap.
5. Copy over the newly generated files rui.key, rui.crt & rui.pfx onto your vCenter's C:\ProgramData\VMware\VMware VirtualCenter\SSL directory. Back up the 3 files there before overwriting.
6. On your vCenter, navigate to http://localhost/mob/?moid=vpxd-securitymanager&vmodl=1, login with a vC administrator, click reloadSslCertificate then Invoke Method. If you see Result: void, then you're done! Based on VMware KB #2009857.

Congratulations! You have a distributed CA and a vCenter that's using the cert chain.

As stated in VMware View 5.1 Installation guide on page 81, View will always try to check for revoked certs, and automatically won't trust any cert, that does not have a CRL linked or it's revocation URL can not be reached. To circumvent the extreme paranoid behaviour, there is a registry switch to turn this off (if you did not take the optional step, you'll need this).

Fire regedit on the View server, then create a string named "CertificateRevocationCheckType" under "HKLM\Software\VMware, Inc.\VMware VDM\Security" and set it's value to 1.

If you add your vCenter now, it's certificate shall be accepted without hassle.

But if you want to make View trust in itself (no red square on the dashboard for the connection servers in View Administrator), you still have to replace it's certificate too:

1. Generate a request for the View cert: openssl req -new -key rui.key -out view.csr -config openssl.cnf. As before, use FQDN as the Common Name.
2. (Optional) You can modify ext.txt if you created and used it as above.
3. Sign the csr: openssl x509 -req -in view.csr -CA myroot.crt -CAkey rui.key -out view.crt -days 3650 <-extfile ext.txt>
4. Export PKCS12: openssl pkcs12 -export -in view.crt -inkey rui.key -name vdm -passout pass: -out view.pfx
5. From here, see the View 5.1 Install guide pg. 75-77. Digested below:
6. Open the Local Computer Certificates on the View server. Run mmc, click on File->Add/Remove Snap-in..., click on Certificates, Add. Choose Computer Account->Next->Local Computer->Finish. Press OK.
7. Expand Certificates->Personal->Certificates. You shall see View's default generated certificate here. First, right-click on it, choose Properties. Change it's Friendly name to something. Press OK.
8. Right-click on the lower Certificates item, choose All-Tasks->Import...
9. Browse for view.pfx. It's password shall be empty. Enable the Mark this key as exportable checkbox! Next-next-finish.
10. The new key's Friendly name shall already be set to vdm, because of the PKCS12 export parameter -name, but double check it.
11. Stop and start the VMware View Connection Server service (wait for all depending services to stop) or reboot the computer (it is a windows after all).

After login, another nice green square shall await you. What a reward...

If you gone this far, than you probably already hate SSL Certificates. At least I did. Sorry for the long post. Over & out.