«

»

Jan 25

VMware Server 2, Proxysvc SSL Handshake problems

Well this little issue had annoyed me for a while. After upgrading my VMware Server 2.x setup to the latest version and upgrading the host to the most current centOS 5.4 x64 OS, the local secure web interface stopped working. Now it seems that the reason for this problem is simple.

So what’s the deal here, you might wonder?

After running the aforementioned upgrades sometime in November, I tried to connect to the web interface of my VMware Server, only to find out it wasn’t working anymore. The error displayed in my beloved firefox browser was “Connection interrupted”. This was not good, not good at all.

Now I could still connect to the Virtual Machines on the host using the VMware Remote Console (vmrc), so I knew the VM’s were still running fine.  The  /var/log/vmware/hostd.log file showed this fine error when trying to connect:

 Proxysvc SSL Handshake on client connection failed: SSL Exception

Ain’t that a beauty?  Well after trying all sorts of troubleshooting steps, I found out I was not alone, see this VMTN thread:

No WebAccess to VMware Server 2: Proxysvc, SSL Exception, session id context uninitialized

It appeared to be a somewhat common problem, which is sort of good, as you then know it might be resolved soon…

Now further troubleshooting showed that you can still manage the host via the Virtual Infrastructure Client or via -drumroll- a firefox browser from another machine running Windows XP SP3. Hmm…

Of course not a really satisfying solution if you just want to manage VM’s directly from within your host that is running the VMs. Now there is an easy answer to that one as you can connect using the plain http interface. So instead of using https://localhost:8333 you would use http://localhost:8222

As I had other things on my mind and as it is a local lab setup it wasn’t such a big security problem, so I let it go for a while.

Today however someone replied to the VMTN thread and this time I figured that the problem is not just with VMware Server, but that it is due to a combination between the local setup of VMware Sever and the local Firefox install.

So in order to test that theory, I installed Opera on my centOS 5.4 host and low and behold, the secure web interface works, I can log in just fine, so the SSL handshake works with Opera. There must be something fishy with the default firefox setup and SSL on centOS, it would be interesting to see what happens if you install a non repository version of firefox on the host, but I’m running out of time for that experiment now.

Playing a bit more with the Opera browser shows that not everything is working as expected so it certainly is not a solution, but the reason for this is not because of the SSL handshake now as it behaves the same on the plain HTTP interface. Maybe it is due to some settings in Opera, will have to test some more and see.

Edit:

Dayworker replied to the forum post and made a remark about how-to fix the issue on firefox 3.6. It turns out that  SSL2 is disabled in Firefox 3.6, this turned out to be my problem on the default firefox 3.0.x setup as well… He referred to a post in a german VMware forum where shecki found out about this little tid bit.

So if you encounter this then try changing the following in about:config

security.enable.ssl2 from false into true 

For me it solved my firefox problem completely. Thanks guys for sharing this.

3 comments

  1. Andre Ustoopia

    Thanks for this great post! changing security.enable.ssl2 in about:config in firefox 3.6 also solved the issue for me!

  2. Xang Xiong

    Glad you were sharing this useful info. I ran to the same problem and through search using google, I found suggestions to clear my cache and ssl encryption key. They both failed to resolve the issue. I was fortunate enough to have found your blog. It work perfectly. Thanks.

  3. dishu

    Since I upgraded to Vmware 2.0.2 something broke on the server side as well.

    The vmware console jave applet doesnt work properly. First it didn’t accept the keyboard arrows then it kept releaseing grabbing the console. I tried everything like removing the vmware plugin, deleting the whole browser profile, running new firefox 3.5X install on windows xp windows7 but the problem stills remained so I removed 2.0.2 and reinstalled the old 2.0.1.

    It’s the same problem again. Can someone help? Im sure it’s on the server side.

    Thanks

Comments have been disabled.