Oracle VM on VMWare VSphere Hypervisor (ESXi)

Several years ago, I started to build my lab environment on CentOS 5.4 and VMWare server 2.0.
It was rather good and stable, even though performance was not to be targeted. I built a lot of virtual machines, including some Oracle VM Servers for the Peoplesoft VM template testing.
I wrote an article in 2009 describing my all environment here.

Although it was working fine, almost 4 years later, it’s time for a change.

I could continue to work with a based OS (CentOS or Oracle Linux), then have a virtual machine software on top of it (VMWare Player or VirtualBox).
But there was a lot of changes these last few years on the bare-metal virtualization world. VMWare introduced ESXi (the free version of ESX), and Oracle VM is also becoming to play a more important role.

So, let’s get rid off one layer (the bare OS) and go for a straight away virtualization environment, a bare-metal virtualization software. Ok, it will make that all the work which was done on OS level has to be backported to an other level (like the Dell Open Manage Server Administration, we’ll see it later in an other blog entry how to do it).

But which VM software ? I worked a lot with VMWare Server over the last few years as well as Oracle VM (for the Peoplesoft VM template). Whether the second is good, the first looks more GUI friendly and easier to use. My assumption was that the bare-metal VMWare ESXi would be also more GUI friendly.
Nothing wrong to have a try, so let’s go for a look on VMWare ESXi side.

And really, after playing around for a while, I’m not disappointed by my choice.
That’s a good software for me and my needs. Moreover, since I had a dozen of VMs previously built on VMWare Server, I used the VMWare Converter utility to convert them for ESXi compliance and that’s it. All up and running within the new environment.

As of now, I have my server Dell PowerEdge 2900III with VMWare ESXi 5.1 and 14 virtual machines, and I cannot complain, all works as expected.

Well, all worked as expected until I wanted to test my Oracle VM on top of the new ESXi installation (bear in mind, here Oracle VM is nothing but a virtual machine – for a second layer of virtualization here).
1. The Oracle VM Server works like a charm, but the VMs inside the Oracle VM Server were not reachable from outside the VM itself (outside the Oracle VM Console).
Trying to make it clear, I have the following : WMWare ESXi => Oracle VM Server => Peoplesoft virtual machine
So what ? It was working fine on WMWare Server but not here, on VMWare ESXi ?
After digging a lot, I found an option of the network security layer, the Promiscuous Mode, once turned on (to ‘Accepted’, this is not the default), the Peoplesoft VM is finally reachable from ‘outside’ the VM itself.
 

2. My Dell PowerEdge 2900III has two embedded NIC. I configured all my virtual machines to work on both, load balanced.
All VMs worked fine. All except those Peoplesoft VMs in the Oracle VM Servers. The connections to those Peoplesoft VMs (as a second layer of virtualization) were working, ping were working. Until a certain extend that I still cannot explain. The network was just broken for a couple seconds (such as 30 seconds), and up again (for a minute or so) and down again, etc. Of course, when you have an application server connection behind, it cannot work…
I spent several days on it, the solution was really simple though. Making the Oracle VM Server using a dedicated NIC solves this problem. 
  
Now I have a NIC for the Oracle VMs environment (Oracle VM Manager, Oracle VM Server for Peoplesoft databases, Oracle VM Server for applications), and an other NIC for all the other and ‘more’ regular VMs (one layer of virtualization only).

Should I say that’s obviously not a production environment ?
 
Happy, I’m back on board with the Peoplesoft VMs.

Nicolas.