Tuesday, April 13

About da VIBs

VMware's 4th generation hypervisor's patching method is mainly based on a new package format, known as VIB. The abbreviation stands for vSphere Installation Bundle. Basically, a VIB is a unix AR archive. You can unpack any VMware VIB using ar x . By using the unix file command, You will see there are 2 types of VIBs: Debian binary package (format 2.0) and current ar archive. The second one (ar) only used on ESX 4, with the redhat based Service Console. It's VIBs contain 3 files:
  • descriptor.xml - It's name contains it's function pretty much. Current vibVersion for example is 1.4.5. The vibType tag says it is an RPM type vib, which means it may only contain an RPM. The pkgfile tag tells the installer the RPM's original name. Other tags state if the package needs maintenance mode or after-remediation reboot on the server, version numbers of the package or suitable ESX build numbers.
  • sig.pkcs7 - A digital signature of the package. Used to make it cryptographically impossible to fake an installation file. By using -nosigcheck with esxupdate, You can force to ignore this file. (See http://en.wikipedia.org/wiki/PKCS). This also invalidates Your support for the given ESX based on the argument's "unsupported" help message.
  • complete.rpm - The real RPM package, the file renamed from the tag above. See http://www.rpm.org/max-rpm/s1-rpm-file-format-rpm-file-format.html.
The first type, DEB package is a bit more interesting. There are 2 sub-types of this VIB based on descriptor.xml's tag: deb and cross. The deb subtype of Debian based VIBs are used only on ESXi thin hypervisor version, and contain the additional following files (descriptor.xml and sig.pkcs7 are obligatory in every VIB).
  • debian-binary - Debian archive version number: 2.0
  • control.tar.gz - Control files for the installation method.
  • data.tar.gz - Binaries to install/replace.
These are standard Deb package structure files, see http://tldp.org/HOWTO/Debian-Binary-Package-Building-HOWTO/x60.html. There is also another sub-type of the DEB package based VIB, and that is cross. It means, that it contains the 3 debian package structure files, plus an additional short.rpm. This type of VIB may be used on both ESX and ESXi, ESXi probably uses the debian files, while ESX uses the rpm.
This makes a matrix of VIB types/usability/files like this:

VIB main (unix file) VIB sub (vibType tag) Suitable Hypervisor Contained files
ar rpm ESX descriptor.xml, sig.pkcs7, complete.rpm
deb deb ESXi descriptor.xml, sig.pkcs7, debian-binary, control.tar.gz, data.tar.gz
deb cross both descriptor.xml, sig.pkcs7, debian-binary, control.tar.gz, data.tar.gz, short.rpm

Based on this info, and by using a simple VIB as a template, anyone can create a vSphere Installation Bundle. Of course VMware won't grant You a copy of it's PKCS7 signing private key, unless Your company is called, for example, Cisco. So Your custom VIB will fail to install with an error like "error checking PKCS7 signature" or "missing or invalid signature" etc. This is where -nochecksig comes in, and support goes out.
If You have a function on Your custom ESXi farm, that You have to implement on every host after the installation by using the unsupported busybox CLI, than unsigned VIBs may be usable for You. Currently I can think of home-brew scripts for ESXi PXE stateless for example.
Because of the major difference between ESX and ESXi, VMware and 3rd parties have to create most installation/init scripts for every device driver twice. I think most drivers fit into both kernels (at the same build number ofc). Cross VIBs contain the short.rpm as a patch between the debian and redhat install method, for VIBs that have identical scripts and binaries on both hypervisors.

Wednesday, April 7

Workstation, with ESX inside

I'm using VMware Workstation since the 2.x series. Currently 7.1 Beta on Linux x64. I'm now used to the -I parameter which is required for installing/uninstalling, I have some perl lib error or such which breaks the installer script.
The other thing I noticed is the new opengl support on linux. This broke entirely my Windows 7 guest's direct3D. Every time I try to run the Windows Experience Index, it just coredumps. Probably I would need to install some gl libs for the Linux, I'm just lazy.

But get to the point:
My favorite feature is the ability to run ESX in the WKS of course. Did this before with the monitor_control.restrict_backdoor on the 6.x series, but it's now extremely easy. The only thing You have to watch out is promiscuous mode. If You are not running WKS as root (I hope You don't), You will get the message about promiscuous mode is not enabled for security reasons. This is a bit false, it means the user running the process does not have the permission to use a file in the /dev filesystem. It is fairly easy to dodge this ofc., just by reading the KB. The trickier part is when You set up a team, and use a private lan segment. Than You have to chown/chmod the vmnet0 interface, which is not documented anywhere. Also, I set /proc/sys/vm/swapiness on the host to 40, to make it swap as little as possible when running a team of 3 VMs.

If You are running a team with a private lan, than You probably do this to have more than 1 ESX-VM. I used a linked clone of a freshly installed esx (3.5u5) to create a second one, with minimal disk usage. After I set up both of them, and an XP VM as vCenter, I found an interesting scenario. Both of them could ping the vCenter, the vCenter could ping both of them, but they could not each other.
The solution was tricky. As others might know, the service console is a virtual interface. The linked clone was made after the fresh install, with a service console already present. And of course, as a virtual NIC, it has a virtual MAC address too...
So I had 2 ESX boxes with the same SC MAC. First thing to do is to drop the vswif0 from CLI esxcfg-vswif -d vswif0. Then, when I recreated it (esxcfg-vswif -a -p "Service Console" -i 192.168.69.12 -n 255.255.255.0 vswif0), it "generated" the same HW address again! The same happened after I dropped vSwitch0 too (esxcfg-vswitch -d vSwitch0; esxcfg-vswitch -a vSwitch0; esxcfg-vswitch -L vmnic0 vSwitch0; esxcfg-vswitch -A "Service Console" vSwitch0). It would be obvious to use just vswif1 instead, but then, how do I change the MAC of a service console?
Redhat. The vswif0 interface's MAC is in /etc/sysconfig/network-scripts/ifcfg-vswif0. After changing the MACADDR line in there, just run ifdown vswif0 && ifup vswif0 and done.
This is only true for ESX. On  ESXi, You have to edit /etc/vmware/esx.conf, as ESXi does not have ifconfig command, or vswif0 interface, just vmk0 and esxcfg-vmknic.
As stated above, I used VI3, but I'm pretty sure, that things are the same in vSphere.

After I had 2 working ESX VMs, I used an NFS datastore for shared storage. The HA needed das.isolationaddress config, as the private lan segment did not have a gateway. Using the vCenter's IP as a GW works too. Now I can ran Tinycore in a VM in ESX in a VM in WKS. Takes about 10 mins to boot tough...

Tuesday, April 6

/sbin/init. VSZ also.

Woot. I finally started.
I was thinking of writing about these things a while ago. Will see if this takes to anywhere.
Well, TBH I don't have much to say now... "We apologize for the inconvenience."

Maybe just one note. I was reading Duncan's post about vShield Manager. My 5 cents:

  • vShield Zones is basically a set of linux VMs with iptables AFAIK. It's not a bad idea to run a VM as a firewall on every ESX You have, but I think it's got a bit overhead. I would prefer (yet!) segmenting my network zones into VMware clusters with 1 physical firewall appliance (ASA, Checkpoint) between them. I will peek into some details sometime in the future, it's on my list.
  • Better watch out for the version of documentation You read. Not just for VSZ of course. Particularly, vsz_10_admin.pdf got at least 2 versions I know of: EN-000167-01 & some older one (00?). The ancient one did not have the "Securing CLI User Accounts" part (pg. 63), which is an essential step, speaking about a security product.
  • This new part has a KB now: http://kb.vmware.com/kb/1012479