Corvus Crow
The Fireraven
Thursday, 20. November 2008, 11:54
VMware Benchmarks
1. Introduction
Recently, VMware released the E.X.P Server Beta for free. One of the problems is the loss of performance; an operating system will always run faster directly on the hardware. To get an idea of this performance loss, I decided to do some benchmarks on different setups.

The hardware used for this test is not up to date, so the differences might be less noticeable on today's machines. But it still gives an impression of the differences and performance loss, depending on the setup.

1.1 Software
- CentOS 4.2 Server (Kernel 2.6.9-34.EL)
- Microsoft Windows 2000 Server (5.0, Build 2195)
- VMware Server E.X.P Beta (Build 20925)
- SiSoft Sandra Lite 2005.SR3 (Build 10.69)
- CPU-Z 1.32.1

1.2 Hardware
- Intel Pentium III, 700MHz
- Asus CUSI-M (Rev 1.03)
- 1x256MB, 1x128MB RAM (140MHz Front Side Bus)
- Western Digital Caviar WD400BB (EIDE 7200rpm 40GB 8.9ms 2MB ATA100)

1.2 Hardware, according to VMware
- Intel Pentium III E, 500MHz
- Intel 440BX/ZX

2. Windows on Linux

2.1 Installation of the host system
The installation of CentOS is pretty straightforward. Most of the options were left with their default settings and TCP/IP was set to DHCP; changing this to a static value shouldn't affect the results. Same goes for keyboard layout, timezone and language.

The harddrive was partitioned with Disk Druid as shown below. Since everything runs from only one drive, the partition layout doesn't matter that much. Of course an additional drive for the virtual machines would improve the performance. To keep the system as slim as possible, no extras were installed; only a minimal system.

/dev/hda1 /boot ext3 125 1 16
/dev/hda2 / ext3 2047 17 277
/dev/hda3 /virtual ext3 35479 278 4800
/dev/hda4   Extended 510 4801 4865
/dev/hda5   swap 510 4801 4685

To get VMware running we need some sort of desktop. I decided to use XFCE, since it is designed for speed and performance, which leaves more resources for the virtualisation. After the basic installation was done, a yum update will get the installed packages updated to their latest version. It turns out that CentOS does not add a compiler in the minimal setup but one is needed to install VMware; yum install gcc fixes this. The next thing is a X server, which is quickly installed with yum install xorg-x11 and then started with service xfs start. To set the resolution and color depth I ran yum install system-config-display followed by system-config-display and made the correct settings. At this point, yum install xfdesktop xfce-utils will install the desktop. Since only VMware will run in it, there is no need for additional packages. Now the host setup is done and after a reboot, everything should be running fine. VMware can be installed easily, because there is a rpm available. After registration and download, rpm -ivh VMware-server-e.x.p-20925.i386.rpm quickly adds it to the system. The only thing left to do is running vmware-config.pl; I accepted all default settings except the path to the virtual machines. This was set to /virtual. Then startxfce4 brings up the desktop. For convenience, I added a launcher for VMware.

2.2 Installation of the guest system
Now it's time to set up a virtual machine. I decided to use a custom machine with the guest system set to Windows 2000, bridged networking and 176MB RAM. A 2GB SCSI disk was created for the Windows installation; another 1GB disk for the performance tests.

Installing Windows from an ISO took about 45 minutes. Like before, all packages (like IIS and the Index Service) were disabled to keep the size down. The VMware Tools will speed up the system considerably, so it's good to install them as soon as possible. It turns out that the post-installation was the most time consuming part. To get the system up to date, I installed Servicepack 4, Internet Explorer 5.5 (I prefer this one) and DirectX 9. After a few reboots in the process, the latest patches were installed via Windows Update. All this took an additional 2.5 hours. The last thing to install was SiSoft Sandra; then everything was ready for the benchmarks.

2.3 Benchmarks
Every benchmark was done in fullscreen mode and repeated 5 times to get average values.


3. Windows on Windows

3.1 Installation
Like with CentOS, Windows was installed with the default settings. The harddrive was split into two partitions (4GB and 33GB) and formatted with NTFS. The basic installation was of course faster than in VMware, taking 30 minutes. However, Windows had a hard time with the onboard devices; it failed to recognize sound, network and video. Since I did not want to burn drivers and updates onto CD, I added another network card (3com 3C905B-TX PCI) temporarly and later removed it again. Just like the virtual machine, the system was completely updated before testing. All this took almost 2 hours; without counting in the driver downloads from Asus. Their servers are terribly slow, and I had to wait 45 minutes to get the 20MB. If you add this, the post installation under VMware was faster.

Installing VMware turned out not to be the simple click-and-go procedure. After the installer was done, VMware refused to start and complained about a missing MSVCP60.dll library. Even after this file was added manually to the system, it didn't work; this time with another error. However, the error message also suggested that reinstalling VMware might help, and it indeed did.

3.2 Benchmarks
Every test was done five times in fullscreen mode. The major difference, compared to the Linux host system, was the slow boot time of the guest system.


4. Windows only

4.1 Installation
This was the fastest installation, since I only had to roll back an image I made after installing the basic system for the Windows on Windows test. The only thing left to do was setting up the benchmark tools.

4.2 Benchmarks
As always, every benchmark was repeated 5 times. The most surprising result was the filesystem benchmark: according to SiSoft Sandra, this machine only made about 5MB/s, compared to the 25MB/s and more in the virtual systems. This also explains the slow boot of the guest Windows system. To rule out a harddrive problem, I re-installed the Windows on Linux system and repeated the filesystem tests. The results were over 25MB/s again, so obviously Windows had a problem with the drive.


5. Results
For the host system, Linux scored better in 10 of 14 benchmarks; Windows in the other 4 tests. The strangest result was the File System benchmark, where Windows only reached about 20%, compared to the tests in the virtual machines. A check under Linux with hdparm -t -T resulted in 26.73 MB/s, so the 5MB/s of Windows must have been are problem of the operating system itself. Such a low speed affects the usage of the guest system drastically, especially when booting up the virtual machine. In the end, I can only recommend using Linux as a host system for VMware.