I did some benchmarks a while back (August 2007) that I had always intended to post online somewhere, and since I've finally gone and made this blog I'm posting them, albeit a little late.
Now firstly, there have been updates to Parallels and Fusion since August '07, so these benchmarks are out of date, but I don't think that makes them totally worthless. Having updated Parallels and Fusion in April this year, and tested both with Visual Studio 2005, my (subjective) feeling is that the performance (at least for compiling a C++ project in Visual Studio in Windows) has changed very little since August last year. Still, I'm supplying these benchmarks with a few grains of salt...
I ran these benchmarks using SiSoft Sandra, which I've always liked for benchmarking (you get each measure individually, and it doesn't try to combine them all into a single weighted average that has no real-world measurable value - but then again, as an Engineer and Physicist I may be a little biased against calculations that fail on dimensional analysis... sorry, I digress...).
My platform is a MacBook Pro 15", 2.2 GHz Core 2 Duo (4MB), 800 MHz Bus, 2 GiB 667 MHz RAM (2x1 GiB DIMMs), GeForce 8600M GT 128MB 16xPCIe (and so on, as per the stock MacBook Pro from August '07 with the 160 GB HDD upgrade).
The benchmarks were run in SiSoft Sandra as mentioned above, in a Windows XP machine on my MacBook Pro. It was actually the same Windows XP install for all three setups - I installed Windows in Boot Camp and was able to run the system either by launching one of the Virtual Machines (Parallels Desktop or VMWare Fusion) or by booting in via Boot Camp. The upside of this was consistency (and for me, ease of switching between VMs/Boot Camp for my actual work), the downside is that any hardware detection that takes place only at install time may cause a lower than optimal performance on one of the VMs (I mention this only because I recall something about needing to re-install Windows XP if you upgrade to a multi-core processor or one that supports certain new CPU instruction sets if you want to make use of the extra cores/instructions, so it might be relevant... I'm not sure). This would also therefore affect the disk performance for the VMs, since they're directly accessing the disk rather than a virtual disk.
The software versions at the time of benchmarking were
- Parallels Desktop 3.0 build 4560
- VMWare Fusion 1.0 (either RC1 or build 51348 - can't remember which)
- Boot Camp (beta 1.3 or 1.4 - can't remember which)
Where possible the machines were set up identically, however Parallels still only supports a single processor for virtualisation. This obviously caused some major differences in the benchmarks, but is not necessarily a Bad Thing™ when you're using a VM. A single core is fine if the application(s) you use in the VM are, on the whole, restricted by a single thread.
So down to the benchmarks... In each of the images below, the systems are represented as follows
- Parallels - Red bars/lines, labelled "Current X" (e.g. Current Processor) - as the screenshots were taken from Parallels.
- VMWare Fusion - Orange bars/lines, labelled "VMWare Fusion 1.0"
- Boot Camp - Green, labelled "BootCamp"
For a good (professional) explanation of these benchmarks (by the people who made them) you should check out SiSoft's Benchmarking 101 Pages.
First up, Processor Multi-Media - Fusion runs almost native speed, Parallels falls way behind at about 25% of native. Now given Parallels is only running with one processor, one would expect it to be half the native speed, so it seems as if the one core is running about half the speed (when it comes to Multi-Media instructions) compared to the cores when running natively or in Fusion.

Processor Arithmetic - Again Fusion is close to native speed (92.2% Dhrystone, 93.0% Whetstone), and Parallels, as expected, is sitting at about half native speed due to the single processor virtualisation (47.6% Dhrystone, 54,0% Whetstone). Actually, 'per-core' Parallels outperforms Fusion here by a few per cent points.







From my personal experience (here comes the subjective bit...) running Windows XP in Parallels and Fusion, even though Parallels only supports a single processor for virtualisation, Fusion seems to use a lot of its second core for processing graphics. I'm not 100% sure of this, but Parallels seems to do a better job (and this might be due to its OpenGL support? since Fusion lacks OpenGL support... clutching at straws though) of passing the work off to the graphics card, while Fusion does more of the rendering in CPU-land. I've seen Fusion sitting there using something like 80% of one processor while Windows is sitting there doing nothing but displaying a few windows; Parallels was down at 10-20% of the one processor (though the GPU does produce plenty of heat while its running).
A more objective test, and this is what has me sold (for now...) is that the C++ library I spend a lot of time using at work takes about 3-4 minutes to compile in Visual Studio 2005 under Parallels, and about 12 minutes under VMWare Fusion. This is a test I ran a few weeks back (April 2008) with the latest version of both Parallels and Fusion, and I came to the same conclusion (that compiling a large C++ project in VS2005 was faster in Parallels than Fusion) in August 2007 as well. As for Boot Camp, I've given up on that completely - I hate rebooting (and the fact that Windows thinks the system clock should be in local time for some really stupid reasons, despite good arguments to the contrary).
I guess this suggests that compiling (at least the combination of C++, VS2005, Windows XP, and our code-base) is more memory-bandwidth/latency-bound than processor/disk-speed-bound.
Fortunately, I've ported this library and some of our applications to Mac OS X (thanks to Leopard's POSIX compliance) so I don't have to spend much time in the VMs any more (;
On a final note, I've noticed that a lot of people compare them by their features, interface, 'feel' and such, saying that one or the other feels more like a Mac program, or feels more robust, etc. I've avoided any mention of these because I didn't find them different enough in these areas to be able to suggest everyone (or even a significant majority) would feel the same way on these subjective matters.
Apart from graphics support and number of CPUs virtualised, they're very very similar applications on the surface, though performance can differ quite a bit for applications that are bound to the performance of specific components of the system.
2 comments:
great post, thanks for the tips! i'm actually trying to do the visual studio thing on my mac, so that last part really helps me decide to give parallels a try.
Who knows where to download XRumer 5.0 Palladium?
Help, please. All recommend this program to effectively advertise on the Internet, this is the best program!
Post a Comment