Tech Analysis: Windows Vista Sucks Performance

Windows Vista is Microsoft's next-generation desktop operating system. But does it provide next-gen performance on today's PCs vs. its predecessor, Windows XP?

The CRN Test Center set out to compare Vista's performance against XP's. The result: You might not want to move off XP just yet.

Vista is a resource-hungry operating system. Its minimum hardware requirements of an 800 MHz processor, 512 Mbytes of memory and a 20-Gbyte hard drive prove the point. New systems running Vista are heartier still, typically having a dual-core processor of around 2 GHz and 1 Gbyte of memory.

Regardless of the platform, Vista takes a long time to boot compared with XP. That's probably why Vista's shutdown button has been moved from the familiar XP location and replaced by a sleep button. By encouraging users to put systems to sleep rather than turning them off, the systems will seemingly boot much faster. If a system running Vista takes longer to boot than one running XP, could the Vista system take longer to do other tasks as well?

The Test Center decided to check it out. A new system sent to the lab came loaded with Vista. The system, made by Polywell, contained an Asus M2 NBP-VM CSM motherboard with a 2.2 GHz AMD Athlon 64 X2 4200+ dual-core processor and 1 Gbyte of memory. Engineers benchmarked the system using PassMark Software's PerformanceTest benchmark, which can be used free by anyone for up to 30 days.

The PerformanceTest software runs multiple tests on the CPU, graphics subsystem, memory and disk drives to generate composite average scores for each category. The composite averages are then used to generate an overall PassMark rating for the entire system. The latest version of PerformanceTest is compatible with Windows XP and Vista. PerformanceTest results can be saved as image files, and links are provided here for the results of the two tests (click for Vista performance results and for XP performance results).

Running Vista, the Polywell system earned a PassMark rating of 391.3. Next, engineers wiped the system and loaded Windows XP Pro (version 2002 with SP2). Running XP, the system earned a PassMark rating of 468.3; that's 16.4 percent faster than Vista, not an insignificant difference. Race car drivers will go to great lengths to shave a few tenths off their elapsed times but rarely perform upgrades that hurt performance. Yet upgrading to Vista can lessen a system's performance by as much as 58 percent, depending on what's being processed.

The news was not all bad, however. The PerformanceTest benchmark ran 24 individual tests, and XP was faster than Vista in only 18 of those tests. In six of the tests, Vista came out on top. More specifically, Vista beat XP at CPU string sorting, 2D graphics shapes, simple 3D graphics, medium 3D graphics, memory writes and random disk seeks.

For CPU string sorting, XP processed 2064.8 thousand strings per second, while Vista processed 2080.1 thousand strings per second. For the 2D graphics shapes test, XP processed 29.2 thousand shapes per second, and Vista processed 30.4 thousand shapes per second. For the simple 3D graphics test, Vista processed 125.8 frames per second, and XP processed 134.0 frames per second.

In the medium 3D graphics test, XP processed 16.0 frames per second, while Vista processed 18.6 frames per second. For the memory write test, XP processed 950.5 MBps and Vista processed 954.6 MBps. And for the random disk seek test, XP processed 2.91 MBps, while Vista processed 3.88 MBps. Except for the random disk seeking, the difference between the other five tests is negligible.

But the bad news was bad. Of the 18 tests in which XP beat Vista, the difference in seven of them was significant.

For the 2D graphics lines test, Vista processed 76.3 thousand lines per second, whereas XP processed 138.3 thousand lines per second. For the 2D graphics rectangles test, Vista processed 39.7 thousand images per second, and XP processed 94.3 thousand images per second. That's a difference of 58 percent.

In the 2D graphics fonts and text test, Vista processed 115.2 operations per second, while XP processed 172.7 operations per second. For the small block memory allocation test, Vista processed 1349.3 MBps and XP processed 1861.6 MBps. For the large RAM memory test, Vista performed 127.2 operations per second, and XP performed 229.2 operations per second. For the sequential disk read test, Vista processed 32.4 MBps, whereas XP processed 65.4 MBps. For the sequential disk write test, Vista processed 39.4 MBps and XP processed 60.6 MBps.

The significant differences in these individual tests resulted in significant differences for the overall 2D graphics mark and overall disk mark. For the overall 2D graphics mark, Vista scored a composite average of 289.0, while XP scored 466.7. And for the overall disk mark, Vista scored a composite average of 273.4 and XP scored 466.1. The differences in those two main categories greatly contributed to the 16.4 percent overall difference between the two operating systems.

NEXT: Vista Vs. XP In .Net Performance

The low performance of the sequential disk writes hinted that Vista's .Net performance might be affected as well (click to see a comparison of Vista vs. XP .Net performance).

After running PerformanceTest under Vista, engineers installed and ran Microsoft's XML Mark 11 test to measure the .Net runtime performance. They then did the same with XP. The XML Mark 11 test can be downloaded from Microsoft's Visual Studio Download Center. XML Mark 11 simulates a multi-threaded server application running a SAX and a DOM parser. The test measures number of threads, retrieval speed of items in an XML document and various transaction processes. In addition to .Net, the XML Mark includes Java code. However, engineers used the C# portion of the test without modifying the code.

The XML Mark test was originally written by Sun Microsystems about two years ago to show Java's XML processing speed relative to .Net 1.1's XML parsing. With the release of .Net 2.0, however, Microsoft adopted the test and was able to show that C# code ran just as fast as Java code.

Because Vista includes .Net 3.0, engineers had to recompile the C# code using Visual Studio 2005 extensions for .NET Framework 3.0 (WCFWPF), from the November 2006 CTP version. Microsoft does not have a finished .Net 3.0 framework out yet for Visual Studio developers. In addition, engineers had to install the .Net Framework 3.0 Redistributable package on XP.

During compilation, a few warning messages were received because a couple of the XML API methods were obsolete. Microsoft changed some of the XML parsing technologies in its .Net framework between versions 2.0 and 3.0. For XML Mark to compile and run properly on both operating systems, engineers added .Net 3.0 as an additional prerequisite to .Net 2.0, instead of eliminating .Net 2.0 from the compilation process completely.

As suspected, Vista hampered .Net's performance as well.

The XML Mark showed that XML applications run faster on XP than on Vista when both operating systems use .Net 3.0 runtime. However, the test produced different results between the two parsers. With the DOM parser test, XP ran about 20 percent faster than Vista. However, Vista ran about 3 percent slower with the stream parser (SAX) test.

These results were expected and were in line with the PerformanceTest results. The DOM test provided a more precise view of .Net's memory architecture because of the large objects it created and deleted. .Net's garbage collection was far more stressed with the DOM test than with the stream test.

The results show that Vista's memory architecture is significantly slower than XP. Microsoft released a product that will make many .Net applications run slower. At this point, we do not recommend running client-based .Net applications on Vista that require large data processing. And unless it's imperative that users have an operating system with a more exciting look and feel, XP will offer better performance than Vista.