The workstation benchmark plays a limited role when deciding how to upgrade.

By Alex Herrera

When it comes to marketing professional-caliber workstations, we’re all accustomed—as either suppliers or consumers—to look toward benchmarks as a primary component of the decision-making process. Benchmarks have always been a first-order means for marketeers to sell potential buyers on the value of a new machine and for buyers to help justify that purchase. But their high profile belies the reality in which both parties criticize—and complain quite validly about—the veracity and complexity of their use. And in today’s technology landscape, there are ever more reasons to avoid heavy reliance on benchmark results when it comes to considering the investment in new hardware.

Whenever I review a new workstation, I pull out my battery of benchmarks to assess its performance, both in comparison to its predecessors as well as its current peers. And when I do, I also pull out my bag of disclaimers that emphasize the limitations of benchmarks, and the risks of relying too heavily on any one, or even set, of benchmarks. Benchmarks are an absolutely useful tool in the right hands and right context, but in and of themselves, they don’t answer anything.

The best way to put benchmarks in the right context is to take a step back and think about the big picture. What are benchmarks supposed to indicate, anyway? They are intended to measure performance, and to varying degrees, they can do that. They’re not perfect indicators of performance for a variety of reasons, the most notable of which is that benchmarks have to hard-wire test cases, and by definition fixed test cases can’t possibly represent typical usage for every buyer out there.

Still, the right use of the right benchmarks can give you a solid, valuable indication of how the machine is going to perform. But no matter how accurate a bellwether a benchmark might be in predicting the performance you’ll see, it still doesn’t get at the root of what you’re ultimately trying to accomplish when you open up your wallet and fork over cash for a new workstation—or any IT expenditure for that matter.

The ultimate goal is not to render an animated character or brake assembly 20% quicker. The ultimate goal is to get more done in less time, to boost your productivity and generate more revenue that you would have otherwise. And really, it’s more than that, it’s to crank up your revenue per day (or month or year, or whatever your time frame) enough that the incremental income in fairly short order pays for that investment and then starts adding to your bottom line. What we’re really looking for is not frames/sec or FLOPS but ROI, return on investment.

And yes, a benchmark that accurately (or at least reasonably) reflects the performance boost you’ll see is an indicator of ROI. But it’s just one indicator, and often it may be a relatively minor indicator. Because, especially in this age of technology, there are other means—often substantial means—of driving up your productivity that no benchmark measures. Take multiple monitors, for example. Moving to two (or three or more) screens from one is the most sure-fire way to get more done in less time, as any multi-monitor disciple will attest, yet no benchmark can measure that very human aptitude for multi-tasking, an aptitude that extra display real estate plays to.

And speaking of multi-tasking, whether the benefits of  today’s CPU and GPU parallel multithreaded architectures from Intel, AMD and Nvidia show up in benchmarks or not, they can most certainly provide the computational headroom for users to more effectively parallelize key tasks and compress their workflow timeline. This relatively new and evolving breed of multi-core technologies can reduce—often dramatically—the time required to churn through very-measurable computational tasks professionals face on a regular basis, things like rendering, finite element analysis (FEA) and computational fluid dynamics (CFD). But they’re also another great tool for the user to juggle more concurrent tasks as well, a use that’s at least as valuable for shortening schedules and boosting ROI, but one that’s not at all quantifiable by individual benchmarks.

The moral of the story is clear, and it is one that I suspect will grow more valid over time. Don’t treat capital investments in IT, including workstations, any differently than you would any other potential asset. The new asset must pay for itself and then some through higher productivity for the individual or team, with the end goal of achieving better financials. It’s about the return on the investment, and especially with today’s workstation technologies, those returns can depend as much on how the user leverages the tools as the raw capabilities of the tools. Benchmarks only provide an indication of the raw capabilities, and often, it’s within the individual leverage where the greatest potential returns lie.

Alex Herrera is a senior analyst with Jon Peddie Research.