By Bob Cramblitt There’s a statement popularized by Mark Twain that describes the often-deceptive power of numbers: “There are three types of lies: lies, damned lies, and statistics.” Among people concerned with computer performance, the saying rears its head in a slightly altered form: “There are lies, damned lies, and benchmarks.” Like most wide-sweeping statements, … [Read more...] about Lies, damned lies, and benchmarks: What makes a good performance metric
The renamed performance evaluation software features new and updated workloads throughout the suite; it is free of charge to users. By Bob Cramblitt SPECworkstation 3, an all-new version of the benchmark formerly known as SPECwpc, was released on October 31. It is developed and maintained by the SPEC Workstation Performance Characterization (SPECwpc) group, which includes … [Read more...] about Inside the new SPECworkstation 3 benchmark
Hierarchy of benchmarking methodologies explained. By Bob Cramblitt An article last year in Jon Peddie's Tech Watch provided an overview of SPEC/GWPG benchmarks, citing the struggle in the early days of computer graphics between application-based and synthetic benchmarks to measure performance. For the vast majority who are not in the day-to-day performance evaluation … [Read more...] about Why application-based benchmarks matter
Detailed configuration information helps recreate the same performance as that documented on the SPEC website. By Bob Cramblitt A lot has been written about the movement of applications to the cloud, but if you are involved in tasks that demand outstanding GPU, CPU and/or storage performance, you likely rely on a professional-level workstation. The SPEC/GWPG group … [Read more...] about Using SPECwpc website results for workstation performance evaluation
Graphics and workstation vendors have their major interests in performance. By Bob Cramblitt "It's the fox guarding the henhouse." "Sounds like a case of collusion." "Conjures visions of a cabal of hooded individuals meeting secretively in a hotel basement." All these things have been said one time or another about benchmark committees composed primarily of … [Read more...] about Skin in the game: who are the best people to create benchmarks?