Today SPEC released a major update to its server energy efficiency rating tool, SERT. See press release.
Created initially for the requirements of the US EPA Energy Star program, it also takes into account the needs of regulatory agencies worldwide. SPEC also has considered the requirements of the
European Union’s DG ENTR Lot 9 Ecodesign Directive for enterprise servers, the Chinese National Institute of Standardization (CNIS), and Japan’s Ministry of Economy, Trade & Industry’s (METI) Top Runner energy efficiency program. The SERT user guide is available in Chinese as well as English: 服务器功效评估工具(SERT) -用户指南
Figure 2: Memory channel impact on CryptoAES.
From “The SERT Metric and the Impact of Server Configuration“
Copyright © 1988‐2017 SPEC
Among the news in SERT 2 is the addition of a single-value composite metric to quantify server efficiency across different workloads. An excellent description of SERT (referring specifically to the 1.1.0 version) was given at the Beijing 2016 SPEC Summit. See slides.
As SERT adoption spreads, computer vendors will be able to run a single test to produce data to establish compliance with energy regulations. Even when different nations set different threshold criteria and other requirements, the benefit to vendors remains of having to run a single test to sell worldwide instead of 195 different tests for every nation. For consumers the benefit is in having energy regulations tied to a measure that accurately reflects the energy usage of the computing equipment.
A Tool Not a Benchmark
SPEC is clear that SERT is a tool, not a benchmark. That seems like a distinction without a difference because it’s also clear that any benchmark can be used more generally as a tool to investigate performance characteristics, and that SERT scores could be used as benchmark results. The differences are in intent and in license restrictions.
Regulatory programs like US EPA Energy Star like to say if your product meets this value it is compliant with our low energy standards and it may be sold in certain realms, and if it fails to meet that value it is not compliant and other restrictions apply. They want a measure of the power and performance of the server “out of the box.” In traditional benchmarks by contrast we want to show how well the server can perform, typically including expert performance tuning. There are usually restrictions on what tuning may be performed in order that the benchmark result remains representative of the performance ordinary users will see in the real world. But the emphasis is on comparability between systems, hardware platforms and/or software systems.
Fair Use
SERT is run with a limited set of reasonable tuning options reviewed and agreed ahead of time by the SPEC committee. So for the tester it’s simple. Just install SERT and run the test. The scores are meant to determine compliance with energy regulations, not to compare one server or processor or software stack against another. For that purpose you do much better to use an energy efficiency benchmark such as SPECpower_ssj2008 or one of many other benchmarks from SPEC and other organizations which have adopted the SPEC Power Methodology.
The Fair Use Rules are the reason you don’t see lists of SERT results. The software license prohibits making such competitive comparisons. The combination of the legal restriction plus the ready availability of superior measures for marketing comparisons and for academic research has worked very well to keep SERT in its proper place in the regulatory domain.





ting companies. You could basically call us geeks. But then success in SPEC, for the company and the individual, is driven also by those engineers’ ability to work with their competition, understand divergent views on the market and the direction of technological innovation, find common ground, and make compromises. Politics. (If they could do politics in Washington half as well as our bunch of “geeks” things would run so smoothly.) Those who rise to the top like David are those with both an engineer’s understanding of what is necessary and a politician’s understanding of what is possible.