HOW USEFUL ARE STORAGE BENCHMARKS?

The article considers the effectiveness of using industry-standard benchmarks to evaluate storage systems. The author believes that the benchmarking process can be easily manipulated because of the large number of variables that influence the result of performance assessment. The process can also be limited by available methods that can be used to configure storage systems such as the number of spindles, cache size and fault tolerance configuration. Included in the factors that can influence the veracity of these benchmarks is the emergence of do-it-yourself benchmarks. It is recommended that the Storage Performance Council (SPC) and Standard Performance Evaluation Corp. (SPEC) set rules on permissible configurations.