Researchers at Sandia National Laboratories propose a very large family of benchmarks for probing the performance of quantum computers.
They call them Volumetric Benchmarks (VBs) because they generalize IBM’s benchmark for measuring Quantum Volume.
The Quantum Volume benchmark defines a family of square circuits whose depth d and width w are the same. A Volumetric Benchmark defines a family of rectangular quantum circuits, for which d and w are uncoupled to allow the study of time/space performance trade-offs. Each VB defines a mapping from circuit shapes — (w,d) pairs — to test suites C(w,d). A test suite is an ensemble of test circuits that share a common structure. The test suite C for a given circuit shape may be a single circuit C, a specific list of circuits {C1…CN} that must all be run, or a large set of possible circuits equipped with a distribution Pr(C).
The circuits in a given VB share a structure, which is limited only by designers’ creativity. The team has listed some known benchmarks, and other circuit families, that fit into the VB framework: several families of random circuits, periodic circuits, and algorithm-inspired circuits. The last ingredient defining a benchmark is a success criterion that defines when a processor is judged to have passed a given test circuit.
The scientists have discussed several options. Benchmark data can be analyzed in many ways to extract many properties, but they proposed a simple, universal graphical summary of results that illustrates the Pareto frontier of the d vs w trade-off for the processor being benchmarked.