Performance regression of BackendSamplerV2 and BackendEstimatorV2

[t-imamichi]

What should we add?

We updated Primitives V2 to enable SamplerV2 and EstimatorV2 to handle different numbers of shots or precision for each pub.
To address this, I implemented BackendSamplerV2 and BackendEstimatorV2 to call backend.run for each pub.
On the other hand, BackendEstimatorV1 and BackendSamplerV1 assume the same number of shots per run call and pass all circuits to backend.run only once.

Copied from Qiskit/qiskit-ibm-runtime#1631

Qiskit main (7175368)

from timeit import timeit

from qiskit_aer import AerSimulator
from qiskit_ibm_runtime.fake_provider import FakeSherbrooke

from qiskit import QuantumCircuit
from qiskit.primitives import BackendSampler, BackendSamplerV2
from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager

backend = AerSimulator.from_backend(FakeSherbrooke())
shots = 10000
num_copies = 20


def gen_circuit(num_qubits: int, reps: int):
    qc = QuantumCircuit(num_qubits)
    for _ in range(reps):
        qc.h(range(num_qubits))
        for i in range(0, num_qubits - 1, 2):
            qc.cx(i, i + 1)
    qc.measure_all()
    return qc


def bench_sampler_v1(qc: QuantumCircuit):
    print("\nBackendSamplerV1")
    sampler = BackendSampler(backend)
    print(f"{timeit(lambda: sampler.run([qc] * num_copies, shots=shots).result(), number=1)} sec")


def bench_sampler_v2(qc: QuantumCircuit):
    print("\nBackendSamplerV2")
    sampler = BackendSamplerV2(backend=backend)
    print(f"{timeit(lambda: sampler.run([qc] * num_copies, shots=shots).result(), number=1)} sec")


qc = gen_circuit(5, 5)
pm = generate_preset_pass_manager(optimization_level=2, backend=backend)
qc2 = pm.run(qc)
bench_sampler_v1(qc2)
bench_sampler_v2(qc2)

output

BackendSamplerV1
0.7136987919984676 sec

BackendSamplerV2
9.465235792002204 sec

[t-imamichi]

#12291 fixes the performance issue.

output with main

BackendSamplerV1
0.5204217499995138 sec

BackendSamplerV2
0.5375754169945139 sec

Here is also results of estimator #12291 (comment)