Plot expectation value benchmarks #168
Conversation
the new data isn't the same format
I think it's because the ideal values are coming out so close to 0 that the relative errors are blowing up. You can find all the most recent results in ucc/benchmarks/results/expval_2025-01-14_20.csv Lines 11 to 14 in 8471ab3 Not sure what the best course of action here is. |
|
Okay just as a sanity check can you plot the simulated and ideal expectation values and standard deviation, similar to what I did here for #58 (where I was running on real hardware)? This reminds me, we can also simply measure an array of observables in addition to ZZZZZZ like I did there. Maybe just adding in some that measure like XIIIIIII or XXXXXZZZZZ, etc. |
There was a problem hiding this comment.
Trying to understand why the relative errors are in the hundreds? What are the actual expectation values we're getting? Are they what we'd expect?
There was a problem hiding this comment.
Perhaps we shouldn't use the same observable on all circuits if it is giving answers that don't seem meaningful.
There was a problem hiding this comment.
Per @willzeng , we want to split off the customization of specific observables with the different benchmarks into a separate issue. Can complete this issue with the current all Z observable.
There was a problem hiding this comment.
It's a little suspicious how the QAOA, QV, and QCNN circuit results are all basically identical. We should make sure this is real and not an artifact of how we perform simulation (or something else)!
There was a problem hiding this comment.
Did you plot the expectation values themselves and standard deviations as suggested above?
There was a problem hiding this comment.
Let's compare the compiled gate counts between the different compilers in case they are returning approximately the same circuits.
Description
This PR refactors the expectation value benchmarking script to ensure it works with the
run_benchmarks.shscript. It also introduces new circuits to broaden the test of the expectation value testing. Scripts to visualize relative and absolute errors across different compilers over time are added, with one plot added to the README.@jordandsullivan we have the option to plot relative or absolute error, but relative is much higher for
ucc, which I don't understand the reason for.