CritPt

Example Challenge: Quantum Error Detection

Setup

In quantum error correction, you encode quantum states into logical states made of many qubits in order to improve their resilience to errors. In quantum error detection, you do the same but can only detect the presence of errors and not correct them. In this problem, we will consider a single [[4,2,2]] quantum error detection code, which encodes two logical qubits into four physical qubits, and investigate how robust logical quantum operations in this code are to quantum errors.

Our convention is that the four physical qubits in the [[4,2,2]] code are labelled 0,1,2,3. The two logical qubits are labelled A and B. The stabilizers are $XXXX$ and $ZZZZ$, where $X$ and $Z$ are Pauli matrices. The logical $X$ and $Z$ operators on the two qubits are $X_A = XIXI$, $X_B=XXII$, $Z_A = ZZII$, $Z_B = ZIZI$, up to multiplication by stabilizers.

We will consider different state preparation circuits consisting of controlled not $CNOT_{ij}$ gates, where $CNOT_{ij}$ has control qubit $i$ and target qubit $j$. As a simple model of quantum errors in hardware, we will suppose that each $CNOT_{ij}$ gate in the circuit has a two qubit depolarizing error channel following it that produces one of the 15 non-identity two-qubit Paulis with equal probability $p/15$. The probability $p$ indicates the probability of an error in a single two-qubit gate. We will assess the logical infidelity of certain state preparation protocols as a function of the physical infidelity $p$.

Challenge Question

Suppose that we prepare a logical two-qubit $|00\rangle_{AB}$ state in the [[4,2,2]] code. To do so, we introduce an ancilla qubit, qubit 4, and use the following state preparation circuit:

$$M_4 (CNOT_{04}) (CNOT_{34}) (CNOT_{23}) (CNOT_{10}) (CNOT_{12}) (H_1)$$

Note that this equation is written in matrix multiplication order, while the quantum operations in the circuit occur in the reverse order (from right-to-left in the above equation). $H$ is a single-qubit Hadamard gate and $M$ is a single-qubit measurement.

The ancilla is used to detect errors in the state preparation circuit and makes the circuit fault-tolerant. If the ancilla measurement is $|0\rangle$ ($|1\rangle$), the state preparation succeeds (fails).

What is the logical state fidelity of the final 2-qubit logical state at the end of the circuit as a function of two-qubit gate error rate $p$, assuming the state is post-selected on all detectable errors in the code and on the ancilla qubit measuring $|0\rangle$?

Answer:

$$F_{\rm{logical}} = 1-\frac{\tfrac{16}{25} p^2 - \tfrac{128}{125} p^3 + \tfrac{2048}{3375} p^4 - \tfrac{32768}{253125} p^5}{1 - \tfrac{68}{15} p + \tfrac{704}{75} p^2 - \tfrac{32768}{3375} p^3 + \tfrac{253952}{50625} p^4 - \tfrac{262144}{253125} p^5}$$

Checkpoint 1

Suppose that we wish to prepare a logical two-qubit GHZ state $(|00\rangle_{AB}+|11\rangle_{AB})/\sqrt{2}$ in the [[4,2,2]] code. To do so, we use the following state preparation circuit:

$$(CNOT_{03}) (H_0) (CNOT_{21}) (H_2)$$

Note that this equation is written in matrix multiplication order, while the quantum operations in the circuit occur in the reverse order (from right-to-left in the above equation). $H$ is a single-qubit Hadamard gate.

What is the physical state fidelity of the final physical 4-qubit state at the end of the circuit as a function of the two-qubit gate error rate $p$?

Answer:

$$F_{\rm{physical}} = (1-\tfrac{12}{15}p)^2$$

Checkpoint 2

Suppose that we wish to prepare a logical two-qubit GHZ state $(|00\rangle_{AB}+|11\rangle_{AB})/\sqrt{2}$ in the [[4,2,2]] code. To do so, we use the following state preparation circuit:

$$(CNOT_{03}) (H_0) (CNOT_{21}) (H_2)$$

Note that this equation is written in matrix multiplication order, while the quantum operations in the circuit occur in the reverse order (from right-to-left in the above equation). $H$ is a single-qubit Hadamard gate.

What is the logical state fidelity of the final 2-qubit logical state at the end of the circuit as a function of the two-qubit gate error rate $p$, assuming the state is post-selected on all detectable errors in the code?

Answer:

$$F_{\rm{logical}} = 1 - \frac{\tfrac{16}{75}p^2}{1-\tfrac{8}{5}p + \tfrac{64}{75}p^2}$$

Checkpoint 3

Suppose that we prepare a logical two-qubit $|00\rangle_{AB}$ state in the [[4,2,2]] code. To do so, we introduce an ancilla qubit, qubit 4, and use the following state preparation circuit:

$$M_4 (CNOT_{04}) (CNOT_{34}) (CNOT_{23}) (CNOT_{10}) (CNOT_{12}) (H_1)$$

Note that this equation is written in matrix multiplication order, while the quantum operations in the circuit occur in the reverse order (from right-to-left in the above equation). $H$ is a single-qubit Hadamard gate and $M$ is a single-qubit measurement.

The ancilla is used to detect errors in the state preparation circuit and makes the circuit fault-tolerant. If the ancilla measurement is $|0\rangle$ ($|1\rangle$), the state preparation succeeds (fails).

What is the logical state fidelity of the final 2-qubit logical state at the end of the circuit as a function of two-qubit gate error rate $p$, assuming the state is post-selected on all detectable errors in the code and on the ancilla qubit measuring $|0\rangle$?

Answer:

$$F_{\rm{logical}} = 1-\frac{\tfrac{16}{25} p^2 - \tfrac{128}{125} p^3 + \tfrac{2048}{3375} p^4 - \tfrac{32768}{253125} p^5}{1 - \tfrac{68}{15} p + \tfrac{704}{75} p^2 - \tfrac{32768}{3375} p^3 + \tfrac{253952}{50625} p^4 - \tfrac{262144}{253125} p^5}$$

Detailed solution:

Checkpoint 1

There are $15$ two-qubit Paulis that occur with probability $p/15$ in the depolarizing error model after both of the two-qubit CNOT gates. The described quantum circuit prepares two independent GHZ states $(|00\rangle+|11\rangle)/\sqrt{2}$ on disconnected qubit pairs, qubits 0,3 and qubits 1,2. These GHZ states are $+1$ eigenstates of the $XX$, $YY$, and $ZZ$ Paulis on those qubits. Therefore, those $3$ Paulis in the depolarizing error model do not affect the state while the other $15-3=12$ Paulis when applied to the state take it to a different orthogonal state. Since with probability $12p/15$ each depolarizing error model results in a state orthogonal to the ideal state, the density matrix produced by the circuit is

$\rho = \left(1-\frac{12p}{15}\right)^2 |\psi\rangle\langle \psi| + \left[1-\left(1-\frac{12p}{15}\right)^2\right]\rho_\perp$

where $|\psi\rangle = \frac{1}{2}(|00\rangle+|11\rangle)_{03}(|00\rangle+|11\rangle)_{12}$ is the ideal state produced by the circuit and $\rho_{\perp}$ is a quantum state orthogonal to the ideal state. The state fidelity of this physical qubit state is then

$\begin{align} F_{\rm{physical}} &= \langle \psi | \rho | \psi \rangle = \left(1-\frac{12p}{15}\right)^2 \langle \psi|\psi\rangle\langle \psi |\psi\rangle+ \left[1-\left(1-\frac{12p}{15}\right)^2\right]\langle \psi| \rho_\perp | \psi \rangle \\ &= \left(1-\frac{12p}{15}\right)^2. \end{align}$

Checkpoint 2

Let us consider all possible combinations of errors that can occur in the two noisy $CNOT$ gates in the circuit. There are three situations that can occur: (1) neither gate has an error (occurs with probability $(1-p)^2$), (2) one gate has an error (occurs with probability $2p(1-p)$), (3) or both gates have an error (occurs with probability $p^2$). In situation (1), no detectable errors have occured. In situation (2), $3$ of the $15$ Pauli errors (again $XX$,$YY$,$ZZ$) are not detectable errors. Situation (3) is a bit more complicated. When both gates have an error, there are $15^2=225$ possible Pauli errors that can occur in total. Enumerating the Pauli errors, we can check how many of them commute with the stabilizers $XXXX$ and $ZZZZ$. The result is that $57$ out of $225$ Pauli errors commute with the stabilizers and are therefore undetectable.

Altogether, one then sees that the probability of not observing any detectable errors is

$\begin{align} P_{\rm{no\, detected\, errors}} &= (1-p)^2 \cdot 1+ 2p(1-p) \cdot \frac{3}{15} + p^2 \cdot \frac{57}{225} \\ &= 1-\frac{8}{5}p + \frac{64}{75}p^2 \end{align}$

Let us consider the final density matrix produced at the end of the circuit. When both gates have an error, there are $225$ possible Pauli errors, $57$ of which are undetectable. These undetectable errors commute with the stabilizers and so are logical operators or logical operators multiplied by stabilizers. Importantly, the final logical GHZ state being produced by the circuit $(|00\rangle_{AB}+|11\rangle_{AB})/\sqrt{2}$ is a $+1$ eigenstate of the logical $X_A X_B = IXXI$ and $Z_A Z_B=IZZI$ operators. Therefore, any Pauli errors that are logically equivalent to these operators, up to multiplication by stabilizers, do not affect the logical state. By going through the $57$ undetectable Pauli errors, we find that $9$ of them are logically equivalent to $X_A X_B$ and $Z_A Z_B$ (up to stabilizers) and so do not affect the logical state. In summary, when both gates have errors there are $225$ possible Pauli errors, $225-57=168$ of which are detectable, $57-9=48$ of which are undetectable and cause logical errors, and $9$ of which are undetectable and cause no logical error.

Now we know enough information to express the final density matrix. It takes the form

$\begin{align} \rho &= (1-p)^2 \rho_0 \\ &\quad +2p(1-p)\left[\frac{12}{15}\rho_{\rm{det}}^{(1)} + \frac{3}{15}\rho_0\right] \\ &\quad +p^2\left[\frac{168}{225}\rho_{\rm{det}}^{(2)} + \frac{48}{225}\rho_{\rm{logical\, error}} + \frac{9}{225}\rho_{0}\right] \end{align}$

where $\rho_0\equiv |\psi\rangle \langle \psi|$ is the ideal output state of the circuit, $\rho_{\rm{det}}^{(1)}$ is the density matrix post-selected on detectable errors from a single gate error, $\rho_{\rm{det}}^{(2)}$ is the density matrix post-selected on detectable errors from two gate errors, and $\rho_{\rm{logical\, error}}$ is the density matrix post-selected on logical errors (and no detectable errors) from two gate errors.

To compute the logical fidelity post-selected on no detected errors, we need to compute the final state's *logical* density matrix, which is the physical density matrix above projected onto the $+1$ eigenstates of the $XXXX$ and $ZZZZ$ stabilizers. This logical density matrix is

$\begin{align} \rho_{\rm{logical}} &= \frac{1}{P_{\rm{no\, detected\, errors}}}\left\{ (1-p)^2\rho_0 +\frac{6}{15}p(1-p)\rho_0 +p^2\left[\frac{48}{225}\rho_{\rm{logical\, error}} + \frac{9}{225}\rho_{0}\right] \right\} \\ &= \frac{1}{P_{\rm{no\, detected\, errors}}}\left\{ \left[(1-p)^2 +\frac{6}{15}p(1-p)+\frac{9}{225}p^2\right]\rho_0 + \frac{16}{75}p^2\rho_{\rm{logical\, error}} \right\} \\ &= \left(1-\frac{\frac{16}{75}p^2}{P_{\rm{no\, detected\, errors}}}\right)\rho_0 + \frac{\frac{16}{75}p^2}{P_{\rm{no\, detected\, errors}}}\rho_{\rm{logical\, error}} \end{align}$

where $P_{\rm{no\, detected\, errors}}=1-\frac{8}{5}p + \frac{64}{75}p^2$ is a normalization constant that ensures that $\textrm{tr}(\rho_{\rm{logical}})=1$ is a valid normalized state. Note that $\rho_{\rm{logical\, error}}$ is orthogonal to $\rho_0$.

The logical state fidelity is then

$\begin{align} F_{\rm{logical}} &= \langle \psi |\rho_{\rm{logical}}|\psi\rangle = 1-\frac{\frac{16}{75}p^2}{P_{\rm{no\, detected\, errors}}} \\ &=1-\frac{\frac{16}{75}p^2}{1-\frac{8}{5}p + \frac{64}{75}p^2}. \end{align}$

Checkpoint 3

For the state preparation circuit for the $|00\rangle_{AB}$ logical state, the $CNOT$ gates are not simply at the end of the circuit as for the earlier subproblems, so the analysis is more complicated. In order to determine the effects of the errors on the logical state, we need to enumerate all possible $16^5=1048576$ Pauli errors and analyze how they propagate through the circuit. This is feasible because the entire circuit is a Clifford circuit, and the propagation of Pauli operators through Clifford circuits can be efficiently simulated classically.

We solve this problem algorithmically with the Python script attached below. The script applies the transformation rules for how the Clifford $CNOT$ gates in the circuit transform Pauli strings into other Pauli strings for each of the possible $16^5$ Pauli error configuration. For each error configuration, we count how many (non-identity) gate errors occurred $n=0,1,2,3,4,$ or $5$ and propagate the Pauli errors through the circuit to obtain a final Pauli string $P$. We then assess if that Pauli string leads to an error detection event (or ancilla check event) and if the Pauli string leads to a logical error. We check if $P$ triggers an error detection event by checking if it is in the stabilizer group generated by $\{XXXXI, ZZZZI, IIIIZ, ZZIII, ZIZII, XIXII, XXIII\}$, which defines a valid logical state that does not trigger an error detection or ancilla check event. We check if $P$ causes a logical error by checking if is in the stabilizer group generated by $\{XXXXI, ZZZZI, IIIIZ, ZZIII, ZIZII\}$, which defines the logical $|00\rangle_{AB}$ state. The number of undetected error configurations with $n$ errors we call $u_n$; the number of undetected error configurations with $n$ errors that cause logical errors we call $e_n$.

From the Python script, we obtain the results that
$u_n = [1, 7, 282, 4222, 31637, 94923]$
and
$e_n = [0, 0, 144, 3024, 24240, 70896]$.

Note that a configuration with $n$ errors occurs with probability $(1-p)^{5-n}\left(\frac{p}{15}\right)^n$. Therefore, the probability of detecting no errors (or ancilla check events) at the end of the circuit is

$\begin{align} P_{\rm{no\, detected\, errors}} &= \sum_{n=0}^5 u_n (1-p)^{5-n}\left(\frac{p}{15}\right)^n\\ &=(1-p)^5 + 7(1-p)^4\left(\frac{p}{15}\right)+ 282(1-p)^3\left(\frac{p}{15}\right)^2 + 4222(1-p)^2\left(\frac{p}{15}\right)^3 + 31637(1-p)\left(\frac{p}{15}\right)^4 + 94923\left(\frac{p}{15}\right)^5 \\ &= 1 - \frac{68}{15} p + \frac{704}{75} p^2 - \frac{32768}{3375} p^3 + \frac{253952}{50625} p^4 - \frac{262144}{253125} p^5. \end{align}$

Similarly, the probability of detecting no errors and a logical error occuring is

$\begin{align} P_{\rm{no\, detected\, errors; logical\, error}} &= \sum_{n=0}^5 e_n (1-p)^{5-n}\left(\frac{p}{15}\right)^n\\ &=144(1-p)^3\left(\frac{p}{15}\right)^2 + 3024(1-p)^2\left(\frac{p}{15}\right)^3 + 24240(1-p)\left(\frac{p}{15}\right)^4 + 70896\left(\frac{p}{15}\right)^5 \\ &= \frac{16}{25} p^2 - \frac{128}{125} p^3 + \frac{2048}{3375} p^4 - \frac{32768}{253125} p^5. \end{align}$

If we post-select on no error detection events, then the logical fidelity of the $|00\rangle_{AB}$ state is

$\begin{align} F_{\rm{logical}}&=1-\frac{P_{\rm{no\, detected\, errors; logical\, error}}}{P_{\rm{no\, detected\, errors}}} \\ &=1-\frac{\frac{16}{25} p^2 - \frac{128}{125} p^3 + \frac{2048}{3375} p^4 - \frac{32768}{253125} p^5}{1 - \frac{68}{15} p + \frac{704}{75} p^2 - \frac{32768}{3375} p^3 + \frac{253952}{50625} p^4 - \frac{262144}{253125} p^5}. \end{align}$

import numpy as np
import itertools as it

# Represent single-qubit Pauli matrices I, X, Y, Z either as python strings or integers.
pauli_to_index = {"I" : 0, "X" : 1, "Y" : 2, "Z" : 3}
index_to_pauli = ["I", "X", "Y", "Z"]

# A matrix defining how to multiply single-qubit Paulis (represented as integers).
pauli_multi_mat = np.array([
[0,1,2,3],
[1,0,3,2],
[2,3,0,1],
[3,2,1,0]
], dtype=int)

# Multiply multi-qubit Pauli strings P1 and P2 to create new Pauli string P1*P2
# (up to a phase that is ignored). Here Pauli strings are represented as python strings.
def multiply(pauli_string1, pauli_string2):
    new_paulis = []
    for q in range(5):
        ind_new_p = pauli_multi_mat[pauli_to_index[pauli_string1[q]], pauli_to_index[pauli_string2[q]]]
        new_paulis.append(index_to_pauli[ind_new_p])
    return "".join(new_paulis)

# Rules for how two-qubit Pauli strings
# propagate through a CNOT gate, with the first qubit as
# the control qubit and the second as the target.
cnot_rules = {
    "II" : "II",
    "IX" : "IX",
    "IY" : "ZY",
    "IZ" : "ZZ",
    "XI" : "XX",
    "XX" : "XI",
    "XY" : "YZ",
    "XZ" : "YY",
    "YI" : "YX",
    "YX" : "YI",
    "YY" : "XZ",
    "YZ" : "XY",
    "ZI" : "ZI",
    "ZX" : "ZX",
    "ZY" : "IY",
    "ZZ" : "IZ"
}

# A list of all 16 two-qubit Pauli matrices.
tq_paulis = ["II",
    "IX",
    "IY",
    "IZ",
    "XI",
    "XX",
    "XY",
    "XZ",
    "YI",
    "YX",
    "YY",
    "YZ",
    "ZI",
    "ZX",
    "ZY",
    "ZZ"]

# Embed a two-qubit Pauli on sites i and j into
# a five-qubit Pauli string (tensor around those sites with identity).
def tq_pauli_qubits(tq_pauli, i, j):
    pauli_string = "IIIII"
    if i < j:
        return f"{pauli_string[0:i]}{tq_pauli[0]}{pauli_string[(i+1):j]}{tq_pauli[1]}{pauli_string[(j+1):]}"
    else:
        return f"{pauli_string[0:j]}{tq_pauli[1]}{pauli_string[(j+1):i]}{tq_pauli[0]}{pauli_string[(i+1):]}"

# The stabilizer generators that generate a stabilizer group that define a valid logical state of the [[4,2,2]] code.
# Any valid logical state is as +1 eigenstates of these operators or any product of them.
# Note: that IIIIZ is included to indicate that the ancilla qubit must be |0> to have a valid logical state.
error_detection_stabilizer_generators = ["XXXXI", "ZZZZI", "IIIIZ", "ZZIII", "ZIZII", "XIXII", "XXIII"]

# The stabilizer group generated by the above operators.
error_detection_stabilizer_group = set()
for s1 in ["IIIII", error_detection_stabilizer_generators[0]]:
    for s2 in ["IIIII", error_detection_stabilizer_generators[1]]:
        for s3 in ["IIIII", error_detection_stabilizer_generators[2]]:
            for s4 in ["IIIII", error_detection_stabilizer_generators[3]]:
                for s5 in ["IIIII", error_detection_stabilizer_generators[4]]:
                    for s6 in ["IIIII", error_detection_stabilizer_generators[5]]:
                        for s7 in ["IIIII", error_detection_stabilizer_generators[6]]:
                            error_detection_stabilizer_group.add(multiply(s1, multiply(s2, multiply(s3, multiply(s4, multiply(s5, multiply(s6, s7)))))))

# The stabilizer generators that generate a stabilizer group that defines the particular
# logical state of interest, |00>_AB for the [[4,2,2]] code.
# Note: that IIIIZ is included to indicate that the ancilla qubit must be |0> to have a valid logical state.
logical_state_stabilizer_generators = ["XXXXI", "ZZZZI", "IIIIZ", "ZZIII", "ZIZII"]

# The stabilizer group generated by the above operators.
logical_state_stabilizer_group = set()
for s1 in ["IIIII", logical_state_stabilizer_generators[0]]:
    for s2 in ["IIIII", logical_state_stabilizer_generators[1]]:
        for s3 in ["IIIII", logical_state_stabilizer_generators[2]]:
            for s4 in ["IIIII", logical_state_stabilizer_generators[3]]:
                for s5 in ["IIIII", logical_state_stabilizer_generators[4]]:
                    logical_state_stabilizer_group.add(multiply(s1, multiply(s2, multiply(s3, multiply(s4, s5)))))

# Propagates a multi-qubit Pauli string through a CNOT_{ij} gate with control qubit i and target qubit j.
def cnot(i,j,pauli_string):
    sij     = f"{pauli_string[i]}{pauli_string[j]}"
    new_sij = cnot_rules[sij]

    if i < j:
        return f"{pauli_string[0:i]}{new_sij[0]}{pauli_string[(i+1):j]}{new_sij[1]}{pauli_string[(j+1):]}"
    else:
        return f"{pauli_string[0:j]}{new_sij[1]}{pauli_string[(j+1):i]}{new_sij[0]}{pauli_string[(i+1):]}"

# For each number of Pauli errors after two-qubit gates (between 0 to 6), keeps track of:

# how many Pauli error configurations were undetected by the error detection and ancilla check.
num_undetected_configurations                      = np.zeros(6, dtype=int)
# how many Pauli error configurations were undetected by the error detection and ancilla check
# AND lead to an invalid |00>_AB logical state (i.e., had an undetected logical error).
num_undetected_logical_invalidstate_configurations = np.zeros(6, dtype=int)

# After each of the five two-qubit gates, pick one of the 16 possible
# two-qubit Paulis as the error. This defines a Pauli error configuration.
for i1 in range(len(tq_paulis)):
    for i2 in range(len(tq_paulis)):
        for i3 in range(len(tq_paulis)):
            for i4 in range(len(tq_paulis)):
                for i5 in range(len(tq_paulis)):
                    # Count how many errors are in the configuration. (i=0 corresponds to the identity Pauli II)
                    num_errors = (i1 != 0) + (i2 != 0) + (i3 != 0) + (i4 != 0) + (i5 != 0)

                    # Initialize the Pauli string of identities.
                    pauli_string = "IIIII"

                    # For each CNOTs in the circuit,
                    for (i,j, ind_tq_pauli) in [(1,2,i1), (1,0,i2), (2,3,i3), (3,4,i4), (0,4,i5)]:
                        # propagate the Pauli string through the CNOT
                        pauli_string = cnot(i, j, pauli_string)
                        # Pick a two-qubit Pauli error if it is in the chosen configuration
                        tq_pauli     = tq_pauli_qubits(tq_paulis[ind_tq_pauli], i, j)
                        # Apply the error to the current Pauli string
                        pauli_string = multiply(pauli_string, tq_pauli)

                    # After the Pauli string has been propagated through the circuit,

                    # check if it causes an error detection event
                    no_error_detected    = (pauli_string in error_detection_stabilizer_group)
                    # check if it causes the state to be a valid logical |00>_AB state
                    valid_logical_state  = (pauli_string in logical_state_stabilizer_group)

                    # Update the configuration tracking with this info.
                    num_undetected_configurations[num_errors]                      += no_error_detected
                    num_undetected_logical_invalidstate_configurations[num_errors] += (no_error_detected and not valid_logical_state)

# Print the final results
print(f"Number of Pauli error configurations that were undetected:\n{num_undetected_configurations}")
print(f"Number of Pauli error configurations that were undetected and caused a logical error:\n{num_undetected_logical_invalidstate_configurations}")
Model Name Challenge Checkpoint 1 Checkpoint 2 Checkpoint 3 Details
Self
Carryover		 Oracle
Carryover		 Self
Carryover		 Oracle
Carryover
GPT-5 (high, code & web) 2/5 5/5 5/5 5/5 3/5 3/5
GPT-5 (high, code) 4/5 5/5 5/5 5/5 5/5 4/5
GPT-5 (high) 0/5 5/5 5/5 5/5 0/5 0/5
Gemini-2.5 Pro 0/5 3/5 3/5 1/5 0/5 0/5
o3 (high) 0/5 5/5 2/5 2/5 0/5 0/5
DeepSeek R1 0/5 3/5 3/5 2/5 0/5 0/5
o4-mini (high) 0/5 5/5 1/5 0/5 0/5 0/5
Gemini-2.5 Flash 0/5 2/5 0/5 0/5 0/5 0/5
Claude Opus 4 0/5 0/5 0/5 0/5 0/5 0/5
GPT-5 (minimal) 0/5 0/5 0/5 0/5 0/5 0/5
Llama-4 Maverick 0/5 0/5 0/5 0/5 0/5 0/5
GPT-4o 0/5 0/5 0/5 0/5 0/5 0/5