A small optimisation problem running on a quantum computer demonstrates that a workflow can execute. It does not automatically show that the approach improves a business decision, outperforms an appropriate classical method or will scale to the intended workload. Those are separate questions, and a useful pilot is designed to distinguish them.
This article follows a fictional maintenance-planning problem to show how an organisation can evaluate quantum optimisation without either dismissing the technology or making claims the experiment cannot support. The aim is a fair, reproducible comparison with a clear decision at the end. No performance results are asserted here; the examples describe how to design the experiment.
The question: what problem are both options solving?
Our fictional operator must choose maintenance tasks for a fixed planning window. Each task has a value, a duration, required skills and possible conflicts with other tasks. The team wants a feasible plan with high operational value. A list with the largest numerical score is not useful if it requires the same engineer in two places at once.
Separate hard constraints from preferences. A hard constraint defines an unacceptable plan, such as exceeding available staff. A preference helps rank acceptable plans, such as completing more high-priority work. Combining both into a score without preserving their meaning can reward a plan that looks mathematically strong but cannot be used.
Record the instance data and the intended output format. Explain how the business would judge two plans, including ties and trade-offs. If the problem definition changes during the pilot, version that change and rerun the relevant comparisons. Otherwise, different methods may quietly be solving different problems.
The options: choose a credible classical baseline
Begin with the organisation’s current planning method and at least one appropriate computational baseline. Depending on the formulation, this could be a mixed-integer optimiser, a constraint solver or a well-chosen heuristic. For very small instances, exhaustive enumeration can establish the optimum and help check the formulation.
Do not use an intentionally weak baseline because it is easy to beat. Give the baseline comparable data preparation, tuning effort and stopping conditions. Record its software version, hardware, configuration and random seed where applicable. If a specialist solver is available, assess whether it represents the problem more naturally than the encoding chosen for the quantum experiment.
Small-instance ground truth has an important role, but it is not a scaling result. A method that matches the optimum on a handful of tiny cases may still behave differently on larger, tighter or differently structured instances. Include an instance family that reflects the organisation’s real constraints rather than selecting only convenient examples.
The criteria: preserve the business constraints
Many quantum optimisation demonstrations use binary variables and an objective with penalty terms. A binary variable can represent whether a task is selected. A penalty can discourage selecting conflicting tasks. The difficulty is that the penalty strength changes the optimisation landscape, and a poor setting can favour an infeasible plan or obscure differences between useful feasible plans.
Before running hardware, decode candidate solutions and check every original business constraint independently. Do not assume that a low encoded objective implies feasibility. Preserve both the encoded score and the original business score so that the experiment can reveal a mismatch.
The Quantum Approximate Optimisation Algorithm, or QAOA, is one approach to combinatorial optimisation. Its original paper describes parameterised quantum circuits for producing approximate solutions. Using it in a pilot does not itself establish a practical advantage. The formulation, parameter search, sampling and surrounding classical computation all affect what the experiment means.
Side-by-side scope: measure the complete workflows
Measure data preparation, encoding, parameter optimisation, device execution, sampling, decoding and validation. Report which stages are included in any timing claim. A short circuit execution time is not the same as a short time to a usable plan if the workflow performs many optimisation iterations or waits for remote access.
Queue time is operationally relevant but depends on the access arrangement. Report it separately so readers can distinguish algorithmic work from the service conditions of a particular run. Equally, do not quietly omit required classical optimisation merely because the experiment is labelled quantum.
For stochastic methods, run multiple trials and report the spread of results. A single best sample can be useful to inspect, but it should not stand in for the probability of obtaining a feasible, useful plan within the available budget.
Trade-offs: simulator evidence and hardware evidence
A simulator can help verify a circuit, inspect an encoding and test the surrounding workflow. A noiseless simulation, a noise-model simulation and a physical-device run answer different questions. Label them separately, including the simulator settings or device details and the time of execution.
Agreement on small instances is a valuable correctness check. It does not show that a simulator can represent the intended production scale, or that physical hardware will produce equivalent results at that scale. Avoid extending a conclusion beyond the conditions actually tested.
For hardware experiments, retain the circuit before and after compilation, relevant device configuration, sampling settings and any mitigation applied. Changes in compilation or execution conditions can affect results. A reproducible record lets a later reviewer investigate those differences without relying on a presentation slide.
Comparison matrix: judge practical usefulness
| Measure | Question it answers |
|---|---|
| Feasible-plan rate | How often does the workflow satisfy the original hard constraints? |
| Business objective on feasible plans | How useful are the returned plans under the agreed scoring rule? |
| Time to an acceptable plan | Can the workflow support the actual planning window? |
| Variation across runs and instances | Is the result repeatable enough for the intended decision? |
| End-to-end resource use | What computation and operating effort does each route require? |
| Evidence and reproducibility | Can another person reconstruct the claim and check its scope? |
Agree the scorecard before examining the final results. Otherwise, it is easy to change the success measure to whichever chart looks strongest. If quality is compared under a fixed time budget, state that budget. If time is compared at a fixed quality threshold, define the threshold and how a run that never reaches it is counted.
When each fits: decide what the pilot established
A pilot can conclude that the formulation is correct and the workflow runs, while leaving practical value unresolved. It can conclude that the quantum candidate does not justify further work for this instance family under the tested conditions. It can also identify a specific uncertainty worth another experiment. Each outcome can be useful when its scope is clear.
Expansion should answer a question the first pilot could not resolve: a larger instance family, a more representative constraint set or a better-controlled execution comparison. Buying more runtime without changing the evidence question can produce more activity without improving the decision.
Before commissioning a quantum optimisation pilot, ask for the original problem specification, the baseline plan, the evaluation rules and the reproducibility record. Those four items make it possible to learn from the experiment. The technology deserves a fair test, and the business deserves a conclusion that follows from what was measured.
Technical reference
Farhi, Goldstone and Gutmann: A Quantum Approximate Optimization Algorithm introduces QAOA. The maintenance-planning scenario and scorecard here are illustrative experimental-design guidance, not benchmark results or a claim of quantum advantage.

