The sample/decode scoring paths conflate "the decoder returned an error" with "the
decoder produced a wrong correction", and the serial and parallel paths do so in two
different, mutually inconsistent ways. All line references are to dev at
python/pecos-rslib/src/fault_tolerance_bindings.rs.
Serial paths — an error always counts as a logical error.
SampleBatch.decode_count, decode_count_batch, decode_stats, and
decode_stats_parallel score each shot with the pattern
.map_or(true, |p| p != self.extract_obs_mask_wide(i))
(lines 3606, 3700, 3795, 3875): a Err from decode_obs is folded into the
logical-error count with no separate accounting.
Parallel sample-and-decode path — an error is usually a failure, but can count as
correct. sample_decode_count_parallel substitutes the full observable selection
mask as the prediction on error:
let mut predicted = decoder
.decode_obs(&syndrome)
.unwrap_or_else(|_| observable_mask.clone()); // line 4716-4718
predicted &= &observable_mask;so an erroring decoder is scored as if it predicted every observable flipped. That is
counted as a failure on most shots, but counted as correct on any shot whose true
observable pattern happens to flip all observables — an erroring decoder can score
better than an honest one on such shots.
Consequences for logical-error-rate estimation:
- A decoder that errors on some fraction of shots (timeout, unsupported syndrome,
internal failure) silently contaminates the reported LER, with no signal that any
errors occurred, in whichever direction the path's convention pushes.
- The same batch scored through the serial and parallel paths can disagree for reasons
that have nothing to do with decoding quality.
Suggested direction: score decoder errors as their own outcome, never folded into the
logical-error count — either surface an error count alongside the LER, or fail loud if
any shot errors. SampleBatch.compare_decoders on the decoder-eval-harness branch
(PR #432) already treats errors as a third outcome per shot and can serve as the
pattern; this issue covers the pre-existing paths on dev.
The sample/decode scoring paths conflate "the decoder returned an error" with "the
decoder produced a wrong correction", and the serial and parallel paths do so in two
different, mutually inconsistent ways. All line references are to
devatpython/pecos-rslib/src/fault_tolerance_bindings.rs.Serial paths — an error always counts as a logical error.
SampleBatch.decode_count,decode_count_batch,decode_stats, anddecode_stats_parallelscore each shot with the pattern(lines 3606, 3700, 3795, 3875): a
Errfromdecode_obsis folded into thelogical-error count with no separate accounting.
Parallel sample-and-decode path — an error is usually a failure, but can count as
correct.
sample_decode_count_parallelsubstitutes the full observable selectionmask as the prediction on error:
so an erroring decoder is scored as if it predicted every observable flipped. That is
counted as a failure on most shots, but counted as correct on any shot whose true
observable pattern happens to flip all observables — an erroring decoder can score
better than an honest one on such shots.
Consequences for logical-error-rate estimation:
internal failure) silently contaminates the reported LER, with no signal that any
errors occurred, in whichever direction the path's convention pushes.
that have nothing to do with decoding quality.
Suggested direction: score decoder errors as their own outcome, never folded into the
logical-error count — either surface an error count alongside the LER, or fail loud if
any shot errors.
SampleBatch.compare_decoderson thedecoder-eval-harnessbranch(PR #432) already treats errors as a third outcome per shot and can serve as the
pattern; this issue covers the pre-existing paths on
dev.