Commit Graph

1 Commits

Author SHA1 Message Date
raphael 257a148d34 docs: VERIFICATION.md — external validation test plan
Honest accounting of what's currently external vs internal in the
five proofs:

  - 4 of 5 proofs are us-testing-us (unit tests, conformance
    harness, robustness fuzz, YAML validation)
  - Only secsgem-py interop is external, and it covers ~15-20 %
    of the claimed wire surface (skips most of GEM 300, HSMS-GS,
    exception recovery, wafer maps, enhanced commands, every
    wire-level edge case that isn't message-shaped)

Plan documents four additional external validators with goals,
methods, success criteria, scope limits, and effort estimates:

  1. SEMI E5 known-answer tests — hex fixtures from the spec's
     own encoding rules; the strongest single codec test
  2. tshark/Wireshark HSMS dissector — independent third codec
     parsing our pcap captures
  3. secs4j cross-validation — Apache-2.0 Java implementation
     by a different author; catches "we both got it wrong the
     same way" relative to secsgem-py
  4. libFuzzer over secs2::decode + secs2::from_sml — coverage-
     guided structural search for crashes and UB

After all four: 5 external proofs (KAT + tshark + secsgem-py +
secs4j + libFuzzer), three of them on overlapping wire surface
from independent angles.

Plan also explicitly lists what these validators do NOT replace:
GEM RTS certification, per-MES interop sweeps, real-fab wire
trace corroboration.  Those remain customer-side work.

Order of execution: KAT → tshark → secs4j → libFuzzer.  KAT
first because it produces fixtures the others can reuse;
libFuzzer last because it benefits from the KAT corpus.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 15:46:34 +02:00