Commit Graph

2 Commits

Author SHA1 Message Date
raphael 5baf3f4dc7 verify: tshark HSMS dissector validation (independent third codec)
Wireshark's built-in HSMS dissector — written by network-protocol
authors who don't know us, didn't talk to us, and don't share
implementation details with secsgem-py — is a third independent codec
for our framing.  If they parse our pcap without warnings, our HSMS
framing is wire-correct independently of both our internal tests and
the secsgem-py interop path.

interop/tshark_validate.sh:
- Boots secs_server on 127.0.0.1:5099 (away from the demo port)
- Captures the loopback wire traffic with tcpdump
- Runs secs_client through ~24 transactions plus Separate.req +
  TCP FIN
- Parses the pcap with tshark -V using the HSMS dissector
- Asserts: no "Malformed Packet", no "Dissector bug", at least one
  HSMS frame, expected tokens present (Select.req/rsp, Separate.req,
  Data message), reports histogram (count by control type + distinct
  S/F pairs)

Result against the demo: 69 HSMS frames dissected, 49 distinct
S/F pairs (S01F01..S16F28), all clean.

Dockerfile gains tshark + tcpdump.  .gitea/workflows/ci.yml gains a
`tshark-dissector` job that runs this validator as part of every
push to main.  README proof table grows to 6 commands.

VERIFICATION.md §1a documents a follow-up: round-trip the KAT
fixtures through secsgem-py to corroborate that the format codes
we used match an independent implementation.  Strengthens the KAT
proof from "internally consistent" to "confirmed by a second
implementer who read the spec without talking to us."

Plan: VERIFICATION.md §2.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 16:02:38 +02:00
raphael 257a148d34 docs: VERIFICATION.md — external validation test plan
Honest accounting of what's currently external vs internal in the
five proofs:

  - 4 of 5 proofs are us-testing-us (unit tests, conformance
    harness, robustness fuzz, YAML validation)
  - Only secsgem-py interop is external, and it covers ~15-20 %
    of the claimed wire surface (skips most of GEM 300, HSMS-GS,
    exception recovery, wafer maps, enhanced commands, every
    wire-level edge case that isn't message-shaped)

Plan documents four additional external validators with goals,
methods, success criteria, scope limits, and effort estimates:

  1. SEMI E5 known-answer tests — hex fixtures from the spec's
     own encoding rules; the strongest single codec test
  2. tshark/Wireshark HSMS dissector — independent third codec
     parsing our pcap captures
  3. secs4j cross-validation — Apache-2.0 Java implementation
     by a different author; catches "we both got it wrong the
     same way" relative to secsgem-py
  4. libFuzzer over secs2::decode + secs2::from_sml — coverage-
     guided structural search for crashes and UB

After all four: 5 external proofs (KAT + tshark + secsgem-py +
secs4j + libFuzzer), three of them on overlapping wire surface
from independent angles.

Plan also explicitly lists what these validators do NOT replace:
GEM RTS certification, per-MES interop sweeps, real-fab wire
trace corroboration.  Those remain customer-side work.

Order of execution: KAT → tshark → secs4j → libFuzzer.  KAT
first because it produces fixtures the others can reuse;
libFuzzer last because it benefits from the KAT corpus.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 15:46:34 +02:00