raphael
fc3422a4a9
Picks up the file renames that landed alongside the previous commit and fixes everything that pointed at the old root locations: - README.md doc-map updated: every entry now points at docs/X.md, with a new "docs/" lead entry pointing at the guided-tour index. - README inline cross-refs (ARCHITECTURE / INTEGRATION / SECURITY / BENCHMARKS / MES_INTEROP / PROOFS) repointed to docs/. - README "Interop" section rewritten — used to mention only secsgem-py; now covers all four external validators (secsgem-py 31 / secs4java8 55 / tshark 69 frames / libFuzzer 200 k+ runs) with a one-line summary each, plus pointers to interop/README.md and docs/VERIFICATION.md. - README "Deferred follow-ups" cleaned: dropped the explanatory "Listed here so reviewers don't go looking for them in COMPLIANCE.md and find an 'out of scope' entry that sounds defensive" sentence — the section header speaks for itself. - docs/00_index.md "Where the rest of the docs live" table: dropped every `../` prefix since the docs are now siblings. - docs/01_what_is_secs_gem.md PROOFS reference updated to sibling. - docs/02_the_cast.md INTEGRATION + MES_INTEROP refs updated to siblings; dropped the stale "at the repo root" wording. - interop/README.md: VERIFICATION + PROOFS refs updated to ../docs/X.md; stale "~24 + 4 checks" updated to 31 (matches PROOFS.md and README). - examples/pvd_tool/README.md: every doc cross-ref now points at ../../docs/X.md. - Source / data / CI comments mentioning doc names (e.g. "INTEGRATION.md §3", "COMPLIANCE.md gap") rewritten to "docs/INTEGRATION.md §3" etc. — affects 9 files across include/, apps/, tests/, data/, examples/, .gitea/workflows/. Verified: full build under docker passes, 445/445 test cases pass, 2 753/2 753 assertions pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
135 lines
6.1 KiB
Markdown
135 lines
6.1 KiB
Markdown
# External cross-validation harnesses
|
|
|
|
Every harness in this directory exists so a reviewer doesn't have to
|
|
take our word for it. Each one validates our C++ codec / framing /
|
|
dispatch against an **independent third-party implementation** that
|
|
read the SEMI standards without talking to us.
|
|
|
|
See [`../docs/VERIFICATION.md`](../docs/VERIFICATION.md) for the full
|
|
test plan and the honest accounting of which proofs are external vs
|
|
internal.
|
|
|
|
## What's here
|
|
|
|
| Validator | Independence | Coverage |
|
|
|--------------------------------------------|---------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
|
|
| `host_vs_cpp_server.py` + `passive_equipment.py` | secsgem-py 0.3.0 — Python reference impl | 31 checks: S1, S2, S5, S6, S7, S10 happy paths + unsolicited S6F11 / S5F1 |
|
|
| `raw_gem300_harness.py` | secsgem-py with hand-crafted SecsStreamFunctions | 3 checks: S3F17, S16F5, S16F27 (limited by SFDL grammar) |
|
|
| `secs4j/Secs4jHostHarness.java` | secs4java8 — Apache 2.0 Java impl by kenta-shimizu | **55 checks** across S1/S2/S3/S5/S6/S7/S10/S14/S16, including the full E40 body that defeated secsgem-py and unsolicited S6F11/S5F1 observation |
|
|
| `tshark_validate.sh` | Wireshark's built-in HSMS dissector | 69 captured frames dissected with no malformed-packet warnings |
|
|
| `spool_persistence_test.py` | secsgem-py + a docker-restart loop | Restart-survives-spool integrity |
|
|
| ⚙️ `../tests/test_e5_kat.cpp` | SEMI E5 §9 encoding rules | 196 known-answer byte assertions across every format code |
|
|
| ⚙️ `../apps/fuzz_secs2_decode.cpp` + `fuzz_sml_parse.cpp` | libFuzzer + ASan + UBSan | ~70 000 + ~285 000 random inputs per minute, 0 crashes |
|
|
|
|
The ⚙️ entries aren't in `interop/` directly because they don't
|
|
involve a network peer — they're either pure codec round-trips
|
|
(KAT) or coverage-guided fuzzing. Listed here so the full external
|
|
proof inventory lives in one place.
|
|
|
|
## Running each validator
|
|
|
|
### secsgem-py — secsgem-py active host → C++ server
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### secsgem-py — C++ host → secsgem-py equipment
|
|
|
|
```sh
|
|
docker compose up -d equipment_py
|
|
docker compose run --rm builder /app/build/secs_interop_probe \
|
|
--host equipment_py --port 5000 --device 0
|
|
```
|
|
|
|
### secsgem-py — raw GEM 300 frames
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/raw_gem300_harness.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### secs4j — independent Java host → C++ server
|
|
|
|
```sh
|
|
bash interop/secs4j_validate.sh
|
|
```
|
|
|
|
Builds an `eclipse-temurin:21-jdk` sidecar with secs4java8 cloned +
|
|
compiled at image build, then drives 55 checks against
|
|
`compose up server`. See `secs4j/Secs4jHostHarness.java` for the
|
|
list and `secs4j/Dockerfile` for the build.
|
|
|
|
### tshark — Wireshark HSMS dissector
|
|
|
|
```sh
|
|
docker compose run --rm builder bash /app/interop/tshark_validate.sh
|
|
```
|
|
|
|
Captures a pcap of the demo flow, runs `tshark -V` with the HSMS
|
|
dissector forced for the test port, asserts no malformed packets +
|
|
that all expected control/data frames parse.
|
|
|
|
### spool persistence — restart-survives test
|
|
|
|
```sh
|
|
bash interop/spool_persistence_test.py
|
|
```
|
|
|
|
Drops the host link mid-flight, kills the server, restarts it, and
|
|
asserts the spooled S5F1 / S6F11 frames drain to the host on
|
|
reconnect.
|
|
|
|
## What these harnesses caught
|
|
|
|
Real bugs surfaced during interop development (now fixed):
|
|
|
|
1. **Strict U-width parsing rejected U1-encoded identifiers.** SEMI
|
|
E5 declares DATAID, RPTID, VID, CEID, ALID, EXID etc. as
|
|
`U1 | U2 | U4 | U8`; secsgem-py picks the smallest width that
|
|
fits. Our scalar accessors were strict. Now lenient with
|
|
range-checked downcasts (`messages_helpers.hpp::any_unsigned_first`).
|
|
|
|
2. **PPBODY rejected when sent as ASCII.** SEMI allows PPBODY to be
|
|
`ASCII | Binary | List`; secsgem-py defaults to ASCII. Added the
|
|
`BINARY_OR_ASCII` codegen type and the `as_text_or_binary`
|
|
accessor.
|
|
|
|
3. **Missing S1F23 / S1F24 (Collection Event Namelist).** Added the
|
|
wire schema, the `vids_for(ceid)` accessor, and the dispatch
|
|
handler.
|
|
|
|
4. **Missing S10F3 handler (host→equipment Terminal Display).** Our
|
|
server only registered S10F1; per SEMI E5 §13 those are opposite
|
|
directions. Added the missing dispatch.
|
|
|
|
5. **TSan use-after-free in `act_exception_complete`** (test code,
|
|
not library): held a pointer across `fire_internal(RecoveryComplete)`
|
|
which deletes the entry. Found by the ThreadSanitizer lane on
|
|
first run.
|
|
|
|
The C++ test suite stayed green through every one of these fixes —
|
|
the changes were purely permissive widenings or additive features,
|
|
no existing behaviour broke.
|
|
|
|
## When to add a new validator
|
|
|
|
A new third-party SECS implementation, or a new dissector, or a new
|
|
fuzzer target — anything that exercises our wire surface from an
|
|
angle the existing five don't cover — is worth adding. The pattern
|
|
is consistent:
|
|
|
|
1. New script / harness lives here (or a sidecar Docker context for
|
|
non-Python validators).
|
|
2. Wired into `.gitea/workflows/ci.yml` as a separate job.
|
|
3. Listed in this README's table + in `../VERIFICATION.md`.
|
|
4. Surfaced in [`../docs/PROOFS.md`](../docs/PROOFS.md) if it adds a
|
|
meaningful new dimension.
|
|
|
|
Bug reports from a new validator → file at
|
|
`raphael@maenle.net` with the wire trace, the validator's output,
|
|
and the equipment YAML so we can reproduce.
|