raphael
195ecc689f
Fills four documentation gaps surfaced by the doc audit:
1. README "Documentation map" was missing VERIFICATION.md (the file
that backs the proof-of-feature-completeness claims) and is now
pointing at the new files added in this commit too — ARCHITECTURE,
GLOSSARY, FAQ, examples/pvd_tool/ (the last two land next).
2. interop/README.md only documented secsgem-py. Three of the five
external validators (tshark, secs4j, libFuzzer) plus the E5 KAT
were invisible from the directory's own README. Rewritten as a
complete index — what's external, what each catches, how to run,
what bugs they've already surfaced, when to add a new validator.
3. GLOSSARY.md is new. Every SEMI acronym used in the codebase or
the docs gets one row: SVID, DVID, CEID, RPTID, ALID, ECID, PPID,
MID, CARRIERID, PRJOBID, CTLJOBID, SUBSTID, OBJSPEC, OBJTYPE,
MDLN, SOFTREV, EQPTYP, DATAID + every ACK code (COMMACK, ONLACK,
OFLACK, HCACK, CMDA, ACKC5-7-10, DRACK, LRACK, ERACK, EAC, TIACK,
GRANT, ALCD, OBJACK) + stream/function shorthand + HSMS terms +
T-timers + E84 signals + the standards lineup + codebase shortcuts
("the model", "the router", "the proof", etc.). Cuts week-1
onboarding time.
4. FAQ.md is new. Canonical answers to the questions that come up
once per integration: why HSMS unencrypted, SVID vs DVID, PJ vs
CJ, who fires FSM transitions, what runs on which thread, how to
add a new SECS-II message, ASCII vs Binary, common MES quirks,
how spool works, robustness fuzz vs libFuzzer, conformance vs
interop, what's not implemented.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
134 lines
6.1 KiB
Markdown
134 lines
6.1 KiB
Markdown
# External cross-validation harnesses
|
|
|
|
Every harness in this directory exists so a reviewer doesn't have to
|
|
take our word for it. Each one validates our C++ codec / framing /
|
|
dispatch against an **independent third-party implementation** that
|
|
read the SEMI standards without talking to us.
|
|
|
|
See [`../VERIFICATION.md`](../VERIFICATION.md) for the full test plan
|
|
and the honest accounting of which proofs are external vs internal.
|
|
|
|
## What's here
|
|
|
|
| Validator | Independence | Coverage |
|
|
|--------------------------------------------|---------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
|
|
| `host_vs_cpp_server.py` + `passive_equipment.py` | secsgem-py 0.3.0 — Python reference impl | ~24 + 4 checks: S1, S2, S5, S6, S7, S10 happy paths |
|
|
| `raw_gem300_harness.py` | secsgem-py with hand-crafted SecsStreamFunctions | 3 checks: S3F17, S16F5, S16F27 (limited by SFDL grammar) |
|
|
| `secs4j/Secs4jHostHarness.java` | secs4java8 — Apache 2.0 Java impl by kenta-shimizu | **55 checks** across S1/S2/S3/S5/S6/S7/S10/S14/S16, including the full E40 body that defeated secsgem-py and unsolicited S6F11/S5F1 observation |
|
|
| `tshark_validate.sh` | Wireshark's built-in HSMS dissector | 69 captured frames dissected with no malformed-packet warnings |
|
|
| `spool_persistence_test.py` | secsgem-py + a docker-restart loop | Restart-survives-spool integrity |
|
|
| ⚙️ `../tests/test_e5_kat.cpp` | SEMI E5 §9 encoding rules | 196 known-answer byte assertions across every format code |
|
|
| ⚙️ `../apps/fuzz_secs2_decode.cpp` + `fuzz_sml_parse.cpp` | libFuzzer + ASan + UBSan | ~70 000 + ~285 000 random inputs per minute, 0 crashes |
|
|
|
|
The ⚙️ entries aren't in `interop/` directly because they don't
|
|
involve a network peer — they're either pure codec round-trips
|
|
(KAT) or coverage-guided fuzzing. Listed here so the full external
|
|
proof inventory lives in one place.
|
|
|
|
## Running each validator
|
|
|
|
### secsgem-py — secsgem-py active host → C++ server
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### secsgem-py — C++ host → secsgem-py equipment
|
|
|
|
```sh
|
|
docker compose up -d equipment_py
|
|
docker compose run --rm builder /app/build/secs_interop_probe \
|
|
--host equipment_py --port 5000 --device 0
|
|
```
|
|
|
|
### secsgem-py — raw GEM 300 frames
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/raw_gem300_harness.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### secs4j — independent Java host → C++ server
|
|
|
|
```sh
|
|
bash interop/secs4j_validate.sh
|
|
```
|
|
|
|
Builds an `eclipse-temurin:21-jdk` sidecar with secs4java8 cloned +
|
|
compiled at image build, then drives 55 checks against
|
|
`compose up server`. See `secs4j/Secs4jHostHarness.java` for the
|
|
list and `secs4j/Dockerfile` for the build.
|
|
|
|
### tshark — Wireshark HSMS dissector
|
|
|
|
```sh
|
|
docker compose run --rm builder bash /app/interop/tshark_validate.sh
|
|
```
|
|
|
|
Captures a pcap of the demo flow, runs `tshark -V` with the HSMS
|
|
dissector forced for the test port, asserts no malformed packets +
|
|
that all expected control/data frames parse.
|
|
|
|
### spool persistence — restart-survives test
|
|
|
|
```sh
|
|
bash interop/spool_persistence_test.py
|
|
```
|
|
|
|
Drops the host link mid-flight, kills the server, restarts it, and
|
|
asserts the spooled S5F1 / S6F11 frames drain to the host on
|
|
reconnect.
|
|
|
|
## What these harnesses caught
|
|
|
|
Real bugs surfaced during interop development (now fixed):
|
|
|
|
1. **Strict U-width parsing rejected U1-encoded identifiers.** SEMI
|
|
E5 declares DATAID, RPTID, VID, CEID, ALID, EXID etc. as
|
|
`U1 | U2 | U4 | U8`; secsgem-py picks the smallest width that
|
|
fits. Our scalar accessors were strict. Now lenient with
|
|
range-checked downcasts (`messages_helpers.hpp::any_unsigned_first`).
|
|
|
|
2. **PPBODY rejected when sent as ASCII.** SEMI allows PPBODY to be
|
|
`ASCII | Binary | List`; secsgem-py defaults to ASCII. Added the
|
|
`BINARY_OR_ASCII` codegen type and the `as_text_or_binary`
|
|
accessor.
|
|
|
|
3. **Missing S1F23 / S1F24 (Collection Event Namelist).** Added the
|
|
wire schema, the `vids_for(ceid)` accessor, and the dispatch
|
|
handler.
|
|
|
|
4. **Missing S10F3 handler (host→equipment Terminal Display).** Our
|
|
server only registered S10F1; per SEMI E5 §13 those are opposite
|
|
directions. Added the missing dispatch.
|
|
|
|
5. **TSan use-after-free in `act_exception_complete`** (test code,
|
|
not library): held a pointer across `fire_internal(RecoveryComplete)`
|
|
which deletes the entry. Found by the ThreadSanitizer lane on
|
|
first run.
|
|
|
|
The C++ test suite stayed green through every one of these fixes —
|
|
the changes were purely permissive widenings or additive features,
|
|
no existing behaviour broke.
|
|
|
|
## When to add a new validator
|
|
|
|
A new third-party SECS implementation, or a new dissector, or a new
|
|
fuzzer target — anything that exercises our wire surface from an
|
|
angle the existing five don't cover — is worth adding. The pattern
|
|
is consistent:
|
|
|
|
1. New script / harness lives here (or a sidecar Docker context for
|
|
non-Python validators).
|
|
2. Wired into `.gitea/workflows/ci.yml` as a separate job.
|
|
3. Listed in this README's table + in `../VERIFICATION.md`.
|
|
4. Surfaced in the README's proof-of-feature-completeness table if
|
|
it adds a meaningful new dimension.
|
|
|
|
Bug reports from a new validator → file at
|
|
`raphael@maenle.net` with the wire trace, the validator's output,
|
|
and the equipment YAML so we can reproduce.
|