raphael
dae6bfd747
Tone pass across the non-tutorial markdown — README, PROOFS,
ARCHITECTURE, BENCHMARKS, COMPLIANCE, FAQ, MES_INTEROP, SECURITY,
and interop/README. Three patterns came out:
- Bug-history war stories ("Past interop sweeps surfaced…",
"What these harnesses caught: 1. Strict U-width parsing…").
- Chat-with-reader framing ("Don't skip TLS unless…", "Treat as a
punch list", "If you're running in a pod…", "Misconfiguration
incidents drop dramatically").
- Self-referential narration ("we ship", "our codec", "the
codebase's most-tested layer", "three orders of magnitude above
fab load", "the gift that keeps giving").
README also drops the standalone ThreadSanitizer subsection under
Build details (now a single line under the new Testing section).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
99 lines
4.6 KiB
Markdown
99 lines
4.6 KiB
Markdown
# External cross-validation harnesses
|
|
|
|
Each harness in this directory validates the C++ codec, framing, and
|
|
dispatch against an **independent third-party implementation** of
|
|
the same SEMI standards.
|
|
|
|
See [`../docs/VERIFICATION.md`](../docs/VERIFICATION.md) for the
|
|
internal-vs-external breakdown across all validators.
|
|
|
|
## What's here
|
|
|
|
| Validator | Independence | Coverage |
|
|
|--------------------------------------------|---------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
|
|
| `host_vs_cpp_server.py` + `passive_equipment.py` | secsgem-py 0.3.0 — Python reference impl | 31 checks: S1, S2, S5, S6, S7, S10 happy paths + unsolicited S6F11 / S5F1 |
|
|
| `raw_gem300_harness.py` | secsgem-py with hand-crafted SecsStreamFunctions | 3 checks: S3F17, S16F5, S16F27 (limited by SFDL grammar) |
|
|
| `secs4j/Secs4jHostHarness.java` | secs4java8 — Apache 2.0 Java impl by kenta-shimizu | **55 checks** across S1/S2/S3/S5/S6/S7/S10/S14/S16, including the full E40 body that defeated secsgem-py and unsolicited S6F11/S5F1 observation |
|
|
| `tshark_validate.sh` | Wireshark's built-in HSMS dissector | 69 captured frames dissected with no malformed-packet warnings |
|
|
| `spool_persistence_test.py` | secsgem-py + a docker-restart loop | Restart-survives-spool integrity |
|
|
| ⚙️ `../tests/test_e5_kat.cpp` | SEMI E5 §9 encoding rules | 196 known-answer byte assertions across every format code |
|
|
| ⚙️ `../apps/fuzz_secs2_decode.cpp` + `fuzz_sml_parse.cpp` | libFuzzer + ASan + UBSan | ~70 000 + ~285 000 random inputs per minute, 0 crashes |
|
|
|
|
The ⚙️ entries aren't in `interop/` directly because they don't
|
|
involve a network peer — they're either pure codec round-trips
|
|
(KAT) or coverage-guided fuzzing. Listed here so the full external
|
|
proof inventory lives in one place.
|
|
|
|
## Running each validator
|
|
|
|
### secsgem-py — secsgem-py active host → C++ server
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### secsgem-py — C++ host → secsgem-py equipment
|
|
|
|
```sh
|
|
docker compose up -d equipment_py
|
|
docker compose run --rm builder /app/build/secs_interop_probe \
|
|
--host equipment_py --port 5000 --device 0
|
|
```
|
|
|
|
### secsgem-py — raw GEM 300 frames
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/raw_gem300_harness.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### secs4j — independent Java host → C++ server
|
|
|
|
```sh
|
|
bash interop/secs4j_validate.sh
|
|
```
|
|
|
|
Builds an `eclipse-temurin:21-jdk` sidecar with secs4java8 cloned +
|
|
compiled at image build, then drives 55 checks against
|
|
`compose up server`. See `secs4j/Secs4jHostHarness.java` for the
|
|
list and `secs4j/Dockerfile` for the build.
|
|
|
|
### tshark — Wireshark HSMS dissector
|
|
|
|
```sh
|
|
docker compose run --rm builder bash /app/interop/tshark_validate.sh
|
|
```
|
|
|
|
Captures a pcap of the demo flow, runs `tshark -V` with the HSMS
|
|
dissector forced for the test port, asserts no malformed packets +
|
|
that all expected control/data frames parse.
|
|
|
|
### spool persistence — restart-survives test
|
|
|
|
```sh
|
|
bash interop/spool_persistence_test.py
|
|
```
|
|
|
|
Drops the host link mid-flight, kills the server, restarts it, and
|
|
asserts the spooled S5F1 / S6F11 frames drain to the host on
|
|
reconnect.
|
|
|
|
## When to add a new validator
|
|
|
|
A new third-party SECS implementation, dissector, or fuzzer target
|
|
that exercises the wire surface from an angle the existing five
|
|
don't cover is worth adding. The pattern is consistent:
|
|
|
|
1. New script / harness lives here (or a sidecar Docker context for
|
|
non-Python validators).
|
|
2. Wired into `.gitea/workflows/ci.yml` as a separate job.
|
|
3. Listed in this README's table + in `../VERIFICATION.md`.
|
|
4. Surfaced in [`../docs/PROOFS.md`](../docs/PROOFS.md) if it adds a
|
|
meaningful new dimension.
|
|
|
|
Bug reports from a new validator → file at `raphael@maenle.net`
|
|
with the wire trace, the validator's output, and the equipment YAML.
|