raphael
af1a159c59
A large gap had opened between the docs and the code: the README and INTEGRATION guide did not mention the gRPC daemon or the Python client at all (the entire vendor surface), ARCHITECTURE still described secs_server as the ~1200-line canonical wiring example (it is a ~110-line thin main over EquipmentRuntime), and test counts across six files were stale (445/2753 -> 473/3087 core + the separate 125-assertion daemon suite). - README: new "Integrating your tool (pick a tier)" section — Python client / any-language gRPC / embedded C++ — plus daemon tests and tools/run_interop.sh in the Testing section. - ARCHITECTURE: layer diagram gains the vendor-surface and EquipmentRuntime/default_handlers tiers; stale wiring row fixed. - INTEGRATION: three-tier chooser up front (this guide = the C++ tier). - ch30 tour: secs_gemd + secs_gemd_tests in the binaries table. - ch31: example alarm used a nonexistent `alcd:` field with bit 7 set (which the validator forbids) -> real `category:`/`name:` fields, and the roles: block documented. - ch35: handler-location note now points at default_handlers.cpp's 15 per-capability register_* functions. - ch40: built-artifacts list + sample output counts. - ch50: secsgem::gem runtime/default_handlers/handler_slot/name_index includes + new secsgem::daemon namespace section. - PROOFS: test-count table gains the runtime/handlers/daemon row so the tally adds up; daemon suite noted. VERIFICATION/COMPLIANCE counts. - interop/README: the one-command runner + the two daemon-track harnesses (daemon_interop, pyclient_interop). Audited via a docs-vs-code sweep (the audit itself under-reported: it validated counts textually; reality was 473/3087). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
128 lines
5.7 KiB
Markdown
128 lines
5.7 KiB
Markdown
# External cross-validation harnesses
|
|
|
|
Each harness in this directory validates the C++ codec, framing, and
|
|
dispatch against an **independent third-party implementation** of
|
|
the same SEMI standards.
|
|
|
|
See [`../docs/VERIFICATION.md`](../docs/VERIFICATION.md) for the
|
|
internal-vs-external breakdown across all validators.
|
|
|
|
## What's here
|
|
|
|
| Validator | Independence | Coverage |
|
|
|--------------------------------------------|---------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
|
|
| `host_vs_cpp_server.py` + `passive_equipment.py` | secsgem-py 0.3.0 — Python reference impl | 31 checks: S1, S2, S5, S6, S7, S10 happy paths + unsolicited S6F11 / S5F1 |
|
|
| `raw_gem300_harness.py` | secsgem-py with hand-crafted SecsStreamFunctions | 3 checks: S3F17, S16F5, S16F27 (limited by SFDL grammar) |
|
|
| `secs4j/Secs4jHostHarness.java` | secs4java8 — Apache 2.0 Java impl by kenta-shimizu | **55 checks** across S1/S2/S3/S5/S6/S7/S10/S14/S16, including the full E40 body that defeated secsgem-py and unsolicited S6F11/S5F1 observation |
|
|
| `tshark_validate.sh` | Wireshark's built-in HSMS dissector | 69 captured frames dissected with no malformed-packet warnings |
|
|
| `spool_persistence_test.py` | secsgem-py + a docker-restart loop | Restart-survives-spool integrity |
|
|
| ⚙️ `../tests/test_e5_kat.cpp` | SEMI E5 §9 encoding rules | 196 known-answer byte assertions across every format code |
|
|
| ⚙️ `../apps/fuzz_secs2_decode.cpp` + `fuzz_sml_parse.cpp` | libFuzzer + ASan + UBSan | ~70 000 + ~285 000 random inputs per minute, 0 crashes |
|
|
|
|
The ⚙️ entries aren't in `interop/` directly because they don't
|
|
involve a network peer — they're either pure codec round-trips
|
|
(KAT) or coverage-guided fuzzing. Listed here so the full external
|
|
proof inventory lives in one place.
|
|
|
|
## One command for everything
|
|
|
|
`tools/run_interop.sh` (from the repo root) runs every validation step —
|
|
build, both unit suites, all the harnesses below, tshark, and secs4java8 —
|
|
with a PASS/FAIL summary. `SKIP_SECS4J=1` skips the Java image build.
|
|
|
|
## Running each validator
|
|
|
|
### secsgem-py — secsgem-py active host → C++ server
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### daemon bridge — gRPC tool + secsgem-py host → secs_gemd
|
|
|
|
```sh
|
|
docker compose up -d --no-deps gemd
|
|
docker compose run --rm --no-deps interop python3 daemon_interop.py \
|
|
--grpc gemd:50051 --hsms-host gemd
|
|
```
|
|
|
|
Both faces of the daemon at once: 20 checks proving gRPC SetVariables/
|
|
FireEvent/SetAlarm reach the reference host as S6F11/S5F1 over HSMS, and
|
|
the HCACK-4 command loop (host S2F41 → tool stream → completion event).
|
|
|
|
### Python client — published secsgem-client package → secs_gemd
|
|
|
|
```sh
|
|
docker compose up -d --no-deps gemd
|
|
docker compose run --rm --no-deps -e PYTHONPATH=/app/clients/python interop \
|
|
python3 pyclient_interop.py --grpc gemd:50051 --hsms-host gemd
|
|
```
|
|
|
|
13 checks driving the PUBLISHED Python API (eq.set / eq.fire / eq.alarm /
|
|
@eq.on) against a live daemon, with secsgem-py judging the wire.
|
|
|
|
### secsgem-py — C++ host → secsgem-py equipment
|
|
|
|
```sh
|
|
docker compose up -d equipment_py
|
|
docker compose run --rm builder /app/build/secs_interop_probe \
|
|
--host equipment_py --port 5000 --device 0
|
|
```
|
|
|
|
### secsgem-py — raw GEM 300 frames
|
|
|
|
```sh
|
|
docker compose up -d server
|
|
docker compose run --rm interop python3 /app/interop/raw_gem300_harness.py \
|
|
--host server --port 5000 --session-id 0
|
|
```
|
|
|
|
### secs4j — independent Java host → C++ server
|
|
|
|
```sh
|
|
bash interop/secs4j_validate.sh
|
|
```
|
|
|
|
Builds an `eclipse-temurin:21-jdk` sidecar with secs4java8 cloned +
|
|
compiled at image build, then drives 55 checks against
|
|
`compose up server`. See `secs4j/Secs4jHostHarness.java` for the
|
|
list and `secs4j/Dockerfile` for the build.
|
|
|
|
### tshark — Wireshark HSMS dissector
|
|
|
|
```sh
|
|
docker compose run --rm builder bash /app/interop/tshark_validate.sh
|
|
```
|
|
|
|
Captures a pcap of the demo flow, runs `tshark -V` with the HSMS
|
|
dissector forced for the test port, asserts no malformed packets +
|
|
that all expected control/data frames parse.
|
|
|
|
### spool persistence — restart-survives test
|
|
|
|
```sh
|
|
bash interop/spool_persistence_test.py
|
|
```
|
|
|
|
Drops the host link mid-flight, kills the server, restarts it, and
|
|
asserts the spooled S5F1 / S6F11 frames drain to the host on
|
|
reconnect.
|
|
|
|
## When to add a new validator
|
|
|
|
A new third-party SECS implementation, dissector, or fuzzer target
|
|
that exercises the wire surface from an angle the existing five
|
|
don't cover is worth adding. The pattern is consistent:
|
|
|
|
1. New script / harness lives here (or a sidecar Docker context for
|
|
non-Python validators).
|
|
2. Wired into `.gitea/workflows/ci.yml` as a separate job.
|
|
3. Listed in this README's table + in `../VERIFICATION.md`.
|
|
4. Surfaced in [`../docs/PROOFS.md`](../docs/PROOFS.md) if it adds a
|
|
meaningful new dimension.
|
|
|
|
Bug reports from a new validator → file at `raphael@maenle.net`
|
|
with the wire trace, the validator's output, and the equipment YAML.
|