# External cross-validation harnesses

Every harness in this directory exists so a reviewer doesn't have to
take our word for it.  Each one validates our C++ codec / framing /
dispatch against an **independent third-party implementation** that
read the SEMI standards without talking to us.

See [`../VERIFICATION.md`](../VERIFICATION.md) for the full test plan
and the honest accounting of which proofs are external vs internal.

## What's here

| Validator                                  | Independence                                | Coverage                                                                                                            |
|--------------------------------------------|---------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
| `host_vs_cpp_server.py` + `passive_equipment.py` | secsgem-py 0.3.0 — Python reference impl   | ~24 + 4 checks: S1, S2, S5, S6, S7, S10 happy paths                                                                  |
| `raw_gem300_harness.py`                    | secsgem-py with hand-crafted SecsStreamFunctions | 3 checks: S3F17, S16F5, S16F27 (limited by SFDL grammar)                                                            |
| `secs4j/Secs4jHostHarness.java`            | secs4java8 — Apache 2.0 Java impl by kenta-shimizu | **55 checks** across S1/S2/S3/S5/S6/S7/S10/S14/S16, including the full E40 body that defeated secsgem-py and unsolicited S6F11/S5F1 observation |
| `tshark_validate.sh`                       | Wireshark's built-in HSMS dissector         | 69 captured frames dissected with no malformed-packet warnings                                                       |
| `spool_persistence_test.py`                | secsgem-py + a docker-restart loop          | Restart-survives-spool integrity                                                                                     |
| ⚙️  `../tests/test_e5_kat.cpp`              | SEMI E5 §9 encoding rules                   | 196 known-answer byte assertions across every format code                                                            |
| ⚙️  `../apps/fuzz_secs2_decode.cpp` + `fuzz_sml_parse.cpp` | libFuzzer + ASan + UBSan       | ~70 000 + ~285 000 random inputs per minute, 0 crashes                                                              |

The ⚙️ entries aren't in `interop/` directly because they don't
involve a network peer — they're either pure codec round-trips
(KAT) or coverage-guided fuzzing.  Listed here so the full external
proof inventory lives in one place.

## Running each validator

### secsgem-py — secsgem-py active host → C++ server

```sh
docker compose up -d server
docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py \
    --host server --port 5000 --session-id 0
```

### secsgem-py — C++ host → secsgem-py equipment

```sh
docker compose up -d equipment_py
docker compose run --rm builder /app/build/secs_interop_probe \
    --host equipment_py --port 5000 --device 0
```

### secsgem-py — raw GEM 300 frames

```sh
docker compose up -d server
docker compose run --rm interop python3 /app/interop/raw_gem300_harness.py \
    --host server --port 5000 --session-id 0
```

### secs4j — independent Java host → C++ server

```sh
bash interop/secs4j_validate.sh
```

Builds an `eclipse-temurin:21-jdk` sidecar with secs4java8 cloned +
compiled at image build, then drives 55 checks against
`compose up server`.  See `secs4j/Secs4jHostHarness.java` for the
list and `secs4j/Dockerfile` for the build.

### tshark — Wireshark HSMS dissector

```sh
docker compose run --rm builder bash /app/interop/tshark_validate.sh
```

Captures a pcap of the demo flow, runs `tshark -V` with the HSMS
dissector forced for the test port, asserts no malformed packets +
that all expected control/data frames parse.

### spool persistence — restart-survives test

```sh
bash interop/spool_persistence_test.py
```

Drops the host link mid-flight, kills the server, restarts it, and
asserts the spooled S5F1 / S6F11 frames drain to the host on
reconnect.

## What these harnesses caught

Real bugs surfaced during interop development (now fixed):

1. **Strict U-width parsing rejected U1-encoded identifiers.**  SEMI
   E5 declares DATAID, RPTID, VID, CEID, ALID, EXID etc. as
   `U1 | U2 | U4 | U8`; secsgem-py picks the smallest width that
   fits.  Our scalar accessors were strict.  Now lenient with
   range-checked downcasts (`messages_helpers.hpp::any_unsigned_first`).

2. **PPBODY rejected when sent as ASCII.**  SEMI allows PPBODY to be
   `ASCII | Binary | List`; secsgem-py defaults to ASCII.  Added the
   `BINARY_OR_ASCII` codegen type and the `as_text_or_binary`
   accessor.

3. **Missing S1F23 / S1F24 (Collection Event Namelist).**  Added the
   wire schema, the `vids_for(ceid)` accessor, and the dispatch
   handler.

4. **Missing S10F3 handler (host→equipment Terminal Display).**  Our
   server only registered S10F1; per SEMI E5 §13 those are opposite
   directions.  Added the missing dispatch.

5. **TSan use-after-free in `act_exception_complete`** (test code,
   not library): held a pointer across `fire_internal(RecoveryComplete)`
   which deletes the entry.  Found by the ThreadSanitizer lane on
   first run.

The C++ test suite stayed green through every one of these fixes —
the changes were purely permissive widenings or additive features,
no existing behaviour broke.

## When to add a new validator

A new third-party SECS implementation, or a new dissector, or a new
fuzzer target — anything that exercises our wire surface from an
angle the existing five don't cover — is worth adding.  The pattern
is consistent:

1. New script / harness lives here (or a sidecar Docker context for
   non-Python validators).
2. Wired into `.gitea/workflows/ci.yml` as a separate job.
3. Listed in this README's table + in `../VERIFICATION.md`.
4. Surfaced in the README's proof-of-feature-completeness table if
   it adds a meaningful new dimension.

Bug reports from a new validator → file at
`raphael@maenle.net` with the wire trace, the validator's output,
and the equipment YAML so we can reproduce.