# External verification The unit suite is internal regression coverage. Four external validators run alongside it: SEMI E5 known-answer tests, Wireshark's HSMS dissector on a captured pcap, secs4java8 cross-validation, and libFuzzer over the decoder + SML parser. Each runs in CI on every push to `main`. | Channel | Source of independence | |----------------------------------|-------------------------------------------------------| | 445 unit/integration tests | Internal | | 47 conformance harness checks | Internal | | **SEMI E5 KAT** | **External — standards body's encoding rules** | | **Wireshark HSMS dissector** | **External — independent network-protocol authors** | | **secs4java8 interop** (55) | **External — second independent SECS implementation** | | **secsgem-py interop** (31) | **External — Python reference impl** | | **libFuzzer** (ASan + UBSan) | **External — coverage-guided structural search** | | 100 k random tool ops | Internal — property test | | YAML validation | Internal | --- ## 1. SEMI E5 known-answer tests `tests/test_e5_kat.cpp` pins the encoder and decoder to the byte patterns SEMI E5 requires. Each fixture is a `(canonical_hex, expected_item)` pair; `encode(expected_item)` must produce `canonical_hex` and `decode(canonical_hex)` must round-trip back. | Format | Code | Fixtures | |--------|--------|----------------------------------------------------------------| | List | `0x00` | empty, nested, mixed-type | | Binary | `0x20` | empty, 1-byte, 256-byte (2-byte length), 65 536-byte (3-byte length) | | Boolean| `0x24` | TRUE, FALSE, multi-element | | ASCII | `0x40` | empty, single char, 255-byte, 256-byte | | JIS-8 | `0x44` | empty, non-ASCII bytes | | U1/U2/U4/U8 | `0xA4 / 0xA8 / 0xAC / 0xA0` | 0, mid, max, multi-element | | I1/I2/I4/I8 | `0x64 / 0x68 / 0x6C / 0x60` | 0, ±1, INT_MIN, INT_MAX | | F4/F8 | `0x84 / 0x80` | 0.0, ±1.0, NaN, ±Inf, subnormal | Length-byte counts of 1, 2, and 3 are exercised explicitly. **Caveat on authority.** SEMI does not publish official test vectors for E5 (unlike NIST for crypto). The bytes are derived from the encoding rules in the spec, so KAT alone proves the codec is internally consistent with that reading. Independent corroboration of every format code arrives through secs4java8 and Wireshark, both with their own decoders. --- ## 2. Wireshark / tshark HSMS dissector `interop/tshark_validate.sh` starts the C++ server, captures a pcap of the two-container demo with `tcpdump`, then dissects every frame with Wireshark's HSMS dissector. The script fails if `tshark` reports any `Malformed Packet`, `Dissector bug`, or `Unknown PType/SType`, and asserts that `Select.req`, `Linktest.req`, `S1F13`, and `S6F11` each appear at least once. Wireshark's dissector is written by network-protocol authors with no shared code with this repository or with secsgem-py. Clean dissection of the pcap is an independent check on HSMS framing. **Coverage.** HSMS framing (4-byte length prefix + 10-byte header) and control-message shapes (Select / Deselect / Linktest / Separate / Reject). Wireshark renders SECS-II bodies as hex blobs and doesn't decode S/F semantics — KAT and secs4j cover that. **Result.** 69 HSMS frames per run, 0 malformed. Wired into CI as the `tshark-dissector` job. --- ## 3. secs4java8 cross-validation `interop/secs4j/` is a Docker harness wrapping [secs4java8](https://github.com/kenta-shimizu/secs4java8) (Apache 2.0). `Secs4jHostHarness.java` connects as an active HSMS host to the passive C++ server and drives 55 cross-validation checks across S1, S2, S3, S5, S6, S7, S10, S14, and S16. The harness covers the full-body GEM 300 shapes secsgem-py cannot easily drive: E40 process-job creation bodies, E94 control-job create, E87 carrier actions with slot maps, S2F49 enhanced commands, S5F13–F18 exception recovery, and S12 wafer maps. `interop/secs4j_validate.sh` orchestrates the harness against the server image; wired into CI as the `secs4j-interop` job. secsgem-py (Python) and secs4java8 (Java) are independent implementations of the same standards. Agreement on every frame across both peers is wire correctness from two independent angles. --- ## 4. libFuzzer over codec + SML parser `apps/fuzz_secs2_decode.cpp` and `apps/fuzz_sml_parse.cpp` are libFuzzer entry points built with `-DSECSGEM_FUZZ=ON` (`-fsanitize=fuzzer,address,undefined`). The CI lane runs each for 60 seconds — roughly 200 000 inputs through `secs2::decode` and 1.4 M through `try_parse_sml`. The corpus is seeded from the SECS-II hex fixtures shared with the rest of the suite, so the fuzzer starts from a known-good baseline and mutates outward. **Coverage.** Crashes and undefined behaviour on adversarial input — length-byte overflow, malformed format codes, recursive list bombs, truncated frames. A decoder that returns the wrong value silently is invisible to libFuzzer; KAT and the interop harnesses cover that. **Result.** 0 crashes, 0 ASan reports, 0 UBSan flags across both targets. --- ## What this does NOT replace - **A GEM RTS run.** Still required for certification; still costs money and needs hardware. See [MES_INTEROP.md](MES_INTEROP.md) §10. - **Per-MES interop sweeps** against the customer's actual MES (Camstar, FactoryWorks, etc.). Still required for any production deployment. See [MES_INTEROP.md](MES_INTEROP.md). - **Real-fab wire traces.** No public corpus exists; fabs treat their captures as IP. Those remain customer-side work.