raphael
af1a159c59
A large gap had opened between the docs and the code: the README and INTEGRATION guide did not mention the gRPC daemon or the Python client at all (the entire vendor surface), ARCHITECTURE still described secs_server as the ~1200-line canonical wiring example (it is a ~110-line thin main over EquipmentRuntime), and test counts across six files were stale (445/2753 -> 473/3087 core + the separate 125-assertion daemon suite). - README: new "Integrating your tool (pick a tier)" section — Python client / any-language gRPC / embedded C++ — plus daemon tests and tools/run_interop.sh in the Testing section. - ARCHITECTURE: layer diagram gains the vendor-surface and EquipmentRuntime/default_handlers tiers; stale wiring row fixed. - INTEGRATION: three-tier chooser up front (this guide = the C++ tier). - ch30 tour: secs_gemd + secs_gemd_tests in the binaries table. - ch31: example alarm used a nonexistent `alcd:` field with bit 7 set (which the validator forbids) -> real `category:`/`name:` fields, and the roles: block documented. - ch35: handler-location note now points at default_handlers.cpp's 15 per-capability register_* functions. - ch40: built-artifacts list + sample output counts. - ch50: secsgem::gem runtime/default_handlers/handler_slot/name_index includes + new secsgem::daemon namespace section. - PROOFS: test-count table gains the runtime/handlers/daemon row so the tally adds up; daemon suite noted. VERIFICATION/COMPLIANCE counts. - interop/README: the one-command runner + the two daemon-track harnesses (daemon_interop, pyclient_interop). Audited via a docs-vs-code sweep (the audit itself under-reported: it validated counts textually; reality was 473/3087). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
130 lines
6.0 KiB
Markdown
130 lines
6.0 KiB
Markdown
# External verification
|
||
|
||
The unit suite is internal regression coverage. Four external
|
||
validators run alongside it: SEMI E5 known-answer tests, Wireshark's
|
||
HSMS dissector on a captured pcap, secs4java8 cross-validation, and
|
||
libFuzzer over the decoder + SML parser. Each runs in CI on every
|
||
push to `main`.
|
||
|
||
| Channel | Source of independence |
|
||
|----------------------------------|-------------------------------------------------------|
|
||
| 473 unit/integration tests | Internal |
|
||
| 47 conformance harness checks | Internal |
|
||
| **SEMI E5 KAT** | **External — standards body's encoding rules** |
|
||
| **Wireshark HSMS dissector** | **External — independent network-protocol authors** |
|
||
| **secs4java8 interop** (55) | **External — second independent SECS implementation** |
|
||
| **secsgem-py interop** (31) | **External — Python reference impl** |
|
||
| **libFuzzer** (ASan + UBSan) | **External — coverage-guided structural search** |
|
||
| 100 k random tool ops | Internal — property test |
|
||
| YAML validation | Internal |
|
||
|
||
---
|
||
|
||
## 1. SEMI E5 known-answer tests
|
||
|
||
`tests/test_e5_kat.cpp` pins the encoder and decoder to the byte
|
||
patterns SEMI E5 requires. Each fixture is a `(canonical_hex,
|
||
expected_item)` pair; `encode(expected_item)` must produce
|
||
`canonical_hex` and `decode(canonical_hex)` must round-trip back.
|
||
|
||
| Format | Code | Fixtures |
|
||
|--------|--------|----------------------------------------------------------------|
|
||
| List | `0x00` | empty, nested, mixed-type |
|
||
| Binary | `0x20` | empty, 1-byte, 256-byte (2-byte length), 65 536-byte (3-byte length) |
|
||
| Boolean| `0x24` | TRUE, FALSE, multi-element |
|
||
| ASCII | `0x40` | empty, single char, 255-byte, 256-byte |
|
||
| JIS-8 | `0x44` | empty, non-ASCII bytes |
|
||
| U1/U2/U4/U8 | `0xA4 / 0xA8 / 0xAC / 0xA0` | 0, mid, max, multi-element |
|
||
| I1/I2/I4/I8 | `0x64 / 0x68 / 0x6C / 0x60` | 0, ±1, INT_MIN, INT_MAX |
|
||
| F4/F8 | `0x84 / 0x80` | 0.0, ±1.0, NaN, ±Inf, subnormal |
|
||
|
||
Length-byte counts of 1, 2, and 3 are exercised explicitly.
|
||
|
||
**Caveat on authority.** SEMI does not publish official test vectors
|
||
for E5 (unlike NIST for crypto). The bytes are derived from the
|
||
encoding rules in the spec, so KAT alone proves the codec is
|
||
internally consistent with that reading. Independent corroboration
|
||
of every format code arrives through secs4java8 and Wireshark, both
|
||
with their own decoders.
|
||
|
||
---
|
||
|
||
## 2. Wireshark / tshark HSMS dissector
|
||
|
||
`interop/tshark_validate.sh` starts the C++ server, captures a pcap
|
||
of the two-container demo with `tcpdump`, then dissects every frame
|
||
with Wireshark's HSMS dissector. The script fails if `tshark` reports
|
||
any `Malformed Packet`, `Dissector bug`, or `Unknown PType/SType`, and
|
||
asserts that `Select.req`, `Linktest.req`, `S1F13`, and `S6F11` each
|
||
appear at least once.
|
||
|
||
Wireshark's dissector is written by network-protocol authors with
|
||
no shared code with this repository or with secsgem-py. Clean
|
||
dissection of the pcap is an independent check on HSMS framing.
|
||
|
||
**Coverage.** HSMS framing (4-byte length prefix + 10-byte header)
|
||
and control-message shapes (Select / Deselect / Linktest / Separate /
|
||
Reject). Wireshark renders SECS-II bodies as hex blobs and doesn't
|
||
decode S/F semantics — KAT and secs4j cover that.
|
||
|
||
**Result.** 69 HSMS frames per run, 0 malformed. Wired into CI as
|
||
the `tshark-dissector` job.
|
||
|
||
---
|
||
|
||
## 3. secs4java8 cross-validation
|
||
|
||
`interop/secs4j/` is a Docker harness wrapping
|
||
[secs4java8](https://github.com/kenta-shimizu/secs4java8) (Apache 2.0).
|
||
`Secs4jHostHarness.java` connects as an active HSMS host to the
|
||
passive C++ server and drives 55 cross-validation checks across S1,
|
||
S2, S3, S5, S6, S7, S10, S14, and S16.
|
||
|
||
The harness covers the full-body GEM 300 shapes secsgem-py cannot
|
||
easily drive: E40 process-job creation bodies, E94 control-job
|
||
create, E87 carrier actions with slot maps, S2F49 enhanced commands,
|
||
S5F13–F18 exception recovery, and S12 wafer maps.
|
||
`interop/secs4j_validate.sh` orchestrates the harness against the
|
||
server image; wired into CI as the `secs4j-interop` job.
|
||
|
||
secsgem-py (Python) and secs4java8 (Java) are independent
|
||
implementations of the same standards. Agreement on every frame
|
||
across both peers is wire correctness from two independent angles.
|
||
|
||
---
|
||
|
||
## 4. libFuzzer over codec + SML parser
|
||
|
||
`apps/fuzz_secs2_decode.cpp` and `apps/fuzz_sml_parse.cpp` are
|
||
libFuzzer entry points built with `-DSECSGEM_FUZZ=ON`
|
||
(`-fsanitize=fuzzer,address,undefined`). The CI lane runs each for
|
||
60 seconds — roughly 200 000 inputs through `secs2::decode` and 1.4 M
|
||
through `try_parse_sml`.
|
||
|
||
The corpus is seeded from the SECS-II hex fixtures shared with the
|
||
rest of the suite, so the fuzzer starts from a known-good baseline
|
||
and mutates outward.
|
||
|
||
**Coverage.** Crashes and undefined behaviour on adversarial input —
|
||
length-byte overflow, malformed format codes, recursive list bombs,
|
||
truncated frames. A decoder that returns the wrong value silently
|
||
is invisible to libFuzzer; KAT and the interop harnesses cover that.
|
||
|
||
**Result.** 0 crashes, 0 ASan reports, 0 UBSan flags across both
|
||
targets.
|
||
|
||
---
|
||
|
||
## What this does NOT replace
|
||
|
||
- **A GEM RTS run.** Still required for certification; still costs
|
||
money and needs hardware. See
|
||
[MES_INTEROP.md](MES_INTEROP.md) §10.
|
||
- **Per-MES interop sweeps** against the customer's actual MES
|
||
(Camstar, FactoryWorks, etc.). Still required for any production
|
||
deployment. See [MES_INTEROP.md](MES_INTEROP.md).
|
||
- **Real-fab wire traces.** No public corpus exists; fabs treat
|
||
their captures as IP.
|
||
|
||
Those remain customer-side work.
|