Public Access

Files

T

raphael b031f057af docs: customer-ready sweep + README restructure + tshark CI fix

Audit pass over the public-facing surface so a customer can read it
end-to-end without tripping on stale numbers or self-contradictions.

README + docs accuracy:
- Test counts 426 → 445, assertions 2 557 → 2 753 (verified via
  doctest run); E5 row was missing test_e5_kat (19 cases)
- Interop checks 24 → 31, COMPLIANCE.md message count 149 → 164,
  COMPLIANCE.md "291 cases / 1515 assertions" → 445 / 2 753
- README "60+ test IDs" for MES_INTEROP → actual 59
- PVD example counts: 32 SVIDs/17 CEIDs → 29/21, "~40 handlers
  in ~200 lines" → 51 in ~460, "~700 lines" → ~1,100; main.cpp
  header table-of-contents resynced with the actual 7 sections

Out-of-scope honesty (COMPLIANCE.md §8 + FAQ.md):
- Removed HSMS-GS (was both ✅ implemented in §1 and "out of scope"
  in §8; INTEGRATION.md §7 documents using it)
- Removed multi-block SECS-I (split_message/assemble_message exist
  with 4 dedicated tests)
- Added serial-port wiring as the genuine open ⬜ item — FSM is
  tested end-to-end over TCP; only the asio serial_port glue is
  deferred
- COMPLIANCE.md intro now lists E42 and notes "E37 (SS + GS)"

README restructure:
- Moved the 8-command proof table and per-standard test-coverage
  table to a new PROOFS.md (72 lines)
- README now leads with what / Quick start / Documentation map,
  then a one-paragraph "How it's proved" linking to PROOFS.md
- Updated cross-refs in FAQ.md, GLOSSARY.md, VERIFICATION.md, and
  interop/README.md to point at PROOFS.md

CI fix — tshark-dissector job:
- interop/tshark_validate.sh hardcoded /app/build/secs_server etc.
  which only works inside the docker image.  Now derives ROOT from
  the script's own location and accepts BUILD/SERVER/CLIENT/DATA
  env overrides, so CI can run it from the workspace dir
- Verified still passes in docker (69 frames, 0 malformed)

.gitignore:
- Added build-fuzz/ and build-tsan/ (were showing as untracked)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-06-09 18:59:17 +02:00

6.1 KiB

Raw Blame History

External cross-validation harnesses

Every harness in this directory exists so a reviewer doesn't have to take our word for it. Each one validates our C++ codec / framing / dispatch against an independent third-party implementation that read the SEMI standards without talking to us.

See ../VERIFICATION.md for the full test plan and the honest accounting of which proofs are external vs internal.

What's here

Validator	Independence	Coverage
`host_vs_cpp_server.py` + `passive_equipment.py`	secsgem-py 0.3.0 — Python reference impl	~24 + 4 checks: S1, S2, S5, S6, S7, S10 happy paths
`raw_gem300_harness.py`	secsgem-py with hand-crafted SecsStreamFunctions	3 checks: S3F17, S16F5, S16F27 (limited by SFDL grammar)
`secs4j/Secs4jHostHarness.java`	secs4java8 — Apache 2.0 Java impl by kenta-shimizu	55 checks across S1/S2/S3/S5/S6/S7/S10/S14/S16, including the full E40 body that defeated secsgem-py and unsolicited S6F11/S5F1 observation
`tshark_validate.sh`	Wireshark's built-in HSMS dissector	69 captured frames dissected with no malformed-packet warnings
`spool_persistence_test.py`	secsgem-py + a docker-restart loop	Restart-survives-spool integrity
⚙️ `../tests/test_e5_kat.cpp`	SEMI E5 §9 encoding rules	196 known-answer byte assertions across every format code
⚙️ `../apps/fuzz_secs2_decode.cpp` + `fuzz_sml_parse.cpp`	libFuzzer + ASan + UBSan	~70 000 + ~285 000 random inputs per minute, 0 crashes

The ⚙️ entries aren't in interop/ directly because they don't involve a network peer — they're either pure codec round-trips (KAT) or coverage-guided fuzzing. Listed here so the full external proof inventory lives in one place.

Running each validator

secsgem-py — secsgem-py active host → C++ server

docker compose up -d server
docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py \
    --host server --port 5000 --session-id 0

secsgem-py — C++ host → secsgem-py equipment

docker compose up -d equipment_py
docker compose run --rm builder /app/build/secs_interop_probe \
    --host equipment_py --port 5000 --device 0

secsgem-py — raw GEM 300 frames

docker compose up -d server
docker compose run --rm interop python3 /app/interop/raw_gem300_harness.py \
    --host server --port 5000 --session-id 0

secs4j — independent Java host → C++ server

bash interop/secs4j_validate.sh

Builds an eclipse-temurin:21-jdk sidecar with secs4java8 cloned + compiled at image build, then drives 55 checks against compose up server. See secs4j/Secs4jHostHarness.java for the list and secs4j/Dockerfile for the build.

tshark — Wireshark HSMS dissector

docker compose run --rm builder bash /app/interop/tshark_validate.sh

Captures a pcap of the demo flow, runs tshark -V with the HSMS dissector forced for the test port, asserts no malformed packets + that all expected control/data frames parse.

spool persistence — restart-survives test

bash interop/spool_persistence_test.py

Drops the host link mid-flight, kills the server, restarts it, and asserts the spooled S5F1 / S6F11 frames drain to the host on reconnect.

What these harnesses caught

Real bugs surfaced during interop development (now fixed):

Strict U-width parsing rejected U1-encoded identifiers. SEMI E5 declares DATAID, RPTID, VID, CEID, ALID, EXID etc. as U1 | U2 | U4 | U8; secsgem-py picks the smallest width that fits. Our scalar accessors were strict. Now lenient with range-checked downcasts (messages_helpers.hpp::any_unsigned_first).
PPBODY rejected when sent as ASCII. SEMI allows PPBODY to be ASCII | Binary | List; secsgem-py defaults to ASCII. Added the BINARY_OR_ASCII codegen type and the as_text_or_binary accessor.
Missing S1F23 / S1F24 (Collection Event Namelist). Added the wire schema, the vids_for(ceid) accessor, and the dispatch handler.
Missing S10F3 handler (host→equipment Terminal Display). Our server only registered S10F1; per SEMI E5 §13 those are opposite directions. Added the missing dispatch.
TSan use-after-free in act_exception_complete (test code, not library): held a pointer across fire_internal(RecoveryComplete) which deletes the entry. Found by the ThreadSanitizer lane on first run.

The C++ test suite stayed green through every one of these fixes — the changes were purely permissive widenings or additive features, no existing behaviour broke.

When to add a new validator

A new third-party SECS implementation, or a new dissector, or a new fuzzer target — anything that exercises our wire surface from an angle the existing five don't cover — is worth adding. The pattern is consistent:

New script / harness lives here (or a sidecar Docker context for non-Python validators).
Wired into .gitea/workflows/ci.yml as a separate job.
Listed in this README's table + in ../VERIFICATION.md.
Surfaced in ../PROOFS.md if it adds a meaningful new dimension.

Bug reports from a new validator → file at raphael@maenle.net with the wire trace, the validator's output, and the equipment YAML so we can reproduce.

6.1 KiB Raw Blame History