Item 8a — ConfigValidator warns on non-identifier variable/event/alarm/
command names ([A-Za-z_][A-Za-z0-9_]*): language bindings expose names as
kwargs/attributes, so 'Chamber Pressure' would be unusable in the planned
Python client. Warning not error — the wire doesn't care. Tested (4 warning
sites + good-name negative).
Item 4 tail — golden frames for S5F1 (Binary ALCD / U4 ALID / ASCII ALTX)
and a composed S6F11 (the production-critical report shape), bytes hand-
computed from E5 encoding rules: external pins on message composition.
Item 7 — equipment_service.hpp moved to include/secsgem/daemon/ (apps/
include-path hack removed) and a TSan daemon lane added locally + in CI.
tools/tsan.supp suppresses races whose accesses sit entirely inside the
UNinstrumented system libgrpc/libgpr/libabsl (epoll wakeups, absl Mutex
GraphCycles bookkeeping); our frames stay fully checked. The lane earned its
keep on first run: it caught a REAL threading-contract violation — a daemon
test reading model stores from the test thread while the io thread serviced
posted writes — fixed to use read_sync, exactly per the documented contract.
Now TSan-clean under halt_on_error=1 in the full production threading shape.
Suites: core 470/3068, daemon Release+TSan 125/125 each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tools/run_interop.sh runs ALL nine validation steps with a PASS/FAIL summary:
build, unit (464), daemon-unit (41), secsgem-py host vs server (31 checks),
secs_conformance (47), gRPC+secsgem-py daemon bridge, spool persistence
across restart, tshark HSMS dissector, secs4java8 (55 checks). Verified green
end-to-end. The unit suite is partly self-referential (our parsers validate
our builders); these external validators are the real oracle — now they run
with one command instead of by hand. Two bugs found by running it: unbounded
ninja at -O3 OOM-kills cc1plus in memory-constrained Docker VMs (build with
-j 2) and bash-3.2 lacks negative array subscripts.
CI: grpc deps added to the build job so secs_gemd + secs_gemd_tests build and
RUN in CI (previously the daemon silently dropped out — now fails loudly if
missing), plus a python-interop lane running py-host/conformance/daemon
harnesses against localhost in one container (no docker-in-docker).
Service hardening while in there: reject proto Values with no kind set at
the RPC edge (previously silently became ASCII ""), TODO markers for list
element formats and daemon graceful shutdown. New tests: unset-Value guard
+ a property test iterating ALL configured variables via gRPC asserting each
keeps its declared SECS-II format (daemon tests 16 -> 41 assertions).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Picks up the file renames that landed alongside the previous commit
and fixes everything that pointed at the old root locations:
- README.md doc-map updated: every entry now points at docs/X.md,
with a new "docs/" lead entry pointing at the guided-tour index.
- README inline cross-refs (ARCHITECTURE / INTEGRATION / SECURITY /
BENCHMARKS / MES_INTEROP / PROOFS) repointed to docs/.
- README "Interop" section rewritten — used to mention only
secsgem-py; now covers all four external validators (secsgem-py
31 / secs4java8 55 / tshark 69 frames / libFuzzer 200 k+ runs)
with a one-line summary each, plus pointers to interop/README.md
and docs/VERIFICATION.md.
- README "Deferred follow-ups" cleaned: dropped the explanatory
"Listed here so reviewers don't go looking for them in
COMPLIANCE.md and find an 'out of scope' entry that sounds
defensive" sentence — the section header speaks for itself.
- docs/00_index.md "Where the rest of the docs live" table: dropped
every `../` prefix since the docs are now siblings.
- docs/01_what_is_secs_gem.md PROOFS reference updated to sibling.
- docs/02_the_cast.md INTEGRATION + MES_INTEROP refs updated to
siblings; dropped the stale "at the repo root" wording.
- interop/README.md: VERIFICATION + PROOFS refs updated to
../docs/X.md; stale "~24 + 4 checks" updated to 31 (matches
PROOFS.md and README).
- examples/pvd_tool/README.md: every doc cross-ref now points at
../../docs/X.md.
- Source / data / CI comments mentioning doc names (e.g.
"INTEGRATION.md §3", "COMPLIANCE.md gap") rewritten to
"docs/INTEGRATION.md §3" etc. — affects 9 files across
include/, apps/, tests/, data/, examples/, .gitea/workflows/.
Verified: full build under docker passes, 445/445 test cases pass,
2 753/2 753 assertions pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CI log showed:
Run export DEBIAN_FRONTEND=noninteractive
apt-get: command not found
Failure - Main Bootstrap (node + git)
exit status 127
The secs4j-interop job runs on the bare runner (not inside a
`container:`) because it needs the host's docker socket to run
`docker compose up -d server`. The runner image isn't fixed across
deployments — catthehacker/ubuntu has apt-get, but a minimal node
image doesn't. The old script hard-coded `apt-get` and exit 127'd
on anything else.
New bootstrap:
- Checks what's already on PATH (git, node, docker). If all three
are present, exits 0 — most act-runner images come pre-loaded.
- Otherwise picks the right package manager (apt-get or apk) and
installs only the missing pieces.
- Errors out with a useful message if neither package manager
exists, instead of failing on a missing command.
Also updates the inline comment that still said "20 checks" — actual
is 55 (matches the count in README / PROOFS.md / COMPLIANCE.md).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Coverage-guided structural search for crashes and undefined behaviour
on arbitrary input to our two parsers.
What's wired:
- -DSECSGEM_FUZZ=ON CMake option, clang-only. Adds
-fsanitize=fuzzer-no-link,address,undefined to all targets +
-fsanitize=fuzzer to the two fuzz executables.
- apps/fuzz_secs2_decode.cpp — feeds raw bytes to secs2::decode.
Catches secs2::CodecError (expected) but traps on anything else
leaking (would be a hardening bug).
- apps/fuzz_sml_parse.cpp — feeds string to try_parse_sml, which is
contractually nothrow-equivalent; traps on any exception.
- .gitea/workflows/ci.yml — `libfuzzer` job builds with clang and
runs each fuzzer for 60s in CI. Any crash / ASan / UBSan flag
fails the job.
- Dockerfile gains clang + libclang-rt-18-dev so devs can run
locally with the same toolchain.
Result on a fresh 30-second local run:
fuzz_secs2_decode: 70 727 random inputs, 0 crashes
fuzz_sml_parse: 284 950 random inputs, 0 crashes
The coverage-guided search found and synthesized inputs that
exercise: zero-byte, single-byte format tags, all length-byte
counts (1/2/3), nested lists, format bytes with reserved bits, the
"BOOLEAN" SML token, malformed quoted strings, etc. libFuzzer's
recommended dictionary at the end of each run shows what bytes /
substrings the coverage feedback discovered as discriminating —
useful signals if we ever want a hand-curated corpus.
README proof table grows to 8 commands. After this:
- 426 unit tests (internal)
- 47 conformance harness checks (internal)
- 24 secsgem-py interop checks (external — Python ref impl)
- 20 secs4j interop checks (external — independent Java impl)
- 69 frames dissected by Wireshark HSMS dissector (external)
- 196 SEMI E5 KAT assertions (standards body's encoding rules)
- **~70k + ~285k random inputs, 0 crashes (external)**
- 100k random tool ops with all invariants holding (internal)
- YAML validation (internal)
- TSan clean on 2 557 assertions (internal correctness aid)
Five distinct external proofs now, each covering a different angle.
Plan: VERIFICATION.md §4.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
20 cross-validation checks PASS against [secs4java8] (Apache 2.0,
kenta-shimizu) — an independent SECS/HSMS implementation in Java by
a different author from a different language ecosystem. Distinct
implementer = independent spec interpretation. Two libraries
agreeing on wire bytes is much stronger evidence of spec-correctness
than either alone.
Coverage targets the gap the secsgem-py interop deliberately skipped
(secsgem-py's SFDL grammar couldn't easily express GEM 300 bodies
with variable lists of named scalars):
- S1F1/F13/F17/F19/F21/F23 — establish comms + namelists
- S2F17 — clock
- S2F23 — trace init (5-field body)
- S2F49 — enhanced remote command (DATAID + OBJSPEC + RCMD + params)
- S3F17/F19/F25/F27 — full E87 carrier surface (action, slot map
verify, transfer with port pair, cancel)
- S5F13/F17 — exception recovery (EXID + EXRECVRA)
- S14F9/F11 — E94 CJ create with prjobids list, CJ delete
- S16F5/F27 — E40 PJ command, E94 CJ command
- S1F15 — offline cleanup
20/20 PASS against the demo equipment. Reply S/F matches the spec
for every transaction; specific ACK values vary by equipment state
(CarrierIDUnknown for an unknown carrier is just as valid as Accept
for a known one) so we assert on the wire shape, not the result.
Ship layout:
interop/secs4j/Dockerfile — eclipse-temurin:21-jdk + clone
+ build of secs4java8 → Export.jar
interop/secs4j/Secs4jHostHarness.java
— 20 round_trip assertions; uses
Secs2.list/uint4/ascii to build
full GEM 300 bodies; comm.send()
for arbitrary S/F pairs
interop/secs4j_validate.sh — orchestrator: builds image,
compiles harness, starts compose
server, runs Java container on
the secs network against it
.gitea/workflows/ci.yml — secs4j-interop job in CI
README.md — proof table grows to 7 commands
.gitignore — *.class
After this commit our proof chain has:
- SEMI E5 KAT (standards body's own arithmetic)
- tshark dissector (Wireshark's HSMS impl)
- secsgem-py interop (Python reference impl)
- **secs4j interop** (independent Java impl)
+ 426 unit tests, 47 conformance harness checks, 100k random ops,
YAML validation
Four independent external proofs, three of them on overlapping wire
surface from independent angles.
Plan: VERIFICATION.md §3.
[secs4java8]: https://github.com/kenta-shimizu/secs4java8
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wireshark's built-in HSMS dissector — written by network-protocol
authors who don't know us, didn't talk to us, and don't share
implementation details with secsgem-py — is a third independent codec
for our framing. If they parse our pcap without warnings, our HSMS
framing is wire-correct independently of both our internal tests and
the secsgem-py interop path.
interop/tshark_validate.sh:
- Boots secs_server on 127.0.0.1:5099 (away from the demo port)
- Captures the loopback wire traffic with tcpdump
- Runs secs_client through ~24 transactions plus Separate.req +
TCP FIN
- Parses the pcap with tshark -V using the HSMS dissector
- Asserts: no "Malformed Packet", no "Dissector bug", at least one
HSMS frame, expected tokens present (Select.req/rsp, Separate.req,
Data message), reports histogram (count by control type + distinct
S/F pairs)
Result against the demo: 69 HSMS frames dissected, 49 distinct
S/F pairs (S01F01..S16F28), all clean.
Dockerfile gains tshark + tcpdump. .gitea/workflows/ci.yml gains a
`tshark-dissector` job that runs this validator as part of every
push to main. README proof table grows to 6 commands.
VERIFICATION.md §1a documents a follow-up: round-trip the KAT
fixtures through secsgem-py to corroborate that the format codes
we used match an independent implementation. Strengthens the KAT
proof from "internally consistent" to "confirmed by a second
implementer who read the spec without talking to us."
Plan: VERIFICATION.md §2.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a -DSECSGEM_TSAN=ON CMake option that builds every target with
-fsanitize=thread + debug symbols + -O1 + frame pointers. Wires a
dedicated thread-sanitizer job into .gitea/workflows/ci.yml that
builds and runs the full test suite under TSan with
TSAN_OPTIONS=halt_on_error=1 (any flagged race fails the job, not
just warns).
Result against the full 426-case / 2557-assertion suite: 0 warnings,
all green. That converts the existing test_thread_safety.cpp (which
exercised the asio::post-onto-strand pattern) and test_concurrency
(in-flight transaction interleaving) and test_robustness_fuzz (28
random action types × thousands of ticks) from "pattern smoke-tests"
into actual race detection.
The first TSan run caught a real bug in test_robustness_fuzz's
act_exception_complete: it held a pointer to an ExceptionStore
entry across fire_internal(RecoveryComplete), which deletes the
entry. The subsequent state() read was a use-after-free. TSan
flagged it 8 times (4 reads × 2 stack-frame variants). Fix is
scoped lookup + re-check via has() after the mutation; matches the
contract any reasonable caller would follow.
The asio std_fenced_block atomic_thread_fence path generates TSan
"not supported" warnings during compile — those are asio's, not
ours, and don't affect runtime detection.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`actions/checkout@v4` is a JavaScript action — it expects `node` on
PATH in the runner image. Gitea Actions (and local `act`) running
against `ubuntu:24.04` had neither node nor git pre-installed, so
checkout failed with:
❌ Failure - Main actions/checkout@v4
exitcode '127': command not found
The pre-step now installs nodejs + git + ca-certificates from apt
before checkout runs. The rest of the C++ toolchain installs in a
second step after the source tree is on disk.
Doesn't affect GitHub-hosted runners (their images already have node);
doesn't change build behaviour either.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Gitea Actions workflow at .gitea/workflows/ci.yml. Spins up an
ubuntu:24.04 container, installs the same apt deps the Dockerfile uses
(build-essential, cmake, ninja-build, libasio-dev, libyaml-cpp-dev,
python3, python3-yaml), checks out, runs cmake + ninja, and executes
the doctest binary.
Runs the same toolchain as the local Docker setup; no docker-in-docker
required.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raphael