The duplicate-ARRIVED fix from the previous commit returned INVALID_OBJECT
with the message "no substrate 'X'" — a lie, since the substrate exists.
Rewrite ReportSubstrate so ARRIVED has its own ack mapping: a duplicate is
CANNOT_DO_NOW with "substrate 'X' already exists" (a state conflict, not a
missing object), and we never silently re-create over live FSM state.
Coverage gaps closed:
- C++: ARRIVED records carrier_id/slot (now asserted); module NOT_EXECUTING
reset transition; duplicate-ARRIVED expects CANNOT_DO_NOW.
- Interop: @eq.command now drives the real host S2F41 path (was @eq.on, so
the headline decorator had zero wire coverage); @eq.command NameError on
unknown name; eq.names var/alarm + dir() + typo-suggestion; replaced the
two `check(..., True)` tautologies with full E90 journey + AT_DESTINATION
and real error paths (ghost wafer raises, illegal module jump raises).
All 8 daemon test cases (248 assertions) and 24 pyclient interop checks pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
C++ (equipment_service.hpp):
- ReportSubstrate ARRIVED: check CreateResult and return INVALID_OBJECT
when the substrate ID already exists, instead of silently doing nothing
- ReportSubstrate/ReportModule default switch branches: return false
(→ CANNOT_DO_NOW) for unknown enum values instead of silently accepting
Python (_client.py):
- @eq.command: raise NameError (client-side name validation) instead of
SecsGemError (which means "daemon declined a request") — wrong type
- Module docstring: update example to show @eq.command / eq.names API
Test (test_daemon_service.cpp):
- Add duplicate-ARRIVED check (expects INVALID_OBJECT)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Completes the daemon's GEM300 surface and adds two new test tiers.
D10 — E87 carriers: CarrierStore gains the HandlerSlot observer pattern
(add_id/slot_map/access_handler). The daemon's id-observer forwards host
S3F17 decisions onto the Subscribe stream as CarrierAction (PROCEED on a
Confirmed transition, CANCEL on CancelCarrier); ReportCarrier drives the
flow tool-side: WAITING creates the carrier + records the slot map,
IN_ACCESS/COMPLETE advance the access FSM (INVALID_OBJECT on unknown,
CANNOT_DO_NOW on an illegal transition).
E16 — operations RPCs: Describe (full name inventory: variables/events/
alarms/commands/constants + device header), FlushSpool (purge or drain),
SendTerminalMessage (S10F1 tool->host, honest CANNOT_DO_NOW when no host
and stream 10 isn't spoolable).
Stream responsiveness: Subscribe/WatchHealth poll at 100ms (was 500ms) so a
cancelled stream frees its sync-server worker thread promptly — this was
found by the new stress test, which hung under Subscribe churn at 500ms.
Tests:
- A randomized concurrent RPC stress case: 4 threads x 250 seeded ops
(set/get/fire/alarm/control-state/describe + Subscribe churn), asserts no
failed RPC and a still-responsive engine afterward; prints its seed; a
strong TSan target.
- A virtual fab (interop/virtual_fab.py + the `fab` compose service /
tools/spawn_fab.sh): N daemons, each with a secsgem-py host AND a
secsgem_client tool, driven by seeded random traffic with end-to-end
invariant checks (set/get round-trips, event->S6F11 and alarm->S5F1
delivery, command->tool->completion). Verified green at N=3 (~150 ops/eq,
all commands round-tripped, 0 violations). Wired into run_interop.sh
(now 13 steps).
Also fixes the CI break from the previous commit: the Python-client lane's
test_values.py step lacked PYTHONPATH=clients/python (now step-level env).
Two bugs found and fixed while building this, both mine from this batch:
1. carrier test hung on a CancelCarrier of a still-NotConfirmed carrier — a
self-transition the FSM doesn't signal, so the observer never fired and
the stream Read blocked forever. Fixed to cancel a Confirmed carrier;
the NotConfirmed edge is documented as a known E87 limitation.
2. the 500ms stream poll above.
Daemon suite 7 cases / 214 assertions; core 475 / 3097; virtual fab green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Semantics settled and documented: v1 is observe-and-report. The engine keeps
acking S16/S3/S7/S2F15 from its FSM tables — exactly the behaviour both
reference implementations validated — while the tool observes lifecycle
events on the Subscribe stream and reports physical progress back. Gating
stays the documented v2 deferred-reply item.
Engine: two new store observers (HandlerSlot pattern) — RecipeStore fires
(ppid, body) after an add (S7F3 downloads), EquipmentConstantStore fires
(id, value) on ACCEPTED S2F15 writes only. Unit-tested.
Daemon: the service registers PJ/recipe/EC observers (io thread; add_
observers coexist with register_default_handlers' primaries) and fans the
new HostRequest variants out via push_request (fire-and-forget, no-
buffering contract). ProcessJob carries action (Start->START, Resume->
RESUME, Paused->PAUSE, Stopping->STOP, Aborting->ABORT) + recipe + material
bindings read store-side on the io thread. ReportProcessJob maps SETTING_UP
->SetupComplete, COMPLETE->ProcessComplete, ABORTED->AbortComplete via
read_sync; PROCESSING is informational; unknown job => INVALID_OBJECT,
table-rejected transition => CANNOT_DO_NOW. Carriers deferred (CarrierStore
has no observer machinery; ReportCarrier stays UNIMPLEMENTED) — roadmap.
Python client: on_process_job / on_recipe / on_constant_change decorators +
report_job(job_id, state); ProcessJob dataclass exported.
Tests: daemon suite 141 -> 175 assertions — the full in-process loop
(S16F11 create -> tool setup -> S16F5 PJSTART -> stream ProcessJob with
recipe+carriers -> ReportProcessJob(COMPLETE) -> FSM at ProcessComplete),
rejection paths, S7F3 -> ProcessProgram, S2F15 -> ConstantChange with the
configured name. Core 475/3097 (observer units). Live regression: daemon
interop 20 checks + pyclient 13 checks still green against the running
daemon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
B7 — the daemon's HSMS face under the Java reference: Dockerfile.server now
bakes secs_gemd alongside secs_server (grpc deps in both stages), and
secs4j_validate.sh gains TARGET=gemd to point the 55-check secs4java8 suite
at the daemon instead. Result: 55/55 green. With secsgem-py already
validating both faces, byte-identical GEM between secs_server and secs_gemd
is now proven by both reference implementations, not inferred from shared
code. CI runs the daemon target as an extra step (image layers shared).
Second client — clients/cpp: a header-only C++ twin of the Python client
over the same proto. eq.set("ChamberPressure", 2.5) with bare literals
(integral/floating dispatch avoids variant ambiguity), get/fire/alarm/
clear, control_state/request_control_state/health, on("START", fn) +
listen()/listen_async()/stop() with auto-CompleteCommand, SecsGemError
carrying the daemon's message. cpp_mini_tool (~30 lines) mirrors the
Python mini_tool. Tested end-to-end over real loopback TCP against the
service inside secs_gemd_tests — now 4 cases / 141 assertions — including
set/get round-trips, error text, alarm-by-name into the model, health,
and the full HCACK-4 command loop with parameters.
(Build note: two grpc-heavy TUs at -O3 OOM even at -j2 on Docker Desktop;
built -j1. Known environment limitation, roadmap-documented.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
register_default_handlers was a relocated app main(): one 1086-line function,
all-or-nothing. It is now 15 per-capability registration functions along the
lines GEM itself defines (S1F19): identification, equipment constants, clock,
event reports, remote commands, trace/limits, spooling, alarms, exceptions,
material tracking (E90/E116/E157), carriers (E87), recipes, object services
(E39), jobs (E40/E94), terminal services. A sensor-class tool registers three
functions instead of carrying carrier/job handlers it doesn't have;
register_default_handlers composes all 15. Each function derives exactly the
runtime aliases its handlers use (generated programmatically from the moved
bodies with boundary/substitution guards — zero hand-retyping).
Magic constants are gone: the control-state/clock SVIDs (were hardcoded 1/2)
and the CJ Executing/Completed CEIDs (were 400/401) now come from a "roles:"
block in equipment.yaml via EquipmentDescriptor, with historical defaults
when absent, loader parsing, and validation (CEID roles must name declared
events). The coupling is now visible in ONE file instead of silently split
between YAML and C++ — the exact drift class this repo's spec-as-data
philosophy exists to kill.
Tests: capability subsetting, role-driven SVID refresh via S1F3, roles
loader (shipped/custom/absent). Battery: core 473/3087 incl. the 53-handler
conformance sweep, daemon 125/125, live GEM300 demo (client exit 0), daemon
interop 20/20 vs secsgem-py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Item 8a — ConfigValidator warns on non-identifier variable/event/alarm/
command names ([A-Za-z_][A-Za-z0-9_]*): language bindings expose names as
kwargs/attributes, so 'Chamber Pressure' would be unusable in the planned
Python client. Warning not error — the wire doesn't care. Tested (4 warning
sites + good-name negative).
Item 4 tail — golden frames for S5F1 (Binary ALCD / U4 ALID / ASCII ALTX)
and a composed S6F11 (the production-critical report shape), bytes hand-
computed from E5 encoding rules: external pins on message composition.
Item 7 — equipment_service.hpp moved to include/secsgem/daemon/ (apps/
include-path hack removed) and a TSan daemon lane added locally + in CI.
tools/tsan.supp suppresses races whose accesses sit entirely inside the
UNinstrumented system libgrpc/libgpr/libabsl (epoll wakeups, absl Mutex
GraphCycles bookkeeping); our frames stay fully checked. The lane earned its
keep on first run: it caught a REAL threading-contract violation — a daemon
test reading model stores from the test thread while the io thread serviced
posted writes — fixed to use read_sync, exactly per the documented contract.
Now TSan-clean under halt_on_error=1 in the full production threading shape.
Suites: core 470/3068, daemon Release+TSan 125/125 each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The HCACK-4 contract, implemented end to end. For every YAML-declared
command the service registers a forwarding handler (new HostCommandRegistry
names()/spec() accessors): with a subscribed tool client the command is
queued onto the Subscribe stream (id + name + params via from_item) and the
host is answered S2F42 HCACK=4 immediately — never blocking the io thread or
the T3 window; with NO subscriber the command takes its declarative YAML ack
(the honest pre-daemon behaviour). Settled + documented in the proto: v1 is
a firehose with no buffering/replay. CompleteCommand correlates the pending
id (audit; unknown id => PARAMETER_INVALID). Side effects stay suppressed on
HCACK-4 (router applies them only on Accept), so the completion event the
TOOL fires is the host's real signal — exactly E30's intent.
Tests (daemon suite 101 -> 124 assertions): a real S2F41 dispatched through
the full default-handler router ON the io thread under run_async — HCACK 4
with subscriber + params on the stream, declarative Accept without,
CompleteCommand known/unknown, fallback restored after unsubscribe.
Interop (now 20 checks, all green): the complete conformant loop against
the secsgem-py reference host — S2F41 START -> S2F42 HCACK=4 -> tool
receives Command(name=START, id=1) -> CompleteCommand -> FireEvent -> host
receives S6F11.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A2 — alarms: optional 'name:' on alarm config (a LOCAL key — SEMI only
defines numeric ALID + freetext ALTX; field appended last so existing
{id, text, category} brace-inits compile unchanged), parsed by the loader,
checked by the validator, shipped in equipment.yaml. SetAlarm/ClearAlarm
RPCs resolve config name OR stringified ALID via a constructor snapshot.
A3 — control state + health: RequestControlState fires operator events on
the io thread (read_sync) and reports what the E30 table actually did —
ACCEPT iff the equipment landed in the requested state, CANNOT_DO_NOW naming
the actual state otherwise (the shipped table has no operator path to
EquipmentOffline; the test pins that honesty). ATTEMPT_ONLINE is rejected as
transient. WatchHealth streams an immediate snapshot then pushes on link/
control-state changes via service observers (add_link_observer +
add_control_state_observer — the HandlerSlot work paying off), spool depth
sampled at the 500ms poll; ends on cancel or engine stop.
Tests: daemon suite 61 -> 101 assertions (alarm lifecycle by name/id/unknown,
WatchHealth initial + change push, all four RequestControlState semantics);
loader test for the alarm name (present + absent fallback); core 467/3055.
Interop now 15 checks incl. gRPC SetAlarm -> host receives S5F1 ALCD=0x84
ALID=1, and RequestControlState(HOST_OFFLINE) -> GetControlState confirms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
EquipmentRuntime::read_sync establishes THE pattern for reading mutable
engine state from gRPC/binding threads (Phase 0 item 6): post the read onto
the io thread (the model's single owner), wait on a future with a deadline,
nullopt => UNAVAILABLE at the RPC edge. Always truthful, no cache to
invalidate; milliseconds are irrelevant at SECS rates.
GetVariables: name resolution against the service snapshot (empty query =
all; unknown name => INVALID_ARGUMENT naming the offender), values read via
read_sync, converted by the new from_item reverse conversion (single-element
numeric arrays => scalars, multi-element => List; Boolean/Binary/text per
format; C2-as-integer and U8>2^63 wrap documented as TODOs).
Tests run the engine in run_async — the daemon's PRODUCTION threading mode,
previously untested — and round-trip through both conversions: SetVariables
(declared-format write) then GetVariables (read) over a real in-process
channel. Daemon suite 41 -> 61 assertions. daemon_interop.py gains a live
GetVariables round-trip check vs the running daemon (verified green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One ordered in-process scenario drives 53 of the 56 registered handlers
through Router::dispatch — S1 identification/comms/control, S2 ECs/clock/
event-config/commands/trace/limits/spool, S5 alarms+exceptions, S6 reports,
S7 recipes, S10 terminal, S14/S16 E39+E40/E94 jobs, S3 carriers — asserting
every reply is the paired (stream, function+1) with a body, plus targeted
state checks (OnlineRemote after S1F17, PJ exists after S16F11, HostOffline
after S1F15) and the Router's SxF0 abort fallback for unregistered W=1
primaries. Same flow secs_conformance runs over a live socket, but cheap
enough for every build; closes the '56 handlers, 4 direct tests' gap from
the design review.
Also seeds message-level golden frames: S1F13's body pinned to bytes
hand-computed from the E5 encoding rules — an external check on message
composition, not our codec validating itself (TODO: S5F1, composed S6F11).
Suite: 466 cases / 3052 assertions (+236), all green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tools/run_interop.sh runs ALL nine validation steps with a PASS/FAIL summary:
build, unit (464), daemon-unit (41), secsgem-py host vs server (31 checks),
secs_conformance (47), gRPC+secsgem-py daemon bridge, spool persistence
across restart, tshark HSMS dissector, secs4java8 (55 checks). Verified green
end-to-end. The unit suite is partly self-referential (our parsers validate
our builders); these external validators are the real oracle — now they run
with one command instead of by hand. Two bugs found by running it: unbounded
ninja at -O3 OOM-kills cc1plus in memory-constrained Docker VMs (build with
-j 2) and bash-3.2 lacks negative array subscripts.
CI: grpc deps added to the build job so secs_gemd + secs_gemd_tests build and
RUN in CI (previously the daemon silently dropped out — now fails loudly if
missing), plus a python-interop lane running py-host/conformance/daemon
harnesses against localhost in one container (no docker-in-docker).
Service hardening while in there: reject proto Values with no kind set at
the RPC edge (previously silently became ASCII ""), TODO markers for list
element formats and daemon graceful shutdown. New tests: unset-Value guard
+ a property test iterating ALL configured variables via gRPC asserting each
keeps its declared SECS-II format (daemon tests 16 -> 41 assertions).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The single-slot set_*_handler pattern was a structural blocker, hit twice:
the daemon could not observe control-state changes because
register_default_handlers owns the slot, forcing GetControlState to read the
FSM cross-thread (a data race), and blocking WatchHealth and the Subscribe
stream's ControlStateChange variant.
HandlerSlot<Args...> keeps a primary slot with exact legacy semantics
(set_ replaces — one existing test depends on replacement) plus an
append-only observer list (add_) that survives set_ calls. Fire sites are
textually unchanged (operator bool / operator() / assign-from-function).
Applied to ControlStateMachine + ProcessJobStore + ControlJobStore (the
roadmap-critical three; the remaining single-slot classes follow the same
3-line pattern as needed). EquipmentRuntime gains an atomic control-state
mirror registered as an observer — control_state() is now safe from any
thread, retiring the GetControlState race — plus add_control_state_observer
and add_link_observer (selected/closed fan-out), the hooks WatchHealth and
Subscribe need.
Tests: observer ordering, set-replaces-primary-but-observers-survive,
observers-without-primary, PJ-store coexistence, and the runtime scenario
that was previously impossible (mirror + observer + default-handlers set_).
Core 464/464 (2816 assertions), daemon 16/16, live GEM300 demo passes with
single-fire control-state transitions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit fixes for two real bugs in the gRPC service:
1. Format compliance: to_item() wrote F8/I8 regardless of the variable's
declared wire format, so values contradicted the S1F11/S1F21 namelists
(ChamberPressure is F4, WaferCounter U4; the interop trace showed <F8 2.5>
on the wire). Conversion now targets the declared format — verified
end-to-end: secsgem-py now receives <F4 2.5> in S6F11.
2. Thread safety: gRPC handler threads called resolve_variable/resolve_event,
copying live store entries (including Item values) while the io thread
mutates them. The service now snapshots the immutable name->id/format maps
at construction (before run_async, per the documented ordering); all writes
already post to the io thread. Remaining known narrow race (GetControlState
enum read) documented in DAEMON_ROADMAP.
Also: drop a stale tools/run_interop.sh reference from docker-compose.yml.
Tests: daemon in-process 16/16 (new F4/U4 format assertions), core 459/459,
secsgem-py interop green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- name_index: add resolve_event(name) -> CEID (unit-tested).
- equipment_service.hpp: extract the gRPC service + value/state conversion
into a shared header; add FireEvent (optional per-fire variable values,
then trigger the collection event by name). secs_gemd slims to main().
- test_daemon_service: real in-process gRPC integration test (client stub ->
service -> EquipmentRuntime) proving SetVariables lands in the model,
GetControlState reports the state, FireEvent and unknown-name paths behave.
Separate secs_gemd_tests target (links grpc++/proto), gated on the daemon.
Core suite 459/459 (2799 assertions); daemon gRPC tests 15/15.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the SECS/GEM engine wiring out of the secs_server app into a
reusable class, and stand up a language-agnostic gRPC daemon on top so a
tool's software (any language) can drive the equipment without linking C++
or knowing SEMI. Foundation for replacing a vendor's SECS/GEM server.
Engine reuse:
- EquipmentRuntime (include/secsgem/gem/runtime.hpp, src/gem/runtime.cpp):
owns io_context, passive Server, model, control-state machine, Router;
thread-safe outbound API (set_variable/emit_event/set_alarm/clear_alarm),
on_command hook, deliver_or_spool, run()/run_async()/poll()/stop().
- register_default_handlers (src/gem/default_handlers.cpp): the 56 GEM
handlers + domain emitters, relocated from secs_server so the app and the
daemon speak byte-identical GEM. secs_server.cpp reduced ~1270 -> 113 lines.
- name_index.hpp: resolve_variable(name) -> VID (the name->id binding layer).
Daemon (apps/secs_gemd.cpp, proto/secsgem/v1/equipment.proto):
- runs the engine + HSMS link on a background thread; serves the gRPC
Equipment service. Increment 1: SetVariables (name-resolved, plain
value->Item) and GetControlState. proto carries the full v1 surface
(universal + carrier/recipe/job tiers); remaining RPCs + the Subscribe
command stream are next (docs/DAEMON_ROADMAP.md).
- CMake: opt-in SECSGEM_DAEMON, protoc/grpc_cpp_plugin codegen, gracefully
skipped where protobuf/grpc++ are absent. Dockerfile gains the grpc deps.
Tests (proof): test_runtime, test_default_handlers (S1F1->S1F2, S2F41->hook),
test_name_index. Full suite 458/458, 2795 assertions; live server<->client
GEM300 demo still passes on the refactored server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Host commands were declarative-only: dispatch() returned the YAML-defined
HCACK plus side effects, and ignored the command parameters entirely (the
param list was a commented-out argument). Equipment could acknowledge a
command but never run anything in response — the pvd_tool example worked
around this by hard-coding behaviour in a C++ router handler.
Add set_handler(rcmd, fn): a registered handler receives the live CPNAME/
CPVAL parameters and returns the HCACK, overriding the declarative default.
Live on S2F41/F21/F49 via the shared dispatch(). No handler => byte-for-byte
the previous declarative behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Picks up the file renames that landed alongside the previous commit
and fixes everything that pointed at the old root locations:
- README.md doc-map updated: every entry now points at docs/X.md,
with a new "docs/" lead entry pointing at the guided-tour index.
- README inline cross-refs (ARCHITECTURE / INTEGRATION / SECURITY /
BENCHMARKS / MES_INTEROP / PROOFS) repointed to docs/.
- README "Interop" section rewritten — used to mention only
secsgem-py; now covers all four external validators (secsgem-py
31 / secs4java8 55 / tshark 69 frames / libFuzzer 200 k+ runs)
with a one-line summary each, plus pointers to interop/README.md
and docs/VERIFICATION.md.
- README "Deferred follow-ups" cleaned: dropped the explanatory
"Listed here so reviewers don't go looking for them in
COMPLIANCE.md and find an 'out of scope' entry that sounds
defensive" sentence — the section header speaks for itself.
- docs/00_index.md "Where the rest of the docs live" table: dropped
every `../` prefix since the docs are now siblings.
- docs/01_what_is_secs_gem.md PROOFS reference updated to sibling.
- docs/02_the_cast.md INTEGRATION + MES_INTEROP refs updated to
siblings; dropped the stale "at the repo root" wording.
- interop/README.md: VERIFICATION + PROOFS refs updated to
../docs/X.md; stale "~24 + 4 checks" updated to 31 (matches
PROOFS.md and README).
- examples/pvd_tool/README.md: every doc cross-ref now points at
../../docs/X.md.
- Source / data / CI comments mentioning doc names (e.g.
"INTEGRATION.md §3", "COMPLIANCE.md gap") rewritten to
"docs/INTEGRATION.md §3" etc. — affects 9 files across
include/, apps/, tests/, data/, examples/, .gitea/workflows/.
Verified: full build under docker passes, 445/445 test cases pass,
2 753/2 753 assertions pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Hex-string fixtures constructed directly from the SEMI E5 §9
format-byte encoding rules:
format_byte = (format_code << 2) | length_byte_count
length_byte_count ∈ {1, 2, 3}
Coverage:
- Every format code (L, B, BOOLEAN, A, J, C, U1-U8, I1-I8, F4, F8)
- Every length-byte-count variant (1, 2, 3 bytes — exercises the
255 → 256 → 65 536 transitions)
- Numeric edges: 0, ±1, MIN, MAX, ±Inf, NaN, -0.0, multi-element vectors
- Empty and single-element variants
- Nested lists
- A "format byte layout per format code" regression tripwire that
pins every code → byte mapping
19 test cases, 196 assertions. Every fixture round-trips
byte-identical against the codec.
Why this is the strongest single codec test: every other validator
(secsgem-py interop, conformance harness, in-house unit tests) is
one implementer's interpretation. KAT is the standard's own
arithmetic. If our encoder matches these canonical bytes and our
decoder reverses them to the same Item, our SECS-II layer is wire-
compatible with anything else that obeys E5 §9.
NaN / signed-zero / Inf use a bit-pattern compare (IEEE NaN != NaN
breaks the default Item == path) — decode the canonical, re-encode
the decoded, assert byte-identical.
The 3-byte-length fixture (ASCII 65 536 × 'X') generates a ~200 KB
expected-bytes string in the test — slow to write but trivial to
check and forces the 3-byte length-prefix path that 99 % of real
traffic doesn't exercise.
Plan: VERIFICATION.md §1.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a -DSECSGEM_TSAN=ON CMake option that builds every target with
-fsanitize=thread + debug symbols + -O1 + frame pointers. Wires a
dedicated thread-sanitizer job into .gitea/workflows/ci.yml that
builds and runs the full test suite under TSan with
TSAN_OPTIONS=halt_on_error=1 (any flagged race fails the job, not
just warns).
Result against the full 426-case / 2557-assertion suite: 0 warnings,
all green. That converts the existing test_thread_safety.cpp (which
exercised the asio::post-onto-strand pattern) and test_concurrency
(in-flight transaction interleaving) and test_robustness_fuzz (28
random action types × thousands of ticks) from "pattern smoke-tests"
into actual race detection.
The first TSan run caught a real bug in test_robustness_fuzz's
act_exception_complete: it held a pointer to an ExceptionStore
entry across fire_internal(RecoveryComplete), which deletes the
entry. The subsequent state() read was a use-after-free. TSan
flagged it 8 times (4 reads × 2 stack-frame variants). Fix is
scoped lookup + re-check via has() after the mutation; matches the
contract any reasonable caller would follow.
The asio std_fenced_block atomic_thread_fence path generates TSan
"not supported" warnings during compile — those are asio's, not
ours, and don't affect runtime detection.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Property-based robustness test that drives long sequences of random
tool operations against EquipmentDataModel and verifies invariants +
persistence round-trip after every action. Replaces hand-written
state-pinning tests with a generative approach that explores
combinations no human author would think to write.
Action menu (28 weighted actions covering the full standard surface):
- PJ create / event / dequeue (E40)
- CJ create / event / delete (E94)
- Carrier create / id / slot (E87)
- Substrate create / location / proc (E90)
- Alarm set / clear / enable toggle (E5 §13)
- SVID updates (E30 §6.13)
- Define-report / link-event / enable (E30 §6.6)
- Exception post / recover / complete (E5 §9, S5F9-F18)
- Module event (E157)
- EPT event (E116)
- Spool enqueue / drain / force-toggle (E30 §6.22)
Every action is "adjusted": it picks a verb at random, then checks
state-machine legality before applying. A Pause is only fired on a
Processing PJ; a Recover only on a Posted exception; pj_dequeue
skips PJs bound to active CJs (mirrors E94's "can't dequeue
CJ-bound PJ" rule the fuzz itself discovered when the first run
flagged a CJ→missing-PJ reference).
Invariants checked every 64 ticks:
- Every tracked PJ exists in the store (size matches)
- Every CJ's prjobids all exist in PJ store
- No FSM in NoState sentinel
- EPT bucket total monotonically non-decreasing
- Defined reports' VIDs all exist
- Substrate / carrier counts match enumeration
Persistence round-trip every 500 ticks:
- Fresh shadow EquipmentDataModel loads from the same journal dir
- Diffs PJ + CJ states one-by-one + carrier/substrate/exception
counts against the live model
- Catches any "mutation didn't reach disk" or
"replay didn't reconstruct state correctly" bugs
Reproducibility:
- Each TEST_CASE uses a fixed seed (0x1, 0xdeadbeef, 0xfeedface,
0xc0ffee — 8000 ops total in the fast suite)
- World keeps a rolling 20-action trace, printed on invariant
violation so the failing sequence can be pasted into a targeted
regression test
- SECSGEM_ROBUSTNESS_SOAK=1 enables a 100k-tick soak case
(~3-5 minutes in Docker; not run by default)
The very first run found a real edge case: act_pj_dequeue removed
PJs that were bound to active CJs, leaving dangling refs. Fixed
the fuzz to filter; the underlying behavior is intentional (store
trusts the application to gate), but the fuzz now mirrors the
correct E94 contract.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The codebase has supported HSMS-GS since the original landing
(test_hsms_gs.cpp covers the wire-level Select.req-per-session
walk-list, the per-session Reject(EntityNotSelected) behaviour,
and session-routed data dispatch). But the documentation said
exactly one line about it ("Connection::add_session(device_id)
registers extra sessions on one TCP socket") and there was no
end-to-end test using the Server/Client API customers actually
build against.
INTEGRATION.md §7 is a new section showing the realistic pattern:
- Server-side: register the primary session via Server::Config,
then `add_session` for the second MES in the on_connection
callback. Per-session message handler + selected handler so
each MES gets its own router (or its own per-session data view
over a shared EquipmentDataModel).
- Active-mode: same `add_session` on the host-side Connection
for multi-tool fleet controllers.
- Equipment-initiated push: pick the session_id when sending
unsolicited primaries (S5F1, S6F11, S10F1).
- Pointer to the wire tests + the new integration test for
customers who want to see the failure modes.
tests/test_hsms_gs_integration.cpp drives two MES sessions
(device_id 1 + 2) through the Server/Client API end to end:
- Both sessions complete Select.req independently
- S1F1 sent on each session returns a distinct MDLN
("EQUIP-SESS-1" vs "EQUIP-SESS-2"), proving per-session
dispatch routes correctly
- Per-session router fires exactly once per session, no
cross-talk
Pre-existing §§8-10 in INTEGRATION.md got bumped to §§9-11 to
make room.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ProcessJobStore and SubstrateStore already implemented the
loader-accepts-any-version-in-[1, kVersion] pattern. The other five
stores (ControlJobStore, CarrierStore, LoadPortStore, ExceptionStore,
SpoolStore) used strict `header[1] != kVersion` rejection, meaning
a future kVersion bump there would silently nuke every persisted
record on first replay. That's a footgun the test_persistence_upgrade
test already flagged as a tripwire.
This commit flips the strict checks to `< 1 || > kVersion`, mirroring
PJ + Substrate. No format change (kVersion stays at 1 across the
five stores), but:
- Future v2 of any store now Just Works: add fields at the end of
write_record_, bump kVersion to 2, gate the new reads behind
`if (version >= 2)`. Old v1 records on disk continue to replay
with the new fields defaulted.
- Future versions beyond kVersion still get rejected (downgrade
protection — older code can't try to decode trailers it doesn't
understand).
Comment blocks on each kVersion declaration now describe the upgrade
discipline so the next contributor doesn't reinvent it.
Test additions:
- Positive test that v1 ControlJob records load on current code
(will continue to pass when kVersion bumps to 2, proving v1 is
still readable)
- ExceptionStore rejects a v9 (future) record, matching CJ + Carrier
- The existing tripwire tests get retitled from "rejects unknown
version" to "rejects a future version" to reflect the new contract
README §6 gets honest: every store is now multi-version-aware, not
just PJ + Substrate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
README §3 promised a monitoring story ("aggregate into Prometheus via
a sidecar that polls the data model"). Nothing shipped. Customers
running a real fab without a metrics pipeline find out about T7
storms, spool blowups, and stalled CJs after their MES does — not
the position you want SRE in.
This commit ships:
- include/secsgem/metrics/prometheus.hpp: header-only. A Registry
(counters + gauges + HELP/TYPE descriptions, label-keyed,
mutex-guarded so updates from the io thread and scrape renders from
the same io serialize cleanly) plus a PrometheusServer (asio
acceptor, replies to any GET with the text-exposition rendering,
no auth — drop nginx in front for that).
- tests/test_metrics_prometheus.cpp: 3 cases / 19 assertions.
Render counter+gauge with labels, scrape via raw TCP and parse the
HTTP body, verify live updates land on subsequent scrapes.
- INTEGRATION.md §6.4: worked example that pairs the exporter with the
Connection + EquipmentDataModel hooks documented in §6.1/§6.2.
Shows the wrap-around-handler trick for message counters, a 5s
polling timer for gauges (spool depth, active alarms), and the
expected /metrics output.
Deliberately *not* shipped:
- A StandardMetrics helper that auto-wires everything — would force
a single hook owner per store, breaking customers who want
composable observers. Customers wire what they need; the registry
gives them counters + gauges + an HTTP endpoint, no policy.
- TLS / auth on the HTTP endpoint. Reverse-proxy territory.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The existing loader throws ConfigError on the first problem it hits.
A customer with a tool-specific equipment.yaml that has six issues
sees one, fixes, restarts, sees the next, fixes, restarts — six
edit-restart cycles before the server even binds. Day-1 friction
is the top support ticket source in fab integrations.
This commit adds a parallel validator that does a separate read-only
pass and surfaces *every* issue at once:
$ secs_server --validate-config \
--config equipment.yaml \
--state-table control_state.yaml
[error] equipment.yaml:5 svids[0].type — unknown SECS-II type `WTF`
[error] equipment.yaml:7 alarms[0].category — value 200 out of range [0, 127]
[error] equipment.yaml:9 host_commands[0].emit_ceid — CEID 999 not declared in `ceids` section
3 error(s), 0 warning(s) across 4 files
What it catches:
- Missing required fields (device.model_name, .software_rev, …)
- Range violations (alarm category must be 0–127, spool streams 1–127,
device.id fits u16, etc.)
- Unknown enum values (SECS-II types, HCACK values, control/PJ/CJ
state and event names — using the right case + snake convention
the runtime parsers enforce)
- Duplicate IDs within svids / dvids / ecids / ceids / alarms,
duplicate PPIDs in recipes, duplicate command names in host_commands
- Referential integrity: host_commands[*].emit_ceid must exist in
ceids; host_commands[*].set_alarm must exist in alarms;
emit_on_control_change must exist in ceids
- PJ-table-specific: `NoState` sentinel rejected as `initial`,
`from`, or `to` (matches loader's existing runtime check)
- yaml-cpp Mark → 1-based line numbers when available
What it doesn't catch (out of scope this round):
- JSON Schema for editor red-squigglies (future)
- Deep semantic checks across state-table reachability
- ECID min/max value parsing (would need numeric type coupling)
Tests cover: clean file passes; multi-error YAML surfaces every issue
on a single pass; line numbers populate; control_state /
process_job_state / control_job_state casing conventions;
format_issues_to renders both severities; the shipped
data/equipment.yaml etc. validate cleanly (regression tripwire if
anyone breaks the demo configs).
INTEGRATION.md §2.3 calls out the flag and suggests CI use.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
README §6 claimed bidirectional forward-compat for journal records.
Reality is narrower:
- ProcessJobStore (kVersion=2) and SubstrateStore (kVersion=2) accept
v1 records on replay — their loaders explicitly switch on the version
byte and treat the v2 trailer fields as empty when absent. This is
the actual upgrade path the README half-described.
- ControlJobStore, CarrierStore, LoadPortStore, ExceptionStore, and
SpoolStore use strict `header[1] != kVersion` rejection. A future
kVersion bump there without a matching loader-side dispatch would
silently nuke every replayed record. The README sold this as a
feature; it isn't yet.
This commit adds:
- tests/test_persistence_upgrade.cpp: five cases that craft journal
records byte-by-byte so format drift is caught (no codec round-trip
hiding the field layout). PJ v1 -> v2 read; PJ v1 rewrite stamps
current kVersion=2; PJ unknown future version rejected; Substrate
v1 read with empty history trailer; CJ + Carrier reject unknown
versions (tripwire for the strict-version stores).
- README §6: replaces the rosy "newer versions ignore unknown
trailers" claim with what's actually implemented — multi-version
reads on PJ + Substrate, strict equality elsewhere — and points
at the test as the contract anchor.
When the strict-version stores grow their own v2, the rejection
tests will need to flip to acceptance; the layout is right there in
the test so the edit is mechanical.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
INTEGRATION.md §3 used to show a sensor-poll thread calling
model->svids.set_value() directly while the io_context thread reads
the same SVID for an inbound S1F3. That's a data race — there are
zero locks anywhere in EquipmentDataModel and there's no intention
to add them. The library is single-threaded by design; the doc was
just inviting trouble.
This commit makes the actual contract explicit:
- INTEGRATION.md §3: thread-safety callout box. All access must run
on the io_context that drives the HSMS connection. Sensor updates
from other threads marshal via asio::post(io.get_executor(), ...).
Same applies to set_*_change_handler callbacks (they fire on the
io_context thread; observers must be thread-safe or hand work off).
- README.md §3 (Monitoring & observability): added a paragraph noting
that hooks fire on the io_context thread, blocking I/O inside a
handler stalls the dispatcher, and metrics exporters must respect
the same contract.
- tests/test_thread_safety.cpp: two scenarios that exercise the
canonical pattern — N producer threads asio::post sensor updates
onto a worker-driven io_context; reads marshal back through the
io. Catches obvious regressions (e.g. someone adding a
"convenience" cross-thread mutator that bypasses the strand).
A passing run isn't proof of race-freedom under ThreadSanitizer —
it pins down the *pattern* customers should follow. TSan integration
is a separate workstream.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The E84StateMachine timers landed last commit but stayed theoretical —
arming was delivered via abstract callbacks the application had to
glue to a real clock. This commit ships the canonical glue:
- include/secsgem/gem/e84_asio_timers.hpp: header-only
E84AsioTimers wraps three asio::steady_timers, wires set_timer_handlers
on attach(), routes async_wait expiry back into fsm.on_timeout().
detach() cancels everything cleanly.
- tests/test_e84_asio_timers.cpp: four scenarios exercised through a
real asio::io_context with wall-clock timers — TA1 expiry,
signal-driven cancel before TA1 fires, TA3 expiry from the
Transferring state, and detach() halting further transitions.
These cover the integration the synthetic unit tests in
test_e84_timers.cpp can't reach.
- INTEGRATION.md §4.6: the vendor-side recipe — create the port,
set timeouts, make_shared<E84AsioTimers>(...)::attach(), feed signals
from your I/O bridge.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
E84StateMachine had the full signal-level handshake but no timer
enforcement. In a real AMHS that's a deadlock: if equipment is slow to
assert L_REQ / U_REQ, or AMHS is slow to assert BUSY / COMPT, neither
side notices — the wires just sit stuck. SEMI E84 §6 mandates three
timers that bound each leg of the dance.
TA1 — armed in ValidAsserted, cancelled in Load/UnloadReady.
AMHS bounds how long equipment takes to acknowledge VALID.
TA2 — armed in Load/UnloadReady, cancelled in Transferring.
Equipment bounds how long AMHS takes to start the transfer.
TA3 — armed in Transferring, cancelled on Complete.
Equipment bounds the BUSY-phase duration.
The FSM stays I/O-free (it's the design invariant): arm/cancel are
delivered via callbacks, the application owns the asio::steady_timer,
and the application calls `fsm.on_timeout(id)` when its real clock
fires. Stale on_timeout calls (post-cancel race) are no-ops.
On expiry, the FSM transitions to a new `HandoffFault` state, records
the `E84Fault` reason, fires the optional fault_handler, and latches
the fault until `reset()`. Signal jitter on the wires cannot silently
clear a recorded handshake timeout — once you've crossed the timer,
you stop.
Defaults are all-zero, which disables arming. This is what every
existing test relies on, and what back-to-back simulation (no
wall-clock) needs. Production tools call `set_timeouts({2s, 2s, 60s})`
or whatever their port spec dictates.
12 new test cases / 59 assertions: arming per state, cancelling per
exit, expiry-to-fault for all three timers, ES cancels everything,
stale-expiry no-op, fault latching across signal jitter, and a
full-cycle arm/cancel trace.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
E42 was an explicit out-of-scope item in the prior COMPLIANCE.md.
This commit closes it.
Wire messages added via the catalog:
S7F23 Formatted PP Send (H↔E, W=1)
S7F24 Formatted PP Ack (ProcessProgramAck)
S7F25 Formatted PP Request (PPID, W=1)
S7F26 Formatted PP Data (E→H, no reply)
Body shape: <L,4 PPID MDLN SOFTREV <L,n <L,2 CCODE <L,m <L,2
PNAME PVAL>>>>>. PVAL is declared ITEM so any SECS-II Item type
round-trips — proven by a test that mixes ASCII, BOOLEAN, U4, F8,
Binary, and nested List values in one step.
RecipeStore extension:
add_formatted(ppid, FormattedRecipe{mdln, softrev, steps})
get_formatted(ppid) -> optional<FormattedRecipe>
has_formatted(ppid) -> bool
Formatted + opaque views live alongside each other: a PPID can carry
both, size() counts unique PPIDs. remove() kills both views.
Six new tests cover wire round-trip per function, every
ProcessProgramAck code, ITEM passthrough, and the store's dual-view
semantics.
COMPLIANCE.md updated: E30 §6.17 row mentions S7F23-F26, S5 message
table grows two rows, §8 "out of scope" entry for E42 removed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The SECS-I Protocol FSM now enforces T3 (reply timeout) and T4
(inter-block timeout) directly, instead of leaving them as
upper-layer hooks.
T3: on complete_send, if the block we just acked had W=1, record its
system_bytes in awaiting_reply_sys_ and emit ActionStartTimer{T3}.
deliver_recv cancels T3 when a block arrives whose system_bytes
match the outstanding request. EventTimeout{T3} aborts the FSM with
"T3 reply timeout".
T4: deliver_recv emits ActionStartTimer{T4} whenever the delivered
block has end_block=false. The next block's deliver_recv cancels
the timer; EventTimeout{T4} aborts with "T4 inter-block timeout".
abort() now also cancels T3/T4 and clears the tracking state.
Test changes:
- Old "T3/T4 are FSM-level no-ops" test → REPLACED by four new
tests: T3 arm+expire, T3 arm+matching-reply cancels, T4
arm+expire, T4 arm+next-block cancels.
- Two new observer accessors on Protocol (awaiting_reply,
awaiting_next_block) so the tests can assert tracking state
without poking internals.
COMPLIANCE.md §1a: T3 + T4 rows go ⬜ → ✅.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
E84 (Parallel I/O) is fundamentally per-load-port: each port has its
own ten-wire handshake with the AMHS. Earlier revisions modeled it
as a single equipment-wide FSM; this commit refactors to a per-port
store, so multi-LP tools can run independent handshakes in parallel.
Public API change in EquipmentDataModel:
E84StateMachine e84; -> removed
E84PortStore e84_ports; // create(port_id), get(port_id), ...
Convenience pass-throughs: E84PortStore::on_signal_change auto-creates
the port on first use (ergonomic for demos); applications should call
create() explicitly with their full port set.
The two existing callsites (test_gem300_scenario, test_e87_wire_scenarios)
are updated. The multi-LP test now demonstrates the actual win:
interleaved LP1 load + LP2 unload handshakes that reach their
respective Ready states without sequencing, and an ES on LP1 that
does NOT affect LP2 — exactly the failure mode the previous design
couldn't catch.
Five new dedicated tests in test_e84_ports.cpp for the store itself.
COMPLIANCE.md §4i updated: row now reflects per-port design.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the last terminal-services message: a multi-line broadcast push
to all terminals, no reply. Same TID+lines body as S10F5, W=0.
Generated via the catalog: data/messages.yaml schema entry +
auto-generated s10f7_terminal_display_broadcast / parse_s10f7.
Test round-trips TID and a 3-line broadcast through the builder
and parser, confirms W=0.
COMPLIANCE.md updated: S10F7 row in §5 added; §8 "out of scope"
entry removed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Connection now supports both HSMS-SS (single session — the
constructor's behaviour, unchanged) and HSMS-GS (multi-session).
add_session(device_id) registers additional sessions; each one has
its own NotSelected/Selected state and its own message/selected
handlers. In GS mode the Select.req carries session_id=device_id;
in SS mode it stays at 0xFFFF (legacy). Linktest/Separate remain
connection-scope per spec.
Public API additions:
add_session(device_id)
set_session_message_handler(device_id, h)
set_session_selected_handler(device_id, h)
session_state(device_id) -> State
is_session_selected(device_id) -> bool
send_request(device_id, msg, cb)
send_data(device_id, msg)
Internal refactor: state_/on_message_/on_selected_ folded into a
SessionSlot map keyed by device_id; SS-style getters/setters route
through the primary session. T7 + linktest are connection-scope —
T7 fires only when no session is selected; linktest runs while at
least one is.
Five wire-level tests:
- passive: two sessions selected independently via Select.req
with their own session_id
- GS Select.req for an unregistered session id is Rejected
(EntityNotSelected)
- data routed by session_id; data on a not-selected session is
Rejected
- active: two registered sessions both end up selected via
serialized Select.req per session
- SS legacy: existing single-session API still works (session_id
0xFFFF in Select.req)
COMPLIANCE.md §1 updated: HSMS-GS row goes ⬜ → ✅.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-substrate transition history now survives restart. Each entry's
steady_clock timestamp is written as a system_clock-millis snapshot;
on replay the steady_clock time_point is reconstructed relative to
the current (steady_now, system_now) pair, so inter-event spacing
is preserved across restarts even if the FSM is in a different
process. Absolute wall-clock accuracy degrades by any NTP step
that happened between write and read; that's a documented caveat.
Record format goes v1 → v2. v1 (history-less) records still load,
just with empty history.
Test updates:
- the old "history is NOT journaled" test is REPLACED with one
that asserts every axis + event + label round-trips.
- hand-crafted v1 record on disk still loads (proves backwards
compat).
- 15 ms-spaced events restore with their spacing intact (±slop
for scheduler jitter).
Closes the "substrate history persistence" caveat from the post-#1-13
status writeup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the v1 caveat: the optional E40-0705 trailers on S16F11 —
recipe variables (RcpVar) and process parameters (ProcessParam),
each carrying a secs2::Item value of arbitrary type — now survive
restart.
Record format bumps to v2:
v2 header = v1 header
+ [u16 rcpvar_count][repeat: u16 name_len, name, u32 enc_len,
secs2::encode(value)]
+ [u16 ppparam_count][...same shape]
v1 records are still accepted by load_record_ (no extras come back).
Two new tests:
- round-trip mixed F4 / ASCII / U4 / nested-list values through
rcpvars + prprocessparams
- hand-crafted v1 record on disk still loads cleanly, just with
empty extras (proves backwards compat)
Closes the "PJ rcpvars / prprocessparams persistence" caveat from
the post-#1-13 status writeup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Real GEM sessions don't serialize requests — the host can have many
primaries outstanding, replies may arrive in any order, and both
peers can talk at once. Connection demuxes via system_bytes per
E37 §8.3; this commit pins the behaviour with four wire tests:
- 5 in-flight requests; equipment buffers all primaries before
replying — proves Connection holds the pending map correctly
even when no replies are coming.
- 7 pipelined primaries with synchronous in-handler replies;
every host callback fires with the correct function and stream.
- Bidirectional in-flight: host issues 3 primaries while equipment
issues 3 of its own; all 6 callbacks resolve with the right
replies.
- 100-burst sequential cycle; confirms the pending_requests_ map
doesn't leak entries (every reply delivered ⇒ map drained).
Closes#13 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SEMI E5 allows identifier fields (DATAID, RPTID, VID, CEID, ALID,
EXID, OBJID, …) to be encoded as U1, U2, U4, or U8. Our parsers
route through any_unsigned_first<T> in messages_helpers.hpp. The
existing per-message round-trip tests prove the U4 path; this
commit adds the cross-width matrix that the interop incident with
secsgem-py demanded:
- as_u4_scalar accepts U1/U2/U4/U8 inputs for the same value
- as_u8_scalar accepts every narrower width
- as_u1_scalar accepts wider widths when the value fits
- as_u1_scalar / as_u2_scalar REJECT out-of-range values rather
than silently truncating
- codec round-trip preserves the format byte AND the value
- signed counterparts (as_i4_scalar) follow the same rule for I1/I2
If a future code-gen change hard-codes a single width on any
identifier field, the rejection case here breaks loudly.
Closes#12 in the test-gap backlog (renumbered: this is gap entry
"identifier wildcard matrix").
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Four new test cases:
* S3F19 verify with matching map → SlotMapVerifyAck::Accept and
CSMS lands in Read on the equipment side.
* S3F19 verify with disagreeing map → Mismatch ack and CSMS lands
in Mismatched.
* 4 LPs + 4 carriers, host verifies CAR-1 (mismatch) and CAR-3
(match) — only those two carriers move on the CSMS axis;
CAR-2/CAR-4 stay NotRead. Confirms per-carrier independence.
* Multi-LP E84 handshake sequencing (load then unload) round-trips
through Idle. Documents that the current E84StateMachine is
per-equipment, not per-port — a future per-port FSM would
update this test alongside.
Closes#11 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
test_gem300_scenario.cpp drives EquipmentDataModel in-memory. This
companion test does the same lifecycle through actual hsms::Connection
frames on a loopback socket pair:
S1F13/F14 establish comm
S3F17/F18 carrier action ProceedWithCarrier (E87)
S16F11/F12 process job create (E40)
S14F9/F10 control job create (E94)
S16F27/F28 CJSTART → CJ → Executing
S6F11 ControlJobExecuting CEID auto-emitted on transition
CJ → Completed via internal AllJobsComplete
EquipmentEmulator owns the data model + a passive Connection,
registers state-change handlers that synthesize S6F11/S16F9 on
transitions, and dispatches the inbound primaries above. HostEmulator
wraps the active Connection and captures everything the equipment
sends unsolicited.
This is the wire-level equivalent of the existing in-memory scenario,
which closes the gap between "FSM works" and "full GEM 300 stack
works on a wire".
Closes#10 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
FSM unit tests already verified state transitions fire the change
handler — but they don't prove the frame leaves the socket with the
right CEID and linked report payload. This commit wires a passive
equipment Connection to an EquipmentDataModel via a small emitter,
drives transitions, and asserts on what the host peer receives.
Six new tests:
EPT → Productive ⇒ S6F11(kCeidProductive) with the linked report
EPT (no subscription) ⇒ no S6F11 (proves disable gate)
PJ Queued→SettingUp ⇒ S16F9 PRJobAlert with PRJOBID + state byte
PJ alert_enabled=false ⇒ no S16F9 (per-PJ gate works)
CJ → Executing ⇒ S6F11(ControlJobExecuting) on the wire
Substrate StartProcessing ⇒ S6F11(SubstrateInProcess) on the wire
All use the generated parse_s6f11 / parse_s16f9 to decode the
incoming frame and assert against typed fields (CEID, PRJOBID, etc.)
rather than poking variant internals — that ties the test to the
schema-as-data rather than to wire byte offsets.
Closes#9 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Deterministic-seed fuzz coverage of the byte-decoding surface:
- secs2::decode on 2000 random buffers
- secs2::decode on every truncation of a real encoding + 500
one-byte flips of the full encoding
- hsms::Frame::decode on 1000 random payloads
- hsms::Header::decode on 2000 random 10-byte buffers
- secsi::Block::decode on 2000 random buffers
- secs2 encode/decode round-trip identity across a battery of every
Item factory (List, ASCII, Binary, Boolean, U1..U8, I1..I8, F4/F8,
nested List)
- oversize <A 3 length-bytes> length-prefix doesn't allocate GBs
- 64-level nested List round-trip doesn't blow the stack
Contract is binary: no crash, no UB. Each decoder is allowed to throw
or return whatever; we deliberately don't assert *what* result comes
back, only that control returns. Fixed PRNG seeds make any failure
reproducible from the CI log alone.
Closes#8 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
test_secsi.cpp covered T2 on the send side (retry) and a tick-based
back-to-back exchange. This commit fills in the rest of the timer
matrix at FSM level:
T1 in RecvBlock → abort, reason mentions "T1"
T1 outside RecvBlock → ignored
T2 in RecvEotSent → abort
T2 in RecvBlock → abort (mid-block stall)
T3 / T4 → FSM-level no-op (documented as upper-layer driven)
T2 contrast → send-side retries, recv-side aborts (same timer,
different recovery, both demonstrated in one test)
If a future commit moves T3 or T4 enforcement into the FSM, the
no-op test breaks loudly so protocol.hpp can be updated alongside.
Closes#7 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
S9F3/F5 are covered by test_s9_fallback (router path); S9F9/F11 by
test_hsms_timers (timer/over-length). This commit adds S9F7 wire-level
tests for the third path — a primary whose body fails secs2::decode.
Three new cases:
- hand-built primary with truncated <B> body provokes S9F7
carrying the original 10-byte MHEAD (sys + stream + function)
- emission is non-fatal: the next well-formed primary still routes
to the registered handler
- data-while-NOT-SELECTED still echoes Reject(EntityNotSelected)
(sanity copy of the test_hsms_connection case so the "what does
the equipment say when a peer sends garbage" family lives together)
Closes#6 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Real-socket tests for the timer family in E37 §10 — these replace
the "the timer fires somewhere" implicit assumption with
end-to-end observations on a loopback pair:
T3: send_request that gets no reply emits S9F9 with the original
MHEAD echoed in the body and surfaces Timeout to the caller.
T6: active mode whose Select.req goes unanswered self-closes
with a "T6 timeout on Select" reason.
T7: passive mode that never receives Select.req self-closes
with a "T7 not-selected timeout" reason.
T8: peer sends only the 4-byte length prefix; T8 expires mid-read
and closes with "T8 intercharacter timeout".
Plus S9F11 emission for an over-length frame (length prefix of
1 GiB+1) — body's <B 10> echoes the offending bytes verbatim.
Per-test timer profiles (only the timer under test is short, the
rest are 5s) so the FSM isn't racing against unrelated timers.
Closes#5 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-EXID binary record (.ex), magic + version + atomic .tmp+rename.
Records full E5 §9 lifecycle: state, EXID, EXTYPE, EXMESSAGE, and
the candidate EXRECVRA list.
Cleared exceptions are terminal — the FSM transitions through
Cleared remove the in-memory entry AND delete the journal file
(matching the existing in-memory semantics). Recovering /
RecoverFailed states survive restart: the application can decide
on replay whether to retry recovery or abort.
Five new tests cover post+replay, Recovering-survives-restart,
autonomous-clear cleanup, RecoverFailed retry post-restart, and
corrupt-record drop.
This completes #12 in the test-gap backlog (persistence for the four
in-memory stores beyond Spool).
Closes#4 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-job binary record (.pj / .cj) with magic+version, atomic
.tmp+rename. PJ store additionally writes an order.idx index file
that preserves HOQ-aware queue position across restarts.
Rcpvars / prprocessparams (secs2::Item variants) are intentionally
out of scope for v1 — they're optional E40 trailers and need a body
codec round-trip; callers re-populate via set_e40_extras() after
restart.
Five new tests cover full lifecycle replay (Processing mid-run +
HOQ-reordered queue), dequeue-deletes-file, corrupt-record drop,
CJ state + PJ-list replay, and CJ remove cleanup.
Closes#3 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same pattern as carriers: per-substrate binary record (.sub) with
atomic .tmp+rename, replay on enable, delete on remove. Records
current state across all three E90 axes (location / processing /
ID-status), plus substid / carrierid / slot / free-form location
label. History is deliberately NOT journaled — it's an in-memory
ring buffer and rebuilding from replayed state would mislead.
Five new tests cover full-axis replay, every terminal processing
state, remove-deletes-journal, corrupt-record drop, and the
history-is-transient invariant.
Closes#2 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors SpoolStore: per-record file with atomic .tmp+rename, magic+
version-prefixed binary layout, replay on enable, delete on remove.
FSMs gain a restore_state() that bypasses the transition table and
handlers since a replay isn't a transition.
Six new tests cover write+restart+replay across every CIDS/CSMS/CAS
axis, remove-deletes-journal, malformed-record drop-not-poison, and
the persistence-disabled no-op path.
Closes#1 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raphael