Semantics settled and documented: v1 is observe-and-report. The engine keeps
acking S16/S3/S7/S2F15 from its FSM tables — exactly the behaviour both
reference implementations validated — while the tool observes lifecycle
events on the Subscribe stream and reports physical progress back. Gating
stays the documented v2 deferred-reply item.
Engine: two new store observers (HandlerSlot pattern) — RecipeStore fires
(ppid, body) after an add (S7F3 downloads), EquipmentConstantStore fires
(id, value) on ACCEPTED S2F15 writes only. Unit-tested.
Daemon: the service registers PJ/recipe/EC observers (io thread; add_
observers coexist with register_default_handlers' primaries) and fans the
new HostRequest variants out via push_request (fire-and-forget, no-
buffering contract). ProcessJob carries action (Start->START, Resume->
RESUME, Paused->PAUSE, Stopping->STOP, Aborting->ABORT) + recipe + material
bindings read store-side on the io thread. ReportProcessJob maps SETTING_UP
->SetupComplete, COMPLETE->ProcessComplete, ABORTED->AbortComplete via
read_sync; PROCESSING is informational; unknown job => INVALID_OBJECT,
table-rejected transition => CANNOT_DO_NOW. Carriers deferred (CarrierStore
has no observer machinery; ReportCarrier stays UNIMPLEMENTED) — roadmap.
Python client: on_process_job / on_recipe / on_constant_change decorators +
report_job(job_id, state); ProcessJob dataclass exported.
Tests: daemon suite 141 -> 175 assertions — the full in-process loop
(S16F11 create -> tool setup -> S16F5 PJSTART -> stream ProcessJob with
recipe+carriers -> ReportProcessJob(COMPLETE) -> FSM at ProcessComplete),
rejection paths, S7F3 -> ProcessProgram, S2F15 -> ConstantChange with the
configured name. Core 475/3097 (observer units). Live regression: daemon
interop 20 checks + pyclient 13 checks still green against the running
daemon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The HCACK-4 contract, implemented end to end. For every YAML-declared
command the service registers a forwarding handler (new HostCommandRegistry
names()/spec() accessors): with a subscribed tool client the command is
queued onto the Subscribe stream (id + name + params via from_item) and the
host is answered S2F42 HCACK=4 immediately — never blocking the io thread or
the T3 window; with NO subscriber the command takes its declarative YAML ack
(the honest pre-daemon behaviour). Settled + documented in the proto: v1 is
a firehose with no buffering/replay. CompleteCommand correlates the pending
id (audit; unknown id => PARAMETER_INVALID). Side effects stay suppressed on
HCACK-4 (router applies them only on Accept), so the completion event the
TOOL fires is the host's real signal — exactly E30's intent.
Tests (daemon suite 101 -> 124 assertions): a real S2F41 dispatched through
the full default-handler router ON the io thread under run_async — HCACK 4
with subscriber + params on the stream, declarative Accept without,
CompleteCommand known/unknown, fallback restored after unsubscribe.
Interop (now 20 checks, all green): the complete conformant loop against
the secsgem-py reference host — S2F41 START -> S2F42 HCACK=4 -> tool
receives Command(name=START, id=1) -> CompleteCommand -> FireEvent -> host
receives S6F11.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A2 — alarms: optional 'name:' on alarm config (a LOCAL key — SEMI only
defines numeric ALID + freetext ALTX; field appended last so existing
{id, text, category} brace-inits compile unchanged), parsed by the loader,
checked by the validator, shipped in equipment.yaml. SetAlarm/ClearAlarm
RPCs resolve config name OR stringified ALID via a constructor snapshot.
A3 — control state + health: RequestControlState fires operator events on
the io thread (read_sync) and reports what the E30 table actually did —
ACCEPT iff the equipment landed in the requested state, CANNOT_DO_NOW naming
the actual state otherwise (the shipped table has no operator path to
EquipmentOffline; the test pins that honesty). ATTEMPT_ONLINE is rejected as
transient. WatchHealth streams an immediate snapshot then pushes on link/
control-state changes via service observers (add_link_observer +
add_control_state_observer — the HandlerSlot work paying off), spool depth
sampled at the 500ms poll; ends on cancel or engine stop.
Tests: daemon suite 61 -> 101 assertions (alarm lifecycle by name/id/unknown,
WatchHealth initial + change push, all four RequestControlState semantics);
loader test for the alarm name (present + absent fallback); core 467/3055.
Interop now 15 checks incl. gRPC SetAlarm -> host receives S5F1 ALCD=0x84
ALID=1, and RequestControlState(HOST_OFFLINE) -> GetControlState confirms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The single-slot set_*_handler pattern was a structural blocker, hit twice:
the daemon could not observe control-state changes because
register_default_handlers owns the slot, forcing GetControlState to read the
FSM cross-thread (a data race), and blocking WatchHealth and the Subscribe
stream's ControlStateChange variant.
HandlerSlot<Args...> keeps a primary slot with exact legacy semantics
(set_ replaces — one existing test depends on replacement) plus an
append-only observer list (add_) that survives set_ calls. Fire sites are
textually unchanged (operator bool / operator() / assign-from-function).
Applied to ControlStateMachine + ProcessJobStore + ControlJobStore (the
roadmap-critical three; the remaining single-slot classes follow the same
3-line pattern as needed). EquipmentRuntime gains an atomic control-state
mirror registered as an observer — control_state() is now safe from any
thread, retiring the GetControlState race — plus add_control_state_observer
and add_link_observer (selected/closed fan-out), the hooks WatchHealth and
Subscribe need.
Tests: observer ordering, set-replaces-primary-but-observers-survive,
observers-without-primary, PJ-store coexistence, and the runtime scenario
that was previously impossible (mirror + observer + default-handlers set_).
Core 464/464 (2816 assertions), daemon 16/16, live GEM300 demo passes with
single-fire control-state transitions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Host commands were declarative-only: dispatch() returned the YAML-defined
HCACK plus side effects, and ignored the command parameters entirely (the
param list was a commented-out argument). Equipment could acknowledge a
command but never run anything in response — the pvd_tool example worked
around this by hard-coding behaviour in a C++ router handler.
Add set_handler(rcmd, fn): a registered handler receives the live CPNAME/
CPVAL parameters and returns the HCACK, overriding the declarative default.
Live on S2F41/F21/F49 via the shared dispatch(). No handler => byte-for-byte
the previous declarative behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Picks up the file renames that landed alongside the previous commit
and fixes everything that pointed at the old root locations:
- README.md doc-map updated: every entry now points at docs/X.md,
with a new "docs/" lead entry pointing at the guided-tour index.
- README inline cross-refs (ARCHITECTURE / INTEGRATION / SECURITY /
BENCHMARKS / MES_INTEROP / PROOFS) repointed to docs/.
- README "Interop" section rewritten — used to mention only
secsgem-py; now covers all four external validators (secsgem-py
31 / secs4java8 55 / tshark 69 frames / libFuzzer 200 k+ runs)
with a one-line summary each, plus pointers to interop/README.md
and docs/VERIFICATION.md.
- README "Deferred follow-ups" cleaned: dropped the explanatory
"Listed here so reviewers don't go looking for them in
COMPLIANCE.md and find an 'out of scope' entry that sounds
defensive" sentence — the section header speaks for itself.
- docs/00_index.md "Where the rest of the docs live" table: dropped
every `../` prefix since the docs are now siblings.
- docs/01_what_is_secs_gem.md PROOFS reference updated to sibling.
- docs/02_the_cast.md INTEGRATION + MES_INTEROP refs updated to
siblings; dropped the stale "at the repo root" wording.
- interop/README.md: VERIFICATION + PROOFS refs updated to
../docs/X.md; stale "~24 + 4 checks" updated to 31 (matches
PROOFS.md and README).
- examples/pvd_tool/README.md: every doc cross-ref now points at
../../docs/X.md.
- Source / data / CI comments mentioning doc names (e.g.
"INTEGRATION.md §3", "COMPLIANCE.md gap") rewritten to
"docs/INTEGRATION.md §3" etc. — affects 9 files across
include/, apps/, tests/, data/, examples/, .gitea/workflows/.
Verified: full build under docker passes, 445/445 test cases pass,
2 753/2 753 assertions pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ProcessJobStore and SubstrateStore already implemented the
loader-accepts-any-version-in-[1, kVersion] pattern. The other five
stores (ControlJobStore, CarrierStore, LoadPortStore, ExceptionStore,
SpoolStore) used strict `header[1] != kVersion` rejection, meaning
a future kVersion bump there would silently nuke every persisted
record on first replay. That's a footgun the test_persistence_upgrade
test already flagged as a tripwire.
This commit flips the strict checks to `< 1 || > kVersion`, mirroring
PJ + Substrate. No format change (kVersion stays at 1 across the
five stores), but:
- Future v2 of any store now Just Works: add fields at the end of
write_record_, bump kVersion to 2, gate the new reads behind
`if (version >= 2)`. Old v1 records on disk continue to replay
with the new fields defaulted.
- Future versions beyond kVersion still get rejected (downgrade
protection — older code can't try to decode trailers it doesn't
understand).
Comment blocks on each kVersion declaration now describe the upgrade
discipline so the next contributor doesn't reinvent it.
Test additions:
- Positive test that v1 ControlJob records load on current code
(will continue to pass when kVersion bumps to 2, proving v1 is
still readable)
- ExceptionStore rejects a v9 (future) record, matching CJ + Carrier
- The existing tripwire tests get retitled from "rejects unknown
version" to "rejects a future version" to reflect the new contract
README §6 gets honest: every store is now multi-version-aware, not
just PJ + Substrate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
E42 was an explicit out-of-scope item in the prior COMPLIANCE.md.
This commit closes it.
Wire messages added via the catalog:
S7F23 Formatted PP Send (H↔E, W=1)
S7F24 Formatted PP Ack (ProcessProgramAck)
S7F25 Formatted PP Request (PPID, W=1)
S7F26 Formatted PP Data (E→H, no reply)
Body shape: <L,4 PPID MDLN SOFTREV <L,n <L,2 CCODE <L,m <L,2
PNAME PVAL>>>>>. PVAL is declared ITEM so any SECS-II Item type
round-trips — proven by a test that mixes ASCII, BOOLEAN, U4, F8,
Binary, and nested List values in one step.
RecipeStore extension:
add_formatted(ppid, FormattedRecipe{mdln, softrev, steps})
get_formatted(ppid) -> optional<FormattedRecipe>
has_formatted(ppid) -> bool
Formatted + opaque views live alongside each other: a PPID can carry
both, size() counts unique PPIDs. remove() kills both views.
Six new tests cover wire round-trip per function, every
ProcessProgramAck code, ITEM passthrough, and the store's dual-view
semantics.
COMPLIANCE.md updated: E30 §6.17 row mentions S7F23-F26, S5 message
table grows two rows, §8 "out of scope" entry for E42 removed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
E84 (Parallel I/O) is fundamentally per-load-port: each port has its
own ten-wire handshake with the AMHS. Earlier revisions modeled it
as a single equipment-wide FSM; this commit refactors to a per-port
store, so multi-LP tools can run independent handshakes in parallel.
Public API change in EquipmentDataModel:
E84StateMachine e84; -> removed
E84PortStore e84_ports; // create(port_id), get(port_id), ...
Convenience pass-throughs: E84PortStore::on_signal_change auto-creates
the port on first use (ergonomic for demos); applications should call
create() explicitly with their full port set.
The two existing callsites (test_gem300_scenario, test_e87_wire_scenarios)
are updated. The multi-LP test now demonstrates the actual win:
interleaved LP1 load + LP2 unload handshakes that reach their
respective Ready states without sequencing, and an ES on LP1 that
does NOT affect LP2 — exactly the failure mode the previous design
couldn't catch.
Five new dedicated tests in test_e84_ports.cpp for the store itself.
COMPLIANCE.md §4i updated: row now reflects per-port design.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-substrate transition history now survives restart. Each entry's
steady_clock timestamp is written as a system_clock-millis snapshot;
on replay the steady_clock time_point is reconstructed relative to
the current (steady_now, system_now) pair, so inter-event spacing
is preserved across restarts even if the FSM is in a different
process. Absolute wall-clock accuracy degrades by any NTP step
that happened between write and read; that's a documented caveat.
Record format goes v1 → v2. v1 (history-less) records still load,
just with empty history.
Test updates:
- the old "history is NOT journaled" test is REPLACED with one
that asserts every axis + event + label round-trips.
- hand-crafted v1 record on disk still loads (proves backwards
compat).
- 15 ms-spaced events restore with their spacing intact (±slop
for scheduler jitter).
Closes the "substrate history persistence" caveat from the post-#1-13
status writeup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the v1 caveat: the optional E40-0705 trailers on S16F11 —
recipe variables (RcpVar) and process parameters (ProcessParam),
each carrying a secs2::Item value of arbitrary type — now survive
restart.
Record format bumps to v2:
v2 header = v1 header
+ [u16 rcpvar_count][repeat: u16 name_len, name, u32 enc_len,
secs2::encode(value)]
+ [u16 ppparam_count][...same shape]
v1 records are still accepted by load_record_ (no extras come back).
Two new tests:
- round-trip mixed F4 / ASCII / U4 / nested-list values through
rcpvars + prprocessparams
- hand-crafted v1 record on disk still loads cleanly, just with
empty extras (proves backwards compat)
Closes the "PJ rcpvars / prprocessparams persistence" caveat from
the post-#1-13 status writeup.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-EXID binary record (.ex), magic + version + atomic .tmp+rename.
Records full E5 §9 lifecycle: state, EXID, EXTYPE, EXMESSAGE, and
the candidate EXRECVRA list.
Cleared exceptions are terminal — the FSM transitions through
Cleared remove the in-memory entry AND delete the journal file
(matching the existing in-memory semantics). Recovering /
RecoverFailed states survive restart: the application can decide
on replay whether to retry recovery or abort.
Five new tests cover post+replay, Recovering-survives-restart,
autonomous-clear cleanup, RecoverFailed retry post-restart, and
corrupt-record drop.
This completes #12 in the test-gap backlog (persistence for the four
in-memory stores beyond Spool).
Closes#4 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-job binary record (.pj / .cj) with magic+version, atomic
.tmp+rename. PJ store additionally writes an order.idx index file
that preserves HOQ-aware queue position across restarts.
Rcpvars / prprocessparams (secs2::Item variants) are intentionally
out of scope for v1 — they're optional E40 trailers and need a body
codec round-trip; callers re-populate via set_e40_extras() after
restart.
Five new tests cover full lifecycle replay (Processing mid-run +
HOQ-reordered queue), dequeue-deletes-file, corrupt-record drop,
CJ state + PJ-list replay, and CJ remove cleanup.
Closes#3 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same pattern as carriers: per-substrate binary record (.sub) with
atomic .tmp+rename, replay on enable, delete on remove. Records
current state across all three E90 axes (location / processing /
ID-status), plus substid / carrierid / slot / free-form location
label. History is deliberately NOT journaled — it's an in-memory
ring buffer and rebuilding from replayed state would mislead.
Five new tests cover full-axis replay, every terminal processing
state, remove-deletes-journal, corrupt-record drop, and the
history-is-transient invariant.
Closes#2 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors SpoolStore: per-record file with atomic .tmp+rename, magic+
version-prefixed binary layout, replay on enable, delete on remove.
FSMs gain a restore_state() that bypasses the transition table and
handlers since a replay isn't a transition.
Six new tests cover write+restart+replay across every CIDS/CSMS/CAS
axis, remove-deletes-journal, malformed-record drop-not-poison, and
the persistence-disabled no-op path.
Closes#1 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds opt-in disk persistence to SpoolStore. `enable_persistence(dir)`
turns every enqueue into a single `<seq>.spool` file alongside the
in-memory queue; drain and clear delete the matching files; restart
replays the directory sorted by seq.
Writes are atomic: serialize the message via the SECS-II codec, write
to `.tmp`, then `std::filesystem::rename` to the final name. Malformed
records are dropped silently so a single bad file can't poison the
whole spool.
`secs_server --spool-dir <path>` enables persistence at startup.
Without the flag the behaviour is identical to before (in-memory only).
Two new tests: enqueue → restart → replay → drain restores the wire
order, and clear deletes the journal files.
Test suite: 291 cases / 1515 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the simplified <L,3 PRJOBID PPID MTRLOUTSPEC> demo body with
the full SEMI E40-0705 §10.2 shape:
<L,5 PRJOBID MF PRRECIPEMETHOD
<L,2 PPID <L,n <L,2 RCPPARNM RCPPARVAL>>>
<L,n MTRLOUTSPEC>
<L,n <L,2 PARAMNAME PARAMVAL>>>
ProcessJob now carries the extra fields (MaterialFlag, ProcessRecipeMethod,
RcpVar[], ProcessParam[]) so a tool's recipe engine can later consume
the recipe-variable overrides and per-job process parameters. Server
S16F11 dispatch populates them via the new ProcessJobStore::set_e40_extras
helper after a successful create.
MaterialFlag + ProcessRecipeMethod enums live in their own tiny header
(`e40_constants.hpp`) so process_jobs.hpp (the store) can use them
without dragging in messages_helpers.hpp (which would create a circular
include via data_model.hpp).
The simplified 3-arg HostHandler::send_create_process_job convenience
remains; it constructs a sensible-default PRJobCreateRequest internally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a Docker-based interop harness that drives the C++ server with
secsgem-py 0.3.0 as the active host and probes a secsgem-py-passive
equipment from a minimal C++ active client. Surfaces and fixes four
interoperability bugs uncovered by cross-testing:
* SEMI E5 identifier formatcodes are a U1|U2|U4|U8 wildcard;
secsgem-py picks the narrowest fitting width while our parsers
only accepted U4. `as_uN_scalar` / `as_iN_scalar` now accept
any unsigned/signed width and range-check the downcast.
* PPBODY (S7F3/F6) is "ASCII | Binary | List" per the spec;
secsgem-py defaults to ASCII. Added BINARY_OR_ASCII codegen
item type with `as_text_or_binary` accessor.
* S1F23/F24 Collection Event Namelist was unimplemented; added
schema + `vids_for(ceid)` accessor on EventReportSubscriptions
plus the dispatch handler.
* S10F1 was registered as a host->equipment handler, but per
SEMI E5 §12 S10F1 is equipment->host; S10F3 is the actual
host->equipment Terminal Display Single. Added an S10F3
handler alongside (we keep S10F1 too for backward compat).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Extends the existing Clock with the metrics a host needs to gate
time-sensitive data against the equipment's sync state (E148 §6.3):
offset_seconds() current applied offset vs system clock
last_drift_seconds() signed drift observed at the most recent sync
sync_count() how many successful syncs have happened
sync_quality() Synchronized (|drift|<=1s) /
Drifting (<=60s) / Unsynchronized (>60s or
never synced)
The thresholds are tuneable per call; the defaults match typical fab
practice but the application can pass tighter bounds for tracelog-
sensitive flows. set_time_string() now snapshots the apparent delta
between the previously-applied offset and the new one as
last_drift_seconds_ at the moment of resync; no background timer.
Three new test cases cover the initial Unsynchronized state, a large
forward drift registering as Unsynchronized, and a same-value resync
landing as Synchronized.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each Substrate now retains an append-only history of state transitions
(both location and processing axes), the triggering event captured as
a std::variant<SubstrateEvent, SubstrateProcessingEvent>, the location
label at the time, and a steady_clock timestamp.
E90 §6.6 requires the equipment to be able to report a wafer's
processing history — typically queried via S6F11 batched reports or
SVID reads. This commit lays the runtime substrate; wire query
plumbing is the natural follow-up.
set_history_limit(n) caps per-substrate retention (default 256, 0 =
unbounded). Oldest entries are dropped when the cap is reached;
vector-erase is fine at this scale (typical wafer lifecycle is a few
dozen transitions).
Two new test cases cover the recording invariants (every fire results
in one history entry on the right axis) and history_limit eviction.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-module process-tracking state machine. An E157 instance models a
single recipe step at a single module, with the canonical lifecycle:
NotExecuting -> GeneralExecuting (StartGeneral)
-> StepExecuting (StartStep)
-> StepCompleted (CompleteStep)
Plus universal escape hatches: Reset returns any state to
NotExecuting; Abort terminates from any state to StepCompleted.
ModuleStore wraps the FSM with the now-standard pattern:
- non-movable (this-capture lambdas)
- per-module bind() carries current_substid + recipe_step
- fire(module_id, event) delegates to the FSM
- set_state_change_handler observes every transition with module_id
Joins EquipmentDataModel. 5 test cases cover happy path, Reset from
each interior state, Abort, store-level create dedup + bind, and the
multi-module change handler keying.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Hierarchical object tree for equipment self-description. Each object
carries a CemObjectType (Equipment / Subsystem / IODevice / Module /
MaterialLocation / Other), an optional parent_objid, and a flat
attribute map keyed by name (the wire shape S14F1 / F3 returns).
Operations covered:
add(CemObject) - dedup'd, validates parent exists
get / has - lookup by objid
get_attr / set_attr - E14 GetAttr / SetAttr semantics
children(parent) - tree traversal; empty parent = roots
The flat-map representation matches how E14 ObjectService traffic
addresses nodes (by OBJSPEC string). Wiring S14F1/F2 GetAttr and
S14F3/F4 SetAttr to this store is a downstream commit; the data model
is what was missing.
Joins EquipmentDataModel alongside the other top-level stores. Three
test cases cover hierarchical add+dedup, children() traversal, and
get/set/missing attribute semantics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ALCD's lower 7 bits are a bitmap of category flags per E5 §10.3 and
E30 §6.13; a single alarm may carry multiple categories at once
(e.g. an irrecoverable equipment-safety fault is 0x10 | 0x02).
Adds:
enum class AlarmSeverity : uint8_t
PersonalSafety EquipmentSafety ParameterError ParameterWarning
Irrecoverable EquipmentStatus Attention
has_severity(alcd, bit), severity_bits(alcd)
Alarm::has(bit), Alarm::is_safety()
constexpr severity_mask = 0x7F
Tests cover single-category alarms, multi-category combos, and that
the bit-7 SET/CLEAR flag is correctly excluded from category bits.
Closes Tranche C (E5 alarm/exception state model).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-EXID exception lifecycle for E5 §9. States mirror the wire flow:
Posted equipment sent S5F9, awaiting host or autonomous clear
Recovering host's S5F13 accepted; equipment running recovery
RecoverFailed S5F15 reported a failed result; host may retry
Cleared terminal — store removes the row
Events:
Created synthetic NoState->Posted observer signal
Recover host's S5F13 (Posted/RecoverFailed -> Recovering)
RecoveryComplete equipment internal (Recovering -> Cleared)
RecoveryFailed equipment internal (Recovering -> RecoverFailed)
RecoveryAbort host's S5F17 (Recovering -> Posted)
Clear equipment internal (Posted/RecoverFailed -> Cleared)
ExceptionStore mirrors ProcessJobStore: per-EXID FSMs heap-allocated via
unique_ptr, non-movable to keep `this`-captures safe, synthetic Created
fires after the row lands so observers can decide whether to emit S5F9
out of band. on_recover validates EXRECVRA against the candidates the
post advertised.
The store joins EquipmentDataModel alongside process_jobs / control_jobs.
S5F9-F18 server-side dispatch lands in C2.
Tests (12 cases) cover FSM transitions including retry, abort, and
autonomous clear, plus store-level duplicate-rejection, EXRECVRA
validation, and Cleared-removes-the-row semantics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
GEM host-side counterpart to the existing equipment server: wraps an
HSMS Connection (Active mode), installs an inbound dispatch table that
auto-acks the messages a host is expected to passively accept, and
exposes the GEM workflow primitives.
Inbound dispatch:
S5F1 Alarm Report observe (alarm handler) + S5F2 Accept
S6F11 Event Report observe (event handler) + S6F12 Accept
S6F25 Spool Data Ready S6F26 Accept (host policy: pull on demand)
S10F1 Terminal Display observe + S10F2 Accepted
S9F* Equipment errors observe (s9 handler); no ack (one-way)
Workflow shortcuts:
establish_communication() S1F13 -> S1F14
go_remote() S1F17 -> S1F18
go_offline() S1F15 -> S1F16
Plus a low-level send_request() escape hatch so the senders coming in
B2/B3 don't have to friend the connection internals.
Drive-by: event_reports.hpp was missing `<optional>` (worked transitively
through the equipment-side include chain but not when included from the
host-side standalone).
secsgem-py has `gem/hosthandler.py`; this mirrors its surface for the
inbound-ack and lifecycle parts. Outbound senders land in B2/B3.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
GEM300 layer: SEMI E40-0705 Process Job and E94-0705 Control Job
state machines, plus the E30 §6.1 communication-state machine that
sits between HSMS SELECT and full GEM communication. Data-driven
via data/process_job_state.yaml and data/control_job_state.yaml,
mirroring the existing control_state.yaml pattern.
Wire coverage:
S14F9/F10 CreateObject (CJ) host -> equipment
S14F11/F12 DeleteObject (CJ) host -> equipment
S16F5/F6 PRJobCommand host -> equipment
S16F9 PRJobAlert equipment -> host
S16F11/F12 PRJobCreate (simplified body) host -> equipment
S16F13/F14 PRJobDequeue host -> equipment
S16F27/F28 CJobCommand host -> equipment
Process Job FSM exposes 8 states matching PRJOBSTATE bytes (E40 §10.3.2);
HOQ is reorder-aware (move-to-head against an insertion-order vector);
Stop/Abort on a Queued PJ routes through ABORTING so the host observes
PRJOBSTATE=7 on the wire (§6.3); alert_enabled is settable per-PJ for
PRALERT control; FSM dispatches through ProcessJobStore::on_change_
dynamically so a late set_state_change_handler() reaches existing PJs.
Hardening: loader rejects NoState (sentinel) as initial/from/to and
rejects `on: created` rows; static_asserts pin enum values to wire
bytes; ProcessJobStore is non-movable to keep the per-PJ this-capture
safe.
Server simulator cascades the full CJ -> PJ lifecycle on CJSTART so
the wire trace exercises every legal state. CEIDs 400/401 fire on CJ
state changes via the existing event-report pipeline.
Tests: 60+ new assertions across test_process_jobs, test_control_jobs,
test_communication_state, test_hsms_connection, plus loader and
messages round-trip coverage.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New TraceStore keyed by TRID; each entry is a TraceConfig with
DSPER + TOTSMP + REPGSZ + SVID list. S2F23 validates that every SVID
exists (TIAACK=4 otherwise) and registers the trace.
S6F1's body is L,4 of {TRID U4, SMPLN U4, STIME ASCII, list_of <Item>}
— the application chooses whether each value Item is a scalar SVID
value or a packed batch.
The periodic sampling timer that turns an active TraceConfig into
S6F1 emissions is intentionally left to the application (E5 doesn't
mandate a specific scheduler and vendors typically already have one).
Four new SxFy in the catalog.
COMPLIANCE.md: Trace Data Collection Additional capability flips ✅.
Tests: 82 cases / 477 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New LimitMonitorStore keyed by VID; each entry is a vector of
LimitDefinition (LIMITID + upper/lower deadband as arbitrary Items).
S2F45/F46 set, S2F47/F48 read. VLAACK validates each VID exists.
Four new SxFy in the catalog; codegen handles the nested
list-of-(VID, list-of-LimitDefinition) shape. LimitDefinition is
defined in store/limits.hpp and referenced as external_struct so the
data model and the message codecs share one type.
The actual "value crossed limit" detection + CEID emission is left to
the application's set_value path (E30 §6.21 leaves *how* the equipment
detects crossings up to the implementer).
COMPLIANCE.md: Limits Monitoring Additional capability flips ✅.
Tests: 80 cases / 465 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements the largest functional gap from the compliance audit. The
equipment now queues events the host can't immediately receive (either
because there's no SELECTED session or because the demo's force-spool
flag is on) and transmits the queue on host request.
What's new
include/secsgem/gem/store/spool.hpp
SpoolStore: a deque queue with a configurable per-stream whitelist
(so only streams 5+6 spool by default), a max_size cap with FIFO
eviction on overflow, and a `force_spool` test flag. Enqueue
returns one of Queued / Dropped_NotSpoolable / Dropped_Full so the
caller can fall back to live delivery when appropriate. Drain
pops the entire queue in FIFO order. Two new ack enums:
ResetSpoolAck (S2F44 RSPACK) and SpoolRequestAck (S6F24 RSDA), plus
SpoolRequestCode (S6F23 RSDC, Transmit/Purge).
data/messages.yaml + auto-regenerated messages.hpp
S2F43 W <L,n <B stream>> Reset Spooling
S2F44 <L,2 <B RSPACK> <L,a ...>> Reset Spooling Ack
S6F23 W <B RSDC> Request Spooled Data
S6F24 <B RSDA> Request Spooled Data Ack
data/equipment.yaml
`spool:` section: max_size + spoolable_streams list. Two new host
commands SPOOL_ON / SPOOL_OFF that flip the force-spool flag (these
stand in for "host link down" in the demo without dropping TCP).
include/secsgem/gem/store/host_commands.hpp
Spec/Result gain an optional<bool> force_spool field. S2F41
dispatch returns the result, the server applies it after S2F42 is
queued.
src/config/loader.cpp
Reads `spool:` from equipment.yaml; reads `force_spool` from each
host_commands entry; populates SpoolStore + CommandSpec.
apps/secs_server.cpp
New `deliver_or_spool(msg, what)` helper. emit_event and
emit_alarm_set funnel through it: if force_spool is on (or there's
no active session), msg.stream is checked against the spoolable
list and the message is enqueued; otherwise it's sent live.
Two new handlers:
S2F43 parses the stream list, updates SpoolStore, replies S2F44
S6F23 RSDC=Transmit drains and re-sends each as a fresh primary
(posted on the executor so the S6F24 ack flushes first);
RSDC=Purge clears the queue and acks.
The S2F41 handler now also propagates result.force_spool into the
SpoolStore.
apps/secs_client.cpp
Demo extended with 4 new steps after the FAULT branch:
SPOOL_ON -> S2F42 Accept
START -> S2F42 Accept; CEID 300 emission spooled (no live S6F11)
SPOOL_OFF -> S2F42 Accept; queue still has the message
S6F23(Transmit) -> S6F24 Accept; spooled S6F11 arrives next
Then the existing S7F19/S7F5/S10F1/S1F15/Separate flow continues.
tests/test_data_model.cpp
Four new TEST_CASEs for SpoolStore (whitelist, FIFO eviction at
max_size, drain ordering, force flag).
tests/test_loader.cpp
Confirms equipment.yaml's `spool:` section populates the store and
`force_spool: true/false` flows through to dispatch results.
COMPLIANCE.md
Spooling moves from ⬜ to 🟡. Adds S2F43/F44 + S6F23/F24 as ✅ in
the message coverage matrix; calls out what's still missing
(S6F25/F26 notification, automatic activation on HSMS NOT-SELECTED,
persistent on-disk spool).
Verified
- Tests: 73 cases / 383 assertions pass (+4 spool cases).
- Demo (docker compose up server client) walks the full happy path
and the spool path, observed in the server log as:
spool: force_spool=true (depth=0)
spool: S6F11 CEID=300 queued (depth=1)
spool: force_spool=false (depth=1)
S6F23 transmit: draining 1 messages
and on the host side as the queued S6F11 arriving in the correct
order after S6F24.
Known limitations (logged in COMPLIANCE.md)
- Spool activation is manual via SPOOL_ON/OFF rather than
automatically triggered by HSMS NOT-SELECTED.
- No S6F25/F26 spooled-data-ready notification on re-SELECT.
- In-memory only; an equipment restart loses queued events.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The god-class is gone. Each capability is now its own focused store:
StatusVariableStore, DataVariableStore, EquipmentConstantStore (with EAC
range validation), EventReportSubscriptions, AlarmRegistry, RecipeStore,
Clock, HostCommandRegistry. Each is independently testable.
EquipmentDataModel becomes a small composite that holds one of each store
as a public member, plus three convenience methods (vid_value, vid_exists,
compose_reports_for) that span SVIDs+DVIDs and inject the right callbacks
into the EventReportSubscriptions.
New under include/secsgem/gem/store/:
status_variables.hpp StatusVariable, StatusVariableStore,
DataVariable, DataVariableStore
equipment_constants.hpp EquipmentConstant, EquipmentConstantStore,
EquipmentAck. set_value() now validates
numeric values against min_str/max_str and
returns EAC=4 on out-of-range — closes the
COMPLIANCE.md gap about EC range validation.
event_reports.hpp CollectionEvent, Report, ReportData,
EventReportSubscriptions + DefineReportAck,
LinkEventAck, EnableEventAck. The store is
pure data; VidLookup / VidExists callbacks
are injected at define / emit time so the
service doesn't back-reference the SVID
store.
alarms.hpp Alarm, AlarmAck, AlarmRegistry.
Encapsulates the (enabled, active) sets and
ALCD byte computation.
recipes.hpp ProcessProgramAck, RecipeStore.
clock.hpp TimeAck, Clock. set_time_string applies an
offset so subsequent reads reflect the host
time without mutating system clock.
host_commands.hpp HostCmdAck, CommandParameter,
HostCommandRegistry with Spec/Result types.
include/secsgem/gem/data_model.hpp shrinks to a 50-line composite:
struct EquipmentDataModel {
StatusVariableStore svids;
DataVariableStore dvids;
EquipmentConstantStore ecids;
EventReportSubscriptions events;
AlarmRegistry alarms;
RecipeStore recipes;
Clock clock;
HostCommandRegistry commands;
/* + vid_value, vid_exists, compose_reports_for sugar */
};
src/gem/data_model.cpp is gone — every store is inline header-only.
include/secsgem/gem/messages_helpers.hpp picks up EventReportAck and
TerminalAck (S6F12 / S10F2-F4 ack enums that aren't tied to any one
store).
Call-site updates:
apps/secs_server.cpp model->status_variable(id) -> model->svids.get(id),
model->equipment_constant(id) -> model->ecids.get(id),
model->alarm_set(id) -> model->alarms.set_active(id),
model->dispatch_command(...) -> model->commands.dispatch(...),
and similar across every handler. Plus
model->current_time_string() -> model->clock....
src/config/loader.cpp model.add_status_variable(sv) -> model.svids.add(sv),
and similar. HostCommandRegistry::Spec replaces
EquipmentDataModel::CommandSpec.
apps/secs_client.cpp std::vector<EquipmentDataModel::CommandParam> ->
std::vector<CommandParameter>.
tests/test_data_model.cpp Rewritten around the individual stores;
each gets its own TEST_CASE block. Adds three
new cases covering EC range validation (in
range / out of range / non-numeric skipped).
tests/test_loader.cpp m.has_event(100) -> m.events.has_event(100),
etc.
Verified:
- Tests: 69 cases / 370 assertions pass (was 67 / 384; -14 stale
composite-API assertions + 16 new store-level assertions covering
EC range validation and the per-store add/get/list/delete paths).
- Demo: byte-identical behaviour across the full 17-step flow.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raphael