Deterministic-seed fuzz coverage of the byte-decoding surface:
- secs2::decode on 2000 random buffers
- secs2::decode on every truncation of a real encoding + 500
one-byte flips of the full encoding
- hsms::Frame::decode on 1000 random payloads
- hsms::Header::decode on 2000 random 10-byte buffers
- secsi::Block::decode on 2000 random buffers
- secs2 encode/decode round-trip identity across a battery of every
Item factory (List, ASCII, Binary, Boolean, U1..U8, I1..I8, F4/F8,
nested List)
- oversize <A 3 length-bytes> length-prefix doesn't allocate GBs
- 64-level nested List round-trip doesn't blow the stack
Contract is binary: no crash, no UB. Each decoder is allowed to throw
or return whatever; we deliberately don't assert *what* result comes
back, only that control returns. Fixed PRNG seeds make any failure
reproducible from the CI log alone.
Closes#8 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
test_secsi.cpp covered T2 on the send side (retry) and a tick-based
back-to-back exchange. This commit fills in the rest of the timer
matrix at FSM level:
T1 in RecvBlock → abort, reason mentions "T1"
T1 outside RecvBlock → ignored
T2 in RecvEotSent → abort
T2 in RecvBlock → abort (mid-block stall)
T3 / T4 → FSM-level no-op (documented as upper-layer driven)
T2 contrast → send-side retries, recv-side aborts (same timer,
different recovery, both demonstrated in one test)
If a future commit moves T3 or T4 enforcement into the FSM, the
no-op test breaks loudly so protocol.hpp can be updated alongside.
Closes#7 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
S9F3/F5 are covered by test_s9_fallback (router path); S9F9/F11 by
test_hsms_timers (timer/over-length). This commit adds S9F7 wire-level
tests for the third path — a primary whose body fails secs2::decode.
Three new cases:
- hand-built primary with truncated <B> body provokes S9F7
carrying the original 10-byte MHEAD (sys + stream + function)
- emission is non-fatal: the next well-formed primary still routes
to the registered handler
- data-while-NOT-SELECTED still echoes Reject(EntityNotSelected)
(sanity copy of the test_hsms_connection case so the "what does
the equipment say when a peer sends garbage" family lives together)
Closes#6 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Real-socket tests for the timer family in E37 §10 — these replace
the "the timer fires somewhere" implicit assumption with
end-to-end observations on a loopback pair:
T3: send_request that gets no reply emits S9F9 with the original
MHEAD echoed in the body and surfaces Timeout to the caller.
T6: active mode whose Select.req goes unanswered self-closes
with a "T6 timeout on Select" reason.
T7: passive mode that never receives Select.req self-closes
with a "T7 not-selected timeout" reason.
T8: peer sends only the 4-byte length prefix; T8 expires mid-read
and closes with "T8 intercharacter timeout".
Plus S9F11 emission for an over-length frame (length prefix of
1 GiB+1) — body's <B 10> echoes the offending bytes verbatim.
Per-test timer profiles (only the timer under test is short, the
rest are 5s) so the FSM isn't racing against unrelated timers.
Closes#5 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-EXID binary record (.ex), magic + version + atomic .tmp+rename.
Records full E5 §9 lifecycle: state, EXID, EXTYPE, EXMESSAGE, and
the candidate EXRECVRA list.
Cleared exceptions are terminal — the FSM transitions through
Cleared remove the in-memory entry AND delete the journal file
(matching the existing in-memory semantics). Recovering /
RecoverFailed states survive restart: the application can decide
on replay whether to retry recovery or abort.
Five new tests cover post+replay, Recovering-survives-restart,
autonomous-clear cleanup, RecoverFailed retry post-restart, and
corrupt-record drop.
This completes #12 in the test-gap backlog (persistence for the four
in-memory stores beyond Spool).
Closes#4 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-job binary record (.pj / .cj) with magic+version, atomic
.tmp+rename. PJ store additionally writes an order.idx index file
that preserves HOQ-aware queue position across restarts.
Rcpvars / prprocessparams (secs2::Item variants) are intentionally
out of scope for v1 — they're optional E40 trailers and need a body
codec round-trip; callers re-populate via set_e40_extras() after
restart.
Five new tests cover full lifecycle replay (Processing mid-run +
HOQ-reordered queue), dequeue-deletes-file, corrupt-record drop,
CJ state + PJ-list replay, and CJ remove cleanup.
Closes#3 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same pattern as carriers: per-substrate binary record (.sub) with
atomic .tmp+rename, replay on enable, delete on remove. Records
current state across all three E90 axes (location / processing /
ID-status), plus substid / carrierid / slot / free-form location
label. History is deliberately NOT journaled — it's an in-memory
ring buffer and rebuilding from replayed state would mislead.
Five new tests cover full-axis replay, every terminal processing
state, remove-deletes-journal, corrupt-record drop, and the
history-is-transient invariant.
Closes#2 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors SpoolStore: per-record file with atomic .tmp+rename, magic+
version-prefixed binary layout, replay on enable, delete on remove.
FSMs gain a restore_state() that bypasses the transition table and
handlers since a replay isn't a transition.
Six new tests cover write+restart+replay across every CIDS/CSMS/CAS
axis, remove-deletes-journal, malformed-record drop-not-poison, and
the persistence-disabled no-op path.
Closes#1 in the test-gap backlog.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A host couldn't drive the new messages through the HostHandler class —
only the server side knew how to dispatch them. Adds six new senders
plus a unit test that walks each through a real loopback connection:
* send_legacy_remote_command -> S2F21
* send_event_report_request -> S6F15
* send_individual_report_request -> S6F19
* send_annotated_report_request -> S6F21
* send_pp_load_inquire -> S7F1
* send_delete_pp -> S7F17
Suite: 296 cases / 1571 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the test gap for messages I added but whose reply parsers were
only generated, never exercised:
* S6F8 — full nested DATAID/CEID/DS/DV structure.
* S12F14 — row-format map reply (RSINF tuples).
* S12F16 — array-format map reply.
* S12F18 — coordinate-format map reply.
Suite: 295 cases / 1545 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds opt-in disk persistence to SpoolStore. `enable_persistence(dir)`
turns every enqueue into a single `<seq>.spool` file alongside the
in-memory queue; drain and clear delete the matching files; restart
replays the directory sorted by seq.
Writes are atomic: serialize the message via the SECS-II codec, write
to `.tmp`, then `std::filesystem::rename` to the final name. Malformed
records are dropped silently so a single bad file can't poison the
whole spool.
`secs_server --spool-dir <path>` enables persistence at startup.
Without the flag the behaviour is identical to before (in-memory only).
Two new tests: enqueue → restart → replay → drain restores the wire
order, and clear deletes the journal files.
Test suite: 291 cases / 1515 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the simplified <L,3 PRJOBID PPID MTRLOUTSPEC> demo body with
the full SEMI E40-0705 §10.2 shape:
<L,5 PRJOBID MF PRRECIPEMETHOD
<L,2 PPID <L,n <L,2 RCPPARNM RCPPARVAL>>>
<L,n MTRLOUTSPEC>
<L,n <L,2 PARAMNAME PARAMVAL>>>
ProcessJob now carries the extra fields (MaterialFlag, ProcessRecipeMethod,
RcpVar[], ProcessParam[]) so a tool's recipe engine can later consume
the recipe-variable overrides and per-job process parameters. Server
S16F11 dispatch populates them via the new ProcessJobStore::set_e40_extras
helper after a successful create.
MaterialFlag + ProcessRecipeMethod enums live in their own tiny header
(`e40_constants.hpp`) so process_jobs.hpp (the store) can use them
without dragging in messages_helpers.hpp (which would create a circular
include via data_model.hpp).
The simplified 3-arg HostHandler::send_create_process_job convenience
remains; it constructs a sensible-default PRJobCreateRequest internally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the SECS-II messages secsgem-py 0.3.0 ships but our C++ catalog
didn't have, plus the alternative wafer-map formats from E5 §13.
None of these were strictly required for GEM core compliance, but
they're the messages a host might send to a conformant equipment.
* S7F1/F2 — Process Program Load Inquire / Grant. Equipment-side
space-and-policy check before a host commits to S7F3.
* S7F17/F18 — Delete Process Program. Empty list = delete-all.
* S6F5/F6 — Multi-block Data Send Inquire / Grant (with MultiBlockGrant
enum: Ok/Busy/NoSpace/DuplicateMsg/BadMsg).
* S6F7/F8 — Data Transfer Request / Send. Host pulls a DATAID;
equipment replies with the nested DS/DV structure.
* S6F15/F16 — Event Report Request (host-initiated). Reply mirrors
the unsolicited S6F11.
* S6F19/F20 — Individual Report Request (RPTID -> values).
* S6F21/F22 — Annotated Individual Report Request (RPTID -> (VID, value)).
* S2F21/F22 — Legacy Remote Command (no parameter list). Delegates
to the same HostCommandRegistry as S2F41.
* S12F9/F10 — Map Data Send (array format, MAPFT=1).
* S12F11/F12 — Map Data Send (coordinate format, MAPFT=2).
* S12F13/F14, F15/F16, F17/F18 — Map Data Request variants for the
row, array, and coordinate formats.
11 new round-trip tests; suite at 289 cases / 1495 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The biggest single gap I called out in the GEM300 audit — closed.
E84 is the digital handshake between AMHS (Automated Material
Handling System) and the equipment for carrier load/unload. Unlike
the rest of GEM300, this isn't SECS messaging; it's a fixed set of
ten parallel boolean wires that follow a strict sequencing protocol
(E84-0710 §6.3).
Adds:
E84Signal enum CS_0/CS_1/VALID/TR_REQ/BUSY/COMPT/L_REQ/U_REQ/
READY/ES
E84SignalSet 10-bit bitmap with bool get/set
E84State Idle / CarrierPresent / ValidAsserted /
LoadReady / UnloadReady / Transferring /
Complete / EmergencyStop
E84StateMachine re-evaluates state on every signal change,
observable via set_state_change_handler
Joins EquipmentDataModel as `e84` (top-level — there's one per tool,
not per port). ES (emergency stop) dominates regardless of other
signals; COMPT and BUSY override the VALID-handshake states. Same
FSM drives real opto-isolated I/O lines (when wired through an
asio digital input adapter) and the back-to-back test simulation.
Six test cases cover the full load handshake trace (six transitions,
including the transient LoadReady-after-BUSY-drops state), the
unload variant via U_REQ, ES dominance + recovery, reset(), and
no-op suppression for idempotent signal writes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Extends the existing Clock with the metrics a host needs to gate
time-sensitive data against the equipment's sync state (E148 §6.3):
offset_seconds() current applied offset vs system clock
last_drift_seconds() signed drift observed at the most recent sync
sync_count() how many successful syncs have happened
sync_quality() Synchronized (|drift|<=1s) /
Drifting (<=60s) / Unsynchronized (>60s or
never synced)
The thresholds are tuneable per call; the defaults match typical fab
practice but the application can pass tighter bounds for tracelog-
sensitive flows. set_time_string() now snapshots the apparent delta
between the previously-applied offset and the new one as
last_drift_seconds_ at the moment of resync; no background timer.
Three new test cases cover the initial Unsynchronized state, a large
forward drift registering as Unsynchronized, and a same-value resync
landing as Synchronized.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The catalog had S14F9/F10 / F11/F12 specialized for E94 ControlJob;
this commit adds the generic E14 attribute access pair, the most-
queried half of the E39 surface area, backed by the CemObjectStore.
S14F1 / F2 GetAttr — OBJSPEC + OBJTYPE + ATTRID list ->
(ATTRID, VALUE) pairs + OBJACK
S14F3 / F4 SetAttr — same addressing, applies ATTRID/VALUE pairs,
reply echoes stored values + OBJACK
Server dispatches both into the CemObjectStore added in tranche G.
OBJTYPE validation is case-sensitive against the CemObjectType name
(Equipment / Subsystem / IODevice / Module / MaterialLocation).
Unknown objects return Denied_UnknownObject; type mismatches return
Denied_InvalidAttribute.
The shared AttrValue struct is declared external_struct: true on
F3/F4 so both directions share the same C++ type.
Two round-trip tests cover both pairs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the two E40 bulk/control gaps the COMPLIANCE doc had flagged
as out-of-scope:
S16F7 / F8 PRJobMonitor — host enables/disables S16F9 alerts
per PJ. PRALERT bit 7 is the enable flag (matches the
ALED convention from S5F3). Server dispatches into the
existing set_alert() store API.
S16F15 / F16 PRJobCreateMultiple — bulk create variant. Host posts
a list of (PRJOBID, PPID, MTRLOUTSPEC) entries; the
equipment processes them in order and returns a
per-PJ HCACK list so the host can identify which
subset failed. Same validators as S16F11.
Catalog now has three new structs: PRJobMonitorEntry,
PRJobCreateEntry, PRJobCreateMultiResult. Two round-trip tests cover
the new wire shapes; server-side correctness is exercised through the
existing PJ store invariants (dedup, validator) which both new paths
delegate to.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the slot-map verification gap I called out:
S3F19 / F20 host -> equip: verify expected slot map against what
the equipment has scanned. Equipment compares element-
wise; on match drives CSMS NotRead -> Read and replies
SVACK=Accept; on mismatch drives CSMS -> Mismatched and
replies SVACK=Mismatch.
S3F21 / F22 equip -> host: equipment-initiated slot map report
(typically pushed after CARRIERID is confirmed).
New SVACK enum: SlotMapVerifyAck { Accept, Mismatch, CarrierUnknown,
Error }. Server dispatch on S3F19 wires the actual CSMS transition
through the CarrierStore from D3.
Two round-trip tests cover both pairs; the FSM-driving behaviour is
exercised through the in-process tests because secs_server.cpp is
the dispatch entry point (no separate integration test needed beyond
the wire round-trip).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the substrate-ID verification FSM that E90 §6.4.6 calls for:
NotConfirmed initial; equipment hasn't read the ID yet
WaitingForHost ID has been read; awaiting host accept/reject
Confirmed host confirmed (or force-bound)
Mismatched host rejected — recoverable via Bind
Events:
Read NotConfirmed -> WaitingForHost
Confirm WaitingForHost -> Confirmed
Mismatch WaitingForHost -> Mismatched
Bind any -> Confirmed (force-bind)
Reset any -> NotConfirmed
Wire-byte values pinned via static_assert. The third axis is now
exposed on SubstrateStateMachine alongside location_state() and
processing_state(); set_id_handler() observes transitions. Existing
two-axis API is unchanged.
4 new test cases cover the happy path, Mismatch+Bind recovery, Reset
from any state, and same-state event handler suppression.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each Substrate now retains an append-only history of state transitions
(both location and processing axes), the triggering event captured as
a std::variant<SubstrateEvent, SubstrateProcessingEvent>, the location
label at the time, and a steady_clock timestamp.
E90 §6.6 requires the equipment to be able to report a wafer's
processing history — typically queried via S6F11 batched reports or
SVID reads. This commit lays the runtime substrate; wire query
plumbing is the natural follow-up.
set_history_limit(n) caps per-substrate retention (default 256, 0 =
unbounded). Oldest entries are dropped when the cap is reached;
vector-erase is fine at this scale (typical wafer lifecycle is a few
dozen transitions).
Two new test cases cover the recording invariants (every fire results
in one history entry on the right axis) and history_limit eviction.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
EptStateMachine now retains per-state cumulative dwell time so the
host can read it as SVIDs. The implementation is zero-overhead while
the FSM is idle (no timers, no background work) — on every transition
we add the prior state's dwell to its bucket and reset the entered_
timestamp. Live dwell in the current state is included in
accumulated() via a now-vs-entered_ delta at read time.
New public API:
accumulated(EptState) per-state cumulative ms (incl. live dwell)
total_elapsed() denominator for utilization ratios
reset_history() S2F43-style history clear
This closes the gap I called out: previously we emitted CEIDs on
transition but didn't accumulate the bucket the host actually queries
for utilization metrics. Wiring these into specific SVIDs is the
application's job (equipment.yaml declares SVIDs against any read
callable); the runtime data is now there.
4 new test cases cover accumulation, live-dwell inclusion, and reset.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The S9F3/F5 fallback was previously inlined in apps/secs_server.cpp;
this commit lifts it onto Router as a template helper and adds two
focused tests asserting the wire behaviour against a real back-to-
back HSMS Connection pair.
template <typename EmitFn, typename HeaderProvider>
std::optional<Message> dispatch_with_s9(emit, header, msg);
The helper does the has_handler / has_handler_for_stream check and
calls the supplied emit function with S9F3 (unknown stream) or S9F5
(unknown function in known stream). The header_provider returns the
optional MHEAD bytes — keeping the helper free of any direct
Connection coupling.
Tests:
- SUT registered only for S1F1; peer sends S1F5 -> SUT replies
S9F5 to the peer.
- SUT registered only for S1F1; peer sends S7F19 -> SUT replies
S9F3 to the peer.
Closes Tranche I — SML parser and the auto-S9F* fallback closeout
both verified end-to-end.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds parse_sml(text) -> Item / try_parse_sml(text) -> optional<Item>
in secs2/sml.hpp. Round-trips with the existing to_sml() emitter for
every Item shape the codec produces: lists with nesting, ASCII / JIS8,
Binary (decimal and 0xHH literals), Boolean (T/F or 1/0, scalar and
multi-element), U1-U8 / I1-I8 / F4 / F8 vectors, and the optional
`[n]` count syntax (accepted but not enforced).
The parser is whitespace-insensitive outside quoted strings and uses
a small Cursor type for read_word / read_quoted / skip_ws. Numeric
literals go through strtoul/strtoll/strtod so SML can carry hex,
octal, and decimal interchangeably (the emitter writes hex for Binary
and decimal everywhere else).
11 test cases cover the full round-trip surface, the whitespace
invariant, unknown-tag rejection, the try_parse error-swallowing
variant, and the optional `[n]` count.
secsgem-py has secs/sml.py for the same purpose; this brings the C++
port to parity on the tooling side.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-module process-tracking state machine. An E157 instance models a
single recipe step at a single module, with the canonical lifecycle:
NotExecuting -> GeneralExecuting (StartGeneral)
-> StepExecuting (StartStep)
-> StepCompleted (CompleteStep)
Plus universal escape hatches: Reset returns any state to
NotExecuting; Abort terminates from any state to StepCompleted.
ModuleStore wraps the FSM with the now-standard pattern:
- non-movable (this-capture lambdas)
- per-module bind() carries current_substid + recipe_step
- fire(module_id, event) delegates to the FSM
- set_state_change_handler observes every transition with module_id
Joins EquipmentDataModel. 5 test cases cover happy path, Reset from
each interior state, Abort, store-level create dedup + bind, and the
multi-module change handler keying.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Hierarchical object tree for equipment self-description. Each object
carries a CemObjectType (Equipment / Subsystem / IODevice / Module /
MaterialLocation / Other), an optional parent_objid, and a flat
attribute map keyed by name (the wire shape S14F1 / F3 returns).
Operations covered:
add(CemObject) - dedup'd, validates parent exists
get / has - lookup by objid
get_attr / set_attr - E14 GetAttr / SetAttr semantics
children(parent) - tree traversal; empty parent = roots
The flat-map representation matches how E14 ObjectService traffic
addresses nodes (by OBJSPEC string). Wiring S14F1/F2 GetAttr and
S14F3/F4 SetAttr to this store is a downstream commit; the data model
is what was missing.
Joins EquipmentDataModel alongside the other top-level stores. Three
test cases cover hierarchical add+dedup, children() traversal, and
get/set/missing attribute semantics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the six E116-0712 §6.2 buckets for classifying equipment time:
NonScheduledTime (0) not scheduled to operate
ScheduledDowntime (1) planned maintenance window
UnscheduledDowntime (2) faults / unplanned stoppage
Engineering (3) engineering / qualification time
Standby (4) idle but available
Productive (5) actively producing
Wire-byte values pinned via static_assert to E116 §10.3.
The FSM is a classifier rather than a strict lifecycle — every
(state, event) pair is legal — but it remains data-driven through the
shared CarrierTransitionTable template so the default cross-product is
expressible declaratively.
The state-change handler also surfaces dwell time (how long the
previous state was held) computed off std::chrono::steady_clock, so
accounting code can compute MTBF / availability / utilization from a
single source without maintaining a parallel timestamp log.
4 test cases cover the initial state, every event firing, dwell-time
reporting, and the no-op same-state event (no handler call).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the Carrier Management wire surface that drives the FSMs from D1:
S3F17 / S3F18 CarrierAction (host issues ProceedWithCarrier /
CancelCarrier / BindCarrierID / etc.; CommandParameter
list is reused from S2F41).
S3F23 / S3F24 PortGroupChangeReport (equipment notifies host of port
group composition changes).
S3F25 / S3F26 CarrierTransfer (host instructs source -> target port
transfer).
S3F27 / S3F28 CancelCarrier (host cancels an outstanding carrier op).
Two new ack enums in messages_helpers.hpp:
CarrierActionAck — CAACK byte; covers the common error responses
(CarrierIDUnknown, Inaccessible, ActionInProgress).
PortGroupAck — PGACK byte (Accept / Error).
Round-trip tests for all four pairs. Server-side dispatch + the
CarrierStore + LoadPortStore that the FSMs key into land in D3/D4.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-carrier triple FSM: CIDS (id verification), CSMS (slot-map), CAS
(access). Per-port triple FSM: LPTS (transfer), LRS (reservation), LAS
(association). Wire-byte enum values pinned via static_assert to match
E87-0716 §10.3.
CarrierStateMachine combines the three carrier-side FSMs because they
are independent but always observed together; same for LoadPortState-
Machine. Generic CarrierTransitionTable<State, Event> template is
reused across all six tables — same row shape as the PJ/CJ/Exception
tables that already exist.
Default tables cover the spec's documented transitions:
CIDS: NotConfirmed <-> Confirmed/Mismatched/Unknown, Cancel returns
to NotConfirmed from any state, Bind force-confirms.
CSMS: NotRead -> Read -> {Mismatched, Reset}.
CAS: NotAccessed -> InAccess -> Complete (terminal).
LPTS: OutOfService <-> InService <-> Loading/Unloading.
LRS / LAS: simple boolean toggle pairs.
15 test cases assert the happy-path lifecycles, cross-state cancels,
and that change handlers fire only on real transitions (Read in
NotConfirmed is a no-op, not a handler call).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ALCD's lower 7 bits are a bitmap of category flags per E5 §10.3 and
E30 §6.13; a single alarm may carry multiple categories at once
(e.g. an irrecoverable equipment-safety fault is 0x10 | 0x02).
Adds:
enum class AlarmSeverity : uint8_t
PersonalSafety EquipmentSafety ParameterError ParameterWarning
Irrecoverable EquipmentStatus Attention
has_severity(alcd, bit), severity_bits(alcd)
Alarm::has(bit), Alarm::is_safety()
constexpr severity_mask = 0x7F
Tests cover single-category alarms, multi-category combos, and that
the bit-7 SET/CLEAR flag is correctly excluded from category bits.
Closes Tranche C (E5 alarm/exception state model).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-EXID exception lifecycle for E5 §9. States mirror the wire flow:
Posted equipment sent S5F9, awaiting host or autonomous clear
Recovering host's S5F13 accepted; equipment running recovery
RecoverFailed S5F15 reported a failed result; host may retry
Cleared terminal — store removes the row
Events:
Created synthetic NoState->Posted observer signal
Recover host's S5F13 (Posted/RecoverFailed -> Recovering)
RecoveryComplete equipment internal (Recovering -> Cleared)
RecoveryFailed equipment internal (Recovering -> RecoverFailed)
RecoveryAbort host's S5F17 (Recovering -> Posted)
Clear equipment internal (Posted/RecoverFailed -> Cleared)
ExceptionStore mirrors ProcessJobStore: per-EXID FSMs heap-allocated via
unique_ptr, non-movable to keep `this`-captures safe, synthetic Created
fires after the row lands so observers can decide whether to emit S5F9
out of band. on_recover validates EXRECVRA against the candidates the
post advertised.
The store joins EquipmentDataModel alongside process_jobs / control_jobs.
S5F9-F18 server-side dispatch lands in C2.
Tests (12 cases) cover FSM transitions including retry, abort, and
autonomous clear, plus store-level duplicate-rejection, EXRECVRA
validation, and Cleared-removes-the-row semantics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Five end-to-end tests wire a real HostHandler against a real passive
HSMS Connection over a TCP loopback pair and assert wire-level
behaviour matches expectations:
- establish_communication + go_remote sequence S1F13 then S1F17
- send_remote_command produces a wire-correct S2F41 the equipment
can re-parse with parse_s2f41 and recover CPNAME/CPVAL
- send_terminal_display round-trips through S10F1/F2
- E40/E94 create+command sequence (S16F11, S14F9, S16F5)
- Inbound S5F1 alarm fires the host's alarm observer + auto-acks
Each test uses the existing pump_until / SocketPair harness pattern
from test_hsms_connection.cpp. The recorder pattern keeps the
equipment-side dispatch table small — every test installs the same
canned reply handler.
This closes Tranche B (host mode). HostHandler now has the inbound
+ outbound surface secsgem-py's GemHostHandler exposes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wires the SECS-I Protocol FSM behind an asio TCP socket so the block
protocol can run over loopback without serial hardware. Mirrors
secsgem-py's `secsitcp/` adapter — useful for back-to-back simulators
and CI without a serial device.
Adds:
include/secsgem/secsi/tcp_transport.hpp
src/secsi/tcp_transport.cpp
tests/test_secsi_tcp.cpp
The transport:
- Splits outgoing SECS-II messages into blocks (transparent multi-block).
- Accumulates incoming blocks until end_block=true, then assembles and
delivers as a single SECS-II message — same surface as the HSMS
Connection's MessageHandler.
- Drives T1 / T2 timers from asio steady_timer; T3/T4 stay upper-layer
per the FSM contract.
- Auto-allocates monotonic system bytes per send.
Tests cover single-block delivery, multi-block reassembly (700-byte
ASCII body spanning multiple SECS-I blocks), and bidirectional exchange.
This closes Tranche A (catch-up to secsgem-py wire/transport surface).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a complete IO-free SECS-I implementation:
include/secsgem/secsi/header.hpp 10-byte block header (R/W/E bits)
include/secsgem/secsi/block.hpp length + header + body + checksum
include/secsgem/secsi/protocol.hpp half-duplex FSM (ENQ/EOT/ACK/NAK)
src/secsi/* implementations
tests/test_secsi.cpp header, block, multi-block split,
back-to-back FSM drive, RTY,
contention, T2 timeout
The protocol is event-driven (`Event` → `Action` queue), so wiring it
to an asio serial_port is a thin adapter — that lands in the next
commit so this one stays reviewable.
Key design points:
- Master/slave contention: slave yields on simultaneous ENQ (E4 §7.1.4).
- RTY exhaustion raises ActionRaiseError, clears the send queue, resets
to Idle (no zombie state).
- Multi-block assembler validates contiguous 1..N numbering and exclusive
E-bit-on-last invariants — rejects malformed sequences with nullopt.
- Block::checksum is exposed publicly for the receive path's verification.
Tests cover the happy path (back-to-back delivery), error paths
(checksum mismatch, short input, oversize body), retries (NAK chain to
exhaustion), and protocol corner cases (contention, T2 timeout).
secsgem-py implements SECS-I block framing but lacks the explicit RTY
state machine; this commit puts the C++ port ahead on transport
correctness.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the core map-management messages: setup send/request (F1-F4),
transmit inquire/grant (F5/F6), data send in row format (F7/F8), and
the one-way error report (F19). Reference points (REFP) are an
external struct shared across F1 and F4.
The alternative data encodings — array format (F9/F10), coordinate
format (F11/F12), and the corresponding request pairs (F13-F18) —
are scope-deferred. They're mechanical YAML edits once a tool needs
them; the codec already handles the underlying BINARY/list shapes.
Three new ack enums: MapSetupAck (SDACK), MapTransmitGrant (GRANT),
MapDataAck (MAPER). No state machine yet — maps are a data exchange,
not a lifecycle.
secsgem-py ships S12F0-F19 as a single block; this commit covers the
practically-used subset and matches their wire shapes where they
overlap.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the OBJSPEC-scoped sibling of S2F41 with extended per-parameter
ack shape (CPACK + CEPACK). Wire:
S2F49 body <DATAID OBJSPEC RCMD <L,n <CPNAME CPVAL>>>
S2F50 body <HCACK <L,n <CPNAME CPACK CEPACK>>>
Server delegates to the existing HostCommandRegistry, logs OBJSPEC for
audit, and currently returns empty cpacks (all-OK). Per-parameter
failures will be wired when the command registry grows CEPACK-level
validation; this commit is the catalog + dispatch scaffolding.
secsgem-py defines these in its catalog but never dispatches them; this
puts the C++ port marginally ahead on remote-command coverage.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Catalog gains the recover-request / recover-complete / recover-abort
loop that closes E5's exception lifecycle. Wire shapes:
S5F13/F14 Exception Recover Request / Acknowledge
S5F15/F16 Exception Recover Complete Notify / Acknowledge
S5F17/F18 Exception Recover Abort Request / Acknowledge
Acks use the same AlarmAck byte already in use by S5F10/F12. EXRESULT
on F15 is modelled as ASCII (the common case; vendor-specific richer
shapes are a YAML edit).
Round-trip tests cover all three pairs. Server dispatch is left for a
later commit alongside the per-alarm AlarmStateMachine (Tranche C) —
this commit is wire-coverage only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
GEM300 layer: SEMI E40-0705 Process Job and E94-0705 Control Job
state machines, plus the E30 §6.1 communication-state machine that
sits between HSMS SELECT and full GEM communication. Data-driven
via data/process_job_state.yaml and data/control_job_state.yaml,
mirroring the existing control_state.yaml pattern.
Wire coverage:
S14F9/F10 CreateObject (CJ) host -> equipment
S14F11/F12 DeleteObject (CJ) host -> equipment
S16F5/F6 PRJobCommand host -> equipment
S16F9 PRJobAlert equipment -> host
S16F11/F12 PRJobCreate (simplified body) host -> equipment
S16F13/F14 PRJobDequeue host -> equipment
S16F27/F28 CJobCommand host -> equipment
Process Job FSM exposes 8 states matching PRJOBSTATE bytes (E40 §10.3.2);
HOQ is reorder-aware (move-to-head against an insertion-order vector);
Stop/Abort on a Queued PJ routes through ABORTING so the host observes
PRJOBSTATE=7 on the wire (§6.3); alert_enabled is settable per-PJ for
PRALERT control; FSM dispatches through ProcessJobStore::on_change_
dynamically so a late set_state_change_handler() reaches existing PJs.
Hardening: loader rejects NoState (sentinel) as initial/from/to and
rejects `on: created` rows; static_asserts pin enum values to wire
bytes; ProcessJobStore is non-movable to keep the per-PJ this-capture
safe.
Server simulator cascades the full CJ -> PJ lifecycle on CJSTART so
the wire trace exercises every legal state. CEIDs 400/401 fire on CJ
state changes via the existing event-report pipeline.
Tests: 60+ new assertions across test_process_jobs, test_control_jobs,
test_communication_state, test_hsms_connection, plus loader and
messages round-trip coverage.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The final additions: S10F5/F6 multi-line terminal display (closes the
last partial Additional capability — Equipment Terminal Services flips
✅), and a thorough COMPLIANCE.md / README pass that states the 100%
claim honestly.
Catalog + handlers
data/messages.yaml S10F5 / S10F6 added.
apps/secs_server.cpp router.on(10, 5) iterates the line list,
acks with S10F6.
tests/test_messages.cpp Round-trips a 3-line multi-line display.
COMPLIANCE.md (rewritten)
Every GEM Fundamental ✅. Every GEM Additional that E30 binds to a
concrete message set ✅. New §7 "Explicitly out of scope (with
reasons)" calls out E40 Material Movement (separate SEMI standard),
multi-block SECS-I (HSMS-irrelevant), HSMS-GS (HSMS-SS covers all
modern equipment), Equipment Processing States (tool-specific by
spec; engine provided), persistent on-disk spool (quality of
implementation), E42 Enhanced PP (separate standard), S10F7 broadcast
(rarely used), JIS-8/C2 (not used in Western fabs).
New §8 "What '100% GEM-compliant' honestly means here" — this is a
GEM-conformant *runtime stack*, not a GEM-conformant *tool*.
Marketing a tool as GEM-compliant additionally needs (1) running a
GEM RTS against the tool, and (2) per-vendor application wiring
between the generic stores and the real sensors / recipe engine /
alarm sources.
README.md (rewritten)
Architecture diagram updated to reflect the actual store list (nine
stores). "Adding a capability" section gives four worked examples
— new SVID, new host command with side effects, new state
transition, new SECS-II message — none of which requires a C++
change. Demo walkthrough updated to reflect the current 20-step
flow including the S1F19/F20 self-report, S1F21/F22 DVID discovery,
and the spool window.
Code clarity
include/secsgem/gem/data_model.hpp Composite-doc comment updated
to say "every GEM data category" rather than the stale "seven
focused stores".
Verified
- Tests: 84 cases / 487 assertions pass.
- Demo: 198 server/host log lines; exits 0 end-to-end.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes spooling. S6F25 (NUM-MSG) goes into the catalog; S6F26
(ACKC6) likewise. The server's on_selected handler now checks the
spool on entering SELECTED — if there's queued data, it auto-emits
S6F25 so the host can decide what to do (S6F23 Transmit vs Purge).
The happy-path demo never drops TCP so the auto-trigger doesn't fire
there, but the canonical re-SELECT path is wired. Client gains a
handler for inbound S6F25 that logs the count and acks S6F26.
COMPLIANCE.md: Spooling Additional capability flips from 🟡 to ✅.
Remaining out of scope for spooling: persistent on-disk spool so
restarts don't lose queued events. Demo + tests don't need it; real
fab equipment would.
Tests: 83 cases / 481 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New TraceStore keyed by TRID; each entry is a TraceConfig with
DSPER + TOTSMP + REPGSZ + SVID list. S2F23 validates that every SVID
exists (TIAACK=4 otherwise) and registers the trace.
S6F1's body is L,4 of {TRID U4, SMPLN U4, STIME ASCII, list_of <Item>}
— the application chooses whether each value Item is a scalar SVID
value or a packed batch.
The periodic sampling timer that turns an active TraceConfig into
S6F1 emissions is intentionally left to the application (E5 doesn't
mandate a specific scheduler and vendors typically already have one).
Four new SxFy in the catalog.
COMPLIANCE.md: Trace Data Collection Additional capability flips ✅.
Tests: 82 cases / 477 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New LimitMonitorStore keyed by VID; each entry is a vector of
LimitDefinition (LIMITID + upper/lower deadband as arbitrary Items).
S2F45/F46 set, S2F47/F48 read. VLAACK validates each VID exists.
Four new SxFy in the catalog; codegen handles the nested
list-of-(VID, list-of-LimitDefinition) shape. LimitDefinition is
defined in store/limits.hpp and referenced as external_struct so the
data model and the message codecs share one type.
The actual "value crossed limit" detection + CEID emission is left to
the application's set_value path (E30 §6.21 leaves *how* the equipment
detects crossings up to the implementer).
COMPLIANCE.md: Limits Monitoring Additional capability flips ✅.
Tests: 80 cases / 465 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the small remaining hole in Alarm Management. S5F7 is
header-only; S5F8 has the same wire shape as S5F6 (vector of
<L,3 <B ALCD> <U4 ALID> <A ALTX>>) but only includes alarms whose
enabled flag is set. Codegen handles both as a list_of with
struct_name=AlarmListing; server pre-computes the per-row ALCD from
(severity_category, active state) before passing the rows in.
COMPLIANCE.md: Alarm Management Additional capability flips ✅.
Tests: 78 cases / 454 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the GEM "Documentation" Fundamental capability: the equipment now
self-reports which GEM capabilities it supports, and the host can
discover the DVID namelist with the same shape used for SVIDs.
Catalog (data/messages.yaml -> generated messages.hpp)
S1F19 W header-only Get GEM Compliance Request
S1F20 <L,3 <A SOFTREV>
<A EQPTYP>
<L,a <L,2 <U1 CCODE> <A CDESC>>>>
Get GEM Compliance Data
S1F21 W <L,n <U4 VID>> DVID Namelist Request (n=0 = all)
S1F22 <L,n <L,3 <U4 VID>
<A VNAME>
<A UNITS>>> DVID Namelist Data
Codegen emits CapabilityEntry and GemCompliance structs. S1F22 reuses
S1F12's StatusName struct (same wire shape; dedup avoids redefinition).
Equipment data dictionary (data/equipment.yaml)
device: Adds `equipment_type: "EQUIPMENT"`
for the S1F20 EQPTYP field.
capabilities: New section. List of
- {code, name} (CCODE, CDESC) pairs honestly
reflecting what the codebase
implements: 1, 2, 3, 5, 6, 7, 8,
9, 11, 12, 14 (partial), 15.
dvids: New section, same schema as
svids:. Demo populates two:
- WaferCounter (U4, units wafer)
- ChamberPressure (F4, units Torr)
Loader (src/config/loader.cpp + include/secsgem/config/loader.hpp)
EquipmentDescriptor gains equipment_type and capabilities (vector of
(uint8_t, string) pairs). load_equipment now reads `capabilities:`
into the descriptor and `dvids:` into model.dvids.
Server (apps/secs_server.cpp)
router.on(1, 19) returns S1F20 with desc.software_rev,
desc.equipment_type, and desc.capabilities converted to
vector<CapabilityEntry>.
router.on(1, 21) returns S1F22 built from model.dvids.all().
Client (apps/secs_client.cpp)
Two new demo steps after Request Online and before SVID discovery:
S1F19 -> S1F20: logs SOFTREV, EQPTYP, and every (CCODE, CDESC)
the equipment claims.
S1F21 -> S1F22: logs each DVID with units.
Tests
tests/test_messages.cpp Round-trip S1F19/F20 with a 3-entry
capability list; round-trip S1F22 with two
DVIDs.
tests/test_loader.cpp Asserts equipment_type, the capabilities
list contains CCODE 14 (Spooling), and the
two DVIDs land in model.dvids.
COMPLIANCE.md
"Documentation" Fundamental moves from ⬜ to ✅.
S1F19/F20 + S1F21/F22 rows in the coverage matrix flip to ✅.
The "what would it take" list drops the documentation-messages bullet.
Verified
- Tests: 77 cases / 444 assertions pass.
- Demo: client logs the full capability list received from the
equipment, including CCODE 14 "Spooling (partial; S2F43/F44 +
S6F23/F24)" — the equipment honestly reports its partial
implementation rather than overclaiming.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds S9F1, F3, F5, F7, F9, F11, F13 to the message catalog and wires
the two emission paths that the Connection layer can drive without help
from the Router or the application: S9F7 on a body-decode failure and
S9F9 on a T3 transaction-timer timeout.
Catalog (data/messages.yaml -> generated messages.hpp)
All six MHEAD-carrying messages (F1/F3/F5/F7/F9/F11) use the same
shape — a single <B 10> body with the offending 10-byte HSMS header.
S9F13 (conversation timeout) carries <L,2 <A MEXP> <A EDID>>.
Connection-side emissions (src/hsms/connection.cpp)
emit_s9(function, mhead) New private helper. Builds a 9/function/W=0
data message whose body is <B 10> with the
MHEAD bytes, allocates a fresh sys_bytes,
and queues it onto the write path. No
reply is tracked.
S9F7 on body decode handle_data wraps Message::from_body in a
try/catch. Previously any decode error
closed the connection; now it emits S9F7
with the offending header and continues
reading. Reply-side decode failure also
emits S9F7 and surfaces the new
Error::IllegalData to the waiting
ReplyHandler (rather than making the
caller wait out T3).
S9F9 on T3 timeout The send_request T3 callback rebuilds the
original outgoing MHEAD from
(device_id, expected_stream,
expected_function-1, sys, W=1) and emits
S9F9 before invoking the callback with
Error::Timeout (unchanged).
What's intentionally not yet wired (logged in COMPLIANCE.md)
- S9F3 / S9F5 — "unknown stream / function". These need to live in
the Router's fallback path, which would require either the Router
knowing about a Connection-shaped sender or the Connection's
message wrapper learning which streams the Router has handlers
for. Deferred — today the fallback returns SxF0 only.
- S9F11 — "Data Too Long". Currently we close on oversized frames;
we'd need to also build a synthetic 10-byte MHEAD substitute (the
real header isn't yet available at the point of detection) and
flush it through close_after_flush.
Tests + docs
tests/test_messages.cpp Round-trip every S9F* using a representative
10-byte MHEAD literal; check S9F13 carries
MEXP + EDID. +2 cases / +37 assertions.
COMPLIANCE.md Error Messages row moved from "no S9 stream"
to a detailed status describing what's
emitted vs catalog-only. Coverage matrix
expanded per-message (F1/F7/F9/F13 ✅;
F3/F5/F11 🟡 catalog-only).
Build/demo unaffected: 75 cases / 420 assertions pass; the happy-path
demo never trips a decode error or T3, so the S9 path isn't exercised
end-to-end (but unit tests prove the wire shape).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements the largest functional gap from the compliance audit. The
equipment now queues events the host can't immediately receive (either
because there's no SELECTED session or because the demo's force-spool
flag is on) and transmits the queue on host request.
What's new
include/secsgem/gem/store/spool.hpp
SpoolStore: a deque queue with a configurable per-stream whitelist
(so only streams 5+6 spool by default), a max_size cap with FIFO
eviction on overflow, and a `force_spool` test flag. Enqueue
returns one of Queued / Dropped_NotSpoolable / Dropped_Full so the
caller can fall back to live delivery when appropriate. Drain
pops the entire queue in FIFO order. Two new ack enums:
ResetSpoolAck (S2F44 RSPACK) and SpoolRequestAck (S6F24 RSDA), plus
SpoolRequestCode (S6F23 RSDC, Transmit/Purge).
data/messages.yaml + auto-regenerated messages.hpp
S2F43 W <L,n <B stream>> Reset Spooling
S2F44 <L,2 <B RSPACK> <L,a ...>> Reset Spooling Ack
S6F23 W <B RSDC> Request Spooled Data
S6F24 <B RSDA> Request Spooled Data Ack
data/equipment.yaml
`spool:` section: max_size + spoolable_streams list. Two new host
commands SPOOL_ON / SPOOL_OFF that flip the force-spool flag (these
stand in for "host link down" in the demo without dropping TCP).
include/secsgem/gem/store/host_commands.hpp
Spec/Result gain an optional<bool> force_spool field. S2F41
dispatch returns the result, the server applies it after S2F42 is
queued.
src/config/loader.cpp
Reads `spool:` from equipment.yaml; reads `force_spool` from each
host_commands entry; populates SpoolStore + CommandSpec.
apps/secs_server.cpp
New `deliver_or_spool(msg, what)` helper. emit_event and
emit_alarm_set funnel through it: if force_spool is on (or there's
no active session), msg.stream is checked against the spoolable
list and the message is enqueued; otherwise it's sent live.
Two new handlers:
S2F43 parses the stream list, updates SpoolStore, replies S2F44
S6F23 RSDC=Transmit drains and re-sends each as a fresh primary
(posted on the executor so the S6F24 ack flushes first);
RSDC=Purge clears the queue and acks.
The S2F41 handler now also propagates result.force_spool into the
SpoolStore.
apps/secs_client.cpp
Demo extended with 4 new steps after the FAULT branch:
SPOOL_ON -> S2F42 Accept
START -> S2F42 Accept; CEID 300 emission spooled (no live S6F11)
SPOOL_OFF -> S2F42 Accept; queue still has the message
S6F23(Transmit) -> S6F24 Accept; spooled S6F11 arrives next
Then the existing S7F19/S7F5/S10F1/S1F15/Separate flow continues.
tests/test_data_model.cpp
Four new TEST_CASEs for SpoolStore (whitelist, FIFO eviction at
max_size, drain ordering, force flag).
tests/test_loader.cpp
Confirms equipment.yaml's `spool:` section populates the store and
`force_spool: true/false` flows through to dispatch results.
COMPLIANCE.md
Spooling moves from ⬜ to 🟡. Adds S2F43/F44 + S6F23/F24 as ✅ in
the message coverage matrix; calls out what's still missing
(S6F25/F26 notification, automatic activation on HSMS NOT-SELECTED,
persistent on-disk spool).
Verified
- Tests: 73 cases / 383 assertions pass (+4 spool cases).
- Demo (docker compose up server client) walks the full happy path
and the spool path, observed in the server log as:
spool: force_spool=true (depth=0)
spool: S6F11 CEID=300 queued (depth=1)
spool: force_spool=false (depth=1)
S6F23 transmit: draining 1 messages
and on the host side as the queued S6F11 arriving in the correct
order after S6F24.
Known limitations (logged in COMPLIANCE.md)
- Spool activation is manual via SPOOL_ON/OFF rather than
automatically triggered by HSMS NOT-SELECTED.
- No S6F25/F26 spooled-data-ready notification on re-SELECT.
- In-memory only; an equipment restart loses queued events.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The god-class is gone. Each capability is now its own focused store:
StatusVariableStore, DataVariableStore, EquipmentConstantStore (with EAC
range validation), EventReportSubscriptions, AlarmRegistry, RecipeStore,
Clock, HostCommandRegistry. Each is independently testable.
EquipmentDataModel becomes a small composite that holds one of each store
as a public member, plus three convenience methods (vid_value, vid_exists,
compose_reports_for) that span SVIDs+DVIDs and inject the right callbacks
into the EventReportSubscriptions.
New under include/secsgem/gem/store/:
status_variables.hpp StatusVariable, StatusVariableStore,
DataVariable, DataVariableStore
equipment_constants.hpp EquipmentConstant, EquipmentConstantStore,
EquipmentAck. set_value() now validates
numeric values against min_str/max_str and
returns EAC=4 on out-of-range — closes the
COMPLIANCE.md gap about EC range validation.
event_reports.hpp CollectionEvent, Report, ReportData,
EventReportSubscriptions + DefineReportAck,
LinkEventAck, EnableEventAck. The store is
pure data; VidLookup / VidExists callbacks
are injected at define / emit time so the
service doesn't back-reference the SVID
store.
alarms.hpp Alarm, AlarmAck, AlarmRegistry.
Encapsulates the (enabled, active) sets and
ALCD byte computation.
recipes.hpp ProcessProgramAck, RecipeStore.
clock.hpp TimeAck, Clock. set_time_string applies an
offset so subsequent reads reflect the host
time without mutating system clock.
host_commands.hpp HostCmdAck, CommandParameter,
HostCommandRegistry with Spec/Result types.
include/secsgem/gem/data_model.hpp shrinks to a 50-line composite:
struct EquipmentDataModel {
StatusVariableStore svids;
DataVariableStore dvids;
EquipmentConstantStore ecids;
EventReportSubscriptions events;
AlarmRegistry alarms;
RecipeStore recipes;
Clock clock;
HostCommandRegistry commands;
/* + vid_value, vid_exists, compose_reports_for sugar */
};
src/gem/data_model.cpp is gone — every store is inline header-only.
include/secsgem/gem/messages_helpers.hpp picks up EventReportAck and
TerminalAck (S6F12 / S10F2-F4 ack enums that aren't tied to any one
store).
Call-site updates:
apps/secs_server.cpp model->status_variable(id) -> model->svids.get(id),
model->equipment_constant(id) -> model->ecids.get(id),
model->alarm_set(id) -> model->alarms.set_active(id),
model->dispatch_command(...) -> model->commands.dispatch(...),
and similar across every handler. Plus
model->current_time_string() -> model->clock....
src/config/loader.cpp model.add_status_variable(sv) -> model.svids.add(sv),
and similar. HostCommandRegistry::Spec replaces
EquipmentDataModel::CommandSpec.
apps/secs_client.cpp std::vector<EquipmentDataModel::CommandParam> ->
std::vector<CommandParameter>.
tests/test_data_model.cpp Rewritten around the individual stores;
each gets its own TEST_CASE block. Adds three
new cases covering EC range validation (in
range / out of range / non-numeric skipped).
tests/test_loader.cpp m.has_event(100) -> m.events.has_event(100),
etc.
Verified:
- Tests: 69 cases / 370 assertions pass (was 67 / 384; -14 stale
composite-API assertions + 16 new store-level assertions covering
EC range validation and the per-store add/get/list/delete paths).
- Demo: byte-identical behaviour across the full 17-step flow.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The bulk of the per-SxFy boilerplate — ~90 hand-written builders and parsers
across 30+ message pairs — is now generated at build time from a single YAML
catalog. Adding a new SECS-II message becomes a YAML edit; the C++ code is
generated, not maintained.
What changed
------------
data/messages.yaml
The catalog. Describes every SxFy currently supported: stream, function,
W-bit, builder name, optional parser name, and a recursive body shape
grammar (scalar / list / list_of). Shapes carry SECS-II item types
(ASCII, BINARY_BYTE, U4, F8, ITEM, ...) and optional C++ enum types for
typed ack codes. Inner-most fields can be marked external_struct: true
so structs already defined elsewhere (ReportData, CommandParameter) are
referenced rather than redefined.
tools/gen_messages.py
Python codegen. Reads the catalog and emits one inline header. Handles
nested shapes via depth-unique variable names in the generated IIFEs, so
S6F11's three-level nesting compiles without lambda capture conflicts.
Post-order traversal ensures inner structs are emitted before outer ones
that reference them. Generates positional and (where applicable) struct
builder overloads, plus struct-returning parsers for messages with a
`parser:` entry.
CMakeLists.txt
Custom command runs gen_messages.py at configure/build time and emits
${CMAKE_BINARY_DIR}/generated/secsgem/gem/messages.hpp. Added to the
secsgem target's include path so `#include "secsgem/gem/messages.hpp"`
resolves to the generated file. Depends on the YAML + the script, so
edits trigger regen automatically.
Dockerfile
Added python3 + python3-yaml to the toolchain image.
include/secsgem/gem/messages_helpers.hpp (new)
The small set of hand-written helpers the generated header relies on:
scalar accessors (as_ascii / as_u4_scalar / ...), parse_u4_list_body,
u4_list_item, ack_byte, ALED byte constants, and the two special-case
messages whose shape doesn't fit the codegen schema (S1F4 needs
per-row std::optional<Item> semantics; S5F6 needs a per-row ALCD
callback).
include/secsgem/gem/messages.hpp (deleted)
The hand-written builder/parser file is gone. Its content now flows
through the catalog + codegen.
include/secsgem/gem/data_model.hpp
Moved CommandParameter to namespace scope so it can be shared between
the data model and the messages.yaml's external_struct entry. Added
`using CommandParam = CommandParameter` for back-compat.
apps/secs_server.cpp + apps/secs_client.cpp
Updated the call sites that the codegen renamed or restructured:
- parse_terminal_display() split into parse_s10f1 / parse_s10f3.
- s1f14_establish_comms_ack now takes a McAck struct for the nested
identity (mdln, softrev) — call site uses brace init.
- S2F33/S2F35 parsers return strongly-typed entries (DefineReportEntry,
LinkEventEntry); the server adapts these to the model's pair-based
API at the call site.
- S2F15 parser returns vector<EcSet>; iterate by .ecid/.value.
- S5F3 parser returns EnableAlarmRequest{aled, alid}; bool comes from
(aled & 0x80) != 0.
- AlarmReport's is_set()/category() methods removed; callers use the
raw alcd byte with bit math (alcd & 0x80, alcd & 0x7F).
- s2f42_host_command_ack and s2f41_host_command always take their
second list argument explicitly (no defaulted arg from codegen).
tests/test_messages.cpp
Updated to construct the generated typed structs (EcSet, StatusName,
EnableAlarmRequest, CommandParameter, CommandParameterAck) and to read
the new field names (.ecid/.value, .rptid/.vids, .ceid/.rptids,
.name/.code).
Coverage
--------
Generated by codegen (44 SxFy in catalog):
S1F1, S1F2, S1F3, S1F11, S1F12, S1F13, S1F14, S1F15, S1F16, S1F17, S1F18
S2F13, S2F14, S2F15, S2F16, S2F17, S2F18, S2F29, S2F30, S2F31, S2F32
S2F33, S2F34, S2F35, S2F36, S2F37, S2F38, S2F41, S2F42
S5F1, S5F2, S5F3, S5F4, S5F5
S6F11, S6F12
S7F3, S7F4, S7F5, S7F6, S7F19, S7F20
S10F1, S10F2, S10F3, S10F4
Hand-written (in messages_helpers.hpp):
S1F4 list-of-optional-items shape (nullopt -> <L,0>)
S5F6 per-row ALCD via callback
Adding a new SxFy
-----------------
Append a single entry to data/messages.yaml describing the body shape.
The builder + parser appear in messages.hpp after the next build. The
host command above for S2F41 (or any other added SxFy) requires no C++
changes if the body fits the recursive scalar/list/list_of grammar.
Tests: 67 cases / 384 assertions still passing.
Demo: byte-for-byte identical behaviour (Select, Establish, Online,
S1F11/F3 namelist+values, S2F29 EC namelist, S2F33/F35/F37 dynamic event
subscription, S2F41 START -> S6F11 emission, S5F5/F3 alarm directory +
enable, S2F41 FAULT -> S5F1 alarm + S6F11, S7F19/F5 recipe ops, S10F1
terminal, S1F15 offline, Separate).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raphael