72da1dc77fb0a60cca3cbf8e5da152ebd5cefd96
20 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 raphael
|
cfa2d1e531 |
BB1: full E40 S16F11 body — MF, PRRECIPEMETHOD, RCPVARS, PRPROCESSPARAMS
Replaces the simplified <L,3 PRJOBID PPID MTRLOUTSPEC> demo body with
the full SEMI E40-0705 §10.2 shape:
<L,5 PRJOBID MF PRRECIPEMETHOD
<L,2 PPID <L,n <L,2 RCPPARNM RCPPARVAL>>>
<L,n MTRLOUTSPEC>
<L,n <L,2 PARAMNAME PARAMVAL>>>
ProcessJob now carries the extra fields (MaterialFlag, ProcessRecipeMethod,
RcpVar[], ProcessParam[]) so a tool's recipe engine can later consume
the recipe-variable overrides and per-job process parameters. Server
S16F11 dispatch populates them via the new ProcessJobStore::set_e40_extras
helper after a successful create.
MaterialFlag + ProcessRecipeMethod enums live in their own tiny header
(`e40_constants.hpp`) so process_jobs.hpp (the store) can use them
without dragging in messages_helpers.hpp (which would create a circular
include via data_model.hpp).
The simplified 3-arg HostHandler::send_create_process_job convenience
remains; it constructs a sensible-default PRJobCreateRequest internally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
4197cdfb25 |
AA: catalog growth — S7F1/F17, S6F5-F22, S2F21, S12F9-F18
Adds the SECS-II messages secsgem-py 0.3.0 ships but our C++ catalog
didn't have, plus the alternative wafer-map formats from E5 §13.
None of these were strictly required for GEM core compliance, but
they're the messages a host might send to a conformant equipment.
* S7F1/F2 — Process Program Load Inquire / Grant. Equipment-side
space-and-policy check before a host commits to S7F3.
* S7F17/F18 — Delete Process Program. Empty list = delete-all.
* S6F5/F6 — Multi-block Data Send Inquire / Grant (with MultiBlockGrant
enum: Ok/Busy/NoSpace/DuplicateMsg/BadMsg).
* S6F7/F8 — Data Transfer Request / Send. Host pulls a DATAID;
equipment replies with the nested DS/DV structure.
* S6F15/F16 — Event Report Request (host-initiated). Reply mirrors
the unsolicited S6F11.
* S6F19/F20 — Individual Report Request (RPTID -> values).
* S6F21/F22 — Annotated Individual Report Request (RPTID -> (VID, value)).
* S2F21/F22 — Legacy Remote Command (no parameter list). Delegates
to the same HostCommandRegistry as S2F41.
* S12F9/F10 — Map Data Send (array format, MAPFT=1).
* S12F11/F12 — Map Data Send (coordinate format, MAPFT=2).
* S12F13/F14, F15/F16, F17/F18 — Map Data Request variants for the
row, array, and coordinate formats.
11 new round-trip tests; suite at 289 cases / 1495 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
2d60571a9c |
interop: secsgem-py cross-validation harness + lenient identifier parsing
Adds a Docker-based interop harness that drives the C++ server with
secsgem-py 0.3.0 as the active host and probes a secsgem-py-passive
equipment from a minimal C++ active client. Surfaces and fixes four
interoperability bugs uncovered by cross-testing:
* SEMI E5 identifier formatcodes are a U1|U2|U4|U8 wildcard;
secsgem-py picks the narrowest fitting width while our parsers
only accepted U4. `as_uN_scalar` / `as_iN_scalar` now accept
any unsigned/signed width and range-check the downcast.
* PPBODY (S7F3/F6) is "ASCII | Binary | List" per the spec;
secsgem-py defaults to ASCII. Added BINARY_OR_ASCII codegen
item type with `as_text_or_binary` accessor.
* S1F23/F24 Collection Event Namelist was unimplemented; added
schema + `vids_for(ceid)` accessor on EventReportSubscriptions
plus the dispatch handler.
* S10F1 was registered as a host->equipment handler, but per
SEMI E5 §12 S10F1 is equipment->host; S10F3 is the actual
host->equipment Terminal Display Single. Added an S10F3
handler alongside (we keep S10F1 too for backward compat).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
af2c60663e |
N: E39 generic ObjectService — S14F1/F2 GetAttr + S14F3/F4 SetAttr
The catalog had S14F9/F10 / F11/F12 specialized for E94 ControlJob;
this commit adds the generic E14 attribute access pair, the most-
queried half of the E39 surface area, backed by the CemObjectStore.
S14F1 / F2 GetAttr — OBJSPEC + OBJTYPE + ATTRID list ->
(ATTRID, VALUE) pairs + OBJACK
S14F3 / F4 SetAttr — same addressing, applies ATTRID/VALUE pairs,
reply echoes stored values + OBJACK
Server dispatches both into the CemObjectStore added in tranche G.
OBJTYPE validation is case-sensitive against the CemObjectType name
(Equipment / Subsystem / IODevice / Module / MaterialLocation).
Unknown objects return Denied_UnknownObject; type mismatches return
Denied_InvalidAttribute.
The shared AttrValue struct is declared external_struct: true on
F3/F4 so both directions share the same C++ type.
Two round-trip tests cover both pairs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
3b45bade8f |
M: S16F7/F8 PRJobMonitor + S16F15/F16 PRJobCreateMultiple
Closes the two E40 bulk/control gaps the COMPLIANCE doc had flagged
as out-of-scope:
S16F7 / F8 PRJobMonitor — host enables/disables S16F9 alerts
per PJ. PRALERT bit 7 is the enable flag (matches the
ALED convention from S5F3). Server dispatches into the
existing set_alert() store API.
S16F15 / F16 PRJobCreateMultiple — bulk create variant. Host posts
a list of (PRJOBID, PPID, MTRLOUTSPEC) entries; the
equipment processes them in order and returns a
per-PJ HCACK list so the host can identify which
subset failed. Same validators as S16F11.
Catalog now has three new structs: PRJobMonitorEntry,
PRJobCreateEntry, PRJobCreateMultiResult. Two round-trip tests cover
the new wire shapes; server-side correctness is exercised through the
existing PJ store invariants (dedup, validator) which both new paths
delegate to.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
5a3f5ca6da |
L: E87 slot-map verification wire (S3F19/F20 + F21/F22)
Closes the slot-map verification gap I called out:
S3F19 / F20 host -> equip: verify expected slot map against what
the equipment has scanned. Equipment compares element-
wise; on match drives CSMS NotRead -> Read and replies
SVACK=Accept; on mismatch drives CSMS -> Mismatched and
replies SVACK=Mismatch.
S3F21 / F22 equip -> host: equipment-initiated slot map report
(typically pushed after CARRIERID is confirmed).
New SVACK enum: SlotMapVerifyAck { Accept, Mismatch, CarrierUnknown,
Error }. Server dispatch on S3F19 wires the actual CSMS transition
through the CarrierStore from D3.
Two round-trip tests cover both pairs; the FSM-driving behaviour is
exercised through the in-process tests because secs_server.cpp is
the dispatch entry point (no separate integration test needed beyond
the wire round-trip).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
5f1444c756 |
D2: E87 wire messages (S3F17/F18 + F23/F24 + F25/F26 + F27/F28)
Adds the Carrier Management wire surface that drives the FSMs from D1:
S3F17 / S3F18 CarrierAction (host issues ProceedWithCarrier /
CancelCarrier / BindCarrierID / etc.; CommandParameter
list is reused from S2F41).
S3F23 / S3F24 PortGroupChangeReport (equipment notifies host of port
group composition changes).
S3F25 / S3F26 CarrierTransfer (host instructs source -> target port
transfer).
S3F27 / S3F28 CancelCarrier (host cancels an outstanding carrier op).
Two new ack enums in messages_helpers.hpp:
CarrierActionAck — CAACK byte; covers the common error responses
(CarrierIDUnknown, Inaccessible, ActionInProgress).
PortGroupAck — PGACK byte (Accept / Error).
Round-trip tests for all four pairs. Server-side dispatch + the
CarrierStore + LoadPortStore that the FSMs key into land in D3/D4.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
ec478ac9cb |
A3: S12 Wafer Maps stream (E5 §13)
Adds the core map-management messages: setup send/request (F1-F4), transmit inquire/grant (F5/F6), data send in row format (F7/F8), and the one-way error report (F19). Reference points (REFP) are an external struct shared across F1 and F4. The alternative data encodings — array format (F9/F10), coordinate format (F11/F12), and the corresponding request pairs (F13-F18) — are scope-deferred. They're mechanical YAML edits once a tool needs them; the codec already handles the underlying BINARY/list shapes. Three new ack enums: MapSetupAck (SDACK), MapTransmitGrant (GRANT), MapDataAck (MAPER). No state machine yet — maps are a data exchange, not a lifecycle. secsgem-py ships S12F0-F19 as a single block; this commit covers the practically-used subset and matches their wire shapes where they overlap. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
fafbd2abd2 |
A2: S2F49/F50 Enhanced Remote Command
Adds the OBJSPEC-scoped sibling of S2F41 with extended per-parameter ack shape (CPACK + CEPACK). Wire: S2F49 body <DATAID OBJSPEC RCMD <L,n <CPNAME CPVAL>>> S2F50 body <HCACK <L,n <CPNAME CPACK CEPACK>>> Server delegates to the existing HostCommandRegistry, logs OBJSPEC for audit, and currently returns empty cpacks (all-OK). Per-parameter failures will be wired when the command registry grows CEPACK-level validation; this commit is the catalog + dispatch scaffolding. secsgem-py defines these in its catalog but never dispatches them; this puts the C++ port marginally ahead on remote-command coverage. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
b59c62bbc9 |
A1: S5F13-F18 exception recovery messages (E5 §9.5-9.7)
Catalog gains the recover-request / recover-complete / recover-abort loop that closes E5's exception lifecycle. Wire shapes: S5F13/F14 Exception Recover Request / Acknowledge S5F15/F16 Exception Recover Complete Notify / Acknowledge S5F17/F18 Exception Recover Abort Request / Acknowledge Acks use the same AlarmAck byte already in use by S5F10/F12. EXRESULT on F15 is modelled as ASCII (the common case; vendor-specific richer shapes are a YAML edit). Round-trip tests cover all three pairs. Server dispatch is left for a later commit alongside the per-alarm AlarmStateMachine (Tranche C) — this commit is wire-coverage only. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
90c177b7ce |
E40 Process Jobs + E94 Control Jobs + E30 communication state
GEM300 layer: SEMI E40-0705 Process Job and E94-0705 Control Job state machines, plus the E30 §6.1 communication-state machine that sits between HSMS SELECT and full GEM communication. Data-driven via data/process_job_state.yaml and data/control_job_state.yaml, mirroring the existing control_state.yaml pattern. Wire coverage: S14F9/F10 CreateObject (CJ) host -> equipment S14F11/F12 DeleteObject (CJ) host -> equipment S16F5/F6 PRJobCommand host -> equipment S16F9 PRJobAlert equipment -> host S16F11/F12 PRJobCreate (simplified body) host -> equipment S16F13/F14 PRJobDequeue host -> equipment S16F27/F28 CJobCommand host -> equipment Process Job FSM exposes 8 states matching PRJOBSTATE bytes (E40 §10.3.2); HOQ is reorder-aware (move-to-head against an insertion-order vector); Stop/Abort on a Queued PJ routes through ABORTING so the host observes PRJOBSTATE=7 on the wire (§6.3); alert_enabled is settable per-PJ for PRALERT control; FSM dispatches through ProcessJobStore::on_change_ dynamically so a late set_state_change_handler() reaches existing PJs. Hardening: loader rejects NoState (sentinel) as initial/from/to and rejects `on: created` rows; static_asserts pin enum values to wire bytes; ProcessJobStore is non-movable to keep the per-PJ this-capture safe. Server simulator cascades the full CJ -> PJ lifecycle on CJSTART so the wire trace exercises every legal state. CEIDs 400/401 fire on CJ state changes via the existing event-report pipeline. Tests: 60+ new assertions across test_process_jobs, test_control_jobs, test_communication_state, test_hsms_connection, plus loader and messages round-trip coverage. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
1f67aad985 |
100%/F: S10F5/F6 multi-line + honest 100% in COMPLIANCE.md + README pass
tests / build-and-test (push) Failing after 33s
The final additions: S10F5/F6 multi-line terminal display (closes the last partial Additional capability — Equipment Terminal Services flips ✅), and a thorough COMPLIANCE.md / README pass that states the 100% claim honestly. Catalog + handlers data/messages.yaml S10F5 / S10F6 added. apps/secs_server.cpp router.on(10, 5) iterates the line list, acks with S10F6. tests/test_messages.cpp Round-trips a 3-line multi-line display. COMPLIANCE.md (rewritten) Every GEM Fundamental ✅. Every GEM Additional that E30 binds to a concrete message set ✅. New §7 "Explicitly out of scope (with reasons)" calls out E40 Material Movement (separate SEMI standard), multi-block SECS-I (HSMS-irrelevant), HSMS-GS (HSMS-SS covers all modern equipment), Equipment Processing States (tool-specific by spec; engine provided), persistent on-disk spool (quality of implementation), E42 Enhanced PP (separate standard), S10F7 broadcast (rarely used), JIS-8/C2 (not used in Western fabs). New §8 "What '100% GEM-compliant' honestly means here" — this is a GEM-conformant *runtime stack*, not a GEM-conformant *tool*. Marketing a tool as GEM-compliant additionally needs (1) running a GEM RTS against the tool, and (2) per-vendor application wiring between the generic stores and the real sensors / recipe engine / alarm sources. README.md (rewritten) Architecture diagram updated to reflect the actual store list (nine stores). "Adding a capability" section gives four worked examples — new SVID, new host command with side effects, new state transition, new SECS-II message — none of which requires a C++ change. Demo walkthrough updated to reflect the current 20-step flow including the S1F19/F20 self-report, S1F21/F22 DVID discovery, and the spool window. Code clarity include/secsgem/gem/data_model.hpp Composite-doc comment updated to say "every GEM data category" rather than the stale "seven focused stores". Verified - Tests: 84 cases / 487 assertions pass. - Demo: 198 server/host log lines; exits 0 end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
1e7105a9e0 |
100%/E: Spool S6F25/F26 + auto-trigger on re-SELECT
tests / build-and-test (push) Failing after 31s
Closes spooling. S6F25 (NUM-MSG) goes into the catalog; S6F26 (ACKC6) likewise. The server's on_selected handler now checks the spool on entering SELECTED — if there's queued data, it auto-emits S6F25 so the host can decide what to do (S6F23 Transmit vs Purge). The happy-path demo never drops TCP so the auto-trigger doesn't fire there, but the canonical re-SELECT path is wired. Client gains a handler for inbound S6F25 that logs the count and acks S6F26. COMPLIANCE.md: Spooling Additional capability flips from 🟡 to ✅. Remaining out of scope for spooling: persistent on-disk spool so restarts don't lose queued events. Demo + tests don't need it; real fab equipment would. Tests: 83 cases / 481 assertions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
6cedaa10dc |
100%/D: Trace Data Collection (S2F23/F24 + S6F1/F2, E30 §6.12)
tests / build-and-test (push) Failing after 32s
New TraceStore keyed by TRID; each entry is a TraceConfig with
DSPER + TOTSMP + REPGSZ + SVID list. S2F23 validates that every SVID
exists (TIAACK=4 otherwise) and registers the trace.
S6F1's body is L,4 of {TRID U4, SMPLN U4, STIME ASCII, list_of <Item>}
— the application chooses whether each value Item is a scalar SVID
value or a packed batch.
The periodic sampling timer that turns an active TraceConfig into
S6F1 emissions is intentionally left to the application (E5 doesn't
mandate a specific scheduler and vendors typically already have one).
Four new SxFy in the catalog.
COMPLIANCE.md: Trace Data Collection Additional capability flips ✅.
Tests: 82 cases / 477 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
224130d99f |
100%/C: Limits Monitoring (S2F45–F48, E30 §6.21)
tests / build-and-test (push) Failing after 35s
New LimitMonitorStore keyed by VID; each entry is a vector of LimitDefinition (LIMITID + upper/lower deadband as arbitrary Items). S2F45/F46 set, S2F47/F48 read. VLAACK validates each VID exists. Four new SxFy in the catalog; codegen handles the nested list-of-(VID, list-of-LimitDefinition) shape. LimitDefinition is defined in store/limits.hpp and referenced as external_struct so the data model and the message codecs share one type. The actual "value crossed limit" detection + CEID emission is left to the application's set_value path (E30 §6.21 leaves *how* the equipment detects crossings up to the implementer). COMPLIANCE.md: Limits Monitoring Additional capability flips ✅. Tests: 80 cases / 465 assertions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
88205037ec |
100%/A: S5F7/F8 list enabled alarms
tests / build-and-test (push) Failing after 32s
Closes the small remaining hole in Alarm Management. S5F7 is header-only; S5F8 has the same wire shape as S5F6 (vector of <L,3 <B ALCD> <U4 ALID> <A ALTX>>) but only includes alarms whose enabled flag is set. Codegen handles both as a list_of with struct_name=AlarmListing; server pre-computes the per-row ALCD from (severity_category, active state) before passing the rows in. COMPLIANCE.md: Alarm Management Additional capability flips ✅. Tests: 78 cases / 454 assertions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
813e011409 |
Close COMPLIANCE.md gap: Documentation (S1F19-F22)
tests / build-and-test (push) Failing after 32s
Adds the GEM "Documentation" Fundamental capability: the equipment now
self-reports which GEM capabilities it supports, and the host can
discover the DVID namelist with the same shape used for SVIDs.
Catalog (data/messages.yaml -> generated messages.hpp)
S1F19 W header-only Get GEM Compliance Request
S1F20 <L,3 <A SOFTREV>
<A EQPTYP>
<L,a <L,2 <U1 CCODE> <A CDESC>>>>
Get GEM Compliance Data
S1F21 W <L,n <U4 VID>> DVID Namelist Request (n=0 = all)
S1F22 <L,n <L,3 <U4 VID>
<A VNAME>
<A UNITS>>> DVID Namelist Data
Codegen emits CapabilityEntry and GemCompliance structs. S1F22 reuses
S1F12's StatusName struct (same wire shape; dedup avoids redefinition).
Equipment data dictionary (data/equipment.yaml)
device: Adds `equipment_type: "EQUIPMENT"`
for the S1F20 EQPTYP field.
capabilities: New section. List of
- {code, name} (CCODE, CDESC) pairs honestly
reflecting what the codebase
implements: 1, 2, 3, 5, 6, 7, 8,
9, 11, 12, 14 (partial), 15.
dvids: New section, same schema as
svids:. Demo populates two:
- WaferCounter (U4, units wafer)
- ChamberPressure (F4, units Torr)
Loader (src/config/loader.cpp + include/secsgem/config/loader.hpp)
EquipmentDescriptor gains equipment_type and capabilities (vector of
(uint8_t, string) pairs). load_equipment now reads `capabilities:`
into the descriptor and `dvids:` into model.dvids.
Server (apps/secs_server.cpp)
router.on(1, 19) returns S1F20 with desc.software_rev,
desc.equipment_type, and desc.capabilities converted to
vector<CapabilityEntry>.
router.on(1, 21) returns S1F22 built from model.dvids.all().
Client (apps/secs_client.cpp)
Two new demo steps after Request Online and before SVID discovery:
S1F19 -> S1F20: logs SOFTREV, EQPTYP, and every (CCODE, CDESC)
the equipment claims.
S1F21 -> S1F22: logs each DVID with units.
Tests
tests/test_messages.cpp Round-trip S1F19/F20 with a 3-entry
capability list; round-trip S1F22 with two
DVIDs.
tests/test_loader.cpp Asserts equipment_type, the capabilities
list contains CCODE 14 (Spooling), and the
two DVIDs land in model.dvids.
COMPLIANCE.md
"Documentation" Fundamental moves from ⬜ to ✅.
S1F19/F20 + S1F21/F22 rows in the coverage matrix flip to ✅.
The "what would it take" list drops the documentation-messages bullet.
Verified
- Tests: 77 cases / 444 assertions pass.
- Demo: client logs the full capability list received from the
equipment, including CCODE 14 "Spooling (partial; S2F43/F44 +
S6F23/F24)" — the equipment honestly reports its partial
implementation rather than overclaiming.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
547fd2116b |
Close COMPLIANCE.md gap: S9 error stream
tests / build-and-test (push) Failing after 43s
Adds S9F1, F3, F5, F7, F9, F11, F13 to the message catalog and wires
the two emission paths that the Connection layer can drive without help
from the Router or the application: S9F7 on a body-decode failure and
S9F9 on a T3 transaction-timer timeout.
Catalog (data/messages.yaml -> generated messages.hpp)
All six MHEAD-carrying messages (F1/F3/F5/F7/F9/F11) use the same
shape — a single <B 10> body with the offending 10-byte HSMS header.
S9F13 (conversation timeout) carries <L,2 <A MEXP> <A EDID>>.
Connection-side emissions (src/hsms/connection.cpp)
emit_s9(function, mhead) New private helper. Builds a 9/function/W=0
data message whose body is <B 10> with the
MHEAD bytes, allocates a fresh sys_bytes,
and queues it onto the write path. No
reply is tracked.
S9F7 on body decode handle_data wraps Message::from_body in a
try/catch. Previously any decode error
closed the connection; now it emits S9F7
with the offending header and continues
reading. Reply-side decode failure also
emits S9F7 and surfaces the new
Error::IllegalData to the waiting
ReplyHandler (rather than making the
caller wait out T3).
S9F9 on T3 timeout The send_request T3 callback rebuilds the
original outgoing MHEAD from
(device_id, expected_stream,
expected_function-1, sys, W=1) and emits
S9F9 before invoking the callback with
Error::Timeout (unchanged).
What's intentionally not yet wired (logged in COMPLIANCE.md)
- S9F3 / S9F5 — "unknown stream / function". These need to live in
the Router's fallback path, which would require either the Router
knowing about a Connection-shaped sender or the Connection's
message wrapper learning which streams the Router has handlers
for. Deferred — today the fallback returns SxF0 only.
- S9F11 — "Data Too Long". Currently we close on oversized frames;
we'd need to also build a synthetic 10-byte MHEAD substitute (the
real header isn't yet available at the point of detection) and
flush it through close_after_flush.
Tests + docs
tests/test_messages.cpp Round-trip every S9F* using a representative
10-byte MHEAD literal; check S9F13 carries
MEXP + EDID. +2 cases / +37 assertions.
COMPLIANCE.md Error Messages row moved from "no S9 stream"
to a detailed status describing what's
emitted vs catalog-only. Coverage matrix
expanded per-message (F1/F7/F9/F13 ✅;
F3/F5/F11 🟡 catalog-only).
Build/demo unaffected: 75 cases / 420 assertions pass; the happy-path
demo never trips a decode error or T3, so the S9 path isn't exercised
end-to-end (but unit tests prove the wire shape).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
0721db9542 |
Close COMPLIANCE.md gap: spooling (E30 §6.22)
tests / build-and-test (push) Failing after 42s
Implements the largest functional gap from the compliance audit. The
equipment now queues events the host can't immediately receive (either
because there's no SELECTED session or because the demo's force-spool
flag is on) and transmits the queue on host request.
What's new
include/secsgem/gem/store/spool.hpp
SpoolStore: a deque queue with a configurable per-stream whitelist
(so only streams 5+6 spool by default), a max_size cap with FIFO
eviction on overflow, and a `force_spool` test flag. Enqueue
returns one of Queued / Dropped_NotSpoolable / Dropped_Full so the
caller can fall back to live delivery when appropriate. Drain
pops the entire queue in FIFO order. Two new ack enums:
ResetSpoolAck (S2F44 RSPACK) and SpoolRequestAck (S6F24 RSDA), plus
SpoolRequestCode (S6F23 RSDC, Transmit/Purge).
data/messages.yaml + auto-regenerated messages.hpp
S2F43 W <L,n <B stream>> Reset Spooling
S2F44 <L,2 <B RSPACK> <L,a ...>> Reset Spooling Ack
S6F23 W <B RSDC> Request Spooled Data
S6F24 <B RSDA> Request Spooled Data Ack
data/equipment.yaml
`spool:` section: max_size + spoolable_streams list. Two new host
commands SPOOL_ON / SPOOL_OFF that flip the force-spool flag (these
stand in for "host link down" in the demo without dropping TCP).
include/secsgem/gem/store/host_commands.hpp
Spec/Result gain an optional<bool> force_spool field. S2F41
dispatch returns the result, the server applies it after S2F42 is
queued.
src/config/loader.cpp
Reads `spool:` from equipment.yaml; reads `force_spool` from each
host_commands entry; populates SpoolStore + CommandSpec.
apps/secs_server.cpp
New `deliver_or_spool(msg, what)` helper. emit_event and
emit_alarm_set funnel through it: if force_spool is on (or there's
no active session), msg.stream is checked against the spoolable
list and the message is enqueued; otherwise it's sent live.
Two new handlers:
S2F43 parses the stream list, updates SpoolStore, replies S2F44
S6F23 RSDC=Transmit drains and re-sends each as a fresh primary
(posted on the executor so the S6F24 ack flushes first);
RSDC=Purge clears the queue and acks.
The S2F41 handler now also propagates result.force_spool into the
SpoolStore.
apps/secs_client.cpp
Demo extended with 4 new steps after the FAULT branch:
SPOOL_ON -> S2F42 Accept
START -> S2F42 Accept; CEID 300 emission spooled (no live S6F11)
SPOOL_OFF -> S2F42 Accept; queue still has the message
S6F23(Transmit) -> S6F24 Accept; spooled S6F11 arrives next
Then the existing S7F19/S7F5/S10F1/S1F15/Separate flow continues.
tests/test_data_model.cpp
Four new TEST_CASEs for SpoolStore (whitelist, FIFO eviction at
max_size, drain ordering, force flag).
tests/test_loader.cpp
Confirms equipment.yaml's `spool:` section populates the store and
`force_spool: true/false` flows through to dispatch results.
COMPLIANCE.md
Spooling moves from ⬜ to 🟡. Adds S2F43/F44 + S6F23/F24 as ✅ in
the message coverage matrix; calls out what's still missing
(S6F25/F26 notification, automatic activation on HSMS NOT-SELECTED,
persistent on-disk spool).
Verified
- Tests: 73 cases / 383 assertions pass (+4 spool cases).
- Demo (docker compose up server client) walks the full happy path
and the spool path, observed in the server log as:
spool: force_spool=true (depth=0)
spool: S6F11 CEID=300 queued (depth=1)
spool: force_spool=false (depth=1)
S6F23 transmit: draining 1 messages
and on the host side as the queued S6F11 arriving in the correct
order after S6F24.
Known limitations (logged in COMPLIANCE.md)
- Spool activation is manual via SPOOL_ON/OFF rather than
automatically triggered by HSMS NOT-SELECTED.
- No S6F25/F26 spooled-data-ready notification on re-SELECT.
- In-memory only; an equipment restart loses queued events.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
29db1caedb |
#6 SxFy codegen from YAML message catalog
The bulk of the per-SxFy boilerplate — ~90 hand-written builders and parsers
across 30+ message pairs — is now generated at build time from a single YAML
catalog. Adding a new SECS-II message becomes a YAML edit; the C++ code is
generated, not maintained.
What changed
------------
data/messages.yaml
The catalog. Describes every SxFy currently supported: stream, function,
W-bit, builder name, optional parser name, and a recursive body shape
grammar (scalar / list / list_of). Shapes carry SECS-II item types
(ASCII, BINARY_BYTE, U4, F8, ITEM, ...) and optional C++ enum types for
typed ack codes. Inner-most fields can be marked external_struct: true
so structs already defined elsewhere (ReportData, CommandParameter) are
referenced rather than redefined.
tools/gen_messages.py
Python codegen. Reads the catalog and emits one inline header. Handles
nested shapes via depth-unique variable names in the generated IIFEs, so
S6F11's three-level nesting compiles without lambda capture conflicts.
Post-order traversal ensures inner structs are emitted before outer ones
that reference them. Generates positional and (where applicable) struct
builder overloads, plus struct-returning parsers for messages with a
`parser:` entry.
CMakeLists.txt
Custom command runs gen_messages.py at configure/build time and emits
${CMAKE_BINARY_DIR}/generated/secsgem/gem/messages.hpp. Added to the
secsgem target's include path so `#include "secsgem/gem/messages.hpp"`
resolves to the generated file. Depends on the YAML + the script, so
edits trigger regen automatically.
Dockerfile
Added python3 + python3-yaml to the toolchain image.
include/secsgem/gem/messages_helpers.hpp (new)
The small set of hand-written helpers the generated header relies on:
scalar accessors (as_ascii / as_u4_scalar / ...), parse_u4_list_body,
u4_list_item, ack_byte, ALED byte constants, and the two special-case
messages whose shape doesn't fit the codegen schema (S1F4 needs
per-row std::optional<Item> semantics; S5F6 needs a per-row ALCD
callback).
include/secsgem/gem/messages.hpp (deleted)
The hand-written builder/parser file is gone. Its content now flows
through the catalog + codegen.
include/secsgem/gem/data_model.hpp
Moved CommandParameter to namespace scope so it can be shared between
the data model and the messages.yaml's external_struct entry. Added
`using CommandParam = CommandParameter` for back-compat.
apps/secs_server.cpp + apps/secs_client.cpp
Updated the call sites that the codegen renamed or restructured:
- parse_terminal_display() split into parse_s10f1 / parse_s10f3.
- s1f14_establish_comms_ack now takes a McAck struct for the nested
identity (mdln, softrev) — call site uses brace init.
- S2F33/S2F35 parsers return strongly-typed entries (DefineReportEntry,
LinkEventEntry); the server adapts these to the model's pair-based
API at the call site.
- S2F15 parser returns vector<EcSet>; iterate by .ecid/.value.
- S5F3 parser returns EnableAlarmRequest{aled, alid}; bool comes from
(aled & 0x80) != 0.
- AlarmReport's is_set()/category() methods removed; callers use the
raw alcd byte with bit math (alcd & 0x80, alcd & 0x7F).
- s2f42_host_command_ack and s2f41_host_command always take their
second list argument explicitly (no defaulted arg from codegen).
tests/test_messages.cpp
Updated to construct the generated typed structs (EcSet, StatusName,
EnableAlarmRequest, CommandParameter, CommandParameterAck) and to read
the new field names (.ecid/.value, .rptid/.vids, .ceid/.rptids,
.name/.code).
Coverage
--------
Generated by codegen (44 SxFy in catalog):
S1F1, S1F2, S1F3, S1F11, S1F12, S1F13, S1F14, S1F15, S1F16, S1F17, S1F18
S2F13, S2F14, S2F15, S2F16, S2F17, S2F18, S2F29, S2F30, S2F31, S2F32
S2F33, S2F34, S2F35, S2F36, S2F37, S2F38, S2F41, S2F42
S5F1, S5F2, S5F3, S5F4, S5F5
S6F11, S6F12
S7F3, S7F4, S7F5, S7F6, S7F19, S7F20
S10F1, S10F2, S10F3, S10F4
Hand-written (in messages_helpers.hpp):
S1F4 list-of-optional-items shape (nullopt -> <L,0>)
S5F6 per-row ALCD via callback
Adding a new SxFy
-----------------
Append a single entry to data/messages.yaml describing the body shape.
The builder + parser appear in messages.hpp after the next build. The
host command above for S2F41 (or any other added SxFy) requires no C++
changes if the body fits the recursive scalar/list/list_of grammar.
Tests: 67 cases / 384 assertions still passing.
Demo: byte-for-byte identical behaviour (Select, Establish, Online,
S1F11/F3 namelist+values, S2F29 EC namelist, S2F33/F35/F37 dynamic event
subscription, S2F41 START -> S6F11 emission, S5F5/F3 alarm directory +
enable, S2F41 FAULT -> S5F1 alarm + S6F11, S7F19/F5 recipe ops, S10F1
terminal, S1F15 offline, Separate).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|