The bulk of the per-SxFy boilerplate — ~90 hand-written builders and parsers
across 30+ message pairs — is now generated at build time from a single YAML
catalog. Adding a new SECS-II message becomes a YAML edit; the C++ code is
generated, not maintained.
What changed
------------
data/messages.yaml
The catalog. Describes every SxFy currently supported: stream, function,
W-bit, builder name, optional parser name, and a recursive body shape
grammar (scalar / list / list_of). Shapes carry SECS-II item types
(ASCII, BINARY_BYTE, U4, F8, ITEM, ...) and optional C++ enum types for
typed ack codes. Inner-most fields can be marked external_struct: true
so structs already defined elsewhere (ReportData, CommandParameter) are
referenced rather than redefined.
tools/gen_messages.py
Python codegen. Reads the catalog and emits one inline header. Handles
nested shapes via depth-unique variable names in the generated IIFEs, so
S6F11's three-level nesting compiles without lambda capture conflicts.
Post-order traversal ensures inner structs are emitted before outer ones
that reference them. Generates positional and (where applicable) struct
builder overloads, plus struct-returning parsers for messages with a
`parser:` entry.
CMakeLists.txt
Custom command runs gen_messages.py at configure/build time and emits
${CMAKE_BINARY_DIR}/generated/secsgem/gem/messages.hpp. Added to the
secsgem target's include path so `#include "secsgem/gem/messages.hpp"`
resolves to the generated file. Depends on the YAML + the script, so
edits trigger regen automatically.
Dockerfile
Added python3 + python3-yaml to the toolchain image.
include/secsgem/gem/messages_helpers.hpp (new)
The small set of hand-written helpers the generated header relies on:
scalar accessors (as_ascii / as_u4_scalar / ...), parse_u4_list_body,
u4_list_item, ack_byte, ALED byte constants, and the two special-case
messages whose shape doesn't fit the codegen schema (S1F4 needs
per-row std::optional<Item> semantics; S5F6 needs a per-row ALCD
callback).
include/secsgem/gem/messages.hpp (deleted)
The hand-written builder/parser file is gone. Its content now flows
through the catalog + codegen.
include/secsgem/gem/data_model.hpp
Moved CommandParameter to namespace scope so it can be shared between
the data model and the messages.yaml's external_struct entry. Added
`using CommandParam = CommandParameter` for back-compat.
apps/secs_server.cpp + apps/secs_client.cpp
Updated the call sites that the codegen renamed or restructured:
- parse_terminal_display() split into parse_s10f1 / parse_s10f3.
- s1f14_establish_comms_ack now takes a McAck struct for the nested
identity (mdln, softrev) — call site uses brace init.
- S2F33/S2F35 parsers return strongly-typed entries (DefineReportEntry,
LinkEventEntry); the server adapts these to the model's pair-based
API at the call site.
- S2F15 parser returns vector<EcSet>; iterate by .ecid/.value.
- S5F3 parser returns EnableAlarmRequest{aled, alid}; bool comes from
(aled & 0x80) != 0.
- AlarmReport's is_set()/category() methods removed; callers use the
raw alcd byte with bit math (alcd & 0x80, alcd & 0x7F).
- s2f42_host_command_ack and s2f41_host_command always take their
second list argument explicitly (no defaulted arg from codegen).
tests/test_messages.cpp
Updated to construct the generated typed structs (EcSet, StatusName,
EnableAlarmRequest, CommandParameter, CommandParameterAck) and to read
the new field names (.ecid/.value, .rptid/.vids, .ceid/.rptids,
.name/.code).
Coverage
--------
Generated by codegen (44 SxFy in catalog):
S1F1, S1F2, S1F3, S1F11, S1F12, S1F13, S1F14, S1F15, S1F16, S1F17, S1F18
S2F13, S2F14, S2F15, S2F16, S2F17, S2F18, S2F29, S2F30, S2F31, S2F32
S2F33, S2F34, S2F35, S2F36, S2F37, S2F38, S2F41, S2F42
S5F1, S5F2, S5F3, S5F4, S5F5
S6F11, S6F12
S7F3, S7F4, S7F5, S7F6, S7F19, S7F20
S10F1, S10F2, S10F3, S10F4
Hand-written (in messages_helpers.hpp):
S1F4 list-of-optional-items shape (nullopt -> <L,0>)
S5F6 per-row ALCD via callback
Adding a new SxFy
-----------------
Append a single entry to data/messages.yaml describing the body shape.
The builder + parser appear in messages.hpp after the next build. The
host command above for S2F41 (or any other added SxFy) requires no C++
changes if the body fits the recursive scalar/list/list_of grammar.
Tests: 67 cases / 384 assertions still passing.
Demo: byte-for-byte identical behaviour (Select, Establish, Online,
S1F11/F3 namelist+values, S2F29 EC namelist, S2F33/F35/F37 dynamic event
subscription, S2F41 START -> S6F11 emission, S5F5/F3 alarm directory +
enable, S2F41 FAULT -> S5F1 alarm + S6F11, S7F19/F5 recipe ops, S10F1
terminal, S1F15 offline, Separate).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
secs-gem
A C++20 SECS-II / HSMS / GEM client + server toolchain, fully containerized.
The long-term vision is the "spec-as-data" stack described in
implementation_plan.md: the SEMI E30 behavioral
spec encoded as transition tables, from which a runtime, passive analyzer,
simulator, and conformance test generator are derived. The current code is
the Layer 0 + Layer 1 base — a working HSMS client/server, the SECS-II
codec, the E30 control state machine, and the GEM core capabilities (status
variables, equipment constants, dynamic event reporting, alarms, remote
control, clock, process programs, terminal services).
For a per-capability E5/E30/E37 audit see COMPLIANCE.md.
Quick start
Everything runs in Docker — no compiler or build tools on the host.
docker compose run --rm builder # configure + compile
docker compose run --rm tests # run unit tests (63 cases / 278 assertions)
docker compose up --no-deps server client # live two-container demo
Architecture
The "spec-as-data" first step: equipment capabilities and the E30 control
state machine are loaded from YAML at startup; the SECS dispatch is a
(stream, function) -> handler Router rather than an if-ladder.
┌──────────────────────────────────────────────────────────────┐
│ data/equipment.yaml data/control_state.yaml │
│ SVIDs, ECIDs, CEIDs, alarms, recipes, host commands; │
│ E30 control state transition table │
└──────────────────────┬───────────────────────────────────────┘
│ (loaded at startup)
▼
┌──────────────────────────────────────────────────────────────┐
│ app / demo │
│ apps/secs_server.cpp apps/secs_client.cpp │
│ uses gem::Router for SECS dispatch │
└────────────┬───────────────────────────┬─────────────────────┘
│ │
▼ ▼
┌──────────────────────────────────────────────────────────────┐
│ secsgem::config loader.hpp YAML -> tables + data model │
│ secsgem::gem control_state (table-driven), data_model, │
│ messages (SxFy builders), router │
│ secsgem::hsms Connection (Asio), Header, Frame, Timers │
│ secsgem::secs2 Item, codec (encode/decode), Message │
└──────────────────────────────────────────────────────────────┘
Tree
secs-gem/
├── Dockerfile, docker-compose.yml # toolchain + demo
├── CMakeLists.txt
├── implementation_plan.md # the 7-layer spec-as-data roadmap
├── COMPLIANCE.md # per-capability E5/E30/E37 audit
├── data/
│ ├── equipment.yaml # SVIDs/ECIDs/CEIDs/alarms/recipes/commands
│ └── control_state.yaml # E30 transition table
├── include/secsgem/
│ ├── secs2/{item,codec,message}.hpp
│ ├── hsms/{header,connection}.hpp # header.hpp also holds Frame + Timers
│ ├── gem/{control_state,data_model,messages,router}.hpp
│ ├── config/loader.hpp
│ └── endpoint.hpp
├── src/{secs2,hsms,gem,config}/*.cpp + endpoint.cpp
├── apps/{secs_server,secs_client}.cpp
└── tests/test_*.cpp
Adding a capability without recompiling the server
# data/equipment.yaml — append a new SVID
svids:
- {id: 4, name: ChamberTemp, units: "C", type: U4, value: 25}
# data/equipment.yaml — append a new host command + side effect
host_commands:
- {name: VENT, ack: Accept, emit_ceid: 400, set_alarm: 2}
# data/control_state.yaml — append a new transition
transitions:
- {from: OnlineRemote, on: host_request_offline, to: EquipmentOffline, ack: Accept}
Restart the server; the new behaviour is live. The C++ code unchanged.
What's implemented
HSMS (E37)
SELECT/DESELECT/LINKTEST/SEPARATE/REJECT control messages, the 4-byte length-prefixed frame, the 10-byte header, the NOT-CONNECTED → SELECTED state machine, and the T3/T5/T6/T7/T8 timers. Active (client) and passive (server) modes. System-bytes based reply correlation.
SECS-II (E5)
A variant-based Item covering every standard format (List, ASCII,
Binary, Boolean, I1–I8, U1–U8, F4, F8), big-endian numerics, 1/2/3-byte
length encoding, and an to_sml() rendering for readable logs.
GEM (E30)
- Control state machine — 5 states (EquipmentOffline, AttemptOnline, HostOffline, OnlineLocal, OnlineRemote), operator actions, host requests with SEMI-mandated ACK codes, change-handler callback.
- Status variables — S1F3/F4, S1F11/F12.
- Equipment constants — S2F13/F14, S2F15/F16, S2F29/F30 + EAC enum.
- Clock — S2F17/F18, S2F31/F32 (16-char
YYYYMMDDhhmmsscc). - Establish Comms — S1F13/F14 +
CommAck. - Online/Offline — S1F15/F16 +
OfflineAck, S1F17/F18 +OnlineAck. - Remote control — S2F41/F42 + 7-value
HostCmdAck+ per-param CPACKs. - Dynamic event reporting — S2F33 Define Report (DRACK), S2F35 Link Event (LRACK), S2F37 Enable (ERACK), S6F11 emission, S6F12 ack.
- Alarms — S5F1/F2 emit + ACKC5, S5F3/F4 enable/disable, S5F5/F6 list, ALCD bit-7 set/cleared, lower-7 category.
- Process programs — S7F3/F4 send, S7F5/F6 request, S7F19/F20 list.
- Terminal services — S10F1/F2 host→equipment, S10F3/F4 equipment→host.
Not (yet) implemented
Spooling, S9 error stream, S1F19/F20 + S1F21/F22 documentation, multi-block, trace data collection (S2F23/F24, S6F1/F2), limits monitoring, material movement (E40), EC range validation, S5F7/F8 list-enabled-alarms, S6F15/F16 event-report request. See COMPLIANCE.md for the honest accounting.
Demo session
The demo client walks 17 distinct steps end-to-end. From the live log (condensed):
[host] -> Select.req [equip] <- Select.req
[host] <- Select.rsp (Ok) [equip] -> Select.rsp (Ok)
[host] == SELECTED == [equip] == SELECTED ==
[host] -> S1F13 W [equip] S1F13; replying S1F14 (COMMACK=0)
[host] -> S1F17 W [equip] control: HostOffline -> AttemptOnline
[equip] control: AttemptOnline -> OnlineRemote
[host] S1F18 ONLACK=0 [equip] -> S10F3 W (welcome)
[host] TERMINAL[0] Welcome from SECSGEM-SIM
[host] -> S1F11 W [equip] S1F11; replying S1F12 (status namelist)
[host] SVID 1 ControlState
[host] SVID 2 Clock
[host] SVID 3 EventsEnabled
[host] -> S1F3 W [equip] S1F3; replying S1F4 (3 values)
[host] -> S2F29 W [equip] -> S2F30 (2 EC entries)
[host] -> S2F33 W [equip] -> S2F34 (DRACK=0)
[host] -> S2F35 W [equip] -> S2F36 (LRACK=0)
[host] -> S2F37 W [equip] -> S2F38 (ERACK=0)
[host] -> S2F41 W START [equip] RCMD START; emit S6F11 CEID=300
[host] EVENT CEID=300 (1 reports)
[host] RPTID 1000: <A "OnlineRemote"> <A "2026060122162336"> <BOOLEAN T>
[host] -> S5F5 W [equip] -> S5F6 (2 alarms)
[host] -> S5F3 W [equip] -> S5F4 (ACKC5=0)
[host] -> S2F41 W FAULT [equip] RCMD FAULT triggers alarm 1
[equip] emit S5F1 alarm set ALID=1
[equip] emit S6F11 CEID=200
[host] ALARM SET ALID=1 cat=4 "Chiller Temp High"
[host] EVENT CEID=200 (1 reports)
[host] -> S7F19 W [equip] -> S7F20 (2 PPIDs)
[host] -> S7F5 W [equip] -> S7F6 RECIPE-A
[host] -> S10F1 W [equip] TERMINAL[0] Hello equipment!
[host] -> S1F15 W [equip] control: OnlineRemote -> HostOffline
[host] -> Separate.req [equip] <- Separate.req
[host] exiting
Build details
The toolchain image (Dockerfile) is Ubuntu 24.04 with g++-13, CMake,
Ninja, and libasio-dev. doctest is fetched via CMake FetchContent.
Build artifacts live in a named Docker volume (build:) so the host
filesystem stays clean.
Standalone Asio is used in header-only mode (ASIO_STANDALONE). No Boost
dependency.