docs: chapters 30–36 — the codebase (Part 3 complete)

Seven chapters walking the implementation top-to-bottom. 30 — Repository tour. Top-level layout, directory by directory. The eight built binaries. The dependency graph from TCP socket up through EquipmentDataModel. CMake's role. Test layout. 31 — Spec-as-data and codegen. Why the design choice fits SECS/ GEM specifically. The five YAML files: messages catalog, control/PJ/CJ transition tables, equipment dictionary. How tools/gen_messages.py turns messages.yaml into typed C++ at build time. The --validate-config multi-error validator. How to add a new SVID / CEID / host command / state / message without C++. 32 — Stores and the data model. What a store IS (records + API + change handler + optional persistence). Every store in the codebase mapped to the SEMI standard it serves (table of 21). EquipmentDataModel as plain composition + cross-store convenience methods (vid_value, compose_reports_for). The no-locks single- threaded contract. How to add a new store. 33 — Transport. hsms::Connection read path (length+payload async chain), write path (queue + one outstanding write), timer model (5 steady_timers + per-request T3). The asio executor / strand model and why it's the right shape. secsi::Protocol as the IO- free FSM with Action / Event variants; secsi::TcpTransport as the asio adapter. Pattern repeats for E84 + GEM comm-state. 34 — Codec and SML. The four files (170 + 30 + 52 + 32 lines of header, 229 + 220 lines of impl). Item variant storage layout (11 alternatives, 16 formats, shared storage where E5 permits). encode_into recursion; decode_at with bounds checks throwing CodecError. Message wrapper. SML printer + try_parse_sml + why SML round-trips Items but not necessarily bytes. 35 — State machines and dispatch. gem::Router as a typed (stream, function) dispatch table. How an S2F41 round-trip walks through parser → store dispatch → side-effect → CEID emission → S6F11 build → spool-aware deliver. The 11 FSMs all sharing the same three-property shape (pure data table + pure FSM + observer pattern). CEID cascading from FSM transitions to wire bytes. 36 — Persistence, validation, metrics. Which 7 stores have file journals + why the others don't. Per-record file pattern (atomic rename, partial-write safe). Schema versioning + multi-version read. Multi-error YAML validator (--validate-config) + cross-file reference checks. Prometheus registry + HTTP exporter + worked metric patterns from the PVD example. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 20:23:05 +02:00
parent 40df3067a4
commit cae98d9a7d
7 changed files with 2127 additions and 0 deletions
@@ -0,0 +1,305 @@
+# 35 — State machines and dispatch
+
+← [34 Codec and SML](34_codec_and_sml.md) | [Back to index](00_index.md) | Next: [36 Persistence, validation, metrics](36_persistence_validation_metrics.md) →
+
+We have transport (chapter 33) delivering `secs2::Message`s
+(chapter 34) into the application.  We have stores (chapter 32)
+holding the data.  This chapter is the **glue**: how the Router
+dispatches by `(stream, function)`, how state machines compose with
+stores, and how a Router handler typically looks.
+
+---
+
+## `gem::Router` — the dispatch table
+
+[`include/secsgem/gem/router.hpp`](../include/secsgem/gem/router.hpp):
+
+```cpp
+class Router {
+ public:
+  using Handler = std::function<std::optional<s2::Message>(const s2::Message&)>;
+
+  Router& on(uint8_t stream, uint8_t function, Handler h);
+  Router& fallback(Handler h);
+
+  std::optional<s2::Message> dispatch(const s2::Message& msg) const;
+};
+```
+
+A `std::map<{stream, function}, Handler>`.  Each handler:
+
+- Takes a `const s2::Message&` (the inbound primary).
+- Returns a `std::optional<s2::Message>` (the reply, or `nullopt`
+  for fire-and-forget primaries).
+
+That's it.  No middleware, no decorator chain, no pre/post hooks.
+Just a typed dispatch.
+
+### Registering handlers
+
+```cpp
+// apps/secs_server.cpp — sketch
+auto router = std::make_shared<gem::Router>();
+
+router->on(1, 1, [model](const auto& m) {
+  // S1F1 Are You There — reply with model name + softrev.
+  return messages::s1f2(model->device.mdln, model->device.softrev);
+});
+
+router->on(1, 3, [model](const auto& m) {
+  // S1F3 — read SVID values.
+  auto svid_list = messages::parse_s1f3(m.body());
+  return messages::s1f4(model->svids.values(svid_list.value_or({})));
+});
+
+router->on(2, 41, [model](const auto& m) {
+  // S2F41 Host Command.
+  auto cmd = messages::parse_s2f41(m.body());
+  auto ack = model->commands.dispatch(cmd->rcmd, cmd->params);
+  return messages::s2f42(ack);
+});
+
+// ...one per S/F pair.  apps/secs_server.cpp registers ~50.
+```
+
+The `examples/pvd_tool/main.cpp` §6 register 51 handlers in ~460
+lines.  Each handler is a few lines: parse the body, mutate or read
+a store, build the reply.
+
+### What happens for unhandled primaries
+
+```cpp
+// router.hpp dispatch()
+auto it = handlers_.find({msg.stream, msg.function});
+if (it != handlers_.end()) return it->second(msg);
+if (fallback_) return fallback_(msg);
+if (msg.reply_expected) return s2::Message(msg.stream, 0, false);  // SxF0 Abort
+return std::nullopt;
+```
+
+Three cases:
+
+1. **Registered**: handler runs.
+2. **Fallback installed**: fallback runs.
+3. **Neither**: if the message expects a reply, send `SxF0` (Abort)
+   per E5 convention.  Otherwise silently drop.
+
+### S9 wiring
+
+The transport layer (chapter 11) emits S9F3 / S9F5 for unhandled
+primaries.  Router exposes the introspection:
+
+```cpp
+bool has_handler(uint8_t stream, uint8_t function) const;
+bool has_handler_for_stream(uint8_t stream) const;
+```
+
+And the wrapper that combines them with S9 emission:
+
+```cpp
+template <typename EmitFn, typename HeaderProvider>
+std::optional<s2::Message> dispatch_with_s9(
+    EmitFn emit_s9, HeaderProvider header_provider,
+    const s2::Message& msg) const {
+  if (!has_handler(msg.stream, msg.function)) {
+    if (auto mhead = header_provider()) {
+      const uint8_t f = has_handler_for_stream(msg.stream) ? 5 : 3;
+      emit_s9(f, *mhead);
+    }
+  }
+  return dispatch(msg);
+}
+```
+
+Used by `Connection::on_data_message` → `Router::dispatch_with_s9`,
+which calls back into `Connection::emit_s9` for the actual S9
+emission.  Tested by
+[`tests/test_s9_fallback.cpp`](../tests/test_s9_fallback.cpp) (2
+cases — unknown stream → S9F3, unknown function in known stream
+→ S9F5).
+
+---
+
+## How a typical handler looks end-to-end
+
+`S2F41` Host Command is the most-touched message in production —
+worth tracing in full:
+
+### 1. The codegen'd parser/builder
+
+```cpp
+// build/generated/secsgem/gem/messages.hpp (auto-generated)
+struct RemoteCommand {
+  std::string rcmd;
+  std::vector<std::pair<std::string, secs2::Item>> params;
+};
+
+inline std::optional<RemoteCommand> parse_s2f41(const secs2::Item& body) {
+  // ...auto-generated body walker...
+}
+
+inline secs2::Message s2f42(uint8_t hcack, /* per-param acks */) {
+  // ...auto-generated builder...
+}
+```
+
+### 2. The Router registration
+
+```cpp
+// apps/secs_server.cpp
+router->on(2, 41, [model](const secs2::Message& m) {
+  auto cmd = messages::parse_s2f41(m.body());
+  if (!cmd) {
+    // Body didn't parse — reply S2F42 with HCACK = 1 (invalid).
+    return messages::s2f42(1, {});
+  }
+  auto outcome = model->commands.dispatch(cmd->rcmd, cmd->params);
+  return messages::s2f42(static_cast<uint8_t>(outcome.ack), outcome.cpacks);
+});
+```
+
+### 3. The store dispatch
+
+```cpp
+// include/secsgem/gem/store/host_commands.hpp
+class HostCommandRegistry {
+ public:
+  CommandOutcome dispatch(const std::string& rcmd, const ParamList& params) {
+    auto it = commands_.find(rcmd);
+    if (it == commands_.end()) return {HostCmdAck::InvalidCommand, ...};
+    const auto& cmd = it->second;
+    // Apply configured side effects: emit_ceid, set_alarm, …
+    for (auto ceid : cmd.emit_ceids) on_emit_ceid_(ceid);
+    for (auto alid : cmd.set_alarms) alarm_registry_->set(alid);
+    return {cmd.default_ack, ...};
+  }
+};
+```
+
+### 4. The side-effect dispatcher
+
+Steps in `dispatch` like `on_emit_ceid_(ceid)` call back into
+the EAP:
+
+```cpp
+// Set up at startup:
+model->commands.set_emit_ceid_handler([conn, model](uint32_t ceid) {
+  if (!model->is_event_enabled(ceid)) return;
+  auto reports = model->compose_reports_for(ceid);
+  auto msg = build_s6f11(ceid, reports);
+  conn->send_data(std::move(msg));
+});
+```
+
+### 5. The wire
+
+`conn->send_data(s6f11)` walks through `secs2::encode` →
+`hsms::Frame::encode` → `async_write` to the socket.  Host sees
+the unsolicited S6F11.
+
+---
+
+## The state-machine pattern
+
+Every state machine in the codebase follows the same shape.  Pick
+`ControlStateMachine`:
+
+```cpp
+// include/secsgem/gem/control_state.hpp
+class ControlStateMachine {
+ public:
+  ControlStateMachine(ControlTransitionTable table);
+
+  ControlState state() const;
+
+  // Apply an event; returns the transition row (if any).
+  const ControlTransition* on_event(ControlEvent e);
+
+  // Observer for transitions.
+  using StateChangeHandler =
+      std::function<void(ControlState from, ControlState to, ControlEvent)>;
+  void set_state_change_handler(StateChangeHandler h);
+};
+```
+
+Three properties:
+
+1. **Pure data table** (`ControlTransitionTable`) decides what
+   transitions exist.
+2. **Pure FSM** (`ControlStateMachine`) applies events against the
+   table, updates state, emits the change.
+3. **Observer pattern** — the EAP registers a change handler that
+   does the wire-level work (fire CEID, emit S6F11, log to
+   metrics).
+
+This pattern repeats for:
+
+- `ProcessJobStateMachine` (E40)
+- `ControlJobStateMachine` (E94)
+- `EptStateMachine` (E116)
+- `ExceptionStateMachine` (E5 §13)
+- `CarrierStateMachine` (E87 — actually composes 3 sub-FSMs)
+- `LoadPortStateMachine` (E87 — same)
+- `SubstrateStateMachine` (E90 — same)
+- `ModuleStateMachine` (E157)
+- `E84StateMachine` (E84)
+- `CommunicationStateMachine` (E30 §6.5)
+
+Eleven FSMs.  All follow the same shape.  All testable in
+isolation without IO.
+
+---
+
+## How FSM transitions cascade into CEIDs
+
+A common need: "when PJ-1 transitions to Processing, fire
+CEID=ProcessStarted, which fires an S6F11 with linked reports."
+
+The wiring is set up at startup in the EAP:
+
+```cpp
+// apps/secs_server.cpp — sketch
+model->process_jobs.set_state_change_handler(
+    [conn, model](const std::string& pjid,
+                  ProcessJobState from, ProcessJobState to,
+                  ProcessJobEvent ev) {
+      // Configured per-state CEID from data/equipment.yaml.
+      auto ceid = ceid_for_pj_state(to);
+      if (!ceid || !model->is_event_enabled(*ceid)) return;
+      auto reports = model->compose_reports_for(*ceid);
+      auto msg = build_s6f11(*ceid, reports);
+      deliver_or_spool(*conn, *model, std::move(msg));
+    });
+```
+
+`deliver_or_spool` is the spool-aware send: if the connection isn't
+SELECTED (or the spool is in transmit-disabled mode), the message
+queues into `SpoolStore`; otherwise it goes straight to the wire.
+
+So the chain for "PJ-1 starts processing":
+
+```
+ProcessJobStore.apply(pjid, Start)
+  → ProcessJobStateMachine.on_event(Start)
+  → State: WaitingForStart → Processing
+  → on_change handler fires
+  → looks up CEID ProcessStarted
+  → composes reports
+  → builds S6F11 message
+  → deliver_or_spool → connection or SpoolStore
+  → on the wire (eventually)
+```
+
+Every step is independently testable.  Tests at the wire level:
+[`tests/test_wire_ceid_emission.cpp`](../tests/test_wire_ceid_emission.cpp)
+(6 cases — every cascade from store mutation to socket bytes).
+
+---
+
+## Where to go next
+
+You've now seen everything that makes the runtime *work*.  One
+chapter left in Part 3: the **operational** concerns — persistence,
+config validation, and metrics.
+
+Next: [→ 36 Persistence, validation, metrics](36_persistence_validation_metrics.md)