docs: chapters 30–36 — the codebase (Part 3 complete)

Seven chapters walking the implementation top-to-bottom. 30 — Repository tour. Top-level layout, directory by directory. The eight built binaries. The dependency graph from TCP socket up through EquipmentDataModel. CMake's role. Test layout. 31 — Spec-as-data and codegen. Why the design choice fits SECS/ GEM specifically. The five YAML files: messages catalog, control/PJ/CJ transition tables, equipment dictionary. How tools/gen_messages.py turns messages.yaml into typed C++ at build time. The --validate-config multi-error validator. How to add a new SVID / CEID / host command / state / message without C++. 32 — Stores and the data model. What a store IS (records + API + change handler + optional persistence). Every store in the codebase mapped to the SEMI standard it serves (table of 21). EquipmentDataModel as plain composition + cross-store convenience methods (vid_value, compose_reports_for). The no-locks single- threaded contract. How to add a new store. 33 — Transport. hsms::Connection read path (length+payload async chain), write path (queue + one outstanding write), timer model (5 steady_timers + per-request T3). The asio executor / strand model and why it's the right shape. secsi::Protocol as the IO- free FSM with Action / Event variants; secsi::TcpTransport as the asio adapter. Pattern repeats for E84 + GEM comm-state. 34 — Codec and SML. The four files (170 + 30 + 52 + 32 lines of header, 229 + 220 lines of impl). Item variant storage layout (11 alternatives, 16 formats, shared storage where E5 permits). encode_into recursion; decode_at with bounds checks throwing CodecError. Message wrapper. SML printer + try_parse_sml + why SML round-trips Items but not necessarily bytes. 35 — State machines and dispatch. gem::Router as a typed (stream, function) dispatch table. How an S2F41 round-trip walks through parser → store dispatch → side-effect → CEID emission → S6F11 build → spool-aware deliver. The 11 FSMs all sharing the same three-property shape (pure data table + pure FSM + observer pattern). CEID cascading from FSM transitions to wire bytes. 36 — Persistence, validation, metrics. Which 7 stores have file journals + why the others don't. Per-record file pattern (atomic rename, partial-write safe). Schema versioning + multi-version read. Multi-error YAML validator (--validate-config) + cross-file reference checks. Prometheus registry + HTTP exporter + worked metric patterns from the PVD example. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 20:23:05 +02:00
parent 40df3067a4
commit cae98d9a7d
7 changed files with 2127 additions and 0 deletions
@@ -0,0 +1,292 @@
+# 33 — Transport
+
+← [32 Stores and the data model](32_stores_and_the_data_model.md) | [Back to index](00_index.md) | Next: [34 Codec and SML](34_codec_and_sml.md) →
+
+We covered the standards-level view of HSMS and SECS-I in
+chapters 11 and 12.  This chapter drops down into the
+implementation: how `hsms::Connection` actually moves bytes,
+the asio executor model, the single-threaded strand contract,
+and why the transport layer doesn't need locks.
+
+---
+
+## The two transport modules
+
+```
+include/secsgem/hsms/
+├── header.hpp        — Frame format primitives (length prefix, header, SType).
+└── connection.hpp    — One-socket session manager + T-timers + S9 emission.
+
+include/secsgem/secsi/
+├── header.hpp        — 10-byte SECS-I block header.
+├── block.hpp         — Block split / assemble (multi-block messages).
+├── protocol.hpp      — IO-free line-turnaround FSM.
+└── tcp_transport.hpp — asio TCP wrapper around the FSM (tunnel for testing).
+```
+
+Each module owns one TCP endpoint (or in the SECS-I case, a tunnel
+endpoint).  Both are **single-threaded by design**.
+
+---
+
+## hsms::Connection — top to bottom
+
+### Lifecycle
+
+```cpp
+// apps/secs_server.cpp — passive-equipment startup
+asio::io_context io;
+
+asio::ip::tcp::acceptor acc(io, asio::ip::tcp::endpoint{asio::ip::tcp::v4(), port});
+
+acc.async_accept([&](std::error_code ec, asio::ip::tcp::socket sock) {
+  auto conn = std::make_shared<hsms::Connection>(
+      std::move(sock), Mode::Passive, /*device_id=*/0, timers);
+  conn->set_message_handler(...);
+  conn->set_closed_handler(...);
+  conn->start();
+});
+
+io.run();   // blocks until all work is done
+```
+
+`Connection::start()` either:
+
+- **Passive** — arms T7 (waiting for Select.req) and starts the
+  read loop.
+- **Active** — initiates the Select.req exchange, then starts the
+  read loop.
+
+### Read path
+
+Three async steps repeated forever:
+
+```
+async_read(socket, 4 bytes) → on_length()
+async_read(socket, length bytes) → on_payload()
+handle_frame(decoded Frame) → dispatch
+```
+
+In code, [`src/hsms/connection.cpp`](../src/hsms/connection.cpp):
+
+```cpp
+void Connection::read_length() {
+  asio::async_read(socket_, asio::buffer(len_buf_, 4),
+      [self = shared_from_this()](std::error_code ec, std::size_t n) {
+        self->on_length(ec, n);
+      });
+}
+
+void Connection::on_length(std::error_code ec, std::size_t n) {
+  if (ec) return close("read_length");
+  uint32_t len = decode_be32(len_buf_);
+  payload_.resize(len);
+  asio::async_read(socket_, asio::buffer(payload_),
+      [self = shared_from_this()](std::error_code ec, std::size_t n) {
+        self->on_payload(ec, n);
+      });
+}
+```
+
+Each callback is on the socket's executor.  No locks because
+nothing else can be touching the read state — by construction.
+
+### Write path
+
+A send queue + one outstanding `async_write`:
+
+```cpp
+void Connection::send_frame(Frame frame) {
+  send_queue_.push_back(std::move(frame));
+  if (send_queue_.size() == 1) write_next();
+}
+
+void Connection::write_next() {
+  auto& frame = send_queue_.front();
+  send_buf_ = frame.encode();
+  asio::async_write(socket_, asio::buffer(send_buf_),
+      [self = shared_from_this()](std::error_code ec, std::size_t) {
+        self->send_queue_.pop_front();
+        if (ec) return self->close("write");
+        if (!self->send_queue_.empty()) self->write_next();
+      });
+}
+```
+
+Same single-threaded discipline — `send_queue_` is only touched
+on the executor.  Callers from other threads must `asio::post`.
+
+### Timers
+
+Five `asio::steady_timer`s, one per HSMS T-timer:
+
+```cpp
+// src/hsms/connection.cpp:30
+Connection::Connection(...)
+  : socket_(std::move(sock)),
+    t3_timer_(socket_.get_executor()),
+    t6_timer_(socket_.get_executor()),
+    t7_timer_(socket_.get_executor()),
+    t8_timer_(socket_.get_executor()),
+    linktest_timer_(socket_.get_executor()),
+    timers_(timers) { }
+```
+
+All five share the socket's executor.  When a timer fires, its
+handler runs on the same executor as the read/write loop — so
+again no locks for timer-vs-IO interaction.
+
+T3 is special: there's one T3 timer per in-flight W=1 message
+(correlated by `system_bytes`).  These are short-lived
+`steady_timer`s allocated when the request is sent and destroyed
+when the reply arrives.  See `src/hsms/connection.cpp:447`:
+
+```cpp
+auto t3 = std::make_shared<asio::steady_timer>(socket_.get_executor());
+t3->expires_after(timers_.t3);
+in_flight_.insert({system_bytes, RequestState{std::move(cb), t3, ...}});
+t3->async_wait([self, system_bytes](std::error_code ec) {
+  if (ec) return;  // cancelled (reply arrived)
+  self->on_t3_expire(system_bytes);
+});
+```
+
+---
+
+## The asio executor / strand model
+
+All `Connection` state lives on **one executor**.  In simple cases
+that's just `io_context.get_executor()` — a single-threaded loop.
+In production, an EAP may run multiple `io_context::run()` threads
+*per connection* by wrapping work in an `asio::strand`.
+
+### What a strand is
+
+A **strand** is an executor that guarantees mutual exclusion
+between handlers it dispatches.  Multiple threads can call
+`io.run()`; the strand picks one of them at a time to run the
+next handler in its queue.
+
+For HSMS purposes, `socket_.get_executor()` already gives strand
+semantics if the underlying `io_context` is single-threaded.  For
+multi-threaded `io_context`, the application wraps with
+`asio::make_strand`:
+
+```cpp
+auto strand = asio::make_strand(io);
+asio::ip::tcp::socket sock(strand);
+auto conn = std::make_shared<hsms::Connection>(std::move(sock), ...);
+```
+
+Now `conn` has all its IO running on `strand`, while the
+`io_context` can use 8 threads to handle 100 different connections.
+
+### What this means for the caller
+
+From any thread that isn't the strand's currently-running handler,
+the caller MUST marshal onto the strand:
+
+```cpp
+// From a sensor-thread callback:
+asio::post(conn->executor(), [conn, msg = std::move(msg)] {
+  conn->send_data(std::move(msg));
+});
+```
+
+Calling `conn->send_data` directly from another thread is **a
+race**.  Same for any store mutation.  TSan catches this and the
+test suite enforces it.
+
+The contract is documented in detail in
+[`docs/INTEGRATION.md`](INTEGRATION.md) §3.
+
+---
+
+## secsi::Protocol — the IO-free FSM
+
+SECS-I's protocol layer is structured differently: the FSM has
+**no IO at all**.  It takes events (bytes received, application
+asked to send, timer fired) and produces a list of `Action`s
+(transmit these bytes, arm a timer, deliver a block to the
+application).
+
+```cpp
+// include/secsgem/secsi/protocol.hpp
+struct ActionTransmit       { std::vector<uint8_t> bytes; };
+struct ActionStartTimer     { Timer which; };
+struct ActionCancelTimer    { Timer which; };
+struct ActionDeliverBlock   { Block block; };
+struct ActionRaiseError     { std::string reason; };
+```
+
+The wrapper (`secsi::TcpTransport`) drives the FSM:
+
+```cpp
+void TcpTransport::on_byte(uint8_t b) {
+  auto actions = protocol_.handle(EventByte{b});
+  for (const auto& a : actions) execute(a);
+}
+
+void TcpTransport::execute(const Action& a) {
+  std::visit([this](auto&& v) {
+    using T = std::decay_t<decltype(v)>;
+    if constexpr (std::is_same_v<T, ActionTransmit>)    write_bytes(v.bytes);
+    else if constexpr (std::is_same_v<T, ActionStartTimer>)   arm_timer(v.which);
+    else if constexpr (std::is_same_v<T, ActionCancelTimer>)  cancel_timer(v.which);
+    else if constexpr (std::is_same_v<T, ActionDeliverBlock>) deliver(v.block);
+    else if constexpr (std::is_same_v<T, ActionRaiseError>)   raise(v.reason);
+  }, a);
+}
+```
+
+This design makes the **whole FSM** unit-testable.  No sockets,
+no timers, just `Event` in → `Action` out.  Tests:
+
+- [`tests/test_secsi.cpp`](../tests/test_secsi.cpp) — basic FSM
+  state walks.
+- [`tests/test_secsi_timers.cpp`](../tests/test_secsi_timers.cpp)
+  — every timer scenario via synthetic `EventTimeout` injection.
+- [`tests/test_secsi_tcp.cpp`](../tests/test_secsi_tcp.cpp) —
+  end-to-end via `TcpTransport`.
+
+Same pattern repeats for E84 (chapter 18) and the GEM
+communication-state FSM (chapter 13): IO-free FSM + asio adapter
+ separate test suites for each layer.
+
+---
+
+## Why this is the right shape
+
+### Pros of single-threaded + IO-free
+
+- **No mutexes.**  Anywhere.
+- **Trivial reasoning.**  When you read `Connection::send_frame`,
+  you can be sure nothing else is mutating the queue.
+- **Fast.**  No lock contention, no atomic round-trips.
+- **Testable.**  IO-free FSM lets you exercise every transition
+  without IO.
+
+### Cons
+
+- **Callers must know about the strand.**  Multi-threaded
+  applications need `asio::post` boilerplate.
+- **One slow handler blocks the rest.**  A 100 ms handler delays
+  every other message until it returns.  Fix: don't write slow
+  handlers; if you must, dispatch the slow work to another
+  executor and return immediately.
+
+For a SECS/GEM equipment runtime — where the natural shape is "one
+TCP socket, one event loop" — the pros far outweigh the cons.
+
+---
+
+## Where to go next
+
+You've now seen the bottom layer in detail: how bytes move,
+how state machines drive transitions without IO, how the single-
+threaded contract makes everything safe.  Next chapter goes back
+up one level: **the codec** — the encoder/decoder that turns
+the Item type from chapter 10 into wire bytes, plus the SML
+human-readable form.
+
+Next: [→ 34 Codec and SML](34_codec_and_sml.md)