raphael 0721db9542
tests / build-and-test (push) Failing after 42s
Close COMPLIANCE.md gap: spooling (E30 §6.22)
Implements the largest functional gap from the compliance audit. The
equipment now queues events the host can't immediately receive (either
because there's no SELECTED session or because the demo's force-spool
flag is on) and transmits the queue on host request.

What's new

  include/secsgem/gem/store/spool.hpp
    SpoolStore: a deque queue with a configurable per-stream whitelist
    (so only streams 5+6 spool by default), a max_size cap with FIFO
    eviction on overflow, and a `force_spool` test flag.  Enqueue
    returns one of Queued / Dropped_NotSpoolable / Dropped_Full so the
    caller can fall back to live delivery when appropriate.  Drain
    pops the entire queue in FIFO order.  Two new ack enums:
    ResetSpoolAck (S2F44 RSPACK) and SpoolRequestAck (S6F24 RSDA), plus
    SpoolRequestCode (S6F23 RSDC, Transmit/Purge).

  data/messages.yaml + auto-regenerated messages.hpp
    S2F43 W   <L,n <B stream>>            Reset Spooling
    S2F44     <L,2 <B RSPACK> <L,a ...>>  Reset Spooling Ack
    S6F23 W   <B RSDC>                    Request Spooled Data
    S6F24     <B RSDA>                    Request Spooled Data Ack

  data/equipment.yaml
    `spool:` section: max_size + spoolable_streams list.  Two new host
    commands SPOOL_ON / SPOOL_OFF that flip the force-spool flag (these
    stand in for "host link down" in the demo without dropping TCP).

  include/secsgem/gem/store/host_commands.hpp
    Spec/Result gain an optional<bool> force_spool field.  S2F41
    dispatch returns the result, the server applies it after S2F42 is
    queued.

  src/config/loader.cpp
    Reads `spool:` from equipment.yaml; reads `force_spool` from each
    host_commands entry; populates SpoolStore + CommandSpec.

  apps/secs_server.cpp
    New `deliver_or_spool(msg, what)` helper.  emit_event and
    emit_alarm_set funnel through it: if force_spool is on (or there's
    no active session), msg.stream is checked against the spoolable
    list and the message is enqueued; otherwise it's sent live.
    Two new handlers:
      S2F43  parses the stream list, updates SpoolStore, replies S2F44
      S6F23  RSDC=Transmit drains and re-sends each as a fresh primary
             (posted on the executor so the S6F24 ack flushes first);
             RSDC=Purge clears the queue and acks.
    The S2F41 handler now also propagates result.force_spool into the
    SpoolStore.

  apps/secs_client.cpp
    Demo extended with 4 new steps after the FAULT branch:
      SPOOL_ON  -> S2F42 Accept
      START     -> S2F42 Accept; CEID 300 emission spooled (no live S6F11)
      SPOOL_OFF -> S2F42 Accept; queue still has the message
      S6F23(Transmit) -> S6F24 Accept; spooled S6F11 arrives next
    Then the existing S7F19/S7F5/S10F1/S1F15/Separate flow continues.

  tests/test_data_model.cpp
    Four new TEST_CASEs for SpoolStore (whitelist, FIFO eviction at
    max_size, drain ordering, force flag).

  tests/test_loader.cpp
    Confirms equipment.yaml's `spool:` section populates the store and
    `force_spool: true/false` flows through to dispatch results.

  COMPLIANCE.md
    Spooling moves from  to 🟡.  Adds S2F43/F44 + S6F23/F24 as  in
    the message coverage matrix; calls out what's still missing
    (S6F25/F26 notification, automatic activation on HSMS NOT-SELECTED,
    persistent on-disk spool).

Verified

  - Tests: 73 cases / 383 assertions pass (+4 spool cases).
  - Demo (docker compose up server client) walks the full happy path
    and the spool path, observed in the server log as:
        spool: force_spool=true (depth=0)
        spool: S6F11 CEID=300 queued (depth=1)
        spool: force_spool=false (depth=1)
        S6F23 transmit: draining 1 messages
    and on the host side as the queued S6F11 arriving in the correct
    order after S6F24.

Known limitations (logged in COMPLIANCE.md)

  - Spool activation is manual via SPOOL_ON/OFF rather than
    automatically triggered by HSMS NOT-SELECTED.
  - No S6F25/F26 spooled-data-ready notification on re-SELECT.
  - In-memory only; an equipment restart loses queued events.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 22:06:55 +02:00

secs-gem

A C++20 SECS-II / HSMS / GEM client + server toolchain, fully containerized.

The long-term vision is the "spec-as-data" stack described in implementation_plan.md: the SEMI E30 behavioral spec encoded as transition tables, from which a runtime, passive analyzer, simulator, and conformance test generator are derived. The current code is the Layer 0 + Layer 1 base — a working HSMS client/server, the SECS-II codec, the E30 control state machine, and the GEM core capabilities (status variables, equipment constants, dynamic event reporting, alarms, remote control, clock, process programs, terminal services).

For a per-capability E5/E30/E37 audit see COMPLIANCE.md.

Quick start

Everything runs in Docker — no compiler or build tools on the host.

docker compose run --rm builder     # configure + compile
docker compose run --rm tests       # run unit tests (63 cases / 278 assertions)
docker compose up --no-deps server client   # live two-container demo

Architecture

The "spec-as-data" first step: equipment capabilities and the E30 control state machine are loaded from YAML at startup; the SECS dispatch is a (stream, function) -> handler Router rather than an if-ladder.

   ┌──────────────────────────────────────────────────────────────┐
   │ data/equipment.yaml   data/control_state.yaml                │
   │   SVIDs, ECIDs, CEIDs, alarms, recipes, host commands;       │
   │   E30 control state transition table                         │
   └──────────────────────┬───────────────────────────────────────┘
                          │  (loaded at startup)
                          ▼
   ┌──────────────────────────────────────────────────────────────┐
   │                        app / demo                            │
   │   apps/secs_server.cpp        apps/secs_client.cpp           │
   │   uses gem::Router for SECS dispatch                         │
   └────────────┬───────────────────────────┬─────────────────────┘
                │                           │
                ▼                           ▼
   ┌──────────────────────────────────────────────────────────────┐
   │ secsgem::config   loader.hpp  YAML -> tables + data model    │
   │ secsgem::gem      control_state (table-driven), data_model,  │
   │                   messages (SxFy builders), router           │
   │ secsgem::hsms     Connection (Asio), Header, Frame, Timers   │
   │ secsgem::secs2    Item, codec (encode/decode), Message       │
   └──────────────────────────────────────────────────────────────┘

Tree

secs-gem/
├── Dockerfile, docker-compose.yml      # toolchain + demo
├── CMakeLists.txt
├── implementation_plan.md              # the 7-layer spec-as-data roadmap
├── COMPLIANCE.md                       # per-capability E5/E30/E37 audit
├── data/
│   ├── equipment.yaml                  # SVIDs/ECIDs/CEIDs/alarms/recipes/commands
│   └── control_state.yaml              # E30 transition table
├── include/secsgem/
│   ├── secs2/{item,codec,message}.hpp
│   ├── hsms/{header,connection}.hpp    # header.hpp also holds Frame + Timers
│   ├── gem/{control_state,data_model,messages,router}.hpp
│   ├── config/loader.hpp
│   └── endpoint.hpp
├── src/{secs2,hsms,gem,config}/*.cpp + endpoint.cpp
├── apps/{secs_server,secs_client}.cpp
└── tests/test_*.cpp

Adding a capability without recompiling the server

# data/equipment.yaml — append a new SVID
svids:
  - {id: 4, name: ChamberTemp, units: "C", type: U4, value: 25}
# data/equipment.yaml — append a new host command + side effect
host_commands:
  - {name: VENT, ack: Accept, emit_ceid: 400, set_alarm: 2}
# data/control_state.yaml — append a new transition
transitions:
  - {from: OnlineRemote, on: host_request_offline, to: EquipmentOffline, ack: Accept}

Restart the server; the new behaviour is live. The C++ code unchanged.

What's implemented

HSMS (E37)

SELECT/DESELECT/LINKTEST/SEPARATE/REJECT control messages, the 4-byte length-prefixed frame, the 10-byte header, the NOT-CONNECTED → SELECTED state machine, and the T3/T5/T6/T7/T8 timers. Active (client) and passive (server) modes. System-bytes based reply correlation.

SECS-II (E5)

A variant-based Item covering every standard format (List, ASCII, Binary, Boolean, I1I8, U1U8, F4, F8), big-endian numerics, 1/2/3-byte length encoding, and an to_sml() rendering for readable logs.

GEM (E30)

  • Control state machine — 5 states (EquipmentOffline, AttemptOnline, HostOffline, OnlineLocal, OnlineRemote), operator actions, host requests with SEMI-mandated ACK codes, change-handler callback.
  • Status variables — S1F3/F4, S1F11/F12.
  • Equipment constants — S2F13/F14, S2F15/F16, S2F29/F30 + EAC enum.
  • Clock — S2F17/F18, S2F31/F32 (16-char YYYYMMDDhhmmsscc).
  • Establish Comms — S1F13/F14 + CommAck.
  • Online/Offline — S1F15/F16 + OfflineAck, S1F17/F18 + OnlineAck.
  • Remote control — S2F41/F42 + 7-value HostCmdAck + per-param CPACKs.
  • Dynamic event reporting — S2F33 Define Report (DRACK), S2F35 Link Event (LRACK), S2F37 Enable (ERACK), S6F11 emission, S6F12 ack.
  • Alarms — S5F1/F2 emit + ACKC5, S5F3/F4 enable/disable, S5F5/F6 list, ALCD bit-7 set/cleared, lower-7 category.
  • Process programs — S7F3/F4 send, S7F5/F6 request, S7F19/F20 list.
  • Terminal services — S10F1/F2 host→equipment, S10F3/F4 equipment→host.

Not (yet) implemented

Spooling, S9 error stream, S1F19/F20 + S1F21/F22 documentation, multi-block, trace data collection (S2F23/F24, S6F1/F2), limits monitoring, material movement (E40), EC range validation, S5F7/F8 list-enabled-alarms, S6F15/F16 event-report request. See COMPLIANCE.md for the honest accounting.

Demo session

The demo client walks 17 distinct steps end-to-end. From the live log (condensed):

[host]  -> Select.req          [equip] <- Select.req
[host]  <- Select.rsp (Ok)     [equip] -> Select.rsp (Ok)
[host]  == SELECTED ==         [equip] == SELECTED ==
[host]  -> S1F13 W             [equip] S1F13; replying S1F14 (COMMACK=0)
[host]  -> S1F17 W             [equip] control: HostOffline -> AttemptOnline
                               [equip] control: AttemptOnline -> OnlineRemote
[host]  S1F18 ONLACK=0         [equip] -> S10F3 W   (welcome)
[host]  TERMINAL[0] Welcome from SECSGEM-SIM
[host]  -> S1F11 W             [equip] S1F11; replying S1F12 (status namelist)
[host]    SVID 1 ControlState
[host]    SVID 2 Clock
[host]    SVID 3 EventsEnabled
[host]  -> S1F3 W              [equip] S1F3; replying S1F4 (3 values)
[host]  -> S2F29 W             [equip] -> S2F30 (2 EC entries)
[host]  -> S2F33 W             [equip] -> S2F34 (DRACK=0)
[host]  -> S2F35 W             [equip] -> S2F36 (LRACK=0)
[host]  -> S2F37 W             [equip] -> S2F38 (ERACK=0)
[host]  -> S2F41 W START       [equip] RCMD START; emit S6F11 CEID=300
[host]  EVENT CEID=300 (1 reports)
[host]    RPTID 1000: <A "OnlineRemote"> <A "2026060122162336"> <BOOLEAN T>
[host]  -> S5F5 W              [equip] -> S5F6 (2 alarms)
[host]  -> S5F3 W              [equip] -> S5F4 (ACKC5=0)
[host]  -> S2F41 W FAULT       [equip] RCMD FAULT triggers alarm 1
                               [equip] emit S5F1 alarm set ALID=1
                               [equip] emit S6F11 CEID=200
[host]  ALARM SET ALID=1 cat=4 "Chiller Temp High"
[host]  EVENT CEID=200 (1 reports)
[host]  -> S7F19 W             [equip] -> S7F20 (2 PPIDs)
[host]  -> S7F5 W              [equip] -> S7F6 RECIPE-A
[host]  -> S10F1 W             [equip] TERMINAL[0] Hello equipment!
[host]  -> S1F15 W             [equip] control: OnlineRemote -> HostOffline
[host]  -> Separate.req        [equip] <- Separate.req
[host]  exiting

Build details

The toolchain image (Dockerfile) is Ubuntu 24.04 with g++-13, CMake, Ninja, and libasio-dev. doctest is fetched via CMake FetchContent. Build artifacts live in a named Docker volume (build:) so the host filesystem stays clean.

Standalone Asio is used in header-only mode (ASIO_STANDALONE). No Boost dependency.

S
Description
No description provided
Readme 2.4 MiB
Languages
C++ 84.8%
Python 10.8%
Java 1.9%
Shell 1.4%
CMake 0.9%
Other 0.2%