raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9 Commits 2 Branches 0 Tags
813e01140919a1b8f3a7d542d2c7358c42473c98
Commit Graph

4 Commits

Author SHA1 Message Date
raphael 813e011409 Close COMPLIANCE.md gap: Documentation (S1F19-F22)
tests / build-and-test (push) Failing after 32s
Details 
Adds the GEM "Documentation" Fundamental capability: the equipment now
self-reports which GEM capabilities it supports, and the host can
discover the DVID namelist with the same shape used for SVIDs.

Catalog (data/messages.yaml -> generated messages.hpp)

  S1F19 W   header-only                Get GEM Compliance Request
  S1F20     <L,3 <A SOFTREV>
                 <A EQPTYP>
                 <L,a <L,2 <U1 CCODE> <A CDESC>>>>
                                       Get GEM Compliance Data
  S1F21 W   <L,n <U4 VID>>             DVID Namelist Request (n=0 = all)
  S1F22     <L,n <L,3 <U4 VID>
                       <A VNAME>
                       <A UNITS>>>     DVID Namelist Data

  Codegen emits CapabilityEntry and GemCompliance structs.  S1F22 reuses
  S1F12's StatusName struct (same wire shape; dedup avoids redefinition).

Equipment data dictionary (data/equipment.yaml)

  device:                              Adds `equipment_type: "EQUIPMENT"`
                                       for the S1F20 EQPTYP field.

  capabilities:                        New section.  List of
    - {code, name}                     (CCODE, CDESC) pairs honestly
                                       reflecting what the codebase
                                       implements: 1, 2, 3, 5, 6, 7, 8,
                                       9, 11, 12, 14 (partial), 15.

  dvids:                               New section, same schema as
                                       svids:.  Demo populates two:
    - WaferCounter      (U4, units wafer)
    - ChamberPressure   (F4, units Torr)

Loader (src/config/loader.cpp + include/secsgem/config/loader.hpp)

  EquipmentDescriptor gains equipment_type and capabilities (vector of
  (uint8_t, string) pairs).  load_equipment now reads `capabilities:`
  into the descriptor and `dvids:` into model.dvids.

Server (apps/secs_server.cpp)

  router.on(1, 19) returns S1F20 with desc.software_rev,
  desc.equipment_type, and desc.capabilities converted to
  vector<CapabilityEntry>.
  router.on(1, 21) returns S1F22 built from model.dvids.all().

Client (apps/secs_client.cpp)

  Two new demo steps after Request Online and before SVID discovery:
    S1F19 -> S1F20: logs SOFTREV, EQPTYP, and every (CCODE, CDESC)
                    the equipment claims.
    S1F21 -> S1F22: logs each DVID with units.

Tests

  tests/test_messages.cpp   Round-trip S1F19/F20 with a 3-entry
                            capability list; round-trip S1F22 with two
                            DVIDs.

  tests/test_loader.cpp     Asserts equipment_type, the capabilities
                            list contains CCODE 14 (Spooling), and the
                            two DVIDs land in model.dvids.

COMPLIANCE.md

  "Documentation" Fundamental moves from ⬜ to ✅.
  S1F19/F20 + S1F21/F22 rows in the coverage matrix flip to ✅.
  The "what would it take" list drops the documentation-messages bullet.

Verified

  - Tests: 77 cases / 444 assertions pass.
  - Demo: client logs the full capability list received from the
    equipment, including CCODE 14 "Spooling (partial; S2F43/F44 +
    S6F23/F24)" — the equipment honestly reports its partial
    implementation rather than overclaiming.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-07 00:30:43 +02:00
raphael 547fd2116b Close COMPLIANCE.md gap: S9 error stream
tests / build-and-test (push) Failing after 43s
Details 
Adds S9F1, F3, F5, F7, F9, F11, F13 to the message catalog and wires
the two emission paths that the Connection layer can drive without help
from the Router or the application: S9F7 on a body-decode failure and
S9F9 on a T3 transaction-timer timeout.

Catalog (data/messages.yaml -> generated messages.hpp)

  All six MHEAD-carrying messages (F1/F3/F5/F7/F9/F11) use the same
  shape — a single <B 10> body with the offending 10-byte HSMS header.
  S9F13 (conversation timeout) carries <L,2 <A MEXP> <A EDID>>.

Connection-side emissions (src/hsms/connection.cpp)

  emit_s9(function, mhead)   New private helper.  Builds a 9/function/W=0
                              data message whose body is <B 10> with the
                              MHEAD bytes, allocates a fresh sys_bytes,
                              and queues it onto the write path.  No
                              reply is tracked.

  S9F7 on body decode        handle_data wraps Message::from_body in a
                              try/catch.  Previously any decode error
                              closed the connection; now it emits S9F7
                              with the offending header and continues
                              reading.  Reply-side decode failure also
                              emits S9F7 and surfaces the new
                              Error::IllegalData to the waiting
                              ReplyHandler (rather than making the
                              caller wait out T3).

  S9F9 on T3 timeout         The send_request T3 callback rebuilds the
                              original outgoing MHEAD from
                              (device_id, expected_stream,
                              expected_function-1, sys, W=1) and emits
                              S9F9 before invoking the callback with
                              Error::Timeout (unchanged).

What's intentionally not yet wired (logged in COMPLIANCE.md)

  - S9F3 / S9F5 — "unknown stream / function".  These need to live in
    the Router's fallback path, which would require either the Router
    knowing about a Connection-shaped sender or the Connection's
    message wrapper learning which streams the Router has handlers
    for.  Deferred — today the fallback returns SxF0 only.
  - S9F11 — "Data Too Long".  Currently we close on oversized frames;
    we'd need to also build a synthetic 10-byte MHEAD substitute (the
    real header isn't yet available at the point of detection) and
    flush it through close_after_flush.

Tests + docs

  tests/test_messages.cpp  Round-trip every S9F* using a representative
                           10-byte MHEAD literal; check S9F13 carries
                           MEXP + EDID.  +2 cases / +37 assertions.

  COMPLIANCE.md            Error Messages row moved from "no S9 stream"
                           to a detailed status describing what's
                           emitted vs catalog-only.  Coverage matrix
                           expanded per-message (F1/F7/F9/F13 ✅;
                           F3/F5/F11 🟡 catalog-only).

Build/demo unaffected: 75 cases / 420 assertions pass; the happy-path
demo never trips a decode error or T3, so the S9 path isn't exercised
end-to-end (but unit tests prove the wire shape).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-06 19:14:41 +02:00
raphael 0721db9542 Close COMPLIANCE.md gap: spooling (E30 §6.22)
tests / build-and-test (push) Failing after 42s
Details 
Implements the largest functional gap from the compliance audit. The
equipment now queues events the host can't immediately receive (either
because there's no SELECTED session or because the demo's force-spool
flag is on) and transmits the queue on host request.

What's new

  include/secsgem/gem/store/spool.hpp
    SpoolStore: a deque queue with a configurable per-stream whitelist
    (so only streams 5+6 spool by default), a max_size cap with FIFO
    eviction on overflow, and a `force_spool` test flag.  Enqueue
    returns one of Queued / Dropped_NotSpoolable / Dropped_Full so the
    caller can fall back to live delivery when appropriate.  Drain
    pops the entire queue in FIFO order.  Two new ack enums:
    ResetSpoolAck (S2F44 RSPACK) and SpoolRequestAck (S6F24 RSDA), plus
    SpoolRequestCode (S6F23 RSDC, Transmit/Purge).

  data/messages.yaml + auto-regenerated messages.hpp
    S2F43 W   <L,n <B stream>>            Reset Spooling
    S2F44     <L,2 <B RSPACK> <L,a ...>>  Reset Spooling Ack
    S6F23 W   <B RSDC>                    Request Spooled Data
    S6F24     <B RSDA>                    Request Spooled Data Ack

  data/equipment.yaml
    `spool:` section: max_size + spoolable_streams list.  Two new host
    commands SPOOL_ON / SPOOL_OFF that flip the force-spool flag (these
    stand in for "host link down" in the demo without dropping TCP).

  include/secsgem/gem/store/host_commands.hpp
    Spec/Result gain an optional<bool> force_spool field.  S2F41
    dispatch returns the result, the server applies it after S2F42 is
    queued.

  src/config/loader.cpp
    Reads `spool:` from equipment.yaml; reads `force_spool` from each
    host_commands entry; populates SpoolStore + CommandSpec.

  apps/secs_server.cpp
    New `deliver_or_spool(msg, what)` helper.  emit_event and
    emit_alarm_set funnel through it: if force_spool is on (or there's
    no active session), msg.stream is checked against the spoolable
    list and the message is enqueued; otherwise it's sent live.
    Two new handlers:
      S2F43  parses the stream list, updates SpoolStore, replies S2F44
      S6F23  RSDC=Transmit drains and re-sends each as a fresh primary
             (posted on the executor so the S6F24 ack flushes first);
             RSDC=Purge clears the queue and acks.
    The S2F41 handler now also propagates result.force_spool into the
    SpoolStore.

  apps/secs_client.cpp
    Demo extended with 4 new steps after the FAULT branch:
      SPOOL_ON  -> S2F42 Accept
      START     -> S2F42 Accept; CEID 300 emission spooled (no live S6F11)
      SPOOL_OFF -> S2F42 Accept; queue still has the message
      S6F23(Transmit) -> S6F24 Accept; spooled S6F11 arrives next
    Then the existing S7F19/S7F5/S10F1/S1F15/Separate flow continues.

  tests/test_data_model.cpp
    Four new TEST_CASEs for SpoolStore (whitelist, FIFO eviction at
    max_size, drain ordering, force flag).

  tests/test_loader.cpp
    Confirms equipment.yaml's `spool:` section populates the store and
    `force_spool: true/false` flows through to dispatch results.

  COMPLIANCE.md
    Spooling moves from ⬜ to 🟡.  Adds S2F43/F44 + S6F23/F24 as ✅ in
    the message coverage matrix; calls out what's still missing
    (S6F25/F26 notification, automatic activation on HSMS NOT-SELECTED,
    persistent on-disk spool).

Verified

  - Tests: 73 cases / 383 assertions pass (+4 spool cases).
  - Demo (docker compose up server client) walks the full happy path
    and the spool path, observed in the server log as:
        spool: force_spool=true (depth=0)
        spool: S6F11 CEID=300 queued (depth=1)
        spool: force_spool=false (depth=1)
        S6F23 transmit: draining 1 messages
    and on the host side as the queued S6F11 arriving in the correct
    order after S6F24.

Known limitations (logged in COMPLIANCE.md)

  - Spool activation is manual via SPOOL_ON/OFF rather than
    automatically triggered by HSMS NOT-SELECTED.
  - No S6F25/F26 spooled-data-ready notification on re-SELECT.
  - In-memory only; an equipment restart loses queued events.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-05 22:06:55 +02:00
raphael 96b02f8b50 Initial commit: C++20 SECS-II / HSMS / GEM client + server 
A fully containerised SECS/GEM toolchain. Single docker compose project,
no host build tools. 63 unit-test cases / 278 assertions, two demo
executables, end-to-end two-container demo exercising every implemented
capability.

Architecture (bottom-up):

  secs2/   E5 SECS-II codec
    Item        variant over L/A/B/BOOLEAN/I1-8/U1-8/F4/F8
    encode/decode  big-endian, 1/2/3-byte length encoding
    Message     SxFy + W-bit + optional root item
    to_sml      human-readable text rendering

  hsms/    E37 HSMS transport (TCP)
    Header      10-byte header + SType enum (Data/Select/Deselect/
                Linktest/Reject/Separate)
    Frame       4-byte length prefix + payload encode/decode
    Connection  async Asio TCP, NOT-SELECTED -> SELECTED state machine,
                T3/T5/T6/T7/T8 timers, system-bytes reply correlation,
                graceful close-after-flush separation

  endpoint  active Client (connect with T5 retry) and passive Server
            (accept loop) wrappers over Connection

  gem/     E30 GEM logic
    ControlStateMachine  5-state E30 control model with operator
                         actions, host requests, SEMI-mandated ack
                         codes (OnlineAck, OfflineAck, CommAck), and
                         a state-change handler
    EquipmentDataModel   in-memory dictionary: SVIDs, DVIDs, ECIDs
                         (with EAC), CEIDs, report defs, CEID->report
                         links, enabled-events set, alarm table
                         (ALCD, enabled, active), process programs,
                         host command registry, clock (16-char
                         YYYYMMDDhhmmsscc with offset)
    messages.hpp         builders + parsers for every SxFy below

GEM message coverage (full list):

  S1F1/F2    Are You There / On Line Data
  S1F3/F4    Selected Equipment Status Request / Data
  S1F11/F12  Status Variable Namelist Request / Data
  S1F13/F14  Establish Communications (+ CommAck)
  S1F15/F16  Request OFFLINE (+ OfflineAck)
  S1F17/F18  Request ONLINE (+ OnlineAck)
  S2F13/F14  Equipment Constant Request / Data
  S2F15/F16  EC Send + EquipmentAck (Accept/UnknownEcid/Busy/OutOfRange)
  S2F17/F18  Date and Time Request / Data
  S2F29/F30  Equipment Constant Namelist Request / Data
  S2F31/F32  Date and Time Set Request / TimeAck
  S2F33/F34  Define Report + DefineReportAck (5 enum values)
  S2F35/F36  Link Event Report + LinkEventAck
  S2F37/F38  Enable / Disable Event Report + EnableEventAck
  S2F41/F42  Host Command + HostCmdAck (7 values) + per-param CPACKs
  S5F1/F2    Alarm Report Send + AlarmAck (ALCD bit-7 set/cleared
             + lower-7 category)
  S5F3/F4    Enable/Disable Alarm Send + AlarmAck
  S5F5/F6    List Alarms Request / Data (active alarms tagged in ALCD)
  S6F11/F12  Event Report Send (equipment-initiated CEID emission
             with full report data) + EventReportAck
  S7F3/F4    Process Program Send + ProcessProgramAck (7 values)
  S7F5/F6    Process Program Request / Data
  S7F19/F20  Current EPPD List Request / Data
  S10F1/F2   Terminal Display Single (host->equipment) + TerminalAck
  S10F3/F4   Terminal Display Single (equipment->host)

Demo apps:

  apps/secs_server.cpp   passive equipment. Populates the data model
                         with 3 SVIDs (ControlState, Clock,
                         EventsEnabled), 2 ECIDs, 3 CEIDs
                         (ControlStateChanged, AlarmSetEvent,
                         ProcessStarted), 2 alarms (Chiller Temp High
                         cat 4, Door Open cat 1), 2 recipes
                         (RECIPE-A, RECIPE-B), and 4 host commands
                         (START, STOP, PAUSE, FAULT). Emits S6F11 on
                         every control state transition + on START;
                         emits S5F1 + the AlarmSetEvent CEID on FAULT.
                         Pushes an S10F3 welcome message when the host
                         comes online.

  apps/secs_client.cpp   active host. Walks 17 steps: Establish ->
                         Online -> S1F11 SVID namelist -> S1F3 read ->
                         S2F29 EC namelist -> S2F13 read ->
                         S2F17 clock -> S2F33/S2F35/S2F37 dynamic
                         event subscription -> S2F41 START
                         (-> receives S6F11) -> S5F5 alarm list ->
                         S5F3 enable alarm 1 -> S2F41 FAULT
                         (-> receives S5F1 + S6F11) -> S7F19/S7F5
                         recipe list + body -> S10F1 terminal ->
                         S1F15 Offline -> Separate. Handles inbound
                         S6F11, S5F1, S10F3 primaries.

Testing:

  tests/test_secs2.cpp         codec round-trip for every format,
                               byte-layout assertions for known values,
                               truncation/trailing-byte rejection,
                               nested list round-trip, SML rendering
  tests/test_hsms.cpp          header byte layout, data + control
                               header round-trip, full frame round-
                               trip with length prefix, short-payload
                               rejection
  tests/test_control_state.cpp every (state, event) pair in the E30
                               control state machine, including
                               AlreadyOnline / NotAccept rejections
                               and idempotent offline-while-offline
  tests/test_data_model.cpp    SVID/ECID/Alarm/Recipe CRUD, clock
                               format + parse, host command registry,
                               full event-report pipeline (define ->
                               link -> enable -> compose) with
                               every error path (InvalidVid,
                               UnknownCeid, UnknownRptid), alarm
                               set/clear with ALCD bit-7 semantics
  tests/test_messages.cpp      round-trip + byte-layout for every
                               builder/parser pair, including S6F11
                               event reports with mixed item types

Toolchain:

  Dockerfile          Ubuntu 24.04, g++-13, CMake, Ninja, libasio-dev
  docker-compose.yml  builder / tests / server / client services,
                      source bind-mounted, build artifacts in a
                      named volume so the host tree stays clean
  CMakeLists.txt      C++20, -Wall -Wextra -Wpedantic, standalone
                      Asio (ASIO_STANDALONE), doctest via FetchContent

Documentation:

  README.md           architecture, quick start, demo log
  COMPLIANCE.md       honest per-capability E5/E30/E37 audit with
                      spec section refs. Calls out what's implemented,
                      what's partial (Reject.req, Alarms missing F7/F8,
                      EC range validation, PP without verify, terminal
                      single-line only), and what's intentionally not
                      yet implemented (spooling, S9 error stream,
                      Documentation S1F19/F20+F21/F22, limits monitoring,
                      trace data collection, multi-block, material
                      movement). Does NOT claim "100% GEM-compliant" and
                      lists the work required to honestly make that claim.

This is Layer 0 + the start of Layer 1 from implementation_plan.md.
The transition-table-driven "spec-as-data" architecture (Layer 1
proper) is not yet implemented; the current code uses imperative
state machines that are structurally ready to be refactored onto
tables.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-02 00:21:10 +02:00