raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
84 Commits 2 Branches 0 Tags
06f287b415c180057b9ab8c96ef48a1cec9cf956
Commit Graph

6 Commits

Author SHA1 Message Date
raphael 06f287b415 conformance: standalone secs_conformance harness binary 
The closest thing to an in-repo "RTS" — a runnable executable that
points at any HSMS-SS equipment and walks through every E30
fundamental + additional capability, reporting pass/fail per check
and exiting with the right code for CI / canary use.

  build/secs_conformance --host <ip> --port 5000 --device 0

Each check sends a host-initiated primary and asserts the equipment
replies with the expected stream/function within T3.  Checks chain
forward through async callbacks (each reply handler kicks off the
next check) so the conformance run stays inside one io.run().

Initial check set (mirrors COMPLIANCE.md §3 fundamentals):
  E37 §7.2  SELECT handshake
  E30 §6.5  S1F13/F14 Establish Comms
  E30 §6.7  S1F1/F2 Are You There
  E30 §6.13 S1F11/F12 SVID Namelist
  E30 §6.16 S2F29/F30 ECID Namelist
  E30 §6.20 S2F17/F18 Clock
  E30 §6.14 S5F5/F6 List Alarms
  E30 §6.17 S7F19/F20 PP List
  E30 §6.10 S1F19/F20 GEM Compliance

Validated against the demo server: 9/9 PASS.

README.md §8 (Compliance + certification) updated to point at the
harness as the suggested first-line conformance check.  Tool
vendors fork apps/secs_conformance.cpp and add their own
capability-specific checks alongside.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-09 12:57:37 +02:00
raphael d470442a8c docs: drop implementation_plan.md, rewrite README for fab deployment 
implementation_plan.md was a Layer-0..6 roadmap from the project's
spec-as-data exploration phase; every layer it described is now
shipped (Layer 0 foundations through Layer 4 message catalog +
state machines).  Removed.

README rewritten for the fab-deployment audience.  Sections added:

  1. Persistence directory layout (storage rules, disk budget, DR)
  2. Security (network isolation, TLS tunnels, audit logging,
     config signing)
  3. Monitoring + observability (signals → hooks table, Prometheus
     pattern)
  4. High availability (active/standby on shared persistence)
  5. Deployment patterns (Docker / systemd / k8s)
  6. Upgrade path (YAML reload, code rollout, schema versioning)
  7. Integration with the fab stack (MES / AMHS / OHT / recipe
     engine table)
  8. Compliance + certification (fork COMPLIANCE.md per tool, run
     RTS)
  9. Testing in production (canary, synthetic transactions, shadow
     traffic)
 10. Operational runbook (incident → first check → mitigation)

Stale stats refreshed: test count went 148/794 → 384/2390;
catalog grew to 164 messages; HSMS-GS, SECS-I T3/T4, per-port E84,
E42 formatted PPs all mentioned.

COMPLIANCE.md §9 lost its stale `implementation_plan.md` reference.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-09 12:54:06 +02:00
raphael 90c177b7ce E40 Process Jobs + E94 Control Jobs + E30 communication state 
GEM300 layer: SEMI E40-0705 Process Job and E94-0705 Control Job
state machines, plus the E30 §6.1 communication-state machine that
sits between HSMS SELECT and full GEM communication. Data-driven
via data/process_job_state.yaml and data/control_job_state.yaml,
mirroring the existing control_state.yaml pattern.

Wire coverage:
  S14F9/F10   CreateObject (CJ)              host -> equipment
  S14F11/F12  DeleteObject (CJ)              host -> equipment
  S16F5/F6    PRJobCommand                   host -> equipment
  S16F9       PRJobAlert                     equipment -> host
  S16F11/F12  PRJobCreate (simplified body)  host -> equipment
  S16F13/F14  PRJobDequeue                   host -> equipment
  S16F27/F28  CJobCommand                    host -> equipment

Process Job FSM exposes 8 states matching PRJOBSTATE bytes (E40 §10.3.2);
HOQ is reorder-aware (move-to-head against an insertion-order vector);
Stop/Abort on a Queued PJ routes through ABORTING so the host observes
PRJOBSTATE=7 on the wire (§6.3); alert_enabled is settable per-PJ for
PRALERT control; FSM dispatches through ProcessJobStore::on_change_
dynamically so a late set_state_change_handler() reaches existing PJs.

Hardening: loader rejects NoState (sentinel) as initial/from/to and
rejects `on: created` rows; static_asserts pin enum values to wire
bytes; ProcessJobStore is non-movable to keep the per-PJ this-capture
safe.

Server simulator cascades the full CJ -> PJ lifecycle on CJSTART so
the wire trace exercises every legal state. CEIDs 400/401 fire on CJ
state changes via the existing event-report pipeline.

Tests: 60+ new assertions across test_process_jobs, test_control_jobs,
test_communication_state, test_hsms_connection, plus loader and
messages round-trip coverage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-07 21:00:32 +02:00
raphael 1f67aad985 100%/F: S10F5/F6 multi-line + honest 100% in COMPLIANCE.md + README pass
tests / build-and-test (push) Failing after 33s
Details 
The final additions: S10F5/F6 multi-line terminal display (closes the
last partial Additional capability — Equipment Terminal Services flips
✅), and a thorough COMPLIANCE.md / README pass that states the 100%
claim honestly.

Catalog + handlers

  data/messages.yaml         S10F5 / S10F6 added.
  apps/secs_server.cpp       router.on(10, 5) iterates the line list,
                             acks with S10F6.
  tests/test_messages.cpp    Round-trips a 3-line multi-line display.

COMPLIANCE.md  (rewritten)

  Every GEM Fundamental ✅.  Every GEM Additional that E30 binds to a
  concrete message set ✅.  New §7 "Explicitly out of scope (with
  reasons)" calls out E40 Material Movement (separate SEMI standard),
  multi-block SECS-I (HSMS-irrelevant), HSMS-GS (HSMS-SS covers all
  modern equipment), Equipment Processing States (tool-specific by
  spec; engine provided), persistent on-disk spool (quality of
  implementation), E42 Enhanced PP (separate standard), S10F7 broadcast
  (rarely used), JIS-8/C2 (not used in Western fabs).

  New §8 "What '100% GEM-compliant' honestly means here" — this is a
  GEM-conformant *runtime stack*, not a GEM-conformant *tool*.
  Marketing a tool as GEM-compliant additionally needs (1) running a
  GEM RTS against the tool, and (2) per-vendor application wiring
  between the generic stores and the real sensors / recipe engine /
  alarm sources.

README.md  (rewritten)

  Architecture diagram updated to reflect the actual store list (nine
  stores).  "Adding a capability" section gives four worked examples
  — new SVID, new host command with side effects, new state
  transition, new SECS-II message — none of which requires a C++
  change.  Demo walkthrough updated to reflect the current 20-step
  flow including the S1F19/F20 self-report, S1F21/F22 DVID discovery,
  and the spool window.

Code clarity
  include/secsgem/gem/data_model.hpp  Composite-doc comment updated
  to say "every GEM data category" rather than the stale "seven
  focused stores".

Verified
  - Tests: 84 cases / 487 assertions pass.
  - Demo: 198 server/host log lines; exits 0 end-to-end.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-07 02:35:24 +02:00
raphael b871cd9da2 Table/YAML-driven refactor (Layer 1 start) 
Move equipment capabilities and the E30 control state machine out of C++
code and into YAML data files; introduce a Router for SECS dispatch;
consolidate small files.

Behavioural changes: none.  Demo identical (15 SxFy transactions +
3 equipment-initiated primaries), 67 test cases / 384 assertions still
all green.  Structural changes only.

Why
---

The previous server.cpp held the equipment data dictionary (3 SVIDs,
2 ECIDs, 3 CEIDs, 2 alarms, 2 recipes, 4 host commands) as imperative
C++ in a 50-line `populate()` function, and routed inbound messages
through a 150-line if-ladder.  Adding a new SVID required a recompile.
Adding a new state transition required editing two switch statements
(`operator_*` and `on_host_request_*`).  The control state machine's
behavioural rules were spread across imperative code in two methods.

This is exactly what implementation_plan.md calls out as the wrong
shape: behavioural rules should live in versioned data, and every
runtime/test/analyzer should read from that data rather than re-encode
it.  This commit starts that move.

What's new
----------

data/equipment.yaml
  Equipment data dictionary.  Declarative SVIDs / ECIDs / CEIDs /
  alarms / recipes / host commands.  Host commands carry their HCACK
  ack code plus optional `emit_ceid` and `set_alarm` side-effects.
  Adding a new SVID or command is a YAML edit, no recompile.

data/control_state.yaml
  The E30 §6.2 control state transition table as data.  Each row is
  (from, on) -> (to [, then] [, ack]).  `then` chains an auto-advance
  through the transient AttemptOnline state.  The previous
  imperative switch is gone.

include/secsgem/config/loader.hpp + src/config/loader.cpp
  yaml-cpp-backed loader.  `load_control_state(path)` returns a
  ControlTransitionTable + initial state; `load_equipment(path, model)`
  populates the EquipmentDataModel and returns the device descriptor
  (id, MDLN, SOFTREV, optional auto-emit CEID).  Surfaces config
  errors with file path + field name via ConfigError.

include/secsgem/gem/router.hpp  (header-only)
  Small (stream, function) -> handler map.  Server registers all
  handlers once at startup, then the Connection's message handler is
  just `router.dispatch(msg)`.  Unhandled primaries with W set get
  SxF0 by default.  Replaces the if-ladder in secs_server.cpp.

include/secsgem/gem/control_state.hpp + .cpp
  ControlTransitionTable is the new pure data type.  ControlStateMachine
  is now a thin engine over the table: `fire(event)` looks up the row,
  optionally transitions, optionally chains a `then` transition, returns
  the ack code.  Behaviour rules no longer live in C++ switches.
  The default in-code table matches data/control_state.yaml row for row;
  tests rely on it so they don't need the YAML file.

include/secsgem/gem/data_model.hpp + .cpp
  `register_command(rcmd, CommandSpec)` replaces the function-handler
  signature.  CommandSpec = (HostCmdAck, optional emit_ceid, optional
  set_alarm).  `dispatch_command` returns a CommandResult so the server
  can fire the side-effects after S2F42 is sent.

apps/secs_server.cpp
  No populate(), no if-ladder.  Loads equipment.yaml + control_state.yaml
  at startup (clean error on bad config), wires the Router once,
  delegates dispatch.  Sm change handler reads emit_on_control_change
  from the YAML.  Welcome S10F3 removed for parity with config (a future
  YAML rule could re-introduce it declaratively).

tests/test_loader.cpp  (new)
  Verifies the YAML loader produces the same shape as the in-code
  default table, and that equipment.yaml populates every section
  (SVIDs/ECIDs/CEIDs/alarms/recipes/commands).  SECSGEM_DATA_DIR
  CMake define points at ${CMAKE_SOURCE_DIR}/data so tests don't
  depend on cwd.

CMakeLists.txt, Dockerfile
  find_package(yaml-cpp) and link.  libyaml-cpp-dev added to the
  Ubuntu base image (yaml-cpp 0.8 ships the modern target name).

File consolidation
------------------

Five small files removed; their content lives in fewer headers:

  - secs2/item.cpp        -> inline in secs2/item.hpp
  - secs2/message.cpp     -> inline in secs2/message.hpp
  - hsms/types.hpp        -> merged into hsms/header.hpp
  - hsms/frame.hpp        -> merged into hsms/header.hpp
  - hsms/frame.cpp        -> merged into hsms/header.cpp

hsms/header.hpp is now "the HSMS wire format" in one place: SType + status
enums + Timers + Header + Frame + constants.  All includers updated.

Net effect
----------

Before: equipment data dictionary lived in 50 lines of imperative
populate() inside secs_server.cpp; dispatch in a 20-branch if-ladder.

After: equipment data dictionary lives in 47 lines of YAML; dispatch
is a Router built once.  Adding a new capability is now a YAML edit
in the common case.

Test count up to 67 cases / 384 assertions (+4 cases / +106 assertions)
covering the loader and the new table-driven SM paths.

What's NOT changed
------------------

The per-SxFy reply construction still lives in C++ (each message has a
unique body shape).  Moving those into YAML/JSON message-shape
definitions is the next refactor step but requires a generic typed
encoder/decoder driven by shape descriptors; out of scope here.

Spooling, the S9 error stream, S1F19/F20, and the other gaps in
COMPLIANCE.md remain unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-02 08:57:38 +02:00
raphael 96b02f8b50 Initial commit: C++20 SECS-II / HSMS / GEM client + server 
A fully containerised SECS/GEM toolchain. Single docker compose project,
no host build tools. 63 unit-test cases / 278 assertions, two demo
executables, end-to-end two-container demo exercising every implemented
capability.

Architecture (bottom-up):

  secs2/   E5 SECS-II codec
    Item        variant over L/A/B/BOOLEAN/I1-8/U1-8/F4/F8
    encode/decode  big-endian, 1/2/3-byte length encoding
    Message     SxFy + W-bit + optional root item
    to_sml      human-readable text rendering

  hsms/    E37 HSMS transport (TCP)
    Header      10-byte header + SType enum (Data/Select/Deselect/
                Linktest/Reject/Separate)
    Frame       4-byte length prefix + payload encode/decode
    Connection  async Asio TCP, NOT-SELECTED -> SELECTED state machine,
                T3/T5/T6/T7/T8 timers, system-bytes reply correlation,
                graceful close-after-flush separation

  endpoint  active Client (connect with T5 retry) and passive Server
            (accept loop) wrappers over Connection

  gem/     E30 GEM logic
    ControlStateMachine  5-state E30 control model with operator
                         actions, host requests, SEMI-mandated ack
                         codes (OnlineAck, OfflineAck, CommAck), and
                         a state-change handler
    EquipmentDataModel   in-memory dictionary: SVIDs, DVIDs, ECIDs
                         (with EAC), CEIDs, report defs, CEID->report
                         links, enabled-events set, alarm table
                         (ALCD, enabled, active), process programs,
                         host command registry, clock (16-char
                         YYYYMMDDhhmmsscc with offset)
    messages.hpp         builders + parsers for every SxFy below

GEM message coverage (full list):

  S1F1/F2    Are You There / On Line Data
  S1F3/F4    Selected Equipment Status Request / Data
  S1F11/F12  Status Variable Namelist Request / Data
  S1F13/F14  Establish Communications (+ CommAck)
  S1F15/F16  Request OFFLINE (+ OfflineAck)
  S1F17/F18  Request ONLINE (+ OnlineAck)
  S2F13/F14  Equipment Constant Request / Data
  S2F15/F16  EC Send + EquipmentAck (Accept/UnknownEcid/Busy/OutOfRange)
  S2F17/F18  Date and Time Request / Data
  S2F29/F30  Equipment Constant Namelist Request / Data
  S2F31/F32  Date and Time Set Request / TimeAck
  S2F33/F34  Define Report + DefineReportAck (5 enum values)
  S2F35/F36  Link Event Report + LinkEventAck
  S2F37/F38  Enable / Disable Event Report + EnableEventAck
  S2F41/F42  Host Command + HostCmdAck (7 values) + per-param CPACKs
  S5F1/F2    Alarm Report Send + AlarmAck (ALCD bit-7 set/cleared
             + lower-7 category)
  S5F3/F4    Enable/Disable Alarm Send + AlarmAck
  S5F5/F6    List Alarms Request / Data (active alarms tagged in ALCD)
  S6F11/F12  Event Report Send (equipment-initiated CEID emission
             with full report data) + EventReportAck
  S7F3/F4    Process Program Send + ProcessProgramAck (7 values)
  S7F5/F6    Process Program Request / Data
  S7F19/F20  Current EPPD List Request / Data
  S10F1/F2   Terminal Display Single (host->equipment) + TerminalAck
  S10F3/F4   Terminal Display Single (equipment->host)

Demo apps:

  apps/secs_server.cpp   passive equipment. Populates the data model
                         with 3 SVIDs (ControlState, Clock,
                         EventsEnabled), 2 ECIDs, 3 CEIDs
                         (ControlStateChanged, AlarmSetEvent,
                         ProcessStarted), 2 alarms (Chiller Temp High
                         cat 4, Door Open cat 1), 2 recipes
                         (RECIPE-A, RECIPE-B), and 4 host commands
                         (START, STOP, PAUSE, FAULT). Emits S6F11 on
                         every control state transition + on START;
                         emits S5F1 + the AlarmSetEvent CEID on FAULT.
                         Pushes an S10F3 welcome message when the host
                         comes online.

  apps/secs_client.cpp   active host. Walks 17 steps: Establish ->
                         Online -> S1F11 SVID namelist -> S1F3 read ->
                         S2F29 EC namelist -> S2F13 read ->
                         S2F17 clock -> S2F33/S2F35/S2F37 dynamic
                         event subscription -> S2F41 START
                         (-> receives S6F11) -> S5F5 alarm list ->
                         S5F3 enable alarm 1 -> S2F41 FAULT
                         (-> receives S5F1 + S6F11) -> S7F19/S7F5
                         recipe list + body -> S10F1 terminal ->
                         S1F15 Offline -> Separate. Handles inbound
                         S6F11, S5F1, S10F3 primaries.

Testing:

  tests/test_secs2.cpp         codec round-trip for every format,
                               byte-layout assertions for known values,
                               truncation/trailing-byte rejection,
                               nested list round-trip, SML rendering
  tests/test_hsms.cpp          header byte layout, data + control
                               header round-trip, full frame round-
                               trip with length prefix, short-payload
                               rejection
  tests/test_control_state.cpp every (state, event) pair in the E30
                               control state machine, including
                               AlreadyOnline / NotAccept rejections
                               and idempotent offline-while-offline
  tests/test_data_model.cpp    SVID/ECID/Alarm/Recipe CRUD, clock
                               format + parse, host command registry,
                               full event-report pipeline (define ->
                               link -> enable -> compose) with
                               every error path (InvalidVid,
                               UnknownCeid, UnknownRptid), alarm
                               set/clear with ALCD bit-7 semantics
  tests/test_messages.cpp      round-trip + byte-layout for every
                               builder/parser pair, including S6F11
                               event reports with mixed item types

Toolchain:

  Dockerfile          Ubuntu 24.04, g++-13, CMake, Ninja, libasio-dev
  docker-compose.yml  builder / tests / server / client services,
                      source bind-mounted, build artifacts in a
                      named volume so the host tree stays clean
  CMakeLists.txt      C++20, -Wall -Wextra -Wpedantic, standalone
                      Asio (ASIO_STANDALONE), doctest via FetchContent

Documentation:

  README.md           architecture, quick start, demo log
  COMPLIANCE.md       honest per-capability E5/E30/E37 audit with
                      spec section refs. Calls out what's implemented,
                      what's partial (Reject.req, Alarms missing F7/F8,
                      EC range validation, PP without verify, terminal
                      single-line only), and what's intentionally not
                      yet implemented (spooling, S9 error stream,
                      Documentation S1F19/F20+F21/F22, limits monitoring,
                      trace data collection, multi-block, material
                      movement). Does NOT claim "100% GEM-compliant" and
                      lists the work required to honestly make that claim.

This is Layer 0 + the start of Layer 1 from implementation_plan.md.
The transition-table-driven "spec-as-data" architecture (Layer 1
proper) is not yet implemented; the current code uses imperative
state machines that are structurally ready to be refactored onto
tables.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-02 00:21:10 +02:00