raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: C++20 SECS-II / HSMS / GEM client + server

Browse Source 
A fully containerised SECS/GEM toolchain. Single docker compose project,
no host build tools. 63 unit-test cases / 278 assertions, two demo
executables, end-to-end two-container demo exercising every implemented
capability.

Architecture (bottom-up):

  secs2/   E5 SECS-II codec
    Item        variant over L/A/B/BOOLEAN/I1-8/U1-8/F4/F8
    encode/decode  big-endian, 1/2/3-byte length encoding
    Message     SxFy + W-bit + optional root item
    to_sml      human-readable text rendering

  hsms/    E37 HSMS transport (TCP)
    Header      10-byte header + SType enum (Data/Select/Deselect/
                Linktest/Reject/Separate)
    Frame       4-byte length prefix + payload encode/decode
    Connection  async Asio TCP, NOT-SELECTED -> SELECTED state machine,
                T3/T5/T6/T7/T8 timers, system-bytes reply correlation,
                graceful close-after-flush separation

  endpoint  active Client (connect with T5 retry) and passive Server
            (accept loop) wrappers over Connection

  gem/     E30 GEM logic
    ControlStateMachine  5-state E30 control model with operator
                         actions, host requests, SEMI-mandated ack
                         codes (OnlineAck, OfflineAck, CommAck), and
                         a state-change handler
    EquipmentDataModel   in-memory dictionary: SVIDs, DVIDs, ECIDs
                         (with EAC), CEIDs, report defs, CEID->report
                         links, enabled-events set, alarm table
                         (ALCD, enabled, active), process programs,
                         host command registry, clock (16-char
                         YYYYMMDDhhmmsscc with offset)
    messages.hpp         builders + parsers for every SxFy below

GEM message coverage (full list):

  S1F1/F2    Are You There / On Line Data
  S1F3/F4    Selected Equipment Status Request / Data
  S1F11/F12  Status Variable Namelist Request / Data
  S1F13/F14  Establish Communications (+ CommAck)
  S1F15/F16  Request OFFLINE (+ OfflineAck)
  S1F17/F18  Request ONLINE (+ OnlineAck)
  S2F13/F14  Equipment Constant Request / Data
  S2F15/F16  EC Send + EquipmentAck (Accept/UnknownEcid/Busy/OutOfRange)
  S2F17/F18  Date and Time Request / Data
  S2F29/F30  Equipment Constant Namelist Request / Data
  S2F31/F32  Date and Time Set Request / TimeAck
  S2F33/F34  Define Report + DefineReportAck (5 enum values)
  S2F35/F36  Link Event Report + LinkEventAck
  S2F37/F38  Enable / Disable Event Report + EnableEventAck
  S2F41/F42  Host Command + HostCmdAck (7 values) + per-param CPACKs
  S5F1/F2    Alarm Report Send + AlarmAck (ALCD bit-7 set/cleared
             + lower-7 category)
  S5F3/F4    Enable/Disable Alarm Send + AlarmAck
  S5F5/F6    List Alarms Request / Data (active alarms tagged in ALCD)
  S6F11/F12  Event Report Send (equipment-initiated CEID emission
             with full report data) + EventReportAck
  S7F3/F4    Process Program Send + ProcessProgramAck (7 values)
  S7F5/F6    Process Program Request / Data
  S7F19/F20  Current EPPD List Request / Data
  S10F1/F2   Terminal Display Single (host->equipment) + TerminalAck
  S10F3/F4   Terminal Display Single (equipment->host)

Demo apps:

  apps/secs_server.cpp   passive equipment. Populates the data model
                         with 3 SVIDs (ControlState, Clock,
                         EventsEnabled), 2 ECIDs, 3 CEIDs
                         (ControlStateChanged, AlarmSetEvent,
                         ProcessStarted), 2 alarms (Chiller Temp High
                         cat 4, Door Open cat 1), 2 recipes
                         (RECIPE-A, RECIPE-B), and 4 host commands
                         (START, STOP, PAUSE, FAULT). Emits S6F11 on
                         every control state transition + on START;
                         emits S5F1 + the AlarmSetEvent CEID on FAULT.
                         Pushes an S10F3 welcome message when the host
                         comes online.

  apps/secs_client.cpp   active host. Walks 17 steps: Establish ->
                         Online -> S1F11 SVID namelist -> S1F3 read ->
                         S2F29 EC namelist -> S2F13 read ->
                         S2F17 clock -> S2F33/S2F35/S2F37 dynamic
                         event subscription -> S2F41 START
                         (-> receives S6F11) -> S5F5 alarm list ->
                         S5F3 enable alarm 1 -> S2F41 FAULT
                         (-> receives S5F1 + S6F11) -> S7F19/S7F5
                         recipe list + body -> S10F1 terminal ->
                         S1F15 Offline -> Separate. Handles inbound
                         S6F11, S5F1, S10F3 primaries.

Testing:

  tests/test_secs2.cpp         codec round-trip for every format,
                               byte-layout assertions for known values,
                               truncation/trailing-byte rejection,
                               nested list round-trip, SML rendering
  tests/test_hsms.cpp          header byte layout, data + control
                               header round-trip, full frame round-
                               trip with length prefix, short-payload
                               rejection
  tests/test_control_state.cpp every (state, event) pair in the E30
                               control state machine, including
                               AlreadyOnline / NotAccept rejections
                               and idempotent offline-while-offline
  tests/test_data_model.cpp    SVID/ECID/Alarm/Recipe CRUD, clock
                               format + parse, host command registry,
                               full event-report pipeline (define ->
                               link -> enable -> compose) with
                               every error path (InvalidVid,
                               UnknownCeid, UnknownRptid), alarm
                               set/clear with ALCD bit-7 semantics
  tests/test_messages.cpp      round-trip + byte-layout for every
                               builder/parser pair, including S6F11
                               event reports with mixed item types

Toolchain:

  Dockerfile          Ubuntu 24.04, g++-13, CMake, Ninja, libasio-dev
  docker-compose.yml  builder / tests / server / client services,
                      source bind-mounted, build artifacts in a
                      named volume so the host tree stays clean
  CMakeLists.txt      C++20, -Wall -Wextra -Wpedantic, standalone
                      Asio (ASIO_STANDALONE), doctest via FetchContent

Documentation:

  README.md           architecture, quick start, demo log
  COMPLIANCE.md       honest per-capability E5/E30/E37 audit with
                      spec section refs. Calls out what's implemented,
                      what's partial (Reject.req, Alarms missing F7/F8,
                      EC range validation, PP without verify, terminal
                      single-line only), and what's intentionally not
                      yet implemented (spooling, S9 error stream,
                      Documentation S1F19/F20+F21/F22, limits monitoring,
                      trace data collection, multi-block, material
                      movement). Does NOT claim "100% GEM-compliant" and
                      lists the work required to honestly make that claim.

This is Layer 0 + the start of Layer 1 from implementation_plan.md.
The transition-table-driven "spec-as-data" architecture (Layer 1
proper) is not yet implemented; the current code uses imperative
state machines that are structurally ready to be refactored onto
tables.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-02 00:21:10 +02:00
commit 96b02f8b50
36 changed files with 5210 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
COMPLIANCE.md
+204
View File
@@ -0,0 +1,204 @@
# SECS/GEM Compliance Audit
An honest, per-capability accounting of which parts of SEMI **E5 (SECS-II)**,
**E30 (GEM)**, and **E37 (HSMS)** this codebase implements, which parts it
implements partially, and which parts it does not implement at all.
> **TL;DR.** This is *not* a 100%-conformant GEM equipment. It implements
> every **GEM Fundamental capability** except formal Documentation (S1F19–F22)
> and the S9 error stream, plus most of the high-value **GEM Additional
> capabilities** (Dynamic Event Reporting, Alarms, EC, Clock, Remote Control,
> Process Programs, Terminal). Spooling, Limits Monitoring, Trace Data
> Collection, Multi-block, and Material Movement are intentionally not yet
> implemented. See the per-capability tables below for spec section refs.
Legend:
- ✅ **Full** — message/feature implemented; round-tripped in tests.
- 🟡 **Partial** — implemented in the demo path but with documented limitations.
- ⬜ **Not implemented** — intentionally out of scope for this iteration.
---
## 1. E37 — HSMS transport
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| TCP transport | ✅ | E37 §6 | `hsms::Connection` (Asio). |
| 4-byte length prefix + 10-byte header | ✅ | E37 §8.2 | `hsms::Frame::encode/decode`. |
| Session ID, byte2, byte3, PType, SType, system-bytes | ✅ | E37 §8.3 | `hsms::Header`. |
| `Select.req / .rsp` | ✅ | E37 §7.2 | `SType` 1/2; SelectStatus enum (0–3). |
| `Deselect.req / .rsp` | ✅ | E37 §7.4 | `SType` 3/4; DeselectStatus enum (0–2). |
| `Linktest.req / .rsp` | ✅ | E37 §7.5 | `SType` 5/6; periodic interval configurable. |
| `Separate.req` | ✅ | E37 §7.6 | `SType` 9; graceful close after flush. |
| `Reject.req` | 🟡 | E37 §7.7 | Emitted on data-while-NOT-SELECTED with reason 4; we don't yet reject on PType/SType-not-supported. |
| Connection state machine NOT-CONNECTED → NOT-SELECTED → SELECTED | ✅ | E37 §6.3 | Both Active and Passive modes. |
| T3 reply timeout | ✅ | E37 §10 | per-transaction `steady_timer`. |
| T5 connect separation timeout | ✅ | E37 §10 | `Client::schedule_retry`. |
| T6 control transaction timeout | ✅ | E37 §10 | single slot (no concurrent control transactions). |
| T7 not-selected timeout (passive) | ✅ | E37 §10 | armed on connect / on Deselect.req. |
| T8 intercharacter timeout | ✅ | E37 §10 | bounds the payload read after length prefix. |
| Multi-host / multi-session | ⬜ | E37 §6 | single SELECTED session at a time. |
| HSMS-SS vs HSMS-GS | ⬜ | E37 §11 | implemented HSMS-SS only. |
---
## 2. E5 — SECS-II encoding
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| Format byte + 1/2/3 length bytes | ✅ | E5 §9 | `secs2::encode_into`. |
| List (`L`) | ✅ | E5 §9.3 | recursive. |
| ASCII (`A`) | ✅ | E5 §9.5 | |
| Binary (`B`) | ✅ | E5 §9.5 | |
| Boolean (`BOOLEAN`) | ✅ | E5 §9.5 | |
| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | |
| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | |
| `F4, F8` (IEEE 754 big-endian) | ✅ | E5 §9.5 | bit-cast round-trip. |
| JIS-8, C2 (Unicode) | ⬜ | E5 §9.5 | rarely used in modern fabs. |
| SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. |
---
## 3. E30 — GEM Fundamental capabilities (§5.2)
| Fundamental Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| State models | ✅ | E30 §6.2 | — | Control state machine (5 states) + HSMS comm state. Equipment-processing state is left to the equipment app. |
| Equipment Processing States | ⬜ | E30 §6.3 | — | Standard says equipment **may** model these; tool-specific. Plug-in point not yet exposed. |
| Host-Initiated S1F13/F14 scenario | ✅ | E30 §6.5 | S1F13/F14 | |
| Event Notification | ✅ | E30 §6.6 | S6F11/F12 | Equipment-initiated, host-acknowledged. |
| On-Line Identification | ✅ | E30 §6.7 | S1F1/F2 | MDLN + SOFTREV. |
| Error Messages | 🟡 | E30 §6.9 | S9F* | HSMS Reject.req covers the transport-level case. **No S9 stream (S9F1–F13)** for "Unrecognized Device ID", "Unrecognized Stream Type", T3/T6/T7/T8 violations, etc. |
| Documentation | ⬜ | E30 §6.10| S1F19/F20 (GEM compliance), S1F21/F22 (data variable namelist) | Not implemented. |
| Control (Operator-Initiated) | ✅ | E30 §6.2 | — | `ControlStateMachine::operator_online/offline/local/remote`. |
---
## 4. E30 — GEM Additional capabilities (§5.3)
| Additional Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. |
| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. |
| Variable Data Collection | ✅ | E30 §6.11| — | `DVID` table; values resolvable via `vid_value`. |
| Trace Data Collection | ⬜ | E30 §6.12| S2F23/F24, S6F1/F2 | Not implemented. |
| Status Data Collection | ✅ | E30 §6.13| S1F3/F4, S1F11/F12 | |
| Alarm Management | 🟡 | E30 §6.14| S5F1/F2, S5F3/F4, S5F5/F6 | F1–F6 implemented; **S5F7/F8 list-enabled-alarms not implemented**. ALCD bit-7 set/cleared, lower-7 category supported. |
| Remote Control | ✅ | E30 §6.15| S2F41/F42 | Full HCACK 7-value enum + per-parameter CPACKs. |
| Equipment Constants | ✅ | E30 §6.16| S2F13/F14, S2F15/F16, S2F29/F30 | **EAC range validation against min/max is NOT performed** — set always accepts a known ECID; this would reject out-of-range with EAC=4 in a conformant equipment. |
| Process Program Management | 🟡 | E30 §6.17| S7F3/F4, S7F5/F6, S7F19/F20 | Unformatted PP send/request/list. **No E42 enhanced PP**, **no S7F23/F24/F25/F26 PP verify**, **no PPID validation**. PPBODY treated as raw bytes (Binary item). |
| Material Movement | ⬜ | E30 §6.18| S3F*, E40 | Tied to E40 carrier handling; separate standard. |
| Equipment Terminal Services | 🟡 | E30 §6.19| S10F1/F2, S10F3/F4 | Single-line only. **S10F5/F6 multi-block, S10F7 broadcast not implemented.** |
| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. |
| Limits Monitoring | ⬜ | E30 §6.21| S2F45/F46, S2F47/F48, S6F45/F46 | Not implemented. |
| Spooling | ⬜ | E30 §6.22| S6F23/F24, S6F25/F26, spool state model | Not implemented. The plan calls this out as a Layer-3+ subsystem. |
| Control | ✅ | E30 §6.2 | — | See Fundamental. |
---
## 5. Message coverage matrix
| Pair | Direction | Status | Implemented in | Tested |
|------------------|-----------|--------|----------------|--------|
| S1F1 / S1F2 | H↔E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F3 / S1F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F11 / S1F12 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F13 / S1F14 | H↔E | ✅ | `messages.hpp` | ✅ in demo |
| S1F15 / S1F16 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S1F17 / S1F18 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S1F19 / S1F20 | H→E | ⬜ | — | — |
| S1F21 / S1F22 | H→E | ⬜ | — | — |
| S2F13 / S2F14 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F15 / S2F16 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F17 / S2F18 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F23 / S2F24 | H→E | ⬜ | — | — |
| S2F29 / S2F30 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F31 / S2F32 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F33 / S2F34 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F35 / S2F36 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F37 / S2F38 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F41 / S2F42 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F45–F48 | H→E | ⬜ | — | — |
| S5F1 / S5F2 | E→H | ✅ | `messages.hpp` | ✅ round-trip |
| S5F3 / S5F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S5F5 / S5F6 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S5F7 / S5F8 | H→E | ⬜ | — | — |
| S6F1 / S6F2 | E→H | ⬜ | — | — |
| S6F5 / S6F6 | H↔E | ⬜ | — | multi-block |
| S6F7 / S6F8 | H↔E | ⬜ | — | multi-block |
| S6F11 / S6F12 | E→H | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S6F15 / S6F16 | H→E | ⬜ | — | event report request |
| S6F23 / S6F26 | spool | ⬜ | — | spooling |
| S7F3 / S7F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S7F5 / S7F6 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S7F19 / S7F20 | H→E | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S7F23–F26 | H↔E | ⬜ | — | enhanced PP |
| S9F* | E→H | ⬜ | — | error stream |
| S10F1 / S10F2 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S10F3 / S10F4 | E→H | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S10F5 / S10F6 | H→E | ⬜ | — | multi-line |
---
## 6. Demo evidence
The two-container demo (`docker compose up --no-deps server client`)
exercises this concrete sequence end-to-end:
1. TCP connect → `Select.req` → `Select.rsp(Ok)` → SELECTED on both sides.
2. `S1F13`/`S1F14` Establish Comms.
3. `S1F17`/`S1F18` Request Online; control state transitions
`HostOffline → AttemptOnline → OnlineRemote`.
4. Server pushes `S10F3` welcome → host acks with `S10F4`.
5. `S1F11`/`S1F12` SVID namelist discovery → `S1F3`/`S1F4` SVID read.
6. `S2F29`/`S2F30` EC namelist → `S2F13`/`S2F14` EC read.
7. `S2F17`/`S2F18` clock read.
8. `S2F33`/`S2F34` Define Report 1000 over the 3 SVIDs (DRACK=0).
9. `S2F35`/`S2F36` Link CEIDs 200 and 300 to Report 1000 (LRACK=0).
10. `S2F37`/`S2F38` Enable CEIDs 200, 300 (ERACK=0).
11. `S2F41`/`S2F42` host command **START** (HCACK=0) → server emits
`S6F11(CEID=300)` carrying the linked Report 1000 → host acks `S6F12`.
12. `S5F5`/`S5F6` list alarm directory.
13. `S5F3`/`S5F4` enable alarm 1 (ACKC5=0).
14. `S2F41`/`S2F42` host command **FAULT** (HCACK=0) → server emits
`S5F1` (ALCD=0x84, set + cat 4) → host acks `S5F2`; server also
emits `S6F11(CEID=200)`.
15. `S7F19`/`S7F20` recipe list, `S7F5`/`S7F6` fetch RECIPE-A.
16. `S10F1`/`S10F2` host → equipment terminal display.
17. `S1F15`/`S1F16` Request Offline; control state goes back to
HostOffline. (CEID 100 ControlStateChanged emission is correctly
*suppressed* because the host never enabled CEID 100 — this is the
correct GEM behavior.)
18. `Separate.req` → clean close on both sides.
Unit tests: **63 cases / 278 assertions pass** (`docker compose run --rm tests`).
---
## 7. What it would take to claim "100% GEM-compliant"
The honest list, in priority order:
1. **Implement spooling** (E30 §6.22, S6F23/F24/F25/F26 + spool state
model). This is the single largest missing GEM feature.
2. **Implement the S9 error stream** (S9F1, F3, F5, F7, F9, F11, F13)
for the documented transport/protocol error cases.
3. **Implement Documentation messages** S1F19/F20 (GEM-compliance) and
S1F21/F22 (data variable namelist) — needed for E30 conformance.
4. **Implement EC range validation** in `set_equipment_constant_value`
so out-of-range sets return EAC=4 instead of being silently accepted.
5. **Implement Limits Monitoring** (S2F45–F48, S6F45/F46) if the
target equipment publishes monitored variables.
6. **Implement Trace Data Collection** (S2F23/F24, S6F1/F2).
7. **Implement S5F7/F8** list-enabled-alarms.
8. **Implement multi-block transfers** (S6F5/F6, S6F7/F8).
9. **Implement equipment processing state model** with operator hooks
(E30 §6.3) — the abstract model is in E30 but the concrete states
are equipment-specific.
10. **Run the implementation against a real conformance test generator**
(Layer 4 of the implementation plan) on a representative tool.
After all of the above, a GEM Reference Test specification (RTS) review
would still be needed before any "GEM compliant" marketing claim could
be honestly made.