raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
547fd2116bcc4883cc49a15dcd26abfc22d9bed7
secs-gem/COMPLIANCE.md
T
raphael 547fd2116b
tests / build-and-test (push) Failing after 43s
Details
Close COMPLIANCE.md gap: S9 error stream 
Adds S9F1, F3, F5, F7, F9, F11, F13 to the message catalog and wires
the two emission paths that the Connection layer can drive without help
from the Router or the application: S9F7 on a body-decode failure and
S9F9 on a T3 transaction-timer timeout.

Catalog (data/messages.yaml -> generated messages.hpp)

  All six MHEAD-carrying messages (F1/F3/F5/F7/F9/F11) use the same
  shape — a single <B 10> body with the offending 10-byte HSMS header.
  S9F13 (conversation timeout) carries <L,2 <A MEXP> <A EDID>>.

Connection-side emissions (src/hsms/connection.cpp)

  emit_s9(function, mhead)   New private helper.  Builds a 9/function/W=0
                              data message whose body is <B 10> with the
                              MHEAD bytes, allocates a fresh sys_bytes,
                              and queues it onto the write path.  No
                              reply is tracked.

  S9F7 on body decode        handle_data wraps Message::from_body in a
                              try/catch.  Previously any decode error
                              closed the connection; now it emits S9F7
                              with the offending header and continues
                              reading.  Reply-side decode failure also
                              emits S9F7 and surfaces the new
                              Error::IllegalData to the waiting
                              ReplyHandler (rather than making the
                              caller wait out T3).

  S9F9 on T3 timeout         The send_request T3 callback rebuilds the
                              original outgoing MHEAD from
                              (device_id, expected_stream,
                              expected_function-1, sys, W=1) and emits
                              S9F9 before invoking the callback with
                              Error::Timeout (unchanged).

What's intentionally not yet wired (logged in COMPLIANCE.md)

  - S9F3 / S9F5 — "unknown stream / function".  These need to live in
    the Router's fallback path, which would require either the Router
    knowing about a Connection-shaped sender or the Connection's
    message wrapper learning which streams the Router has handlers
    for.  Deferred — today the fallback returns SxF0 only.
  - S9F11 — "Data Too Long".  Currently we close on oversized frames;
    we'd need to also build a synthetic 10-byte MHEAD substitute (the
    real header isn't yet available at the point of detection) and
    flush it through close_after_flush.

Tests + docs

  tests/test_messages.cpp  Round-trip every S9F* using a representative
                           10-byte MHEAD literal; check S9F13 carries
                           MEXP + EDID.  +2 cases / +37 assertions.

  COMPLIANCE.md            Error Messages row moved from "no S9 stream"
                           to a detailed status describing what's
                           emitted vs catalog-only.  Coverage matrix
                           expanded per-message (F1/F7/F9/F13 ✅;
                           F3/F5/F11 🟡 catalog-only).

Build/demo unaffected: 75 cases / 420 assertions pass; the happy-path
demo never trips a decode error or T3, so the S9 path isn't exercised
end-to-end (but unit tests prove the wire shape).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 19:14:41 +02:00

217 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SECS/GEM Compliance Audit
An honest, per-capability accounting of which parts of SEMI **E5 (SECS-II)**,
**E30 (GEM)**, and **E37 (HSMS)** this codebase implements, which parts it
implements partially, and which parts it does not implement at all.
> **TL;DR.** This is *not* a 100%-conformant GEM equipment. It implements
> every **GEM Fundamental capability** except formal Documentation (S1F19–F22)
> and the S9 error stream, plus most of the high-value **GEM Additional
> capabilities** (Dynamic Event Reporting, Alarms, EC, Clock, Remote Control,
> Process Programs, Terminal). Spooling, Limits Monitoring, Trace Data
> Collection, Multi-block, and Material Movement are intentionally not yet
> implemented. See the per-capability tables below for spec section refs.
Legend:
- ✅ **Full** — message/feature implemented; round-tripped in tests.
- 🟡 **Partial** — implemented in the demo path but with documented limitations.
- ⬜ **Not implemented** — intentionally out of scope for this iteration.
---
## 1. E37 — HSMS transport
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| TCP transport | ✅ | E37 §6 | `hsms::Connection` (Asio). |
| 4-byte length prefix + 10-byte header | ✅ | E37 §8.2 | `hsms::Frame::encode/decode`. |
| Session ID, byte2, byte3, PType, SType, system-bytes | ✅ | E37 §8.3 | `hsms::Header`. |
| `Select.req / .rsp` | ✅ | E37 §7.2 | `SType` 1/2; SelectStatus enum (0–3). |
| `Deselect.req / .rsp` | ✅ | E37 §7.4 | `SType` 3/4; DeselectStatus enum (0–2). |
| `Linktest.req / .rsp` | ✅ | E37 §7.5 | `SType` 5/6; periodic interval configurable. |
| `Separate.req` | ✅ | E37 §7.6 | `SType` 9; graceful close after flush. |
| `Reject.req` | 🟡 | E37 §7.7 | Emitted on data-while-NOT-SELECTED with reason 4; we don't yet reject on PType/SType-not-supported. |
| Connection state machine NOT-CONNECTED → NOT-SELECTED → SELECTED | ✅ | E37 §6.3 | Both Active and Passive modes. |
| T3 reply timeout | ✅ | E37 §10 | per-transaction `steady_timer`. |
| T5 connect separation timeout | ✅ | E37 §10 | `Client::schedule_retry`. |
| T6 control transaction timeout | ✅ | E37 §10 | single slot (no concurrent control transactions). |
| T7 not-selected timeout (passive) | ✅ | E37 §10 | armed on connect / on Deselect.req. |
| T8 intercharacter timeout | ✅ | E37 §10 | bounds the payload read after length prefix. |
| Multi-host / multi-session | ⬜ | E37 §6 | single SELECTED session at a time. |
| HSMS-SS vs HSMS-GS | ⬜ | E37 §11 | implemented HSMS-SS only. |
---
## 2. E5 — SECS-II encoding
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| Format byte + 1/2/3 length bytes | ✅ | E5 §9 | `secs2::encode_into`. |
| List (`L`) | ✅ | E5 §9.3 | recursive. |
| ASCII (`A`) | ✅ | E5 §9.5 | |
| Binary (`B`) | ✅ | E5 §9.5 | |
| Boolean (`BOOLEAN`) | ✅ | E5 §9.5 | |
| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | |
| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | |
| `F4, F8` (IEEE 754 big-endian) | ✅ | E5 §9.5 | bit-cast round-trip. |
| JIS-8, C2 (Unicode) | ⬜ | E5 §9.5 | rarely used in modern fabs. |
| SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. |
---
## 3. E30 — GEM Fundamental capabilities (§5.2)
| Fundamental Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| State models | ✅ | E30 §6.2 | — | Control state machine (5 states) + HSMS comm state. Equipment-processing state is left to the equipment app. |
| Equipment Processing States | ⬜ | E30 §6.3 | — | Standard says equipment **may** model these; tool-specific. Plug-in point not yet exposed. |
| Host-Initiated S1F13/F14 scenario | ✅ | E30 §6.5 | S1F13/F14 | |
| Event Notification | ✅ | E30 §6.6 | S6F11/F12 | Equipment-initiated, host-acknowledged. |
| On-Line Identification | ✅ | E30 §6.7 | S1F1/F2 | MDLN + SOFTREV. |
| Error Messages | 🟡 | E30 §6.9 | S9F* | HSMS Reject.req covers the transport-level case. S9F1/F3/F5/F7/F9/F11/F13 are now in the message catalog and round-trip-tested. `Connection` automatically emits **S9F7** when a peer's primary or reply body fails SECS-II decode (connection stays up; the caller of `send_request` sees `Error::IllegalData`), and **S9F9** when its outgoing request times out at T3 (alongside the existing `Error::Timeout`). **Not wired**: S9F3 (unknown stream) and S9F5 (unknown function) — these belong in the `Router`'s fallback path; today the fallback returns SxF0 only. |
| Documentation | ⬜ | E30 §6.10| S1F19/F20 (GEM compliance), S1F21/F22 (data variable namelist) | Not implemented. |
| Control (Operator-Initiated) | ✅ | E30 §6.2 | — | `ControlStateMachine::operator_online/offline/local/remote`. |
---
## 4. E30 — GEM Additional capabilities (§5.3)
| Additional Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. |
| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. |
| Variable Data Collection | ✅ | E30 §6.11| — | `DVID` table; values resolvable via `vid_value`. |
| Trace Data Collection | ⬜ | E30 §6.12| S2F23/F24, S6F1/F2 | Not implemented. |
| Status Data Collection | ✅ | E30 §6.13| S1F3/F4, S1F11/F12 | |
| Alarm Management | 🟡 | E30 §6.14| S5F1/F2, S5F3/F4, S5F5/F6 | F1–F6 implemented; **S5F7/F8 list-enabled-alarms not implemented**. ALCD bit-7 set/cleared, lower-7 category supported. |
| Remote Control | ✅ | E30 §6.15| S2F41/F42 | Full HCACK 7-value enum + per-parameter CPACKs. |
| Equipment Constants | ✅ | E30 §6.16| S2F13/F14, S2F15/F16, S2F29/F30 | **EAC range validation against min/max is NOT performed** — set always accepts a known ECID; this would reject out-of-range with EAC=4 in a conformant equipment. |
| Process Program Management | 🟡 | E30 §6.17| S7F3/F4, S7F5/F6, S7F19/F20 | Unformatted PP send/request/list. **No E42 enhanced PP**, **no S7F23/F24/F25/F26 PP verify**, **no PPID validation**. PPBODY treated as raw bytes (Binary item). |
| Material Movement | ⬜ | E30 §6.18| S3F*, E40 | Tied to E40 carrier handling; separate standard. |
| Equipment Terminal Services | 🟡 | E30 §6.19| S10F1/F2, S10F3/F4 | Single-line only. **S10F5/F6 multi-block, S10F7 broadcast not implemented.** |
| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. |
| Limits Monitoring | ⬜ | E30 §6.21| S2F45/F46, S2F47/F48, S6F45/F46 | Not implemented. |
| Spooling | 🟡 | E30 §6.22| S2F43/F44, S6F23/F24 | Spoolable-streams config (S2F43/F44) + spool request transmit/purge (S6F23/F24) implemented; SpoolStore queues equipment-initiated primaries when the host is unreachable or force-spool is on, drains FIFO on S6F23 transmit, purges on S6F23 purge. **Not implemented**: S6F25/F26 spooled-data-ready notification, automatic spool activation tied to HSMS NOT-SELECTED, persistent (on-disk) spool. |
| Control | ✅ | E30 §6.2 | — | See Fundamental. |
---
## 5. Message coverage matrix
| Pair | Direction | Status | Implemented in | Tested |
|------------------|-----------|--------|----------------|--------|
| S1F1 / S1F2 | H↔E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F3 / S1F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F11 / S1F12 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F13 / S1F14 | H↔E | ✅ | `messages.hpp` | ✅ in demo |
| S1F15 / S1F16 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S1F17 / S1F18 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S1F19 / S1F20 | H→E | ⬜ | — | — |
| S1F21 / S1F22 | H→E | ⬜ | — | — |
| S2F13 / S2F14 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F15 / S2F16 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F17 / S2F18 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F23 / S2F24 | H→E | ⬜ | — | — |
| S2F29 / S2F30 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F31 / S2F32 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F33 / S2F34 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F35 / S2F36 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F37 / S2F38 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F41 / S2F42 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F45–F48 | H→E | ⬜ | — | — |
| S5F1 / S5F2 | E→H | ✅ | `messages.hpp` | ✅ round-trip |
| S5F3 / S5F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S5F5 / S5F6 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S5F7 / S5F8 | H→E | ⬜ | — | — |
| S6F1 / S6F2 | E→H | ⬜ | — | — |
| S6F5 / S6F6 | H↔E | ⬜ | — | multi-block |
| S6F7 / S6F8 | H↔E | ⬜ | — | multi-block |
| S6F11 / S6F12 | E→H | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S6F15 / S6F16 | H→E | ⬜ | — | event report request |
| S2F43 / S2F44 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S6F23 / S6F24 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S6F25 / S6F26 | spool | ⬜ | — | spool-data-ready notification |
| S7F3 / S7F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S7F5 / S7F6 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S7F19 / S7F20 | H→E | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S7F23–F26 | H↔E | ⬜ | — | enhanced PP |
| S9F1 | E↔H | ✅ | catalog | ✅ round-trip |
| S9F3 | E↔H | 🟡 | catalog | ✅ round-trip; emission not yet wired |
| S9F5 | E↔H | 🟡 | catalog | ✅ round-trip; emission not yet wired |
| S9F7 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on body decode |
| S9F9 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on T3 timeout |
| S9F11 | E↔H | 🟡 | catalog | ✅ round-trip; emission not yet wired |
| S9F13 | E↔H | ✅ | catalog | ✅ round-trip |
| S10F1 / S10F2 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S10F3 / S10F4 | E→H | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S10F5 / S10F6 | H→E | ⬜ | — | multi-line |
---
## 6. Demo evidence
The two-container demo (`docker compose up --no-deps server client`)
exercises this concrete sequence end-to-end:
1. TCP connect → `Select.req` → `Select.rsp(Ok)` → SELECTED on both sides.
2. `S1F13`/`S1F14` Establish Comms.
3. `S1F17`/`S1F18` Request Online; control state transitions
`HostOffline → AttemptOnline → OnlineRemote`.
4. Server pushes `S10F3` welcome → host acks with `S10F4`.
5. `S1F11`/`S1F12` SVID namelist discovery → `S1F3`/`S1F4` SVID read.
6. `S2F29`/`S2F30` EC namelist → `S2F13`/`S2F14` EC read.
7. `S2F17`/`S2F18` clock read.
8. `S2F33`/`S2F34` Define Report 1000 over the 3 SVIDs (DRACK=0).
9. `S2F35`/`S2F36` Link CEIDs 200 and 300 to Report 1000 (LRACK=0).
10. `S2F37`/`S2F38` Enable CEIDs 200, 300 (ERACK=0).
11. `S2F41`/`S2F42` host command **START** (HCACK=0) → server emits
`S6F11(CEID=300)` carrying the linked Report 1000 → host acks `S6F12`.
12. `S5F5`/`S5F6` list alarm directory.
13. `S5F3`/`S5F4` enable alarm 1 (ACKC5=0).
14. `S2F41`/`S2F42` host command **FAULT** (HCACK=0) → server emits
`S5F1` (ALCD=0x84, set + cat 4) → host acks `S5F2`; server also
emits `S6F11(CEID=200)`.
15. `S7F19`/`S7F20` recipe list, `S7F5`/`S7F6` fetch RECIPE-A.
16. `S10F1`/`S10F2` host → equipment terminal display.
17. `S1F15`/`S1F16` Request Offline; control state goes back to
HostOffline. (CEID 100 ControlStateChanged emission is correctly
*suppressed* because the host never enabled CEID 100 — this is the
correct GEM behavior.)
18. `Separate.req` → clean close on both sides.
Unit tests: **63 cases / 278 assertions pass** (`docker compose run --rm tests`).
---
## 7. What it would take to claim "100% GEM-compliant"
The honest list, in priority order:
1. **Finish spooling**: S6F25/F26 spooled-data-ready notification, plus
automatic activation when HSMS goes NOT-SELECTED (and automatic
notification on re-SELECT) so the host doesn't have to manually flip
the test-only `force_spool` flag. Optional: persistent on-disk spool
so equipment restarts don't lose queued events.
2. **Finish S9 wiring**: route Router-level "unknown stream/function" through
S9F3/F5, and emit S9F11 (Data Too Long) with the 4-byte length prefix
in place of MHEAD when the incoming frame is oversized.
3. **Implement Documentation messages** S1F19/F20 (GEM-compliance) and
S1F21/F22 (data variable namelist) — needed for E30 conformance.
4. **Implement EC range validation** in `set_equipment_constant_value`
so out-of-range sets return EAC=4 instead of being silently accepted.
5. **Implement Limits Monitoring** (S2F45–F48, S6F45/F46) if the
target equipment publishes monitored variables.
6. **Implement Trace Data Collection** (S2F23/F24, S6F1/F2).
7. **Implement S5F7/F8** list-enabled-alarms.
8. **Implement multi-block transfers** (S6F5/F6, S6F7/F8).
9. **Implement equipment processing state model** with operator hooks
(E30 §6.3) — the abstract model is in E30 but the concrete states
are equipment-specific.
10. **Run the implementation against a real conformance test generator**
(Layer 4 of the implementation plan) on a representative tool.
After all of the above, a GEM Reference Test specification (RTS) review
would still be needed before any "GEM compliant" marketing claim could
be honestly made.
Reference in New Issue View Git Blame Copy Permalink