raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
813e01140919a1b8f3a7d542d2c7358c42473c98
secs-gem/COMPLIANCE.md
T
raphael 813e011409
tests / build-and-test (push) Failing after 32s
Details
Close COMPLIANCE.md gap: Documentation (S1F19-F22) 
Adds the GEM "Documentation" Fundamental capability: the equipment now
self-reports which GEM capabilities it supports, and the host can
discover the DVID namelist with the same shape used for SVIDs.

Catalog (data/messages.yaml -> generated messages.hpp)

  S1F19 W   header-only                Get GEM Compliance Request
  S1F20     <L,3 <A SOFTREV>
                 <A EQPTYP>
                 <L,a <L,2 <U1 CCODE> <A CDESC>>>>
                                       Get GEM Compliance Data
  S1F21 W   <L,n <U4 VID>>             DVID Namelist Request (n=0 = all)
  S1F22     <L,n <L,3 <U4 VID>
                       <A VNAME>
                       <A UNITS>>>     DVID Namelist Data

  Codegen emits CapabilityEntry and GemCompliance structs.  S1F22 reuses
  S1F12's StatusName struct (same wire shape; dedup avoids redefinition).

Equipment data dictionary (data/equipment.yaml)

  device:                              Adds `equipment_type: "EQUIPMENT"`
                                       for the S1F20 EQPTYP field.

  capabilities:                        New section.  List of
    - {code, name}                     (CCODE, CDESC) pairs honestly
                                       reflecting what the codebase
                                       implements: 1, 2, 3, 5, 6, 7, 8,
                                       9, 11, 12, 14 (partial), 15.

  dvids:                               New section, same schema as
                                       svids:.  Demo populates two:
    - WaferCounter      (U4, units wafer)
    - ChamberPressure   (F4, units Torr)

Loader (src/config/loader.cpp + include/secsgem/config/loader.hpp)

  EquipmentDescriptor gains equipment_type and capabilities (vector of
  (uint8_t, string) pairs).  load_equipment now reads `capabilities:`
  into the descriptor and `dvids:` into model.dvids.

Server (apps/secs_server.cpp)

  router.on(1, 19) returns S1F20 with desc.software_rev,
  desc.equipment_type, and desc.capabilities converted to
  vector<CapabilityEntry>.
  router.on(1, 21) returns S1F22 built from model.dvids.all().

Client (apps/secs_client.cpp)

  Two new demo steps after Request Online and before SVID discovery:
    S1F19 -> S1F20: logs SOFTREV, EQPTYP, and every (CCODE, CDESC)
                    the equipment claims.
    S1F21 -> S1F22: logs each DVID with units.

Tests

  tests/test_messages.cpp   Round-trip S1F19/F20 with a 3-entry
                            capability list; round-trip S1F22 with two
                            DVIDs.

  tests/test_loader.cpp     Asserts equipment_type, the capabilities
                            list contains CCODE 14 (Spooling), and the
                            two DVIDs land in model.dvids.

COMPLIANCE.md

  "Documentation" Fundamental moves from ⬜ to ✅.
  S1F19/F20 + S1F21/F22 rows in the coverage matrix flip to ✅.
  The "what would it take" list drops the documentation-messages bullet.

Verified

  - Tests: 77 cases / 444 assertions pass.
  - Demo: client logs the full capability list received from the
    equipment, including CCODE 14 "Spooling (partial; S2F43/F44 +
    S6F23/F24)" — the equipment honestly reports its partial
    implementation rather than overclaiming.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-07 00:30:43 +02:00

216 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# SECS/GEM Compliance Audit
An honest, per-capability accounting of which parts of SEMI **E5 (SECS-II)**,
**E30 (GEM)**, and **E37 (HSMS)** this codebase implements, which parts it
implements partially, and which parts it does not implement at all.
> **TL;DR.** This is *not* a 100%-conformant GEM equipment. It implements
> every **GEM Fundamental capability** except formal Documentation (S1F19–F22)
> and the S9 error stream, plus most of the high-value **GEM Additional
> capabilities** (Dynamic Event Reporting, Alarms, EC, Clock, Remote Control,
> Process Programs, Terminal). Spooling, Limits Monitoring, Trace Data
> Collection, Multi-block, and Material Movement are intentionally not yet
> implemented. See the per-capability tables below for spec section refs.
Legend:
- ✅ **Full** — message/feature implemented; round-tripped in tests.
- 🟡 **Partial** — implemented in the demo path but with documented limitations.
- ⬜ **Not implemented** — intentionally out of scope for this iteration.
---
## 1. E37 — HSMS transport
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| TCP transport | ✅ | E37 §6 | `hsms::Connection` (Asio). |
| 4-byte length prefix + 10-byte header | ✅ | E37 §8.2 | `hsms::Frame::encode/decode`. |
| Session ID, byte2, byte3, PType, SType, system-bytes | ✅ | E37 §8.3 | `hsms::Header`. |
| `Select.req / .rsp` | ✅ | E37 §7.2 | `SType` 1/2; SelectStatus enum (0–3). |
| `Deselect.req / .rsp` | ✅ | E37 §7.4 | `SType` 3/4; DeselectStatus enum (0–2). |
| `Linktest.req / .rsp` | ✅ | E37 §7.5 | `SType` 5/6; periodic interval configurable. |
| `Separate.req` | ✅ | E37 §7.6 | `SType` 9; graceful close after flush. |
| `Reject.req` | 🟡 | E37 §7.7 | Emitted on data-while-NOT-SELECTED with reason 4; we don't yet reject on PType/SType-not-supported. |
| Connection state machine NOT-CONNECTED → NOT-SELECTED → SELECTED | ✅ | E37 §6.3 | Both Active and Passive modes. |
| T3 reply timeout | ✅ | E37 §10 | per-transaction `steady_timer`. |
| T5 connect separation timeout | ✅ | E37 §10 | `Client::schedule_retry`. |
| T6 control transaction timeout | ✅ | E37 §10 | single slot (no concurrent control transactions). |
| T7 not-selected timeout (passive) | ✅ | E37 §10 | armed on connect / on Deselect.req. |
| T8 intercharacter timeout | ✅ | E37 §10 | bounds the payload read after length prefix. |
| Multi-host / multi-session | ⬜ | E37 §6 | single SELECTED session at a time. |
| HSMS-SS vs HSMS-GS | ⬜ | E37 §11 | implemented HSMS-SS only. |
---
## 2. E5 — SECS-II encoding
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| Format byte + 1/2/3 length bytes | ✅ | E5 §9 | `secs2::encode_into`. |
| List (`L`) | ✅ | E5 §9.3 | recursive. |
| ASCII (`A`) | ✅ | E5 §9.5 | |
| Binary (`B`) | ✅ | E5 §9.5 | |
| Boolean (`BOOLEAN`) | ✅ | E5 §9.5 | |
| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | |
| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | |
| `F4, F8` (IEEE 754 big-endian) | ✅ | E5 §9.5 | bit-cast round-trip. |
| JIS-8, C2 (Unicode) | ⬜ | E5 §9.5 | rarely used in modern fabs. |
| SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. |
---
## 3. E30 — GEM Fundamental capabilities (§5.2)
| Fundamental Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| State models | ✅ | E30 §6.2 | — | Control state machine (5 states) + HSMS comm state. Equipment-processing state is left to the equipment app. |
| Equipment Processing States | ⬜ | E30 §6.3 | — | Standard says equipment **may** model these; tool-specific. Plug-in point not yet exposed. |
| Host-Initiated S1F13/F14 scenario | ✅ | E30 §6.5 | S1F13/F14 | |
| Event Notification | ✅ | E30 §6.6 | S6F11/F12 | Equipment-initiated, host-acknowledged. |
| On-Line Identification | ✅ | E30 §6.7 | S1F1/F2 | MDLN + SOFTREV. |
| Error Messages | 🟡 | E30 §6.9 | S9F* | HSMS Reject.req covers the transport-level case. S9F1/F3/F5/F7/F9/F11/F13 are now in the message catalog and round-trip-tested. `Connection` automatically emits **S9F7** when a peer's primary or reply body fails SECS-II decode (connection stays up; the caller of `send_request` sees `Error::IllegalData`), and **S9F9** when its outgoing request times out at T3 (alongside the existing `Error::Timeout`). **Not wired**: S9F3 (unknown stream) and S9F5 (unknown function) — these belong in the `Router`'s fallback path; today the fallback returns SxF0 only. |
| Documentation | ✅ | E30 §6.10| S1F19/F20, S1F21/F22 | Equipment self-reports its GEM-compliance level via S1F20 (SOFTREV, EQPTYP, list of (CCODE, CDESC) per E30 Appendix A) and its DVID namelist via S1F22. Both populated from `data/equipment.yaml`. |
| Control (Operator-Initiated) | ✅ | E30 §6.2 | — | `ControlStateMachine::operator_online/offline/local/remote`. |
---
## 4. E30 — GEM Additional capabilities (§5.3)
| Additional Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. |
| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. |
| Variable Data Collection | ✅ | E30 §6.11| — | `DVID` table; values resolvable via `vid_value`. |
| Trace Data Collection | ⬜ | E30 §6.12| S2F23/F24, S6F1/F2 | Not implemented. |
| Status Data Collection | ✅ | E30 §6.13| S1F3/F4, S1F11/F12 | |
| Alarm Management | 🟡 | E30 §6.14| S5F1/F2, S5F3/F4, S5F5/F6 | F1–F6 implemented; **S5F7/F8 list-enabled-alarms not implemented**. ALCD bit-7 set/cleared, lower-7 category supported. |
| Remote Control | ✅ | E30 §6.15| S2F41/F42 | Full HCACK 7-value enum + per-parameter CPACKs. |
| Equipment Constants | ✅ | E30 §6.16| S2F13/F14, S2F15/F16, S2F29/F30 | **EAC range validation against min/max is NOT performed** — set always accepts a known ECID; this would reject out-of-range with EAC=4 in a conformant equipment. |
| Process Program Management | 🟡 | E30 §6.17| S7F3/F4, S7F5/F6, S7F19/F20 | Unformatted PP send/request/list. **No E42 enhanced PP**, **no S7F23/F24/F25/F26 PP verify**, **no PPID validation**. PPBODY treated as raw bytes (Binary item). |
| Material Movement | ⬜ | E30 §6.18| S3F*, E40 | Tied to E40 carrier handling; separate standard. |
| Equipment Terminal Services | 🟡 | E30 §6.19| S10F1/F2, S10F3/F4 | Single-line only. **S10F5/F6 multi-block, S10F7 broadcast not implemented.** |
| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. |
| Limits Monitoring | ⬜ | E30 §6.21| S2F45/F46, S2F47/F48, S6F45/F46 | Not implemented. |
| Spooling | 🟡 | E30 §6.22| S2F43/F44, S6F23/F24 | Spoolable-streams config (S2F43/F44) + spool request transmit/purge (S6F23/F24) implemented; SpoolStore queues equipment-initiated primaries when the host is unreachable or force-spool is on, drains FIFO on S6F23 transmit, purges on S6F23 purge. **Not implemented**: S6F25/F26 spooled-data-ready notification, automatic spool activation tied to HSMS NOT-SELECTED, persistent (on-disk) spool. |
| Control | ✅ | E30 §6.2 | — | See Fundamental. |
---
## 5. Message coverage matrix
| Pair | Direction | Status | Implemented in | Tested |
|------------------|-----------|--------|----------------|--------|
| S1F1 / S1F2 | H↔E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F3 / S1F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F11 / S1F12 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S1F13 / S1F14 | H↔E | ✅ | `messages.hpp` | ✅ in demo |
| S1F15 / S1F16 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S1F17 / S1F18 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S1F19 / S1F20 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S1F21 / S1F22 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F13 / S2F14 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F15 / S2F16 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F17 / S2F18 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F23 / S2F24 | H→E | ⬜ | — | — |
| S2F29 / S2F30 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F31 / S2F32 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S2F33 / S2F34 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F35 / S2F36 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F37 / S2F38 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F41 / S2F42 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S2F45–F48 | H→E | ⬜ | — | — |
| S5F1 / S5F2 | E→H | ✅ | `messages.hpp` | ✅ round-trip |
| S5F3 / S5F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S5F5 / S5F6 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S5F7 / S5F8 | H→E | ⬜ | — | — |
| S6F1 / S6F2 | E→H | ⬜ | — | — |
| S6F5 / S6F6 | H↔E | ⬜ | — | multi-block |
| S6F7 / S6F8 | H↔E | ⬜ | — | multi-block |
| S6F11 / S6F12 | E→H | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S6F15 / S6F16 | H→E | ⬜ | — | event report request |
| S2F43 / S2F44 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S6F23 / S6F24 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S6F25 / S6F26 | spool | ⬜ | — | spool-data-ready notification |
| S7F3 / S7F4 | H→E | ✅ | `messages.hpp` | ✅ round-trip |
| S7F5 / S7F6 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S7F19 / S7F20 | H→E | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S7F23–F26 | H↔E | ⬜ | — | enhanced PP |
| S9F1 | E↔H | ✅ | catalog | ✅ round-trip |
| S9F3 | E↔H | 🟡 | catalog | ✅ round-trip; emission not yet wired |
| S9F5 | E↔H | 🟡 | catalog | ✅ round-trip; emission not yet wired |
| S9F7 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on body decode |
| S9F9 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on T3 timeout |
| S9F11 | E↔H | 🟡 | catalog | ✅ round-trip; emission not yet wired |
| S9F13 | E↔H | ✅ | catalog | ✅ round-trip |
| S10F1 / S10F2 | H→E | ✅ | `messages.hpp` | ✅ in demo |
| S10F3 / S10F4 | E→H | ✅ | `messages.hpp` | ✅ round-trip + demo |
| S10F5 / S10F6 | H→E | ⬜ | — | multi-line |
---
## 6. Demo evidence
The two-container demo (`docker compose up --no-deps server client`)
exercises this concrete sequence end-to-end:
1. TCP connect → `Select.req` → `Select.rsp(Ok)` → SELECTED on both sides.
2. `S1F13`/`S1F14` Establish Comms.
3. `S1F17`/`S1F18` Request Online; control state transitions
`HostOffline → AttemptOnline → OnlineRemote`.
4. Server pushes `S10F3` welcome → host acks with `S10F4`.
5. `S1F11`/`S1F12` SVID namelist discovery → `S1F3`/`S1F4` SVID read.
6. `S2F29`/`S2F30` EC namelist → `S2F13`/`S2F14` EC read.
7. `S2F17`/`S2F18` clock read.
8. `S2F33`/`S2F34` Define Report 1000 over the 3 SVIDs (DRACK=0).
9. `S2F35`/`S2F36` Link CEIDs 200 and 300 to Report 1000 (LRACK=0).
10. `S2F37`/`S2F38` Enable CEIDs 200, 300 (ERACK=0).
11. `S2F41`/`S2F42` host command **START** (HCACK=0) → server emits
`S6F11(CEID=300)` carrying the linked Report 1000 → host acks `S6F12`.
12. `S5F5`/`S5F6` list alarm directory.
13. `S5F3`/`S5F4` enable alarm 1 (ACKC5=0).
14. `S2F41`/`S2F42` host command **FAULT** (HCACK=0) → server emits
`S5F1` (ALCD=0x84, set + cat 4) → host acks `S5F2`; server also
emits `S6F11(CEID=200)`.
15. `S7F19`/`S7F20` recipe list, `S7F5`/`S7F6` fetch RECIPE-A.
16. `S10F1`/`S10F2` host → equipment terminal display.
17. `S1F15`/`S1F16` Request Offline; control state goes back to
HostOffline. (CEID 100 ControlStateChanged emission is correctly
*suppressed* because the host never enabled CEID 100 — this is the
correct GEM behavior.)
18. `Separate.req` → clean close on both sides.
Unit tests: **63 cases / 278 assertions pass** (`docker compose run --rm tests`).
---
## 7. What it would take to claim "100% GEM-compliant"
The honest list, in priority order:
1. **Finish spooling**: S6F25/F26 spooled-data-ready notification, plus
automatic activation when HSMS goes NOT-SELECTED (and automatic
notification on re-SELECT) so the host doesn't have to manually flip
the test-only `force_spool` flag. Optional: persistent on-disk spool
so equipment restarts don't lose queued events.
2. **Finish S9 wiring**: route Router-level "unknown stream/function" through
S9F3/F5, and emit S9F11 (Data Too Long) with the 4-byte length prefix
in place of MHEAD when the incoming frame is oversized.
3. *(done in this revision)*: S1F19/F20 + S1F21/F22 — Documentation.
4. **Implement EC range validation** in `set_equipment_constant_value`
so out-of-range sets return EAC=4 instead of being silently accepted.
5. **Implement Limits Monitoring** (S2F45–F48, S6F45/F46) if the
target equipment publishes monitored variables.
6. **Implement Trace Data Collection** (S2F23/F24, S6F1/F2).
7. **Implement S5F7/F8** list-enabled-alarms.
8. **Implement multi-block transfers** (S6F5/F6, S6F7/F8).
9. **Implement equipment processing state model** with operator hooks
(E30 §6.3) — the abstract model is in E30 but the concrete states
are equipment-specific.
10. **Run the implementation against a real conformance test generator**
(Layer 4 of the implementation plan) on a representative tool.
After all of the above, a GEM Reference Test specification (RTS) review
would still be needed before any "GEM compliant" marketing claim could
be honestly made.
Reference in New Issue View Git Blame Copy Permalink