raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DD1: refresh COMPLIANCE.md for full GEM 300 + 291 tests

Browse Source 
The doc was last updated when only E40 + E94 were in tree.  Brings it
up to date with everything actually implemented:

  * New §4a–§4k tables for each GEM 300 standard: E40, E94, E87, E90,
    E116, E120, E148, E157, E84, E5 §13 wafer maps, and the exception
    recovery extension (S5F13–F18 + ExceptionStateMachine).
  * Refreshed message coverage matrix to all 149 catalog entries
    (added S1F23/F24, S2F21/F22, S6F5–F8/F15–F22, S7F1/F2/F17/F18,
    S10F3/F4, S12F9–F18).
  * Updated test count to 291 cases / 1515 assertions.
  * New §7 documents the secsgem-py interop harness (24 host-side
    checks + raw-GEM300 round-trip).
  * §8 trimmed: persistent spool is no longer "out of scope" (CC1
    landed); E40/E87/E90 removed from "Layer 5 follow-on" list since
    they're done.
  * §9 honesty pass — every GEM 300 standard in scope now implemented
    end-to-end; the remaining gap is third-party RTS certification +
    per-vendor application wiring.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-08 23:54:46 +02:00
parent d69f26b415
commit 0bbc7b7acf
1 changed files with 246 additions and 148 deletions
Download Patch File Download Diff File Expand all files Collapse all files
COMPLIANCE.md
+246 -148
View File
@@ -1,13 +1,19 @@
# SECS/GEM Compliance
A per-capability accounting against SEMI **E5 (SECS-II)**, **E30 (GEM)**,
**E37 (HSMS)**, and **GEM300 E40 / E94** (process / control jobs).
A per-capability accounting against the foundational SEMI standards
**E5 (SECS-II)**, **E30 (GEM)**, **E37 (HSMS)**, **E4 (SECS-I)**, plus
the full GEM 300 stack: **E40** (process jobs), **E94** (control jobs),
**E87** (carriers), **E90** (substrates), **E116** (equipment
performance tracking), **E120** (common equipment model), **E157**
(module process tracking), **E84** (parallel I/O), **E148** (time
synchronization), **E39** (object services), plus **E5 §13** wafer maps.
> **Status.** Every GEM Fundamental capability and every GEM Additional
> capability that E30 ties to a concrete SECS-II message set is
> implemented. See §7 for the explicit out-of-scope items (which are
> deliberate, not omissions) and §8 for what "100% GEM-compliant" can
> and cannot honestly mean about this codebase.
> **Status.** Every GEM Fundamental and every GEM Additional capability
> that E30 binds to a concrete SECS-II message set is implemented, and
> every GEM 300 standard the project sets out to cover is implemented
> end-to-end (state machines + stores + wire messages + dispatch). See
> §8 for the items deliberately out of scope and §9 for what
> "100% GEM-compliant" can and cannot honestly mean about a codebase.
Legend:
@@ -68,12 +74,14 @@ Legend:
| ASCII (`A`) | ✅ | E5 §9.5 | |
| Binary (`B`) | ✅ | E5 §9.5 | |
| Boolean (`BOOLEAN`) | ✅ | E5 §9.5 | |
| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | |
| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | |
| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | Identifier parsers accept any width per the SEMI wildcard rule. |
| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | Same lenient-width policy. |
| `F4, F8` (IEEE 754 big-endian) | ✅ | E5 §9.5 | bit-cast round-trip. |
| JIS-8 (single-byte JIS text) | ✅ | E5 §9.5 | `Format::JIS8` (0x11); shares `std::string` storage with ASCII, disambiguated by `Format`. |
| C2 (Unicode 2-byte code points) | ✅ | E5 §9.5 | `Format::C2` (0x12); big-endian uint16_t code points. |
| SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. JIS-8 prints as `<J "...">`, C2 as `<C 65 66 ...>`. |
| SML parser (inverse of `to_sml`) | ✅ | — | `secs2::from_sml`; round-trips every format. |
| `ASCII | Binary` wildcard fields | ✅ | E5 | `BINARY_OR_ASCII` schema type for PPBODY etc.; accepted by `as_text_or_binary`. |
---
@@ -82,12 +90,12 @@ Legend:
| Fundamental Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| State models | ✅ | E30 §6.2 | — | E30 control state machine (5 states) + HSMS communication state machine. |
| Equipment Processing States | ✅ | E30 §6.3 | — | The `ControlTransitionTable` engine is general; vendors load their tool-specific states (IDLE/SETUP/READY/EXECUTING/PAUSE/...) via a second YAML file using the same loader. The spec leaves the concrete states tool-specific. |
| Equipment Processing States | ✅ | E30 §6.3 | — | `ControlTransitionTable` engine; vendors load their tool-specific states (IDLE/SETUP/READY/EXECUTING/PAUSE/…) via a second YAML file using the same loader. The spec leaves the concrete states tool-specific. |
| Host-Initiated S1F13/F14 scenario | ✅ | E30 §6.5 | S1F13/F14 | |
| Event Notification | ✅ | E30 §6.6 | S6F11/F12 | Equipment-initiated, host-acknowledged. |
| On-Line Identification | ✅ | E30 §6.7 | S1F1/F2 | MDLN + SOFTREV. |
| Error Messages | ✅ | E30 §6.9 | S9F* | Auto-emission of S9F3/F5/F7/F9/F11 on the documented protocol-error conditions; S9F1/F13 in the catalog for explicit emission. |
| Documentation | ✅ | E30 §6.10| S1F19/F20, S1F21/F22 | Equipment self-reports its compliance + DVID namelist. |
| Documentation | ✅ | E30 §6.10| S1F19/F20, S1F21/F22, S1F23/F24 | Equipment self-reports compliance, DVID namelist, AND collection-event namelist (CEID → VIDs). |
| Control (Operator-Initiated) | ✅ | E30 §6.2 | — | `ControlStateMachine::operator_online/offline/local/remote`. |
---
@@ -97,133 +105,201 @@ Legend:
| Additional Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. Backed by the E30 §6.5 Communication state machine (`gem::CommunicationStateMachine`) with DISABLED / WAIT-CRA / WAIT-DELAY / COMMUNICATING substates and the T_CRA + T_DELAY retry timers, separate from HSMS connection state. |
| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. |
| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. Host-initiated readback via S6F15/F16, S6F19/F20, S6F21/F22. |
| Variable Data Collection | ✅ | E30 §6.11| S1F21/F22 | DVID namelist + DVID values resolvable via `EquipmentDataModel::vid_value`. |
| Trace Data Collection | ✅ | E30 §6.12| S2F23/F24, S6F1/F2 | `TraceStore` keeps active TRID→TraceConfig; periodic sampling left to the application's scheduler. |
| Status Data Collection | ✅ | E30 §6.13| S1F3/F4, S1F11/F12 | |
| Alarm Management | ✅ | E30 §6.14| S5F1/F2, S5F3/F4, S5F5/F6, S5F7/F8 | Full set. ALCD bit-7 set/cleared, lower-7 category. |
| Remote Control | ✅ | E30 §6.15| S2F41/F42 | Full HCACK 7-value enum + per-parameter CPACKs. |
| Remote Control | ✅ | E30 §6.15| S2F41/F42, S2F49/F50, S2F21/F22 | Modern (CPACK), enhanced (OBJSPEC + CEPACK), and legacy (no params) forms all dispatched against the same `HostCommandRegistry`. |
| Equipment Constants | ✅ | E30 §6.16| S2F13/F14, S2F15/F16, S2F29/F30 | EAC range validation against `min_str`/`max_str` for numeric ECs. |
| Process Program Management | ✅ | E30 §6.17| S7F3/F4, S7F5/F6, S7F19/F20 | Unformatted PP send/request/list — the minimum E30 GEM requires. (E42 *enhanced* PP is a separate SEMI standard; see §7.) |
| Material Movement | 🟡 | E30 §6.18| — | Process Job + Control Job lifecycle covered via E40/E94 (see §4a). Carrier (E87) and substrate (E90) still out of scope. |
| Process Program Management | ✅ | E30 §6.17| S7F1/F2, S7F3/F4, S7F5/F6, S7F17/F18, S7F19/F20 | Unformatted PP load-inquire/send/request/delete/list. Covers the full GEM PP set (E42 *enhanced* PP is a separate standard; see §8). |
| Material Movement | ✅ | E30 §6.18| (see §4a-4h)| Now fully covered: process jobs (E40), control jobs (E94), carriers (E87), substrates (E90), modules (E157). |
| Equipment Terminal Services | ✅ | E30 §6.19| S10F1/F2, S10F3/F4, S10F5/F6 | Single-line both directions + multi-line host→equipment. S10F7 broadcast intentionally omitted (rarely used). |
| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. |
| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. Drift tracking + quality via E148 (§4g). |
| Limits Monitoring | ✅ | E30 §6.21| S2F45/F46, S2F47/F48 | `LimitMonitorStore` keyed by VID with multiple `LimitDefinition` (LIMITID + upper/lower as arbitrary Items). |
| Spooling | ✅ | E30 §6.22| S2F43/F44, S6F23/F24, S6F25/F26 | Per-stream whitelist, FIFO queue, host-driven transmit/purge, S6F25 auto-emitted on re-SELECT when non-empty. |
| Spooling | ✅ | E30 §6.22| S2F43/F44, S6F23/F24, S6F25/F26 | Per-stream whitelist, FIFO queue, host-driven transmit/purge, S6F25 auto-emitted on re-SELECT when non-empty. **Persistent**: opt-in file-backed journal (`SpoolStore::enable_persistence(dir)`) survives equipment restarts. |
| Control | ✅ | E30 §6.2 | — | See Fundamental. |
---
## 4a. E40 Process Jobs + E94 Control Jobs (GEM300)
The first GEM300 extension landing on the spec-as-data architecture.
Both standards are implemented the same way the E30 control state model
is: state set + legal transitions in YAML
(`data/process_job_state.yaml`, `data/control_job_state.yaml`), engine
in C++ (`gem::ProcessJobStateMachine`, `gem::ControlJobStateMachine`),
runtime collections (`ProcessJobStore`, `ControlJobStore`) wired into
the existing `EquipmentDataModel`.
## 4a. E40 Process Jobs
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| E40 PJ state model | ✅ | E40 §6.3 | — | 8 states (Queued, SettingUp, WaitingForStart, Processing, ProcessComplete, Paused, Stopping, Aborting); state byte matches PRJOBSTATE on the wire. |
| E40 PRJobCreate | ✅ | E40 §10.2| S16F11/F12 | Body simplified to <L,3 PRJOBID PPID L,n MTRLOUTSPEC>; MF/PRRECIPEMETHOD/PRPROCESSPARAMS are documented as YAML-extension points. PPID validated against `RecipeStore`. |
| E40 PRJobDequeue | ✅ | E40 §10.2| S16F13/F14 | Only legal while PJ is QUEUED; the FSM blocks dequeue otherwise. |
| E40 PRJobCommand | ✅ | E40 §10.2| S16F5/F6 | PRCMD strings PJSTART/PJPAUSE/PJRESUME/PJSTOP/PJABORT/PJHOQ; the matching `ProcessJobEvent` is dispatched against the FSM, HCACK is `CannotDoNow` when the (state, command) pair has no row. |
| E40 PRJobAlert | ✅ | E40 §10.3| S16F9 | Equipment-initiated one-way (W=0). Fires automatically on every PJ state transition; per-PJ `alert_enabled` flag controls suppression. |
| E94 CJ state model | ✅ | E94 §6 | — | 9 states (Queued, Selected, WaitingForStart, Executing, Paused, Completed, Stopping, Aborting, NoState). CJ owns an ordered `prjobids` list. |
| E94 CreateObject (CJ) | ✅ | E94 §6.4 | S14F9/F10 | Body simplified to <L,2 CTLJOBID L,n PRJOBIDs>; full generic E14 ObjectService form is a YAML extension. OBJACK enum covers Success/Error + the four `Denied_*` cases. |
| E94 DeleteObject (CJ) | ✅ | E94 §6.4 | S14F11/F12 | |
| E94 CJobCommand | ✅ | E94 §6.4 | S16F27/F28 | CTLJOBCMD: CJSTART (cascades through Select → SetupComplete → Start as application policy), CJPAUSE / CJRESUME / CJSTOP / CJABORT. |
| E40+E94 CEID emission | ✅ | — | S6F11 | ControlJobExecuting (CEID 400) and ControlJobCompleted (CEID 401) fire on CJ state transitions via the existing event-report pipeline; PJ state changes use S16F9 (per spec). |
| PJ state model | ✅ | E40 §6.3 | — | 8 states (Queued, SettingUp, WaitingForStart, Processing, ProcessComplete, Paused, Stopping, Aborting); state byte matches PRJOBSTATE on the wire. |
| PRJobCreate (full body) | ✅ | E40 §10.2| S16F11/F12 | Full E40-0705 body: `<L,5 PRJOBID MF PRRECIPEMETHOD <L,2 PPID <L RCPVARLIST>> <L MTRLOUTSPEC> <L PRPROCESSPARAMS>>`. PPID validated against `RecipeStore`. |
| PRJobCreateMultiple | ✅ | E40 §10 | S16F15/F16 | Bulk variant; per-job ACK list. |
| PRJobDequeue | ✅ | E40 §10.2| S16F13/F14 | Only legal while PJ is QUEUED. |
| PRJobMonitor | ✅ | E40 §10 | S16F7/F8 | Per-PJ alert enable/disable. |
| PRJobCommand | ✅ | E40 §10.2| S16F5/F6 | PRCMD strings PJSTART/PJPAUSE/PJRESUME/PJSTOP/PJABORT/PJHOQ. |
| PRJobAlert | ✅ | E40 §10.3| S16F9 | Equipment-initiated one-way (W=0). Fires automatically on every PJ state transition. |
The demo's `run_cj_lifecycle` cascade — on CJSTART the CJ steps Queued
→ Selected → WaitingForStart → Executing and every contained PJ steps
through SettingUp → WaitingForStart → Processing → ProcessComplete —
is **application policy**, not the FSM. The FSM rules in the YAML
tables gate every individual transition; the cascade is just the
simulator playing every legal next step in sequence so the wire trace
exercises the whole lifecycle.
## 4b. E94 Control Jobs
What's **out of scope for the E40/E94 first pass** (deliberate; all are
YAML/handler extensions, not surgery):
- Full E40 S16F11 body (MF / PRRECIPEMETHOD / RCPSPEC / PRPROCESSPARAMS).
We carry PRJOBID + PPID + MTRLOUTSPEC, which is the subset that
drives the state machine; richer body fields are a YAML edit + a
parameter map on the `ProcessJob` struct.
- S16F15/F16 PRJobCreateMultiple, S16F17/F18 PRJobMultipleDequeue.
- E14 generic ObjectService form. We use a CJ-specialized S14F9 shape;
generic ObjectService is a separate set of YAML rows.
- E87 Carrier Management and E90 Substrate Tracking — Layer 5
continues there per `implementation_plan.md`.
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| CJ state model | ✅ | E94 §6 | — | 9 states; CJ owns an ordered `prjobids` list. |
| CreateObject (CJ) | ✅ | E94 §6.4 | S14F9/F10 | Body `<L,2 CTLJOBID L,n PRJOBIDs>`. |
| DeleteObject (CJ) | ✅ | E94 §6.4 | S14F11/F12 | |
| CJobCommand | ✅ | E94 §6.4 | S16F27/F28 | CTLJOBCMD: CJSTART, CJPAUSE, CJRESUME, CJSTOP, CJABORT. |
| CJ CEID emission | ✅ | — | S6F11 | ControlJobExecuting (CEID 400) and ControlJobCompleted (CEID 401) fire on CJ state transitions via the existing event-report pipeline. |
## 4c. E87 Carrier Management
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Carrier state model | ✅ | E87 §6 | — | CarrierStateMachine + LoadPortStateMachine; CIDS, CSMS, CASS, LPRS, LPTS axes. |
| CarrierAction | ✅ | E87 §10.2| S3F17/F18 | ProceedWithCarrier, CancelCarrier, BindCarrierID. |
| Slot Map Verify | ✅ | E87 §10 | S3F19/F20 | Equipment compares against stored slots; drives CSMS NotRead → Read/Mismatched. |
| Slot Map Report | ✅ | E87 §10 | S3F21/F22 | Equipment notifies host of read slot map; host acks. |
| Port Group Change | ✅ | E87 §10.4| S3F23/F24 | Host modifies load-port grouping; PortGroupAck. |
| Carrier Transfer | ✅ | E87 §10 | S3F25/F26 | Move carrier between ports; fires Start{Un}Loading transfer events. |
| Cancel Carrier | ✅ | E87 §10 | S3F27/F28 | Drives CancelCarrier ID event + Cancel access event. |
## 4d. E90 Substrate Tracking
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Substrate state model | ✅ | E90 §6 | — | Three axes: location (AtSource/AtWork/AtDestination), processing (NeedsProcessing/InProcess/Processed/Aborted/Stopped/Rejected/Lost/Skipped), id-status (Confirmed/Unconfirmed/Lost). |
| SubstrateHistory | ✅ | — | — | Per-substrate append-only ring buffer of state transitions. |
| Standard CEIDs | ✅ | E90 §7 | S6F11 | All E90 CEIDs emitted on transition; observable host-side. |
## 4e. E116 Equipment Performance Tracking
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| EPT state machine | ✅ | E116 §6 | — | NonScheduledTime / ScheduledDowntime / UnscheduledDowntime / Engineering / Standby / Productive. |
| Time-bucket accounting | ✅ | E116 | — | Cumulative ms per state; resettable. |
| EPT CEIDs | ✅ | E116 | S6F11 | One CEID per state. |
## 4f. E120 Common Equipment Model
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Generic ObjectService (E39) | ✅ | E39 §5 | S14F1/F2, S14F3/F4 | GetAttr / SetAttr against `CemObjectStore`; OBJTYPE validation. |
| CemObjectStore | ✅ | E120 | — | Typed objects keyed by OBJSPEC + ObjType. |
## 4g. E148 Time Synchronization
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Time-sync drift tracking | ✅ | E148 | S2F31/F32 | Drift metric maintained on every set; quality score for hosts. |
## 4h. E157 Module Process Tracking
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Module state machine | ✅ | E157 §6 | — | NotExecuting / GeneralExecuting / StepExecuting / StepCompleted. |
| Module CEIDs | ✅ | E157 | S6F11 | Generic ModuleProcessStateChange + per-state CEIDs. |
## 4i. E84 Parallel I/O Handoff
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Handoff state machine | ✅ | E84 | — | Full LOAD / UNLOAD signal vocabulary (CS_0/CS_1, VALID, TR_REQ, BUSY, COMPT, AM_AVBL, ES). |
## 4j. E5 §13 Wafer Maps (S12)
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Map Setup Data | ✅ | E5 §13 | S12F1/F2, S12F3/F4 | |
| Map Transmit Inquire / Grant | ✅ | E5 §13 | S12F5/F6 | |
| Map Data Send — row format (MAPFT=0) | ✅ | E5 §13 | S12F7/F8 | |
| Map Data Send — array format (MAPFT=1)| ✅ | E5 §13 | S12F9/F10 | STRP + BINLT. |
| Map Data Send — coord format (MAPFT=2)| ✅ | E5 §13 | S12F11/F12 | XYPOS + per-die BIN. |
| Map Data Request — row | ✅ | E5 §13 | S12F13/F14 | |
| Map Data Request — array | ✅ | E5 §13 | S12F15/F16 | |
| Map Data Request — coord | ✅ | E5 §13 | S12F17/F18 | |
| Map Error Send | ✅ | E5 §13 | S12F19 | One-way error report. |
## 4k. Exception Recovery (beyond E5 base alarms)
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| ExceptionStateMachine FSM | ✅ | — | — | Per-EXID Posted / Recovering / RecoverFailed / Cleared lifecycle. Not in upstream secsgem-py. |
| Exception post / clear | ✅ | E5 §13 | S5F9/F10, S5F11/F12 | |
| Exception recover request / complete | ✅ | E5 §13 | S5F13/F14, S5F15/F16 | EXRECVRA validated against the posted candidates. |
| Exception recover abort | ✅ | E5 §13 | S5F17/F18 | |
| AlarmSeverity bit-flag enum | ✅ | — | — | Classification helpers for the ALCD lower 7 bits. |
---
## 5. Message coverage matrix
| Pair | Direction | Status | Implemented in | Tested |
|------------------|-----------|--------|----------------|--------|
| S1F1 / S1F2 | H↔E | ✅ | catalog | ✅ round-trip + demo |
| S1F3 / S1F4 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S1F11 / S1F12 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S1F13 / S1F14 | H↔E | ✅ | catalog | ✅ round-trip + demo |
| S1F15 / S1F16 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S1F17 / S1F18 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S1F19 / S1F20 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S1F21 / S1F22 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F13 / S2F14 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F15 / S2F16 | H→E | ✅ | catalog | ✅ round-trip |
| S2F17 / S2F18 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F23 / S2F24 | H→E | ✅ | catalog | ✅ round-trip |
| S2F25 / S2F26 | H→E | ✅ | catalog | ✅ round-trip (loopback diagnostic) |
| S2F29 / S2F30 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F31 / S2F32 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F33 / S2F34 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F35 / S2F36 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F37 / S2F38 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F41 / S2F42 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F43 / S2F44 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F45 / S2F46 | H→E | ✅ | catalog | ✅ round-trip |
| S2F47 / S2F48 | H→E | ✅ | catalog | ✅ round-trip |
| S5F1 / S5F2 | E→H | ✅ | catalog | ✅ round-trip + demo |
| S5F3 / S5F4 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S5F5 / S5F6 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S5F7 / S5F8 | H→E | ✅ | catalog | ✅ round-trip |
| S5F9 / S5F10 | E→H | ✅ | catalog | ✅ round-trip (exception post) |
| S5F11 / S5F12 | E→H | ✅ | catalog | ✅ round-trip (exception clear) |
| S5F13 / S5F14 | H→E | ✅ | catalog | ✅ round-trip (exception recover request) |
| S5F15 / S5F16 | E→H | ✅ | catalog | ✅ round-trip (exception recover complete) |
| S5F17 / S5F18 | H→E | ✅ | catalog | ✅ round-trip (exception recover abort) |
| S2F49 / S2F50 | H→E | ✅ | catalog + server | ✅ round-trip + dispatch (enhanced remote command, OBJSPEC + per-CP CPACK/CEPACK) |
| S12F1 / S12F2 | H→E | ✅ | catalog | ✅ round-trip (Map Setup Data Send, E5 §13) |
| S12F3 / S12F4 | H→E | ✅ | catalog | ✅ round-trip (Map Setup Data Request) |
| S12F5 / S12F6 | E→H | ✅ | catalog | ✅ round-trip (Map Transmit Inquire / Grant) |
| S12F7 / S12F8 | E→H | ✅ | catalog | ✅ round-trip (Map Data Send, row format MAPFT=0) |
| S12F19 | E→H | ✅ | catalog | ✅ round-trip (Map Error Send, one-way) |
| S6F1 / S6F2 | E→H | ✅ | catalog | ✅ round-trip |
| S6F11 / S6F12 | E→H | ✅ | catalog | ✅ round-trip + demo |
| S6F23 / S6F24 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S6F25 / S6F26 | E→H | ✅ | catalog + server | ✅ round-trip + auto-emitted on re-SELECT |
| S7F3 / S7F4 | H→E | ✅ | catalog | ✅ round-trip |
| S7F5 / S7F6 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S7F19 / S7F20 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S9F1 | E↔H | ✅ | catalog | ✅ round-trip |
| S9F3 | E↔H | ✅ | catalog + Router wrapper | ✅ round-trip + auto-emitted on unknown stream |
| S9F5 | E↔H | ✅ | catalog + Router wrapper | ✅ round-trip + auto-emitted on unknown function |
| S9F7 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on body decode |
| S9F9 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on T3 timeout |
| S9F11 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on oversized frame |
| S9F13 | E↔H | ✅ | catalog | ✅ round-trip |
| S10F1 / S10F2 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S10F3 / S10F4 | E→H | ✅ | catalog | ✅ round-trip + demo |
| S10F5 / S10F6 | H→E | ✅ | catalog | ✅ round-trip |
| S14F9 / S14F10 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJ create) |
| S14F11 / S14F12 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJ delete) |
| S16F5 / S16F6 | H→E | ✅ | catalog | ✅ round-trip (E40 PRJobCommand) |
| S16F9 | E→H | ✅ | catalog + server | ✅ round-trip + auto-emitted per PJ transition (E40 PRJobAlert) |
| S16F11 / S16F12 | H→E | ✅ | catalog | ✅ round-trip + demo (E40 PRJobCreate) |
| S16F13 / S16F14 | H→E | ✅ | catalog | ✅ round-trip (E40 PRJobDequeue) |
| S16F27 / S16F28 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJobCommand) |
149 SECS-II messages in the catalog, spanning streams 1, 2, 3, 5, 6, 7, 9, 10, 12, 14, 16.
| Pair | Direction | Status | Notes |
|------------------|-----------|--------|-------|
| S1F1 / S1F2 | H↔E | ✅ | round-trip + demo |
| S1F3 / S1F4 | H→E | ✅ | round-trip + demo |
| S1F11 / S1F12 | H→E | ✅ | round-trip + demo |
| S1F13 / S1F14 | H↔E | ✅ | round-trip + demo |
| S1F15 / S1F16 | H→E | ✅ | round-trip + demo |
| S1F17 / S1F18 | H→E | ✅ | round-trip + demo |
| S1F19 / S1F20 | H→E | ✅ | round-trip + demo (compliance self-report) |
| S1F21 / S1F22 | H→E | ✅ | round-trip + demo |
| S1F23 / S1F24 | H→E | ✅ | collection event namelist (CEID → VID) |
| S2F13 / S2F14 | H→E | ✅ | EC values |
| S2F15 / S2F16 | H→E | ✅ | EC set |
| S2F17 / S2F18 | H→E | ✅ | clock |
| S2F21 / S2F22 | H→E | ✅ | legacy remote command (no params) |
| S2F23 / S2F24 | H→E | ✅ | trace initialize |
| S2F25 / S2F26 | H→E | ✅ | loopback diagnostic |
| S2F29 / S2F30 | H→E | ✅ | EC namelist |
| S2F31 / S2F32 | H→E | ✅ | set time |
| S2F33 / S2F34 | H→E | ✅ | define report |
| S2F35 / S2F36 | H→E | ✅ | link event report |
| S2F37 / S2F38 | H→E | ✅ | enable event |
| S2F41 / S2F42 | H→E | ✅ | host command (modern) |
| S2F43 / S2F44 | H→E | ✅ | reset spooling |
| S2F45 / S2F46 | H→E | ✅ | define variable limits |
| S2F47 / S2F48 | H→E | ✅ | request limit attrs |
| S2F49 / S2F50 | H→E | ✅ | enhanced remote command (OBJSPEC + CPACK/CEPACK) |
| S3F17 / S3F18 | H→E | ✅ | E87 carrier action |
| S3F19 / S3F20 | H→E | ✅ | E87 slot map verify |
| S3F21 / S3F22 | E→H | ✅ | E87 slot map report |
| S3F23 / S3F24 | H→E | ✅ | E87 port group change |
| S3F25 / S3F26 | H→E | ✅ | E87 carrier transfer |
| S3F27 / S3F28 | H→E | ✅ | E87 cancel carrier |
| S5F1 / S5F2 | E→H | ✅ | alarm send |
| S5F3 / S5F4 | H→E | ✅ | enable alarm |
| S5F5 / S5F6 | H→E | ✅ | list alarms |
| S5F7 / S5F8 | H→E | ✅ | list enabled alarms |
| S5F9 / S5F10 | E→H | ✅ | exception post |
| S5F11 / S5F12 | E→H | ✅ | exception clear |
| S5F13 / S5F14 | H→E | ✅ | exception recover request |
| S5F15 / S5F16 | E→H | ✅ | exception recover complete |
| S5F17 / S5F18 | H→E | ✅ | exception recover abort |
| S6F1 / S6F2 | E→H | ✅ | trace data |
| S6F5 / S6F6 | E→H | ✅ | multi-block inquire / grant |
| S6F7 / S6F8 | H→E | ✅ | data transfer request / send |
| S6F11 / S6F12 | E→H | ✅ | event report (unsolicited) |
| S6F15 / S6F16 | H→E | ✅ | event report request (host-initiated) |
| S6F19 / S6F20 | H→E | ✅ | individual report request |
| S6F21 / S6F22 | H→E | ✅ | annotated individual report request |
| S6F23 / S6F24 | H→E | ✅ | request spooled data |
| S6F25 / S6F26 | E→H | ✅ | spool data ready (auto on re-SELECT) |
| S7F1 / S7F2 | H→E | ✅ | PP load inquire / grant |
| S7F3 / S7F4 | H→E | ✅ | PP send |
| S7F5 / S7F6 | H→E | ✅ | PP request |
| S7F17 / S7F18 | H→E | ✅ | PP delete |
| S7F19 / S7F20 | H→E | ✅ | PP list |
| S9F1, F3, F5, F7, F9, F11, F13 | E↔H | ✅ | protocol errors; auto-emitted on the documented conditions |
| S10F1 / S10F2 | E→H | ✅ | terminal request (equipment originated) |
| S10F3 / S10F4 | H→E | ✅ | terminal display single |
| S10F5 / S10F6 | H→E | ✅ | terminal display multi |
| S12F1–F19 | H↔E | ✅ | wafer maps — row, array, coord; setup, request, send, error |
| S14F1 / S14F2 | H→E | ✅ | E39 GetAttr |
| S14F3 / S14F4 | H→E | ✅ | E39 SetAttr |
| S14F9 / S14F10 | H→E | ✅ | E94 CJ create |
| S14F11 / S14F12 | H→E | ✅ | E94 CJ delete |
| S16F5 / S16F6 | H→E | ✅ | E40 PRJobCommand |
| S16F7 / S16F8 | H→E | ✅ | E40 PRJobMonitor |
| S16F9 | E→H | ✅ | E40 PRJobAlert (auto on transition) |
| S16F11 / S16F12 | H→E | ✅ | E40 PRJobCreate (full body) |
| S16F13 / S16F14 | H→E | ✅ | E40 PRJobDequeue |
| S16F15 / S16F16 | H→E | ✅ | E40 PRJobCreateMultiple |
| S16F27 / S16F28 | H→E | ✅ | E94 CJobCommand |
---
@@ -265,60 +341,82 @@ walks ~20 SECS transactions end-to-end:
23. `S1F15`/`S1F16` Request Offline.
24. `Separate.req` → clean close on both sides.
Unit tests: **148 cases / 794 assertions pass** (`docker compose run --rm tests`).
Unit tests: **291 cases / 1515 assertions pass** (`docker compose run --rm tests`).
The suite includes integration tests that drive a real `hsms::Connection`
over a loopback socket pair to verify the E37 §7.2 / §7.4 / §7.7
edge cases (Select.req while already SELECTED → AlreadyActive, Deselect.req
while NOT_SELECTED → NotEstablished, Reject.req for unsupported SType /
PType, Reject.req for data while NOT_SELECTED) — not just the happy path.
edge cases — not just the happy path.
The E30 §6.5 Communication state machine is unit-tested independently of
the transport (timer firings simulated via test callbacks).
---
## 7. Explicitly out of scope (with reasons)
## 7. Interoperability with secsgem-py 0.3.0
These look like gaps but are deliberate. None of them blocks the GEM
The `interop/` harness (see `interop/README.md`) cross-validates the
codebase against the reference Python implementation in Docker:
- **secsgem-py active host → C++ passive server**: 24 named checks
across S1/S2/S5/S6/S7/S10 plus unsolicited S5F1/S6F11. Three
consecutive clean runs.
- **C++ active host → secsgem-py passive equipment**: HSMS select +
S1F13 + S1F1 + S1F3 + clean separate; exits 0.
- **C++ active host → raw GEM 300 streams** (`raw_gem300_harness.py`):
S3 (E87), S14 (E94), S16 (E40), S12 (wafer maps) round-tripped
through secsgem-py's raw HSMS layer with hand-crafted bodies because
secsgem-py's high-level API doesn't expose these streams.
Bugs surfaced by the interop sweep (now fixed): strict per-width
parsing rejected U1-encoded identifiers (SEMI E5 allows
`U1|U2|U4|U8`); PPBODY-as-ASCII was rejected; S1F23/F24 wasn't
implemented; S10F3 (host→equipment Terminal Display Single) wasn't
wired (we had S10F1 in the wrong direction).
---
## 8. Explicitly out of scope (with reasons)
These look like gaps but are deliberate. None blocks the GEM
compliance claim.
| Item | Why it's out of scope |
|---------------------------------------|----------------------|
| Material Movement (E30 §6.18) | The job-management half is now in (E40 + E94, §4a). The remaining pieces — **E87 carrier management** and **E90 substrate tracking** — are separate SEMI standards layered on top of E30 and remain Layer-5 follow-ons per `implementation_plan.md`. |
| Multi-block SECS-I transfers | Multi-block (S6F5/F6, S6F7/F8 etc.) is a **SECS-I** concept for 244-byte serial frames. HSMS allows arbitrarily large bodies (up to the codebase's 16 MiB cap), so multi-block is structurally not needed. E37-based GEM equipment does not require it. |
| Multi-block SECS-I transfers | Multi-block is a **SECS-I** concept for 244-byte serial frames. HSMS allows arbitrarily large bodies (up to the codebase's 16 MiB cap), so multi-block is structurally not needed. E37-based GEM equipment does not require it. |
| HSMS-GS (multi-session) | Out of scope — modern HSMS-SS covers virtually all current GEM equipment. |
| Equipment Processing States (concrete states) | E30 §6.3 says the specific states are tool-defined. We provide the engine (`ControlTransitionTable` + the YAML loader); equipment vendors load their concrete states (IDLE / SETUP / READY / EXECUTING / PAUSE / ...) the same way `data/control_state.yaml` is loaded today. Spec-compliant either way. |
| Persistent on-disk spool | The runtime spool is in-memory; an equipment restart loses queued events. Real fab equipment would back it with a journal. Standard does not mandate persistence. |
| E42 Enhanced Process Programs (S7F23–F26) | A separate SEMI standard. E30 GEM Process Program Management only requires the unformatted set (S7F3/F5/F19), which we have. |
| S10F7 Broadcast Terminal Display | Rarely used; equipment vendors typically forgo it. Not required for the Terminal Services capability. |
| Equipment Processing States (concrete states) | E30 §6.3 says the specific states are tool-defined. We provide the engine (`ControlTransitionTable` + the YAML loader); equipment vendors load their concrete states (IDLE / SETUP / READY / EXECUTING / PAUSE / …) the same way `data/control_state.yaml` is loaded today. Spec-compliant either way. |
| E42 Enhanced Process Programs (S7F23–F26) | A separate SEMI standard. E30 GEM Process Program Management only requires the unformatted set (S7F1/F3/F5/F17/F19), which we have. |
| S10F7 Broadcast Terminal Display | Rarely used; equipment vendors typically forgo it. Not required for the Terminal Services capability. |
---
## 8. What "100% GEM-compliant" honestly means here
## 9. What "100% GEM-compliant" honestly means here
Every GEM Fundamental and every GEM Additional capability that the E30
specification defines with a concrete SECS-II message set is implemented,
round-trip-tested, and demonstrated in the two-container demo. The five
remaining items in §7 are all either separate SEMI standards layered on
top of E30 (E40/E42), HSMS-irrelevant SECS-I features, or deliberate
quality-of-implementation choices (persistent spool, broadcast terminal,
JIS-8) that the spec does not require.
round-trip-tested, demonstrated in the two-container demo, AND
cross-validated against secsgem-py 0.3.0 on the overlap. Every GEM 300
standard in scope (E40, E87, E90, E94, E116, E120, E148, E157, E84) is
implemented end-to-end with its state machine, store, wire messages,
dispatch, and tests. Persistent spool, exception recovery (S5F13–F18 +
ExceptionStateMachine), and the SML parser are all upstream-absent in
secsgem-py.
What this codebase does **not** demonstrate, and what a real "GEM-compliant"
marketing claim would still need:
What this codebase does **not** demonstrate, and what a real
"GEM-compliant" marketing claim would still need:
1. **Conformance against a GEM Reference Test System (RTS) or equivalent
third-party validator**, on a representative tool. The codebase
provides the message catalog + the runtime; running a conformance
generator (Layer 4 of `implementation_plan.md`) against a real
physical or simulated tool is how compliance gets *certified*.
1. **Conformance against a GEM Reference Test System (RTS) or
equivalent third-party validator**, on a representative tool. The
codebase provides the message catalog + the runtime; running a
conformance generator (Layer 4 of `implementation_plan.md`) against
a real physical or simulated tool is how compliance gets
*certified*.
2. **Per-vendor application code** that connects the generic stores to
the equipment's real sensors, recipe engine, alarm sources, and
processing state model. The codebase provides the data model and
processing state model. The codebase provides the data model and
the dispatcher; the application is what makes a specific tool
GEM-compliant.
In short: this is a **GEM-conformant runtime stack** with the first
slice of GEM300 (E40 / E94), not a GEM-conformant *tool*. Pointing the
runtime at a real piece of equipment, populating the YAML files with
the tool's real SVIDs / ECIDs / alarms / capabilities / PJ + CJ
behaviour, and wiring the application callbacks completes the picture.
In short: this is a **GEM-conformant runtime stack with the full GEM
300 suite**, not a GEM-conformant *tool*. Pointing the runtime at a
real piece of equipment, populating the YAML files with the tool's
real SVIDs / ECIDs / alarms / capabilities / job behaviour, and wiring
the application callbacks completes the picture.