diff --git a/COMPLIANCE.md b/COMPLIANCE.md index fbaa007..e3c95ae 100644 --- a/COMPLIANCE.md +++ b/COMPLIANCE.md @@ -1,13 +1,19 @@ # SECS/GEM Compliance -A per-capability accounting against SEMI **E5 (SECS-II)**, **E30 (GEM)**, -**E37 (HSMS)**, and **GEM300 E40 / E94** (process / control jobs). +A per-capability accounting against the foundational SEMI standards +**E5 (SECS-II)**, **E30 (GEM)**, **E37 (HSMS)**, **E4 (SECS-I)**, plus +the full GEM 300 stack: **E40** (process jobs), **E94** (control jobs), +**E87** (carriers), **E90** (substrates), **E116** (equipment +performance tracking), **E120** (common equipment model), **E157** +(module process tracking), **E84** (parallel I/O), **E148** (time +synchronization), **E39** (object services), plus **E5 §13** wafer maps. -> **Status.** Every GEM Fundamental capability and every GEM Additional -> capability that E30 ties to a concrete SECS-II message set is -> implemented. See §7 for the explicit out-of-scope items (which are -> deliberate, not omissions) and §8 for what "100% GEM-compliant" can -> and cannot honestly mean about this codebase. +> **Status.** Every GEM Fundamental and every GEM Additional capability +> that E30 binds to a concrete SECS-II message set is implemented, and +> every GEM 300 standard the project sets out to cover is implemented +> end-to-end (state machines + stores + wire messages + dispatch). See +> §8 for the items deliberately out of scope and §9 for what +> "100% GEM-compliant" can and cannot honestly mean about a codebase. Legend: @@ -68,12 +74,14 @@ Legend: | ASCII (`A`) | ✅ | E5 §9.5 | | | Binary (`B`) | ✅ | E5 §9.5 | | | Boolean (`BOOLEAN`) | ✅ | E5 §9.5 | | -| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | | -| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | | +| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | Identifier parsers accept any width per the SEMI wildcard rule. | +| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | Same lenient-width policy. | | `F4, F8` (IEEE 754 big-endian) | ✅ | E5 §9.5 | bit-cast round-trip. | | JIS-8 (single-byte JIS text) | ✅ | E5 §9.5 | `Format::JIS8` (0x11); shares `std::string` storage with ASCII, disambiguated by `Format`. | | C2 (Unicode 2-byte code points) | ✅ | E5 §9.5 | `Format::C2` (0x12); big-endian uint16_t code points. | | SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. JIS-8 prints as ``, C2 as ``. | +| SML parser (inverse of `to_sml`) | ✅ | — | `secs2::from_sml`; round-trips every format. | +| `ASCII | Binary` wildcard fields | ✅ | E5 | `BINARY_OR_ASCII` schema type for PPBODY etc.; accepted by `as_text_or_binary`. | --- @@ -82,12 +90,12 @@ Legend: | Fundamental Capability | Status | Spec ref | Messages | Notes | |---------------------------------------|--------|----------|----------|-------| | State models | ✅ | E30 §6.2 | — | E30 control state machine (5 states) + HSMS communication state machine. | -| Equipment Processing States | ✅ | E30 §6.3 | — | The `ControlTransitionTable` engine is general; vendors load their tool-specific states (IDLE/SETUP/READY/EXECUTING/PAUSE/...) via a second YAML file using the same loader. The spec leaves the concrete states tool-specific. | +| Equipment Processing States | ✅ | E30 §6.3 | — | `ControlTransitionTable` engine; vendors load their tool-specific states (IDLE/SETUP/READY/EXECUTING/PAUSE/…) via a second YAML file using the same loader. The spec leaves the concrete states tool-specific. | | Host-Initiated S1F13/F14 scenario | ✅ | E30 §6.5 | S1F13/F14 | | | Event Notification | ✅ | E30 §6.6 | S6F11/F12 | Equipment-initiated, host-acknowledged. | | On-Line Identification | ✅ | E30 §6.7 | S1F1/F2 | MDLN + SOFTREV. | | Error Messages | ✅ | E30 §6.9 | S9F* | Auto-emission of S9F3/F5/F7/F9/F11 on the documented protocol-error conditions; S9F1/F13 in the catalog for explicit emission. | -| Documentation | ✅ | E30 §6.10| S1F19/F20, S1F21/F22 | Equipment self-reports its compliance + DVID namelist. | +| Documentation | ✅ | E30 §6.10| S1F19/F20, S1F21/F22, S1F23/F24 | Equipment self-reports compliance, DVID namelist, AND collection-event namelist (CEID → VIDs). | | Control (Operator-Initiated) | ✅ | E30 §6.2 | — | `ControlStateMachine::operator_online/offline/local/remote`. | --- @@ -97,133 +105,201 @@ Legend: | Additional Capability | Status | Spec ref | Messages | Notes | |---------------------------------------|--------|----------|----------|-------| | Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. Backed by the E30 §6.5 Communication state machine (`gem::CommunicationStateMachine`) with DISABLED / WAIT-CRA / WAIT-DELAY / COMMUNICATING substates and the T_CRA + T_DELAY retry timers, separate from HSMS connection state. | -| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. | +| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. Host-initiated readback via S6F15/F16, S6F19/F20, S6F21/F22. | | Variable Data Collection | ✅ | E30 §6.11| S1F21/F22 | DVID namelist + DVID values resolvable via `EquipmentDataModel::vid_value`. | | Trace Data Collection | ✅ | E30 §6.12| S2F23/F24, S6F1/F2 | `TraceStore` keeps active TRID→TraceConfig; periodic sampling left to the application's scheduler. | | Status Data Collection | ✅ | E30 §6.13| S1F3/F4, S1F11/F12 | | | Alarm Management | ✅ | E30 §6.14| S5F1/F2, S5F3/F4, S5F5/F6, S5F7/F8 | Full set. ALCD bit-7 set/cleared, lower-7 category. | -| Remote Control | ✅ | E30 §6.15| S2F41/F42 | Full HCACK 7-value enum + per-parameter CPACKs. | +| Remote Control | ✅ | E30 §6.15| S2F41/F42, S2F49/F50, S2F21/F22 | Modern (CPACK), enhanced (OBJSPEC + CEPACK), and legacy (no params) forms all dispatched against the same `HostCommandRegistry`. | | Equipment Constants | ✅ | E30 §6.16| S2F13/F14, S2F15/F16, S2F29/F30 | EAC range validation against `min_str`/`max_str` for numeric ECs. | -| Process Program Management | ✅ | E30 §6.17| S7F3/F4, S7F5/F6, S7F19/F20 | Unformatted PP send/request/list — the minimum E30 GEM requires. (E42 *enhanced* PP is a separate SEMI standard; see §7.) | -| Material Movement | 🟡 | E30 §6.18| — | Process Job + Control Job lifecycle covered via E40/E94 (see §4a). Carrier (E87) and substrate (E90) still out of scope. | +| Process Program Management | ✅ | E30 §6.17| S7F1/F2, S7F3/F4, S7F5/F6, S7F17/F18, S7F19/F20 | Unformatted PP load-inquire/send/request/delete/list. Covers the full GEM PP set (E42 *enhanced* PP is a separate standard; see §8). | +| Material Movement | ✅ | E30 §6.18| (see §4a-4h)| Now fully covered: process jobs (E40), control jobs (E94), carriers (E87), substrates (E90), modules (E157). | | Equipment Terminal Services | ✅ | E30 §6.19| S10F1/F2, S10F3/F4, S10F5/F6 | Single-line both directions + multi-line host→equipment. S10F7 broadcast intentionally omitted (rarely used). | -| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. | +| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. Drift tracking + quality via E148 (§4g). | | Limits Monitoring | ✅ | E30 §6.21| S2F45/F46, S2F47/F48 | `LimitMonitorStore` keyed by VID with multiple `LimitDefinition` (LIMITID + upper/lower as arbitrary Items). | -| Spooling | ✅ | E30 §6.22| S2F43/F44, S6F23/F24, S6F25/F26 | Per-stream whitelist, FIFO queue, host-driven transmit/purge, S6F25 auto-emitted on re-SELECT when non-empty. | +| Spooling | ✅ | E30 §6.22| S2F43/F44, S6F23/F24, S6F25/F26 | Per-stream whitelist, FIFO queue, host-driven transmit/purge, S6F25 auto-emitted on re-SELECT when non-empty. **Persistent**: opt-in file-backed journal (`SpoolStore::enable_persistence(dir)`) survives equipment restarts. | | Control | ✅ | E30 §6.2 | — | See Fundamental. | --- -## 4a. E40 Process Jobs + E94 Control Jobs (GEM300) - -The first GEM300 extension landing on the spec-as-data architecture. -Both standards are implemented the same way the E30 control state model -is: state set + legal transitions in YAML -(`data/process_job_state.yaml`, `data/control_job_state.yaml`), engine -in C++ (`gem::ProcessJobStateMachine`, `gem::ControlJobStateMachine`), -runtime collections (`ProcessJobStore`, `ControlJobStore`) wired into -the existing `EquipmentDataModel`. +## 4a. E40 Process Jobs | Capability | Status | Spec ref | Messages | Notes | |---------------------------------------|--------|----------|----------|-------| -| E40 PJ state model | ✅ | E40 §6.3 | — | 8 states (Queued, SettingUp, WaitingForStart, Processing, ProcessComplete, Paused, Stopping, Aborting); state byte matches PRJOBSTATE on the wire. | -| E40 PRJobCreate | ✅ | E40 §10.2| S16F11/F12 | Body simplified to ; MF/PRRECIPEMETHOD/PRPROCESSPARAMS are documented as YAML-extension points. PPID validated against `RecipeStore`. | -| E40 PRJobDequeue | ✅ | E40 §10.2| S16F13/F14 | Only legal while PJ is QUEUED; the FSM blocks dequeue otherwise. | -| E40 PRJobCommand | ✅ | E40 §10.2| S16F5/F6 | PRCMD strings PJSTART/PJPAUSE/PJRESUME/PJSTOP/PJABORT/PJHOQ; the matching `ProcessJobEvent` is dispatched against the FSM, HCACK is `CannotDoNow` when the (state, command) pair has no row. | -| E40 PRJobAlert | ✅ | E40 §10.3| S16F9 | Equipment-initiated one-way (W=0). Fires automatically on every PJ state transition; per-PJ `alert_enabled` flag controls suppression. | -| E94 CJ state model | ✅ | E94 §6 | — | 9 states (Queued, Selected, WaitingForStart, Executing, Paused, Completed, Stopping, Aborting, NoState). CJ owns an ordered `prjobids` list. | -| E94 CreateObject (CJ) | ✅ | E94 §6.4 | S14F9/F10 | Body simplified to ; full generic E14 ObjectService form is a YAML extension. OBJACK enum covers Success/Error + the four `Denied_*` cases. | -| E94 DeleteObject (CJ) | ✅ | E94 §6.4 | S14F11/F12 | | -| E94 CJobCommand | ✅ | E94 §6.4 | S16F27/F28 | CTLJOBCMD: CJSTART (cascades through Select → SetupComplete → Start as application policy), CJPAUSE / CJRESUME / CJSTOP / CJABORT. | -| E40+E94 CEID emission | ✅ | — | S6F11 | ControlJobExecuting (CEID 400) and ControlJobCompleted (CEID 401) fire on CJ state transitions via the existing event-report pipeline; PJ state changes use S16F9 (per spec). | +| PJ state model | ✅ | E40 §6.3 | — | 8 states (Queued, SettingUp, WaitingForStart, Processing, ProcessComplete, Paused, Stopping, Aborting); state byte matches PRJOBSTATE on the wire. | +| PRJobCreate (full body) | ✅ | E40 §10.2| S16F11/F12 | Full E40-0705 body: `> >`. PPID validated against `RecipeStore`. | +| PRJobCreateMultiple | ✅ | E40 §10 | S16F15/F16 | Bulk variant; per-job ACK list. | +| PRJobDequeue | ✅ | E40 §10.2| S16F13/F14 | Only legal while PJ is QUEUED. | +| PRJobMonitor | ✅ | E40 §10 | S16F7/F8 | Per-PJ alert enable/disable. | +| PRJobCommand | ✅ | E40 §10.2| S16F5/F6 | PRCMD strings PJSTART/PJPAUSE/PJRESUME/PJSTOP/PJABORT/PJHOQ. | +| PRJobAlert | ✅ | E40 §10.3| S16F9 | Equipment-initiated one-way (W=0). Fires automatically on every PJ state transition. | -The demo's `run_cj_lifecycle` cascade — on CJSTART the CJ steps Queued -→ Selected → WaitingForStart → Executing and every contained PJ steps -through SettingUp → WaitingForStart → Processing → ProcessComplete — -is **application policy**, not the FSM. The FSM rules in the YAML -tables gate every individual transition; the cascade is just the -simulator playing every legal next step in sequence so the wire trace -exercises the whole lifecycle. +## 4b. E94 Control Jobs -What's **out of scope for the E40/E94 first pass** (deliberate; all are -YAML/handler extensions, not surgery): -- Full E40 S16F11 body (MF / PRRECIPEMETHOD / RCPSPEC / PRPROCESSPARAMS). - We carry PRJOBID + PPID + MTRLOUTSPEC, which is the subset that - drives the state machine; richer body fields are a YAML edit + a - parameter map on the `ProcessJob` struct. -- S16F15/F16 PRJobCreateMultiple, S16F17/F18 PRJobMultipleDequeue. -- E14 generic ObjectService form. We use a CJ-specialized S14F9 shape; - generic ObjectService is a separate set of YAML rows. -- E87 Carrier Management and E90 Substrate Tracking — Layer 5 - continues there per `implementation_plan.md`. +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| CJ state model | ✅ | E94 §6 | — | 9 states; CJ owns an ordered `prjobids` list. | +| CreateObject (CJ) | ✅ | E94 §6.4 | S14F9/F10 | Body ``. | +| DeleteObject (CJ) | ✅ | E94 §6.4 | S14F11/F12 | | +| CJobCommand | ✅ | E94 §6.4 | S16F27/F28 | CTLJOBCMD: CJSTART, CJPAUSE, CJRESUME, CJSTOP, CJABORT. | +| CJ CEID emission | ✅ | — | S6F11 | ControlJobExecuting (CEID 400) and ControlJobCompleted (CEID 401) fire on CJ state transitions via the existing event-report pipeline. | + +## 4c. E87 Carrier Management + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| Carrier state model | ✅ | E87 §6 | — | CarrierStateMachine + LoadPortStateMachine; CIDS, CSMS, CASS, LPRS, LPTS axes. | +| CarrierAction | ✅ | E87 §10.2| S3F17/F18 | ProceedWithCarrier, CancelCarrier, BindCarrierID. | +| Slot Map Verify | ✅ | E87 §10 | S3F19/F20 | Equipment compares against stored slots; drives CSMS NotRead → Read/Mismatched. | +| Slot Map Report | ✅ | E87 §10 | S3F21/F22 | Equipment notifies host of read slot map; host acks. | +| Port Group Change | ✅ | E87 §10.4| S3F23/F24 | Host modifies load-port grouping; PortGroupAck. | +| Carrier Transfer | ✅ | E87 §10 | S3F25/F26 | Move carrier between ports; fires Start{Un}Loading transfer events. | +| Cancel Carrier | ✅ | E87 §10 | S3F27/F28 | Drives CancelCarrier ID event + Cancel access event. | + +## 4d. E90 Substrate Tracking + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| Substrate state model | ✅ | E90 §6 | — | Three axes: location (AtSource/AtWork/AtDestination), processing (NeedsProcessing/InProcess/Processed/Aborted/Stopped/Rejected/Lost/Skipped), id-status (Confirmed/Unconfirmed/Lost). | +| SubstrateHistory | ✅ | — | — | Per-substrate append-only ring buffer of state transitions. | +| Standard CEIDs | ✅ | E90 §7 | S6F11 | All E90 CEIDs emitted on transition; observable host-side. | + +## 4e. E116 Equipment Performance Tracking + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| EPT state machine | ✅ | E116 §6 | — | NonScheduledTime / ScheduledDowntime / UnscheduledDowntime / Engineering / Standby / Productive. | +| Time-bucket accounting | ✅ | E116 | — | Cumulative ms per state; resettable. | +| EPT CEIDs | ✅ | E116 | S6F11 | One CEID per state. | + +## 4f. E120 Common Equipment Model + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| Generic ObjectService (E39) | ✅ | E39 §5 | S14F1/F2, S14F3/F4 | GetAttr / SetAttr against `CemObjectStore`; OBJTYPE validation. | +| CemObjectStore | ✅ | E120 | — | Typed objects keyed by OBJSPEC + ObjType. | + +## 4g. E148 Time Synchronization + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| Time-sync drift tracking | ✅ | E148 | S2F31/F32 | Drift metric maintained on every set; quality score for hosts. | + +## 4h. E157 Module Process Tracking + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| Module state machine | ✅ | E157 §6 | — | NotExecuting / GeneralExecuting / StepExecuting / StepCompleted. | +| Module CEIDs | ✅ | E157 | S6F11 | Generic ModuleProcessStateChange + per-state CEIDs. | + +## 4i. E84 Parallel I/O Handoff + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| Handoff state machine | ✅ | E84 | — | Full LOAD / UNLOAD signal vocabulary (CS_0/CS_1, VALID, TR_REQ, BUSY, COMPT, AM_AVBL, ES). | + +## 4j. E5 §13 Wafer Maps (S12) + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| Map Setup Data | ✅ | E5 §13 | S12F1/F2, S12F3/F4 | | +| Map Transmit Inquire / Grant | ✅ | E5 §13 | S12F5/F6 | | +| Map Data Send — row format (MAPFT=0) | ✅ | E5 §13 | S12F7/F8 | | +| Map Data Send — array format (MAPFT=1)| ✅ | E5 §13 | S12F9/F10 | STRP + BINLT. | +| Map Data Send — coord format (MAPFT=2)| ✅ | E5 §13 | S12F11/F12 | XYPOS + per-die BIN. | +| Map Data Request — row | ✅ | E5 §13 | S12F13/F14 | | +| Map Data Request — array | ✅ | E5 §13 | S12F15/F16 | | +| Map Data Request — coord | ✅ | E5 §13 | S12F17/F18 | | +| Map Error Send | ✅ | E5 §13 | S12F19 | One-way error report. | + +## 4k. Exception Recovery (beyond E5 base alarms) + +| Capability | Status | Spec ref | Messages | Notes | +|---------------------------------------|--------|----------|----------|-------| +| ExceptionStateMachine FSM | ✅ | — | — | Per-EXID Posted / Recovering / RecoverFailed / Cleared lifecycle. Not in upstream secsgem-py. | +| Exception post / clear | ✅ | E5 §13 | S5F9/F10, S5F11/F12 | | +| Exception recover request / complete | ✅ | E5 §13 | S5F13/F14, S5F15/F16 | EXRECVRA validated against the posted candidates. | +| Exception recover abort | ✅ | E5 §13 | S5F17/F18 | | +| AlarmSeverity bit-flag enum | ✅ | — | — | Classification helpers for the ALCD lower 7 bits. | --- ## 5. Message coverage matrix -| Pair | Direction | Status | Implemented in | Tested | -|------------------|-----------|--------|----------------|--------| -| S1F1 / S1F2 | H↔E | ✅ | catalog | ✅ round-trip + demo | -| S1F3 / S1F4 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S1F11 / S1F12 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S1F13 / S1F14 | H↔E | ✅ | catalog | ✅ round-trip + demo | -| S1F15 / S1F16 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S1F17 / S1F18 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S1F19 / S1F20 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S1F21 / S1F22 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F13 / S2F14 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F15 / S2F16 | H→E | ✅ | catalog | ✅ round-trip | -| S2F17 / S2F18 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F23 / S2F24 | H→E | ✅ | catalog | ✅ round-trip | -| S2F25 / S2F26 | H→E | ✅ | catalog | ✅ round-trip (loopback diagnostic) | -| S2F29 / S2F30 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F31 / S2F32 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F33 / S2F34 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F35 / S2F36 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F37 / S2F38 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F41 / S2F42 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F43 / S2F44 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S2F45 / S2F46 | H→E | ✅ | catalog | ✅ round-trip | -| S2F47 / S2F48 | H→E | ✅ | catalog | ✅ round-trip | -| S5F1 / S5F2 | E→H | ✅ | catalog | ✅ round-trip + demo | -| S5F3 / S5F4 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S5F5 / S5F6 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S5F7 / S5F8 | H→E | ✅ | catalog | ✅ round-trip | -| S5F9 / S5F10 | E→H | ✅ | catalog | ✅ round-trip (exception post) | -| S5F11 / S5F12 | E→H | ✅ | catalog | ✅ round-trip (exception clear) | -| S5F13 / S5F14 | H→E | ✅ | catalog | ✅ round-trip (exception recover request) | -| S5F15 / S5F16 | E→H | ✅ | catalog | ✅ round-trip (exception recover complete) | -| S5F17 / S5F18 | H→E | ✅ | catalog | ✅ round-trip (exception recover abort) | -| S2F49 / S2F50 | H→E | ✅ | catalog + server | ✅ round-trip + dispatch (enhanced remote command, OBJSPEC + per-CP CPACK/CEPACK) | -| S12F1 / S12F2 | H→E | ✅ | catalog | ✅ round-trip (Map Setup Data Send, E5 §13) | -| S12F3 / S12F4 | H→E | ✅ | catalog | ✅ round-trip (Map Setup Data Request) | -| S12F5 / S12F6 | E→H | ✅ | catalog | ✅ round-trip (Map Transmit Inquire / Grant) | -| S12F7 / S12F8 | E→H | ✅ | catalog | ✅ round-trip (Map Data Send, row format MAPFT=0) | -| S12F19 | E→H | ✅ | catalog | ✅ round-trip (Map Error Send, one-way) | -| S6F1 / S6F2 | E→H | ✅ | catalog | ✅ round-trip | -| S6F11 / S6F12 | E→H | ✅ | catalog | ✅ round-trip + demo | -| S6F23 / S6F24 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S6F25 / S6F26 | E→H | ✅ | catalog + server | ✅ round-trip + auto-emitted on re-SELECT | -| S7F3 / S7F4 | H→E | ✅ | catalog | ✅ round-trip | -| S7F5 / S7F6 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S7F19 / S7F20 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S9F1 | E↔H | ✅ | catalog | ✅ round-trip | -| S9F3 | E↔H | ✅ | catalog + Router wrapper | ✅ round-trip + auto-emitted on unknown stream | -| S9F5 | E↔H | ✅ | catalog + Router wrapper | ✅ round-trip + auto-emitted on unknown function | -| S9F7 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on body decode | -| S9F9 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on T3 timeout | -| S9F11 | E↔H | ✅ | catalog + Connection | ✅ round-trip + auto-emitted on oversized frame | -| S9F13 | E↔H | ✅ | catalog | ✅ round-trip | -| S10F1 / S10F2 | H→E | ✅ | catalog | ✅ round-trip + demo | -| S10F3 / S10F4 | E→H | ✅ | catalog | ✅ round-trip + demo | -| S10F5 / S10F6 | H→E | ✅ | catalog | ✅ round-trip | -| S14F9 / S14F10 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJ create) | -| S14F11 / S14F12 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJ delete) | -| S16F5 / S16F6 | H→E | ✅ | catalog | ✅ round-trip (E40 PRJobCommand) | -| S16F9 | E→H | ✅ | catalog + server | ✅ round-trip + auto-emitted per PJ transition (E40 PRJobAlert) | -| S16F11 / S16F12 | H→E | ✅ | catalog | ✅ round-trip + demo (E40 PRJobCreate) | -| S16F13 / S16F14 | H→E | ✅ | catalog | ✅ round-trip (E40 PRJobDequeue) | -| S16F27 / S16F28 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJobCommand) | +149 SECS-II messages in the catalog, spanning streams 1, 2, 3, 5, 6, 7, 9, 10, 12, 14, 16. + +| Pair | Direction | Status | Notes | +|------------------|-----------|--------|-------| +| S1F1 / S1F2 | H↔E | ✅ | round-trip + demo | +| S1F3 / S1F4 | H→E | ✅ | round-trip + demo | +| S1F11 / S1F12 | H→E | ✅ | round-trip + demo | +| S1F13 / S1F14 | H↔E | ✅ | round-trip + demo | +| S1F15 / S1F16 | H→E | ✅ | round-trip + demo | +| S1F17 / S1F18 | H→E | ✅ | round-trip + demo | +| S1F19 / S1F20 | H→E | ✅ | round-trip + demo (compliance self-report) | +| S1F21 / S1F22 | H→E | ✅ | round-trip + demo | +| S1F23 / S1F24 | H→E | ✅ | collection event namelist (CEID → VID) | +| S2F13 / S2F14 | H→E | ✅ | EC values | +| S2F15 / S2F16 | H→E | ✅ | EC set | +| S2F17 / S2F18 | H→E | ✅ | clock | +| S2F21 / S2F22 | H→E | ✅ | legacy remote command (no params) | +| S2F23 / S2F24 | H→E | ✅ | trace initialize | +| S2F25 / S2F26 | H→E | ✅ | loopback diagnostic | +| S2F29 / S2F30 | H→E | ✅ | EC namelist | +| S2F31 / S2F32 | H→E | ✅ | set time | +| S2F33 / S2F34 | H→E | ✅ | define report | +| S2F35 / S2F36 | H→E | ✅ | link event report | +| S2F37 / S2F38 | H→E | ✅ | enable event | +| S2F41 / S2F42 | H→E | ✅ | host command (modern) | +| S2F43 / S2F44 | H→E | ✅ | reset spooling | +| S2F45 / S2F46 | H→E | ✅ | define variable limits | +| S2F47 / S2F48 | H→E | ✅ | request limit attrs | +| S2F49 / S2F50 | H→E | ✅ | enhanced remote command (OBJSPEC + CPACK/CEPACK) | +| S3F17 / S3F18 | H→E | ✅ | E87 carrier action | +| S3F19 / S3F20 | H→E | ✅ | E87 slot map verify | +| S3F21 / S3F22 | E→H | ✅ | E87 slot map report | +| S3F23 / S3F24 | H→E | ✅ | E87 port group change | +| S3F25 / S3F26 | H→E | ✅ | E87 carrier transfer | +| S3F27 / S3F28 | H→E | ✅ | E87 cancel carrier | +| S5F1 / S5F2 | E→H | ✅ | alarm send | +| S5F3 / S5F4 | H→E | ✅ | enable alarm | +| S5F5 / S5F6 | H→E | ✅ | list alarms | +| S5F7 / S5F8 | H→E | ✅ | list enabled alarms | +| S5F9 / S5F10 | E→H | ✅ | exception post | +| S5F11 / S5F12 | E→H | ✅ | exception clear | +| S5F13 / S5F14 | H→E | ✅ | exception recover request | +| S5F15 / S5F16 | E→H | ✅ | exception recover complete | +| S5F17 / S5F18 | H→E | ✅ | exception recover abort | +| S6F1 / S6F2 | E→H | ✅ | trace data | +| S6F5 / S6F6 | E→H | ✅ | multi-block inquire / grant | +| S6F7 / S6F8 | H→E | ✅ | data transfer request / send | +| S6F11 / S6F12 | E→H | ✅ | event report (unsolicited) | +| S6F15 / S6F16 | H→E | ✅ | event report request (host-initiated) | +| S6F19 / S6F20 | H→E | ✅ | individual report request | +| S6F21 / S6F22 | H→E | ✅ | annotated individual report request | +| S6F23 / S6F24 | H→E | ✅ | request spooled data | +| S6F25 / S6F26 | E→H | ✅ | spool data ready (auto on re-SELECT) | +| S7F1 / S7F2 | H→E | ✅ | PP load inquire / grant | +| S7F3 / S7F4 | H→E | ✅ | PP send | +| S7F5 / S7F6 | H→E | ✅ | PP request | +| S7F17 / S7F18 | H→E | ✅ | PP delete | +| S7F19 / S7F20 | H→E | ✅ | PP list | +| S9F1, F3, F5, F7, F9, F11, F13 | E↔H | ✅ | protocol errors; auto-emitted on the documented conditions | +| S10F1 / S10F2 | E→H | ✅ | terminal request (equipment originated) | +| S10F3 / S10F4 | H→E | ✅ | terminal display single | +| S10F5 / S10F6 | H→E | ✅ | terminal display multi | +| S12F1–F19 | H↔E | ✅ | wafer maps — row, array, coord; setup, request, send, error | +| S14F1 / S14F2 | H→E | ✅ | E39 GetAttr | +| S14F3 / S14F4 | H→E | ✅ | E39 SetAttr | +| S14F9 / S14F10 | H→E | ✅ | E94 CJ create | +| S14F11 / S14F12 | H→E | ✅ | E94 CJ delete | +| S16F5 / S16F6 | H→E | ✅ | E40 PRJobCommand | +| S16F7 / S16F8 | H→E | ✅ | E40 PRJobMonitor | +| S16F9 | E→H | ✅ | E40 PRJobAlert (auto on transition) | +| S16F11 / S16F12 | H→E | ✅ | E40 PRJobCreate (full body) | +| S16F13 / S16F14 | H→E | ✅ | E40 PRJobDequeue | +| S16F15 / S16F16 | H→E | ✅ | E40 PRJobCreateMultiple | +| S16F27 / S16F28 | H→E | ✅ | E94 CJobCommand | --- @@ -265,60 +341,82 @@ walks ~20 SECS transactions end-to-end: 23. `S1F15`/`S1F16` Request Offline. 24. `Separate.req` → clean close on both sides. -Unit tests: **148 cases / 794 assertions pass** (`docker compose run --rm tests`). +Unit tests: **291 cases / 1515 assertions pass** (`docker compose run --rm tests`). The suite includes integration tests that drive a real `hsms::Connection` over a loopback socket pair to verify the E37 §7.2 / §7.4 / §7.7 -edge cases (Select.req while already SELECTED → AlreadyActive, Deselect.req -while NOT_SELECTED → NotEstablished, Reject.req for unsupported SType / -PType, Reject.req for data while NOT_SELECTED) — not just the happy path. +edge cases — not just the happy path. The E30 §6.5 Communication state machine is unit-tested independently of the transport (timer firings simulated via test callbacks). --- -## 7. Explicitly out of scope (with reasons) +## 7. Interoperability with secsgem-py 0.3.0 -These look like gaps but are deliberate. None of them blocks the GEM +The `interop/` harness (see `interop/README.md`) cross-validates the +codebase against the reference Python implementation in Docker: + +- **secsgem-py active host → C++ passive server**: 24 named checks + across S1/S2/S5/S6/S7/S10 plus unsolicited S5F1/S6F11. Three + consecutive clean runs. +- **C++ active host → secsgem-py passive equipment**: HSMS select + + S1F13 + S1F1 + S1F3 + clean separate; exits 0. +- **C++ active host → raw GEM 300 streams** (`raw_gem300_harness.py`): + S3 (E87), S14 (E94), S16 (E40), S12 (wafer maps) round-tripped + through secsgem-py's raw HSMS layer with hand-crafted bodies because + secsgem-py's high-level API doesn't expose these streams. + +Bugs surfaced by the interop sweep (now fixed): strict per-width +parsing rejected U1-encoded identifiers (SEMI E5 allows +`U1|U2|U4|U8`); PPBODY-as-ASCII was rejected; S1F23/F24 wasn't +implemented; S10F3 (host→equipment Terminal Display Single) wasn't +wired (we had S10F1 in the wrong direction). + +--- + +## 8. Explicitly out of scope (with reasons) + +These look like gaps but are deliberate. None blocks the GEM compliance claim. | Item | Why it's out of scope | |---------------------------------------|----------------------| -| Material Movement (E30 §6.18) | The job-management half is now in (E40 + E94, §4a). The remaining pieces — **E87 carrier management** and **E90 substrate tracking** — are separate SEMI standards layered on top of E30 and remain Layer-5 follow-ons per `implementation_plan.md`. | -| Multi-block SECS-I transfers | Multi-block (S6F5/F6, S6F7/F8 etc.) is a **SECS-I** concept for 244-byte serial frames. HSMS allows arbitrarily large bodies (up to the codebase's 16 MiB cap), so multi-block is structurally not needed. E37-based GEM equipment does not require it. | +| Multi-block SECS-I transfers | Multi-block is a **SECS-I** concept for 244-byte serial frames. HSMS allows arbitrarily large bodies (up to the codebase's 16 MiB cap), so multi-block is structurally not needed. E37-based GEM equipment does not require it. | | HSMS-GS (multi-session) | Out of scope — modern HSMS-SS covers virtually all current GEM equipment. | -| Equipment Processing States (concrete states) | E30 §6.3 says the specific states are tool-defined. We provide the engine (`ControlTransitionTable` + the YAML loader); equipment vendors load their concrete states (IDLE / SETUP / READY / EXECUTING / PAUSE / ...) the same way `data/control_state.yaml` is loaded today. Spec-compliant either way. | -| Persistent on-disk spool | The runtime spool is in-memory; an equipment restart loses queued events. Real fab equipment would back it with a journal. Standard does not mandate persistence. | -| E42 Enhanced Process Programs (S7F23–F26) | A separate SEMI standard. E30 GEM Process Program Management only requires the unformatted set (S7F3/F5/F19), which we have. | -| S10F7 Broadcast Terminal Display | Rarely used; equipment vendors typically forgo it. Not required for the Terminal Services capability. | +| Equipment Processing States (concrete states) | E30 §6.3 says the specific states are tool-defined. We provide the engine (`ControlTransitionTable` + the YAML loader); equipment vendors load their concrete states (IDLE / SETUP / READY / EXECUTING / PAUSE / …) the same way `data/control_state.yaml` is loaded today. Spec-compliant either way. | +| E42 Enhanced Process Programs (S7F23–F26) | A separate SEMI standard. E30 GEM Process Program Management only requires the unformatted set (S7F1/F3/F5/F17/F19), which we have. | +| S10F7 Broadcast Terminal Display | Rarely used; equipment vendors typically forgo it. Not required for the Terminal Services capability. | --- -## 8. What "100% GEM-compliant" honestly means here +## 9. What "100% GEM-compliant" honestly means here Every GEM Fundamental and every GEM Additional capability that the E30 specification defines with a concrete SECS-II message set is implemented, -round-trip-tested, and demonstrated in the two-container demo. The five -remaining items in §7 are all either separate SEMI standards layered on -top of E30 (E40/E42), HSMS-irrelevant SECS-I features, or deliberate -quality-of-implementation choices (persistent spool, broadcast terminal, -JIS-8) that the spec does not require. +round-trip-tested, demonstrated in the two-container demo, AND +cross-validated against secsgem-py 0.3.0 on the overlap. Every GEM 300 +standard in scope (E40, E87, E90, E94, E116, E120, E148, E157, E84) is +implemented end-to-end with its state machine, store, wire messages, +dispatch, and tests. Persistent spool, exception recovery (S5F13–F18 + +ExceptionStateMachine), and the SML parser are all upstream-absent in +secsgem-py. -What this codebase does **not** demonstrate, and what a real "GEM-compliant" -marketing claim would still need: +What this codebase does **not** demonstrate, and what a real +"GEM-compliant" marketing claim would still need: -1. **Conformance against a GEM Reference Test System (RTS) or equivalent - third-party validator**, on a representative tool. The codebase - provides the message catalog + the runtime; running a conformance - generator (Layer 4 of `implementation_plan.md`) against a real - physical or simulated tool is how compliance gets *certified*. +1. **Conformance against a GEM Reference Test System (RTS) or + equivalent third-party validator**, on a representative tool. The + codebase provides the message catalog + the runtime; running a + conformance generator (Layer 4 of `implementation_plan.md`) against + a real physical or simulated tool is how compliance gets + *certified*. 2. **Per-vendor application code** that connects the generic stores to the equipment's real sensors, recipe engine, alarm sources, and - processing state model. The codebase provides the data model and + processing state model. The codebase provides the data model and the dispatcher; the application is what makes a specific tool GEM-compliant. -In short: this is a **GEM-conformant runtime stack** with the first -slice of GEM300 (E40 / E94), not a GEM-conformant *tool*. Pointing the -runtime at a real piece of equipment, populating the YAML files with -the tool's real SVIDs / ECIDs / alarms / capabilities / PJ + CJ -behaviour, and wiring the application callbacks completes the picture. +In short: this is a **GEM-conformant runtime stack with the full GEM +300 suite**, not a GEM-conformant *tool*. Pointing the runtime at a +real piece of equipment, populating the YAML files with the tool's +real SVIDs / ECIDs / alarms / capabilities / job behaviour, and wiring +the application callbacks completes the picture.