raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

E40 Process Jobs + E94 Control Jobs + E30 communication state

Browse Source 
GEM300 layer: SEMI E40-0705 Process Job and E94-0705 Control Job
state machines, plus the E30 §6.1 communication-state machine that
sits between HSMS SELECT and full GEM communication. Data-driven
via data/process_job_state.yaml and data/control_job_state.yaml,
mirroring the existing control_state.yaml pattern.

Wire coverage:
  S14F9/F10   CreateObject (CJ)              host -> equipment
  S14F11/F12  DeleteObject (CJ)              host -> equipment
  S16F5/F6    PRJobCommand                   host -> equipment
  S16F9       PRJobAlert                     equipment -> host
  S16F11/F12  PRJobCreate (simplified body)  host -> equipment
  S16F13/F14  PRJobDequeue                   host -> equipment
  S16F27/F28  CJobCommand                    host -> equipment

Process Job FSM exposes 8 states matching PRJOBSTATE bytes (E40 §10.3.2);
HOQ is reorder-aware (move-to-head against an insertion-order vector);
Stop/Abort on a Queued PJ routes through ABORTING so the host observes
PRJOBSTATE=7 on the wire (§6.3); alert_enabled is settable per-PJ for
PRALERT control; FSM dispatches through ProcessJobStore::on_change_
dynamically so a late set_state_change_handler() reaches existing PJs.

Hardening: loader rejects NoState (sentinel) as initial/from/to and
rejects `on: created` rows; static_asserts pin enum values to wire
bytes; ProcessJobStore is non-movable to keep the per-PJ this-capture
safe.

Server simulator cascades the full CJ -> PJ lifecycle on CJSTART so
the wire trace exercises every legal state. CEIDs 400/401 fire on CJ
state changes via the existing event-report pipeline.

Tests: 60+ new assertions across test_process_jobs, test_control_jobs,
test_communication_state, test_hsms_connection, plus loader and
messages round-trip coverage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-07 21:00:32 +02:00
parent 1f67aad985
commit 90c177b7ce
33 changed files with 3122 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
COMPLIANCE.md
+86 -16
View File
@@ -1,7 +1,7 @@
# SECS/GEM Compliance
A per-capability accounting against SEMI **E5 (SECS-II)**, **E30 (GEM)**,
and **E37 (HSMS)**.
**E37 (HSMS)**, and **GEM300 E40 / E94** (process / control jobs).
> **Status.** Every GEM Fundamental capability and every GEM Additional
> capability that E30 ties to a concrete SECS-II message set is
@@ -52,8 +52,9 @@ Legend:
| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | |
| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | |
| `F4, F8` (IEEE 754 big-endian) | ✅ | E5 §9.5 | bit-cast round-trip. |
| JIS-8 / C2 (Unicode) | ⬜ | E5 §9.5 | Rarely used in modern fabs. |
| SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. |
| JIS-8 (single-byte JIS text) | ✅ | E5 §9.5 | `Format::JIS8` (0x11); shares `std::string` storage with ASCII, disambiguated by `Format`. |
| C2 (Unicode 2-byte code points) | ✅ | E5 §9.5 | `Format::C2` (0x12); big-endian uint16_t code points. |
| SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. JIS-8 prints as `<J "...">`, C2 as `<C 65 66 ...>`. |
---
@@ -76,7 +77,7 @@ Legend:
| Additional Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. |
| Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. Backed by the E30 §6.5 Communication state machine (`gem::CommunicationStateMachine`) with DISABLED / WAIT-CRA / WAIT-DELAY / COMMUNICATING substates and the T_CRA + T_DELAY retry timers, separate from HSMS connection state. |
| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. |
| Variable Data Collection | ✅ | E30 §6.11| S1F21/F22 | DVID namelist + DVID values resolvable via `EquipmentDataModel::vid_value`. |
| Trace Data Collection | ✅ | E30 §6.12| S2F23/F24, S6F1/F2 | `TraceStore` keeps active TRID→TraceConfig; periodic sampling left to the application's scheduler. |
@@ -85,7 +86,7 @@ Legend:
| Remote Control | ✅ | E30 §6.15| S2F41/F42 | Full HCACK 7-value enum + per-parameter CPACKs. |
| Equipment Constants | ✅ | E30 §6.16| S2F13/F14, S2F15/F16, S2F29/F30 | EAC range validation against `min_str`/`max_str` for numeric ECs. |
| Process Program Management | ✅ | E30 §6.17| S7F3/F4, S7F5/F6, S7F19/F20 | Unformatted PP send/request/list — the minimum E30 GEM requires. (E42 *enhanced* PP is a separate SEMI standard; see §7.) |
| Material Movement | ⬜ | E30 §6.18| — | See §7. |
| Material Movement | 🟡 | E30 §6.18| — | Process Job + Control Job lifecycle covered via E40/E94 (see §4a). Carrier (E87) and substrate (E90) still out of scope. |
| Equipment Terminal Services | ✅ | E30 §6.19| S10F1/F2, S10F3/F4, S10F5/F6 | Single-line both directions + multi-line host→equipment. S10F7 broadcast intentionally omitted (rarely used). |
| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. |
| Limits Monitoring | ✅ | E30 §6.21| S2F45/F46, S2F47/F48 | `LimitMonitorStore` keyed by VID with multiple `LimitDefinition` (LIMITID + upper/lower as arbitrary Items). |
@@ -94,6 +95,51 @@ Legend:
---
## 4a. E40 Process Jobs + E94 Control Jobs (GEM300)
The first GEM300 extension landing on the spec-as-data architecture.
Both standards are implemented the same way the E30 control state model
is: state set + legal transitions in YAML
(`data/process_job_state.yaml`, `data/control_job_state.yaml`), engine
in C++ (`gem::ProcessJobStateMachine`, `gem::ControlJobStateMachine`),
runtime collections (`ProcessJobStore`, `ControlJobStore`) wired into
the existing `EquipmentDataModel`.
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| E40 PJ state model | ✅ | E40 §6.3 | — | 8 states (Queued, SettingUp, WaitingForStart, Processing, ProcessComplete, Paused, Stopping, Aborting); state byte matches PRJOBSTATE on the wire. |
| E40 PRJobCreate | ✅ | E40 §10.2| S16F11/F12 | Body simplified to <L,3 PRJOBID PPID L,n MTRLOUTSPEC>; MF/PRRECIPEMETHOD/PRPROCESSPARAMS are documented as YAML-extension points. PPID validated against `RecipeStore`. |
| E40 PRJobDequeue | ✅ | E40 §10.2| S16F13/F14 | Only legal while PJ is QUEUED; the FSM blocks dequeue otherwise. |
| E40 PRJobCommand | ✅ | E40 §10.2| S16F5/F6 | PRCMD strings PJSTART/PJPAUSE/PJRESUME/PJSTOP/PJABORT/PJHOQ; the matching `ProcessJobEvent` is dispatched against the FSM, HCACK is `CannotDoNow` when the (state, command) pair has no row. |
| E40 PRJobAlert | ✅ | E40 §10.3| S16F9 | Equipment-initiated one-way (W=0). Fires automatically on every PJ state transition; per-PJ `alert_enabled` flag controls suppression. |
| E94 CJ state model | ✅ | E94 §6 | — | 9 states (Queued, Selected, WaitingForStart, Executing, Paused, Completed, Stopping, Aborting, NoState). CJ owns an ordered `prjobids` list. |
| E94 CreateObject (CJ) | ✅ | E94 §6.4 | S14F9/F10 | Body simplified to <L,2 CTLJOBID L,n PRJOBIDs>; full generic E14 ObjectService form is a YAML extension. OBJACK enum covers Success/Error + the four `Denied_*` cases. |
| E94 DeleteObject (CJ) | ✅ | E94 §6.4 | S14F11/F12 | |
| E94 CJobCommand | ✅ | E94 §6.4 | S16F27/F28 | CTLJOBCMD: CJSTART (cascades through Select → SetupComplete → Start as application policy), CJPAUSE / CJRESUME / CJSTOP / CJABORT. |
| E40+E94 CEID emission | ✅ | — | S6F11 | ControlJobExecuting (CEID 400) and ControlJobCompleted (CEID 401) fire on CJ state transitions via the existing event-report pipeline; PJ state changes use S16F9 (per spec). |
The demo's `run_cj_lifecycle` cascade — on CJSTART the CJ steps Queued
→ Selected → WaitingForStart → Executing and every contained PJ steps
through SettingUp → WaitingForStart → Processing → ProcessComplete —
is **application policy**, not the FSM. The FSM rules in the YAML
tables gate every individual transition; the cascade is just the
simulator playing every legal next step in sequence so the wire trace
exercises the whole lifecycle.
What's **out of scope for the E40/E94 first pass** (deliberate; all are
YAML/handler extensions, not surgery):
- Full E40 S16F11 body (MF / PRRECIPEMETHOD / RCPSPEC / PRPROCESSPARAMS).
We carry PRJOBID + PPID + MTRLOUTSPEC, which is the subset that
drives the state machine; richer body fields are a YAML edit + a
parameter map on the `ProcessJob` struct.
- S16F15/F16 PRJobCreateMultiple, S16F17/F18 PRJobMultipleDequeue.
- E14 generic ObjectService form. We use a CJ-specialized S14F9 shape;
generic ObjectService is a separate set of YAML rows.
- E87 Carrier Management and E90 Substrate Tracking — Layer 5
continues there per `implementation_plan.md`.
---
## 5. Message coverage matrix
| Pair | Direction | Status | Implemented in | Tested |
@@ -110,6 +156,7 @@ Legend:
| S2F15 / S2F16 | H→E | ✅ | catalog | ✅ round-trip |
| S2F17 / S2F18 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F23 / S2F24 | H→E | ✅ | catalog | ✅ round-trip |
| S2F25 / S2F26 | H→E | ✅ | catalog | ✅ round-trip (loopback diagnostic) |
| S2F29 / S2F30 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F31 / S2F32 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S2F33 / S2F34 | H→E | ✅ | catalog | ✅ round-trip + demo |
@@ -123,6 +170,8 @@ Legend:
| S5F3 / S5F4 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S5F5 / S5F6 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S5F7 / S5F8 | H→E | ✅ | catalog | ✅ round-trip |
| S5F9 / S5F10 | E→H | ✅ | catalog | ✅ round-trip (exception post) |
| S5F11 / S5F12 | E→H | ✅ | catalog | ✅ round-trip (exception clear) |
| S6F1 / S6F2 | E→H | ✅ | catalog | ✅ round-trip |
| S6F11 / S6F12 | E→H | ✅ | catalog | ✅ round-trip + demo |
| S6F23 / S6F24 | H→E | ✅ | catalog | ✅ round-trip + demo |
@@ -140,6 +189,13 @@ Legend:
| S10F1 / S10F2 | H→E | ✅ | catalog | ✅ round-trip + demo |
| S10F3 / S10F4 | E→H | ✅ | catalog | ✅ round-trip + demo |
| S10F5 / S10F6 | H→E | ✅ | catalog | ✅ round-trip |
| S14F9 / S14F10 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJ create) |
| S14F11 / S14F12 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJ delete) |
| S16F5 / S16F6 | H→E | ✅ | catalog | ✅ round-trip (E40 PRJobCommand) |
| S16F9 | E→H | ✅ | catalog + server | ✅ round-trip + auto-emitted per PJ transition (E40 PRJobAlert) |
| S16F11 / S16F12 | H→E | ✅ | catalog | ✅ round-trip + demo (E40 PRJobCreate) |
| S16F13 / S16F14 | H→E | ✅ | catalog | ✅ round-trip (E40 PRJobDequeue) |
| S16F27 / S16F28 | H→E | ✅ | catalog | ✅ round-trip + demo (E94 CJobCommand) |
---
@@ -169,11 +225,26 @@ walks ~20 SECS transactions end-to-end:
16. Spool window: `SPOOL_ON` → `START` (emission goes to spool) →
`SPOOL_OFF` → `S6F23(Transmit)` → server drains queued S6F11 to host.
17. `S7F19`/`S7F20` recipe list, `S7F5`/`S7F6` fetch RECIPE-A.
18. `S10F1`/`S10F2` host → equipment terminal display.
19. `S1F15`/`S1F16` Request Offline.
20. `Separate.req` → clean close on both sides.
18. `S16F11`/`S16F12` create Process Job `PJ-1` with PPID `RECIPE-A`.
19. `S14F9`/`S14F10` create Control Job `CJ-1` containing `[PJ-1]`.
20. `S16F27`/`S16F28` CJSTART → equipment cascades CJ Queued → Executing
and the contained PJ through SettingUp → WaitingForStart →
Processing → ProcessComplete, emitting one `S16F9 PRJobAlert` per PJ
transition and `S6F11(CEID=400)` / `S6F11(CEID=401)` for CJ Executing
/ Completed.
21. `S14F11`/`S14F12` delete `CJ-1`.
22. `S10F1`/`S10F2` host → equipment terminal display.
23. `S1F15`/`S1F16` Request Offline.
24. `Separate.req` → clean close on both sides.
Unit tests: **84+ cases / 480+ assertions pass** (`docker compose run --rm tests`).
Unit tests: **148 cases / 794 assertions pass** (`docker compose run --rm tests`).
The suite includes integration tests that drive a real `hsms::Connection`
over a loopback socket pair to verify the E37 §7.2 / §7.4 / §7.7
edge cases (Select.req while already SELECTED → AlreadyActive, Deselect.req
while NOT_SELECTED → NotEstablished, Reject.req for unsupported SType /
PType, Reject.req for data while NOT_SELECTED) — not just the happy path.
The E30 §6.5 Communication state machine is unit-tested independently of
the transport (timer firings simulated via test callbacks).
---
@@ -184,14 +255,13 @@ compliance claim.
| Item | Why it's out of scope |
|---------------------------------------|----------------------|
| Material Movement (E30 §6.18) | This is **E40 / E87 / E90** (carrier management, substrate tracking, process jobs) — separate SEMI standards layered *on top of* E30. E30 GEM compliance does not require them. Adding E40 is a Layer-5 deliverable per `implementation_plan.md`. |
| Material Movement (E30 §6.18) | The job-management half is now in (E40 + E94, §4a). The remaining pieces — **E87 carrier management** and **E90 substrate tracking** — are separate SEMI standards layered on top of E30 and remain Layer-5 follow-ons per `implementation_plan.md`. |
| Multi-block SECS-I transfers | Multi-block (S6F5/F6, S6F7/F8 etc.) is a **SECS-I** concept for 244-byte serial frames. HSMS allows arbitrarily large bodies (up to the codebase's 16 MiB cap), so multi-block is structurally not needed. E37-based GEM equipment does not require it. |
| HSMS-GS (multi-session) | Out of scope — modern HSMS-SS covers virtually all current GEM equipment. |
| Equipment Processing States (concrete states) | E30 §6.3 says the specific states are tool-defined. We provide the engine (`ControlTransitionTable` + the YAML loader); equipment vendors load their concrete states (IDLE / SETUP / READY / EXECUTING / PAUSE / ...) the same way `data/control_state.yaml` is loaded today. Spec-compliant either way. |
| Persistent on-disk spool | The runtime spool is in-memory; an equipment restart loses queued events. Real fab equipment would back it with a journal. Standard does not mandate persistence. |
| E42 Enhanced Process Programs (S7F23–F26) | A separate SEMI standard. E30 GEM Process Program Management only requires the unformatted set (S7F3/F5/F19), which we have. |
| S10F7 Broadcast Terminal Display | Rarely used; equipment vendors typically forgo it. Not required for the Terminal Services capability. |
| JIS-8 / C2 (Unicode) SECS-II formats | Not used in modern Western fab tooling. Codec is structured so additional formats are trivial to add. |
---
@@ -219,8 +289,8 @@ marketing claim would still need:
the dispatcher; the application is what makes a specific tool
GEM-compliant.
In short: this is a **GEM-conformant runtime stack**, not a
GEM-conformant *tool*. Pointing the runtime at a real piece of
equipment, populating the YAML files with the tool's real SVIDs / ECIDs
/ alarms / capabilities, and wiring the application callbacks completes
the picture.
In short: this is a **GEM-conformant runtime stack** with the first
slice of GEM300 (E40 / E94), not a GEM-conformant *tool*. Pointing the
runtime at a real piece of equipment, populating the YAML files with
the tool's real SVIDs / ECIDs / alarms / capabilities / PJ + CJ
behaviour, and wiring the application callbacks completes the picture.