raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: chapters 02 + 03 of the guided tour (Part 1 complete)

Browse Source 
02 — The cast of characters: equipment, EAP, MES, fab planner, AMHS,
operator.  Who initiates which conversation, why the equipment is
the passive side of HSMS by convention, how the AMHS handshake is
out-of-band relative to SECS.  Cross-references the relevant
namespace and test files for each actor.

03 — Vocabulary + a wafer's journey: follows one 300 mm wafer
end-to-end through a fab and labels every SECS message and acronym
that fires.  Introduces SVID / DVID / ECID / CEID / RPTID / ALID /
PPID / MDLN / SOFTREV / HCACK / ALCD / OFLACK / CAACK / SMACK / etc.
in context rather than as a list.  Includes one-screen reference
tables for the remaining acknowledge codes, T-timers in all four
contexts (HSMS / SECS-I / E84 / E30 communication state), and a
stream-by-stream summary.

Part 1 (Foundations) of the guided tour is now complete — a reader
who reads chapters 01–03 can describe the protocol stack, identify
the actors, and recognise every acronym they'll meet in Part 2.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-09 19:35:43 +02:00
parent bc54de7711
commit 60fa164626
12 changed files with 1009 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/COMPLIANCE.md
+456
View File
@@ -0,0 +1,456 @@
# SECS/GEM Compliance
A per-capability accounting against the foundational SEMI standards
**E5 (SECS-II)**, **E30 (GEM)**, **E37 (HSMS, SS + GS)**, **E4 (SECS-I)**,
plus the full GEM 300 stack: **E40** (process jobs), **E42** (formatted
process programs), **E94** (control jobs), **E87** (carriers), **E90**
(substrates), **E116** (equipment performance tracking), **E120** (common
equipment model), **E157** (module process tracking), **E84** (parallel
I/O), **E148** (time synchronization), **E39** (object services), plus
**E5 §13** wafer maps.
> **Status.** Every GEM Fundamental and every GEM Additional capability
> that E30 binds to a concrete SECS-II message set is implemented, and
> every GEM 300 standard the project sets out to cover is implemented
> end-to-end (state machines + stores + wire messages + dispatch). See
> §8 for what "100% GEM-compliant" can and cannot honestly mean about a
> codebase, and the README "Deferred follow-ups" section for the
> non-shipped pieces that aren't behavioural gaps.
Legend:
- ✅ **Full** — implemented to the spec; round-trip-tested.
- 🟡 **Partial** — implemented in the demo path with a documented limitation.
- ⬜ **Out of scope** — deliberately not implemented; reason given.
---
## 1. E37 — HSMS transport
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| TCP transport | ✅ | E37 §6 | `hsms::Connection` over standalone Asio. |
| 4-byte length prefix + 10-byte header | ✅ | E37 §8.2 | `hsms::Frame::encode/decode`. |
| Session ID, byte2, byte3, PType, SType, system-bytes | ✅ | E37 §8.3 | `hsms::Header`. |
| `Select.req / .rsp` | ✅ | E37 §7.2 | `SType` 1/2; SelectStatus enum (0–3). |
| `Deselect.req / .rsp` | ✅ | E37 §7.4 | `SType` 3/4; DeselectStatus enum (0–2). |
| `Linktest.req / .rsp` | ✅ | E37 §7.5 | `SType` 5/6; periodic interval configurable. |
| `Separate.req` | ✅ | E37 §7.6 | `SType` 9; graceful close after flush. |
| `Reject.req` | ✅ | E37 §7.7 | Emitted on data-while-NOT-SELECTED. |
| Connection state machine NOT-CONNECTED → NOT-SELECTED → SELECTED | ✅ | E37 §6.3 | Both Active and Passive modes. |
| T3 reply timeout | ✅ | E37 §10 | Per-transaction `steady_timer`. |
| T5 connect separation timeout | ✅ | E37 §10 | `Client::schedule_retry`. |
| T6 control transaction timeout | ✅ | E37 §10 | One concurrent control transaction. |
| T7 not-selected timeout (passive) | ✅ | E37 §10 | Armed on connect / on Deselect.req. |
| T8 intercharacter timeout | ✅ | E37 §10 | Bounds the payload read after length prefix. |
| HSMS-SS (single-session) | ✅ | E37 §11 | Default mode: the constructor registers a single session. |
| HSMS-GS (general-session) | ✅ | E37 §11 | `Connection::add_session(device_id)` registers extra sessions; per-session SELECTED state + message handlers; Select.req carries session_id=device_id in GS mode. |
---
## 1a. E4 — SECS-I transport (block protocol)
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| 10-byte block header (R/W/E bits, system bytes) | ✅ | E4 §6.2 | `secsi::Header` with bit-precise pack/unpack. |
| Length-prefixed block + 2-byte checksum | ✅ | E4 §6.1, §6.3 | `secsi::Block::encode/decode`. |
| Multi-block message split / assemble | ✅ | E4 §7.2.3 | `split_message` / `assemble_message`; E-bit only on the final block. |
| ENQ/EOT/ACK/NAK handshake | ✅ | E4 §7.1 | `secsi::Protocol` half-duplex FSM. |
| RTY retry counter | ✅ | E4 §10.2 | Per-block retry budget, exhaust → ActionRaiseError. |
| T1 inter-character timer hook | ✅ | E4 §10.1 | Drained in `RecvBlock`; host wires the actual asio timer. |
| T2 protocol timer hook | ✅ | E4 §10.1 | Triggers a retry from any send state. |
| T3 reply timer | ✅ | E4 §10.1 | FSM tracks system_bytes of outstanding W=1 primaries; arms T3 on send-complete, cancels on matching reply, aborts on expiry. |
| T4 inter-block timer | ✅ | E4 §10.1 | FSM arms T4 when a block delivers with end_block=false; cancels when the next block arrives, aborts on expiry. |
| Master/slave contention resolution | ✅ | E4 §7.1.4 | Slave yields on simultaneous ENQ; master holds. |
| Serial port wiring (asio) | ⬜ | — | FSM is IO-free; serial integration is a wiring follow-up. |
| TCP tunnel for testing | ✅ | — | `secsi::TcpTransport` wraps the FSM behind an asio TCP socket; mirrors secsgem-py's `secsitcp/`. |
---
## 2. E5 — SECS-II encoding
| Item | Status | Spec ref | Notes |
|---------------------------------------|--------|----------|-------|
| Format byte + 1/2/3 length bytes | ✅ | E5 §9 | `secs2::encode_into`. |
| List (`L`) | ✅ | E5 §9.3 | Recursive. |
| ASCII (`A`) | ✅ | E5 §9.5 | |
| Binary (`B`) | ✅ | E5 §9.5 | |
| Boolean (`BOOLEAN`) | ✅ | E5 §9.5 | |
| `U1, U2, U4, U8` (big-endian) | ✅ | E5 §9.5 | Identifier parsers accept any width per the SEMI wildcard rule. |
| `I1, I2, I4, I8` (big-endian, two's complement) | ✅ | E5 §9.5 | Same lenient-width policy. |
| `F4, F8` (IEEE 754 big-endian) | ✅ | E5 §9.5 | bit-cast round-trip. |
| JIS-8 (single-byte JIS text) | ✅ | E5 §9.5 | `Format::JIS8` (0x11); shares `std::string` storage with ASCII, disambiguated by `Format`. |
| C2 (Unicode 2-byte code points) | ✅ | E5 §9.5 | `Format::C2` (0x12); big-endian uint16_t code points. |
| SML text rendering | ✅ | E5 Annex | `secs2::to_sml`. JIS-8 prints as `<J "...">`, C2 as `<C 65 66 ...>`. |
| SML parser (inverse of `to_sml`) | ✅ | — | `secs2::from_sml`; round-trips every format. |
| `ASCII | Binary` wildcard fields | ✅ | E5 | `BINARY_OR_ASCII` schema type for PPBODY etc.; accepted by `as_text_or_binary`. |
---
## 3. E30 — GEM Fundamental capabilities (§5.2)
| Fundamental Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| State models | ✅ | E30 §6.2 | — | E30 control state machine (5 states) + HSMS communication state machine. |
| Equipment Processing States | ✅ | E30 §6.3 | — | `ControlTransitionTable` engine; vendors load their tool-specific states (IDLE/SETUP/READY/EXECUTING/PAUSE/…) via a second YAML file using the same loader. The spec leaves the concrete states tool-specific. |
| Host-Initiated S1F13/F14 scenario | ✅ | E30 §6.5 | S1F13/F14 | |
| Event Notification | ✅ | E30 §6.6 | S6F11/F12 | Equipment-initiated, host-acknowledged. |
| On-Line Identification | ✅ | E30 §6.7 | S1F1/F2 | MDLN + SOFTREV. |
| Error Messages | ✅ | E30 §6.9 | S9F* | Auto-emission of S9F3/F5/F7/F9/F11 on the documented protocol-error conditions; S9F1/F13 in the catalog for explicit emission. |
| Documentation | ✅ | E30 §6.10| S1F19/F20, S1F21/F22, S1F23/F24 | Equipment self-reports compliance, DVID namelist, AND collection-event namelist (CEID → VIDs). |
| Control (Operator-Initiated) | ✅ | E30 §6.2 | — | `ControlStateMachine::operator_online/offline/local/remote`. |
---
## 4. E30 — GEM Additional capabilities (§5.3)
| Additional Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Establish Communications | ✅ | E30 §6.5 | S1F13/F14 | Both directions modeled; COMMACK enum. Backed by the E30 §6.5 Communication state machine (`gem::CommunicationStateMachine`) with DISABLED / WAIT-CRA / WAIT-DELAY / COMMUNICATING substates and the T_CRA + T_DELAY retry timers, separate from HSMS connection state. |
| Dynamic Event Report Configuration | ✅ | E30 §6.6 | S2F33/F34, S2F35/F36, S2F37/F38 | Full Define-Report / Link-Event / Enable-Event pipeline with all four ack enums. Host-initiated readback via S6F15/F16, S6F19/F20, S6F21/F22. |
| Variable Data Collection | ✅ | E30 §6.11| S1F21/F22 | DVID namelist + DVID values resolvable via `EquipmentDataModel::vid_value`. |
| Trace Data Collection | ✅ | E30 §6.12| S2F23/F24, S6F1/F2 | `TraceStore` keeps active TRID→TraceConfig; periodic sampling left to the application's scheduler. |
| Status Data Collection | ✅ | E30 §6.13| S1F3/F4, S1F11/F12 | |
| Alarm Management | ✅ | E30 §6.14| S5F1/F2, S5F3/F4, S5F5/F6, S5F7/F8 | Full set. ALCD bit-7 set/cleared, lower-7 category. |
| Remote Control | ✅ | E30 §6.15| S2F41/F42, S2F49/F50, S2F21/F22 | Modern (CPACK), enhanced (OBJSPEC + CEPACK), and legacy (no params) forms all dispatched against the same `HostCommandRegistry`. |
| Equipment Constants | ✅ | E30 §6.16| S2F13/F14, S2F15/F16, S2F29/F30 | EAC range validation against `min_str`/`max_str` for numeric ECs. |
| Process Program Management | ✅ | E30 §6.17| S7F1/F2, S7F3/F4, S7F5/F6, S7F17/F18, S7F19/F20, S7F23/F24, S7F25/F26 | Unformatted PP load-inquire/send/request/delete/list + E42 enhanced (formatted) PP send/request via S7F23-F26. RecipeStore carries both views simultaneously. |
| Material Movement | ✅ | E30 §6.18| (see §4a-4h)| Now fully covered: process jobs (E40), control jobs (E94), carriers (E87), substrates (E90), modules (E157). |
| Equipment Terminal Services | ✅ | E30 §6.19| S10F1/F2, S10F3/F4, S10F5/F6 | Single-line both directions + multi-line host→equipment. S10F7 broadcast intentionally omitted (rarely used). |
| Clock | ✅ | E30 §6.20| S2F17/F18, S2F31/F32 | 16-char (`YYYYMMDDhhmmsscc`) and 14-char accepted on set. Drift tracking + quality via E148 (§4g). |
| Limits Monitoring | ✅ | E30 §6.21| S2F45/F46, S2F47/F48 | `LimitMonitorStore` keyed by VID with multiple `LimitDefinition` (LIMITID + upper/lower as arbitrary Items). |
| Spooling | ✅ | E30 §6.22| S2F43/F44, S6F23/F24, S6F25/F26 | Per-stream whitelist, FIFO queue, host-driven transmit/purge, S6F25 auto-emitted on re-SELECT when non-empty. **Persistent**: opt-in file-backed journal (`SpoolStore::enable_persistence(dir)`) survives equipment restarts. |
| Control | ✅ | E30 §6.2 | — | See Fundamental. |
---
## 4a. E40 Process Jobs
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| PJ state model | ✅ | E40 §6.3 | — | 8 states (Queued, SettingUp, WaitingForStart, Processing, ProcessComplete, Paused, Stopping, Aborting); state byte matches PRJOBSTATE on the wire. |
| PRJobCreate (full body) | ✅ | E40 §10.2| S16F11/F12 | Full E40-0705 body: `<L,5 PRJOBID MF PRRECIPEMETHOD <L,2 PPID <L RCPVARLIST>> <L MTRLOUTSPEC> <L PRPROCESSPARAMS>>`. PPID validated against `RecipeStore`. |
| PRJobCreateMultiple | ✅ | E40 §10 | S16F15/F16 | Bulk variant; per-job ACK list. |
| PRJobDequeue | ✅ | E40 §10.2| S16F13/F14 | Only legal while PJ is QUEUED. |
| PRJobMonitor | ✅ | E40 §10 | S16F7/F8 | Per-PJ alert enable/disable. |
| PRJobCommand | ✅ | E40 §10.2| S16F5/F6 | PRCMD strings PJSTART/PJPAUSE/PJRESUME/PJSTOP/PJABORT/PJHOQ. |
| PRJobAlert | ✅ | E40 §10.3| S16F9 | Equipment-initiated one-way (W=0). Fires automatically on every PJ state transition. |
## 4b. E94 Control Jobs
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| CJ state model | ✅ | E94 §6 | — | 9 states; CJ owns an ordered `prjobids` list. |
| CreateObject (CJ) | ✅ | E94 §6.4 | S14F9/F10 | Body `<L,2 CTLJOBID L,n PRJOBIDs>`. |
| DeleteObject (CJ) | ✅ | E94 §6.4 | S14F11/F12 | |
| CJobCommand | ✅ | E94 §6.4 | S16F27/F28 | CTLJOBCMD: CJSTART, CJPAUSE, CJRESUME, CJSTOP, CJABORT. |
| CJ CEID emission | ✅ | — | S6F11 | ControlJobExecuting (CEID 400) and ControlJobCompleted (CEID 401) fire on CJ state transitions via the existing event-report pipeline. |
## 4c. E87 Carrier Management
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Carrier state model | ✅ | E87 §6 | — | CarrierStateMachine + LoadPortStateMachine; CIDS, CSMS, CASS, LPRS, LPTS axes. |
| CarrierAction | ✅ | E87 §10.2| S3F17/F18 | ProceedWithCarrier, CancelCarrier, BindCarrierID. |
| Slot Map Verify | ✅ | E87 §10 | S3F19/F20 | Equipment compares against stored slots; drives CSMS NotRead → Read/Mismatched. |
| Slot Map Report | ✅ | E87 §10 | S3F21/F22 | Equipment notifies host of read slot map; host acks. |
| Port Group Change | ✅ | E87 §10.4| S3F23/F24 | Host modifies load-port grouping; PortGroupAck. |
| Carrier Transfer | ✅ | E87 §10 | S3F25/F26 | Move carrier between ports; fires Start{Un}Loading transfer events. |
| Cancel Carrier | ✅ | E87 §10 | S3F27/F28 | Drives CancelCarrier ID event + Cancel access event. |
## 4d. E90 Substrate Tracking
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Substrate state model | ✅ | E90 §6 | — | Three axes: location (AtSource/AtWork/AtDestination), processing (NeedsProcessing/InProcess/Processed/Aborted/Stopped/Rejected/Lost/Skipped), id-status (Confirmed/Unconfirmed/Lost). |
| SubstrateHistory | ✅ | — | — | Per-substrate append-only ring buffer of state transitions. |
| Standard CEIDs | ✅ | E90 §7 | S6F11 | All E90 CEIDs emitted on transition; observable host-side. |
## 4e. E116 Equipment Performance Tracking
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| EPT state machine | ✅ | E116 §6 | — | NonScheduledTime / ScheduledDowntime / UnscheduledDowntime / Engineering / Standby / Productive. |
| Time-bucket accounting | ✅ | E116 | — | Cumulative ms per state; resettable. |
| EPT CEIDs | ✅ | E116 | S6F11 | One CEID per state. |
## 4f. E120 Common Equipment Model
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Generic ObjectService (E39) | ✅ | E39 §5 | S14F1/F2, S14F3/F4 | GetAttr / SetAttr against `CemObjectStore`; OBJTYPE validation. |
| CemObjectStore | ✅ | E120 | — | Typed objects keyed by OBJSPEC + ObjType. |
## 4g. E148 Time Synchronization
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Time-sync drift tracking | ✅ | E148 | S2F31/F32 | Drift metric maintained on every set; quality score for hosts. |
## 4h. E157 Module Process Tracking
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Module state machine | ✅ | E157 §6 | — | NotExecuting / GeneralExecuting / StepExecuting / StepCompleted. |
| Module CEIDs | ✅ | E157 | S6F11 | Generic ModuleProcessStateChange + per-state CEIDs. |
## 4i. E84 Parallel I/O Handoff
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Handoff state machine | ✅ | E84 | — | Full LOAD / UNLOAD signal vocabulary (CS_0/CS_1, VALID, TR_REQ, BUSY, COMPT, AM_AVBL, ES). Per-port via `E84PortStore` keyed by `port_id`; independent FSMs run in parallel per load port. |
| Handshake timers TA1 / TA2 / TA3 | ✅ | E84 §6 | — | `E84StateMachine::set_timeouts({ta1, ta2, ta3})` + `set_timer_handlers(arm, cancel)`. TA1 armed in ValidAsserted, TA2 in Load/UnloadReady, TA3 in Transferring; cancelled on the matching transition out. Expiry transitions to `HandoffFault` (latched until `reset()`). FSM stays I/O-free — application drives the real clock (asio::steady_timer in the reference server). |
## 4j. E5 §13 Wafer Maps (S12)
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| Map Setup Data | ✅ | E5 §13 | S12F1/F2, S12F3/F4 | |
| Map Transmit Inquire / Grant | ✅ | E5 §13 | S12F5/F6 | |
| Map Data Send — row format (MAPFT=0) | ✅ | E5 §13 | S12F7/F8 | |
| Map Data Send — array format (MAPFT=1)| ✅ | E5 §13 | S12F9/F10 | STRP + BINLT. |
| Map Data Send — coord format (MAPFT=2)| ✅ | E5 §13 | S12F11/F12 | XYPOS + per-die BIN. |
| Map Data Request — row | ✅ | E5 §13 | S12F13/F14 | |
| Map Data Request — array | ✅ | E5 §13 | S12F15/F16 | |
| Map Data Request — coord | ✅ | E5 §13 | S12F17/F18 | |
| Map Error Send | ✅ | E5 §13 | S12F19 | One-way error report. |
## 4k. Exception Recovery (beyond E5 base alarms)
| Capability | Status | Spec ref | Messages | Notes |
|---------------------------------------|--------|----------|----------|-------|
| ExceptionStateMachine FSM | ✅ | — | — | Per-EXID Posted / Recovering / RecoverFailed / Cleared lifecycle. Not in upstream secsgem-py. |
| Exception post / clear | ✅ | E5 §13 | S5F9/F10, S5F11/F12 | |
| Exception recover request / complete | ✅ | E5 §13 | S5F13/F14, S5F15/F16 | EXRECVRA validated against the posted candidates. |
| Exception recover abort | ✅ | E5 §13 | S5F17/F18 | |
| AlarmSeverity bit-flag enum | ✅ | — | — | Classification helpers for the ALCD lower 7 bits. |
---
## 5. Message coverage matrix
164 SECS-II messages in the catalog, spanning streams 1, 2, 3, 5, 6, 7, 9, 10, 12, 14, 16.
| Pair | Direction | Status | Notes |
|------------------|-----------|--------|-------|
| S1F1 / S1F2 | H↔E | ✅ | round-trip + demo |
| S1F3 / S1F4 | H→E | ✅ | round-trip + demo |
| S1F11 / S1F12 | H→E | ✅ | round-trip + demo |
| S1F13 / S1F14 | H↔E | ✅ | round-trip + demo |
| S1F15 / S1F16 | H→E | ✅ | round-trip + demo |
| S1F17 / S1F18 | H→E | ✅ | round-trip + demo |
| S1F19 / S1F20 | H→E | ✅ | round-trip + demo (compliance self-report) |
| S1F21 / S1F22 | H→E | ✅ | round-trip + demo |
| S1F23 / S1F24 | H→E | ✅ | collection event namelist (CEID → VID) |
| S2F13 / S2F14 | H→E | ✅ | EC values |
| S2F15 / S2F16 | H→E | ✅ | EC set |
| S2F17 / S2F18 | H→E | ✅ | clock |
| S2F21 / S2F22 | H→E | ✅ | legacy remote command (no params) |
| S2F23 / S2F24 | H→E | ✅ | trace initialize |
| S2F25 / S2F26 | H→E | ✅ | loopback diagnostic |
| S2F29 / S2F30 | H→E | ✅ | EC namelist |
| S2F31 / S2F32 | H→E | ✅ | set time |
| S2F33 / S2F34 | H→E | ✅ | define report |
| S2F35 / S2F36 | H→E | ✅ | link event report |
| S2F37 / S2F38 | H→E | ✅ | enable event |
| S2F41 / S2F42 | H→E | ✅ | host command (modern) |
| S2F43 / S2F44 | H→E | ✅ | reset spooling |
| S2F45 / S2F46 | H→E | ✅ | define variable limits |
| S2F47 / S2F48 | H→E | ✅ | request limit attrs |
| S2F49 / S2F50 | H→E | ✅ | enhanced remote command (OBJSPEC + CPACK/CEPACK) |
| S3F17 / S3F18 | H→E | ✅ | E87 carrier action |
| S3F19 / S3F20 | H→E | ✅ | E87 slot map verify |
| S3F21 / S3F22 | E→H | ✅ | E87 slot map report |
| S3F23 / S3F24 | H→E | ✅ | E87 port group change |
| S3F25 / S3F26 | H→E | ✅ | E87 carrier transfer |
| S3F27 / S3F28 | H→E | ✅ | E87 cancel carrier |
| S5F1 / S5F2 | E→H | ✅ | alarm send |
| S5F3 / S5F4 | H→E | ✅ | enable alarm |
| S5F5 / S5F6 | H→E | ✅ | list alarms |
| S5F7 / S5F8 | H→E | ✅ | list enabled alarms |
| S5F9 / S5F10 | E→H | ✅ | exception post |
| S5F11 / S5F12 | E→H | ✅ | exception clear |
| S5F13 / S5F14 | H→E | ✅ | exception recover request |
| S5F15 / S5F16 | E→H | ✅ | exception recover complete |
| S5F17 / S5F18 | H→E | ✅ | exception recover abort |
| S6F1 / S6F2 | E→H | ✅ | trace data |
| S6F5 / S6F6 | E→H | ✅ | multi-block inquire / grant |
| S6F7 / S6F8 | H→E | ✅ | data transfer request / send |
| S6F11 / S6F12 | E→H | ✅ | event report (unsolicited) |
| S6F15 / S6F16 | H→E | ✅ | event report request (host-initiated) |
| S6F19 / S6F20 | H→E | ✅ | individual report request |
| S6F21 / S6F22 | H→E | ✅ | annotated individual report request |
| S6F23 / S6F24 | H→E | ✅ | request spooled data |
| S6F25 / S6F26 | E→H | ✅ | spool data ready (auto on re-SELECT) |
| S7F1 / S7F2 | H→E | ✅ | PP load inquire / grant |
| S7F3 / S7F4 | H→E | ✅ | PP send |
| S7F5 / S7F6 | H→E | ✅ | PP request |
| S7F17 / S7F18 | H→E | ✅ | PP delete |
| S7F19 / S7F20 | H→E | ✅ | PP list |
| S7F23 / S7F24 | H→E | ✅ | E42 formatted PP send |
| S7F25 / S7F26 | H→E | ✅ | E42 formatted PP request |
| S9F1, F3, F5, F7, F9, F11, F13 | E↔H | ✅ | protocol errors; auto-emitted on the documented conditions |
| S10F1 / S10F2 | E→H | ✅ | terminal request (equipment originated) |
| S10F3 / S10F4 | H→E | ✅ | terminal display single |
| S10F5 / S10F6 | H→E | ✅ | terminal display multi |
| S10F7 | H→E | ✅ | terminal display broadcast (W=0, no reply) |
| S12F1–F19 | H↔E | ✅ | wafer maps — row, array, coord; setup, request, send, error |
| S14F1 / S14F2 | H→E | ✅ | E39 GetAttr |
| S14F3 / S14F4 | H→E | ✅ | E39 SetAttr |
| S14F9 / S14F10 | H→E | ✅ | E94 CJ create |
| S14F11 / S14F12 | H→E | ✅ | E94 CJ delete |
| S16F5 / S16F6 | H→E | ✅ | E40 PRJobCommand |
| S16F7 / S16F8 | H→E | ✅ | E40 PRJobMonitor |
| S16F9 | E→H | ✅ | E40 PRJobAlert (auto on transition) |
| S16F11 / S16F12 | H→E | ✅ | E40 PRJobCreate (full body) |
| S16F13 / S16F14 | H→E | ✅ | E40 PRJobDequeue |
| S16F15 / S16F16 | H→E | ✅ | E40 PRJobCreateMultiple |
| S16F27 / S16F28 | H→E | ✅ | E94 CJobCommand |
---
## 6. Demo evidence
The two-container demo (`docker compose up --no-deps server client`)
walks ~20 SECS transactions end-to-end:
1. TCP connect → `Select.req` → `Select.rsp(Ok)` → SELECTED on both sides.
2. `S1F13`/`S1F14` Establish Comms.
3. `S1F17`/`S1F18` Request Online; control state transitions
`HostOffline → AttemptOnline → OnlineRemote`.
4. `S1F19`/`S1F20` host fetches the equipment's GEM-compliance self-report.
5. `S1F21`/`S1F22` DVID namelist.
6. `S1F11`/`S1F12` SVID namelist → `S1F3`/`S1F4` values read.
7. `S2F29`/`S2F30` EC namelist → `S2F13`/`S2F14` EC read.
8. `S2F17`/`S2F18` clock read.
9. `S2F33`/`S2F34` Define Report 1000 over the 3 SVIDs.
10. `S2F35`/`S2F36` Link CEIDs 200 and 300 to Report 1000.
11. `S2F37`/`S2F38` Enable CEIDs 200, 300.
12. `S2F41`/`S2F42` host command **START** → server emits
`S6F11(CEID=300)` carrying the linked Report 1000 → host acks `S6F12`.
13. `S5F5`/`S5F6` list alarm directory.
14. `S5F3`/`S5F4` enable alarm 1.
15. `S2F41`/`S2F42` host command **FAULT** → server emits
`S5F1` (ALCD=0x84) + `S6F11(CEID=200)`.
16. Spool window: `SPOOL_ON` → `START` (emission goes to spool) →
`SPOOL_OFF` → `S6F23(Transmit)` → server drains queued S6F11 to host.
17. `S7F19`/`S7F20` recipe list, `S7F5`/`S7F6` fetch RECIPE-A.
18. `S16F11`/`S16F12` create Process Job `PJ-1` with PPID `RECIPE-A`.
19. `S14F9`/`S14F10` create Control Job `CJ-1` containing `[PJ-1]`.
20. `S16F27`/`S16F28` CJSTART → equipment cascades CJ Queued → Executing
and the contained PJ through SettingUp → WaitingForStart →
Processing → ProcessComplete, emitting one `S16F9 PRJobAlert` per PJ
transition and `S6F11(CEID=400)` / `S6F11(CEID=401)` for CJ Executing
/ Completed.
21. `S14F11`/`S14F12` delete `CJ-1`.
22. `S10F1`/`S10F2` host → equipment terminal display.
23. `S1F15`/`S1F16` Request Offline.
24. `Separate.req` → clean close on both sides.
Unit tests: **445 cases / 2753 assertions pass** (`docker compose run --rm tests`).
The suite includes integration tests that drive a real `hsms::Connection`
over a loopback socket pair to verify the E37 §7.2 / §7.4 / §7.7
edge cases — not just the happy path.
The E30 §6.5 Communication state machine is unit-tested independently of
the transport (timer firings simulated via test callbacks).
Live conformance harness: **`build/secs_conformance --host <ip> --port <p>`**
walks 47 host-driven checks against a passive equipment, covering every
E30 fundamental + additional capability that COMPLIANCE.md ✅ — establish
comms, identification, status/DVID/CEID/EC namelists + values, dynamic
event reports (define/link/enable + S6F15/F19/F21 readbacks), unsolicited
S6F11 observation after RCMD, all three remote-command forms
(S2F41/F21/F49), trace init, limits, spool reset + transmit, alarms
(list/list-enabled/enable), exception recover/abort (S5F13/F17), PP
load-inquire/list/request, both terminal-display directions
(S10F3/F5), E40 PJ create/monitor/command/dequeue, E94 CJ
create/command/delete, E87 carrier action/slot-map/transfer/cancel,
and E39 GetAttr. Exits 0 with PASS/FAIL summary; intended to be run
against vendor equipment as the first-line conformance probe.
---
## 7. Interoperability with external implementations
Four independent external validators cross-check the codebase. None
of them shares code with us; three of them are not even C++. Full
test plan in [VERIFICATION.md](VERIFICATION.md); proof commands in
[PROOFS.md](PROOFS.md).
**secsgem-py 0.3.0** (Python reference implementation, Apache 2.0).
Three harnesses under `interop/`:
- **secsgem-py active host → C++ passive server**: 31 named checks
across S1/S2/S5/S6/S7/S10 plus unsolicited S5F1/S6F11.
- **C++ active host → secsgem-py passive equipment**: HSMS select +
S1F13 + S1F1 + S1F3 + clean separate; exits 0.
- **C++ active host → raw GEM 300 streams** (`raw_gem300_harness.py`):
S3 (E87), S14 (E94), S16 (E40), S12 (wafer maps) round-tripped
through secsgem-py's raw HSMS layer with hand-crafted bodies because
secsgem-py's high-level API doesn't expose these streams.
**secs4java8** (independent Java SECS implementation by Kenta
Shimizu, Apache 2.0). 55 cross-validation checks under
`interop/secs4j/` covering S1/S2/S3/S5/S6/S7/S10/S14/S16, the full
E40 PJ body, dynamic event reports + unsolicited S6F11/S5F1
observation, alarm management, spool, PP management, terminal
services, limits, trace, E39, and the GEM 300 streams secsgem-py
couldn't easily drive. This is the only validator that exercises
S2F49 (enhanced remote command) and S5F13–F18 (exception recovery)
end-to-end against a second SECS implementation.
**Wireshark / tshark HSMS dissector** (independent network-protocol
authors). `interop/tshark_validate.sh` captures a pcap of the demo
run, dissects with tshark's built-in HSMS dissector, asserts no
malformed-packet warnings and that every expected control + data
frame parses. 69 HSMS frames dissected cleanly. This catches
framing bugs that two SECS implementations might *both* share but
that a third party reading the bytes would flag.
**libFuzzer + ASan + UBSan** (coverage-guided structural search).
`apps/fuzz_secs2_decode.cpp` and `apps/fuzz_sml_parse.cpp` feed
random inputs to the decoder and SML parser under
AddressSanitizer + UndefinedBehaviorSanitizer. 60-second CI lanes
typically explore 200 000+ inputs through `secs2::decode` and
1 400 000+ through `try_parse_sml`; 0 crashes, 0 ASan/UBSan reports.
Bugs surfaced and fixed across the four channels include: strict
per-width parsing rejected U1-encoded identifiers (SEMI E5 allows
`U1|U2|U4|U8`); PPBODY-as-ASCII was rejected; S1F23/F24 wasn't
implemented; S10F3 (host→equipment Terminal Display Single) wasn't
wired; one HSMS framing edge case caught by the tshark dissector;
several SML edge cases caught by libFuzzer.
---
## 8. What "100% GEM-compliant" honestly means here
Every GEM Fundamental and every GEM Additional capability that the E30
specification defines with a concrete SECS-II message set is implemented,
round-trip-tested, demonstrated in the two-container demo, AND
cross-validated against secsgem-py 0.3.0 on the overlap. Every GEM 300
standard in scope (E40, E87, E90, E94, E116, E120, E148, E157, E84) is
implemented end-to-end with its state machine, store, wire messages,
dispatch, and tests. Persistent spool, exception recovery (S5F13–F18 +
ExceptionStateMachine), and the SML parser are all upstream-absent in
secsgem-py.
What this codebase does **not** demonstrate, and what a real
"GEM-compliant" marketing claim would still need:
1. **Conformance against a GEM Reference Test System (RTS) or
equivalent third-party validator**, on a representative tool. The
codebase provides the message catalog + the runtime; running an
external validator against a real physical or simulated tool is
how compliance gets *certified*.
2. **Per-vendor application code** that connects the generic stores to
the equipment's real sensors, recipe engine, alarm sources, and
processing state model. The codebase provides the data model and
the dispatcher; the application is what makes a specific tool
GEM-compliant.
In short: this is a **GEM-conformant runtime stack with the full GEM
300 suite**, not a GEM-conformant *tool*. Pointing the runtime at a
real piece of equipment, populating the YAML files with the tool's
real SVIDs / ECIDs / alarms / capabilities / job behaviour, and wiring
the application callbacks completes the picture.