From b067a76b80918a64361f97bb33db94fcbdb84575 Mon Sep 17 00:00:00 2001 From: Raphael Maenle Date: Wed, 10 Jun 2026 18:35:53 +0200 Subject: [PATCH] docs: rewrite daemon roadmap as ordered plan with known-issues audit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Status table brought current (format-aware daemon, secsgem-py interop), the stale Layer-0 section replaced, and the path to an excellent GEM300 repo laid out as ordered phases A–F: finish universal RPCs, the Subscribe command stream (HCACK-4 design written down as the implementation contract), the Python client package, GEM300 job/carrier in-the-loop, hardening/CI, and the fab-acceptance track. Known-issues section records what the audit found (GetControlState enum race + why the state-change-handler slot can't be reused, missing alarm name key, pvd_tool predating set_handler, manual interop harnesses, TSan gap). Co-Authored-By: Claude Fable 5 --- docs/DAEMON_ROADMAP.md | 186 +++++++++++++++++++++++++---------------- 1 file changed, 113 insertions(+), 73 deletions(-) diff --git a/docs/DAEMON_ROADMAP.md b/docs/DAEMON_ROADMAP.md index 3e410c7..7ae0df2 100644 --- a/docs/DAEMON_ROADMAP.md +++ b/docs/DAEMON_ROADMAP.md @@ -1,96 +1,136 @@ -# Vendor Daemon & gRPC API — Status and Roadmap to Fab-Readiness +# Vendor Daemon & gRPC API — Status, Known Issues, and Plan to Fab-Readiness > **This is a forward-looking roadmap, not a description of shipped behaviour.** > Every item carries a status marker. Do not read an item as "done" unless it -> says ✅. (Written 2026-06-10.) +> says ✅. (Last full audit: 2026-06-10.) > > Status legend: ✅ done · 🚧 in progress · ⬜ planned · ⚠️ risk/unknown ## What this is -A vendor-facing **daemon** that runs the SECS/GEM engine as its own process and -exposes a small, name-based, language-agnostic API over gRPC, so a tool's -control software (in any language) can drive the equipment without linking C++ -or knowing SEMI. See `proto/secsgem/v1/equipment.proto` for the API. +A vendor-facing **daemon** (`secs_gemd`) that runs the SECS/GEM engine as its +own process and exposes a small, name-based, language-agnostic API over gRPC, +so a tool's control software (in any language) can drive the equipment without +linking C++ or knowing SEMI. See `proto/secsgem/v1/equipment.proto`. The point of the daemon model: it owns the durable HSMS relationship with the host and stays conformant while the tool software restarts/upgrades/crashes. -## Current status (2026-06-10) +## Current status (2026-06-10, end of day) | Piece | Status | Notes | |---|---|---| -| `proto/secsgem/v1/equipment.proto` | ✅ | v1 API surface designed (universal + carrier/recipe/job tiers) | -| `HostCommandRegistry::set_handler` behaviour hook | ✅ | the engine seam the daemon binds to; tested | -| `EquipmentRuntime` (engine owner) | ✅ | infra + outbound API built & tested (`tests/test_runtime.cpp`); **`secs_server` now runs entirely on it** (verified by the live server↔client GEM300 demo — full job/spool/control-state flow, client exit 0). | -| `register_default_handlers` in the library (so the daemon reuses the 56 handlers) | ✅ | relocated into `src/gem/default_handlers.cpp` (programmatic move, zero retype); `secs_server` reduced to ~113 lines and calls it. Tested (`tests/test_default_handlers.cpp`: S1F1→S1F2, S2F41→on_command hook) + live GEM300 demo still passes. | -| gRPC/protobuf in toolchain (Dockerfile + CMake) | ✅ | image rebuilt (grpc++ 1.51, protoc 3.21); CMake proto codegen wired (opt-in `SECSGEM_DAEMON`, graceful skip without grpc). | -| `secs_gemd` daemon | 🚧 | **runs** — engine + HSMS on a background thread, serves gRPC. RPCs done + tested over a real in-process channel: `SetVariables` (name-resolved, value→Item, lands in the model), `FireEvent` (name→CEID), `GetControlState`. `tests/test_daemon_service.cpp` (15 assertions). **Still TODO:** `GetVariables` (needs Item→Value), `SetAlarm`/`ClearAlarm` (needs an alarm `name` config field), `RequestControlState`, `WatchHealth`, and the host→tool **`Subscribe` command stream + `CompleteCommand`** (the hard part). | -| `secs_gemd` daemon implementing the service | ⬜ | translate RPCs ↔ runtime; stream host requests | -| Reference client library (Python) | ⬜ | thin wrapper over generated stubs | -| Daemon interop test vs secsgem-py | ✅ | `interop/daemon_interop.py` — a gRPC tool + a secsgem-py active host both drive a live `secs_gemd`; proves a gRPC `SetVariables`+`FireEvent` reaches the reference host as `S6F11` over HSMS. Run via the `gemd` compose service. **Java (secs4j) equivalent: ⬜.** | +| `proto/secsgem/v1/equipment.proto` | ✅ | v1 surface designed: universal + carrier/recipe/job tiers, `Subscribe` stream, health | +| `HostCommandRegistry::set_handler` behaviour hook | ✅ | the engine seam for command behaviour; tested | +| `EquipmentRuntime` (engine owner) | ✅ | tested (`test_runtime.cpp`); `secs_server` runs entirely on it (live GEM300 demo passes) | +| `register_default_handlers` (the 56 GEM handlers as a library fn) | ✅ | `src/gem/default_handlers.cpp`; tested (`test_default_handlers.cpp`) | +| gRPC/protobuf toolchain (Dockerfile + CMake codegen) | ✅ | grpc++ 1.51 / protoc 3.21; opt-in `SECSGEM_DAEMON`, graceful skip without grpc | +| `secs_gemd`: `SetVariables` / `FireEvent` / `GetControlState` | ✅ | **format-aware** (converts to each variable's declared SECS-II format) and thread-safe (name/format maps snapshotted at construction; all writes post to the io thread). In-process gRPC tests (`test_daemon_service.cpp`, 16 assertions) | +| Daemon interop vs **secsgem-py** reference host | ✅ | `interop/daemon_interop.py` (via `gemd` compose service): gRPC `SetVariables(ChamberPressure=2.5)` + `FireEvent` → host receives `S6F11 CEID 300` carrying `` — value *and declared format* flow gRPC→engine→HSMS→host | +| Daemon interop vs **secs4j** (Java) | ⬜ | mirror the secsgem-py harness against `interop/secs4j` | +| `Subscribe` host→tool command stream | ⬜ | design settled (HCACK-4, see below); not implemented | +| Remaining universal RPCs (`GetVariables`, alarms, `RequestControlState`, `WatchHealth`) | ⬜ | see plan | +| Python client package (the "beautiful API") | ⬜ | thin wrapper over generated stubs | -**Nothing in the proto is wired to the engine yet.** The engine itself is broad -(56 wire handlers across S1/2/3/5/6/7/10/14/16; all GEM300 stores) — the daemon -is about *exposing* that, not building it. +## Known issues (found in the 2026-06-10 audit; honest list) -## Gaps to fab-readiness +- 🚧 **`GetControlState` cross-thread read.** The gRPC handler reads the control + FSM's state enum while the io thread may transition it — a narrow data race + (single enum read; benign on every real ABI, but TSan-visible and sloppy). + Fix: an atomic state mirror in `EquipmentRuntime`. NOTE: cannot piggyback on + `ControlStateMachine::set_state_change_handler` — that is a single slot + already owned by `register_default_handlers`; either add multi-handler + support or update the mirror inside the runtime's own wiring. +- ⬜ **Alarms have no name key.** `equipment.yaml` alarms carry only numeric + `id` + freetext `text` (matches SEMI: ALID/ALTX; there is no standard short + name). The name-based `SetAlarm`/`ClearAlarm` RPCs need an optional local + `name:` field in the alarm config (fallback: stringified id). +- ⬜ **`pvd_tool` predates the behaviour hook.** It still hard-codes + `if (rcmd=="START") recipe->start(...)` in a router handler. Migrate it to + `commands.set_handler` so the flagship example showcases the intended seam. +- ⬜ **Interop harnesses are manual.** `daemon_interop.py` (and the older + host/server harnesses) run via ad-hoc compose invocations; there is no + `tools/run_interop.sh` or CI lane that runs them. Add one script + CI job. +- ⬜ **TSan lane doesn't cover the daemon.** `secs_gemd_tests` should also be + built/run under `-DSECSGEM_TSAN=ON` once the control-state mirror lands. +- ⚠️ **macOS bind-mount staleness can break Docker builds mid-edit** (a build + reading a half-synced source file). Not a product bug; re-run the build. -### Layer 0 — Make it run at all (blocks everything) -- ⬜ Extract `EquipmentRuntime` from `apps/secs_server.cpp` (io_context, Server, - model, router, emit lambdas, `set_handler`). Reduce `secs_server` to a thin - `main()` over it. Verify against the existing test suite. -- ⬜ Add gRPC/protobuf to the Dockerfile + CMake codegen for the proto. -- ⬜ Implement `secs_gemd`: construct the runtime, `io.run()` on a background - thread, map each RPC to a runtime call, route host requests onto the - `Subscribe` stream via `set_handler` + the FSM change handlers. -- ⬜ One reference client (Python) proving the end-to-end loop. +## The `Subscribe` design (settled — implement to this) -### Layer 1 — API completeness (engine supports these; surface them) -- ⬜ **Job/carrier in-the-loop semantics.** The proto has `ProcessJob`/ - `CarrierAction` + report RPCs, but the exact contract is unspecified: who acks - the host's S16F5/S3Fxx, whether the tool *gates* a job start or only observes, - and timing vs. T3. **Design this before implementing the daemon for process tools.** -- ⬜ Trace data collection (engine: `TraceStore`, S2F23/S6F1). -- ⬜ Limits monitoring (engine: `LimitMonitorStore`, S2F45). -- ⬜ Substrate/E90 + module/E157 tracking (engine: `SubstrateStore`, `ModuleStore`). -- ⬜ Terminal services / operator messages (engine: S10F1–F6) — host↔tool HMI text. -- ⬜ Spool depth + force-flush API (engine: `SpoolStore`). -- ⬜ `Describe` RPC: enumerate configured variables/events/alarms/commands at - runtime (diagnostics & tooling). +`S2F42` is an *acknowledgement*, not a completion: SEMI separates "I accept +your command" from "the work finished". The conformant, non-blocking flow: -### Layer 2 — Production hardening -- ⬜ **gRPC auth / exposure.** No auth today. Bind to a Unix domain socket or - localhost-only, or add credentials. Never expose the API on the equipment LAN - unauthenticated. -- ⬜ **`Subscribe` reconnect/replay semantics.** Define what happens to host - requests (commands, jobs) if the tool client disconnects and reconnects: are - they buffered/replayed, or dropped? Required for a 24/7 tool. (Correctness gap.) -- ⬜ Supervised deployment (systemd unit / container), auto-restart; rely on the - existing spool persistence so queued host events survive a daemon restart. -- ⬜ Expose the existing Prometheus metrics + structured logs from the daemon. -- ⬜ Decide multi-host (HSMS-GS) story — engine supports it; v1 assumes one - equipment/session. Probably fine; document the assumption. +1. Host sends `S2F41 START`. The engine's `on_command` handler (registered by + the daemon) runs on the io thread. +2. If no tool client is subscribed → fall back to the YAML declarative ack. + If a tool is subscribed → push the command onto its `Subscribe` stream and + **return `HCACK=4` (AcceptedWillFinishLater) immediately** — never block + the io thread or the T3 window on the tool. +3. The tool does the work and reports the outcome via `FireEvent` (success + event) / `SetAlarm` (failure) — exactly how secsgem-py applications and + commercial gateways do it. +4. `CompleteCommand` therefore only correlates/audits the command lifecycle in + v1. A *synchronous gating* mode (tool decides HCACK 0/2 before the S2F42 + goes out) requires a deferred-reply mechanism in the engine — explicitly a + v2 refinement, not needed for conformance. -### Layer 3 — Actual fab acceptance (the hard gate) -- ⚠️ **Standards correctness is unverified.** The SECS/GEM behaviour in this repo - was substantially reconstructed without access to the SEMI standard texts. - Interop tests (secsgem-py, secs4java8, Wireshark) mitigate but do not prove - conformance. Subtle wire/state-machine deviations could fail a real host. This - is the #1 fab-readiness risk and it is *verification*, not features. -- ⬜ Pass a specific fab's **MES qualification suite** against their real host - (see `docs/MES_INTEROP.md` for the punch-list). Fab acceptance is empirical - and per-fab. -- ⬜ Produce the GEM **compliance statement** (S1F19/F20) + written GEM manual - matching the tool's actual data dictionary. -- ⬜ Finish the **SECS-I serial driver** (FSM done; asio `serial_port` adapter - missing) — only if a target tool uses RS-232 rather than HSMS/TCP. -- ⬜ Per-tool `equipment.yaml` authored to match the tool's real SVIDs/CEIDs/ - ECIDs/alarms/recipes and the fab's spec (vendor work; the config validator helps). +Open sub-decisions to settle while implementing: +- Per-command routing (subscribe to specific RCMDs?) or one firehose? (v1: firehose.) +- Reconnect semantics: buffer commands while no subscriber (bounded queue + + declarative fallback after timeout) or reject with HCACK 2? Must be decided + and TESTED before calling the stream production-ready. -## Sequencing recommendation +## Plan — ordered next steps -Layer 0 in order (runtime → deps → daemon → client), then Layer 1's job/carrier -semantics, then Layer 2 hardening. Layer 3 runs in parallel and is gated by -access to real standards and a real host — treat it as the thing that decides -whether any of this is truly fab-ready. +### Phase A — finish the universal daemon surface (small, unblock vendors) +1. ⬜ `GetVariables` — needs the reverse `Item → proto Value` conversion + (read via post-to-io + future, or serve from a daemon-side cache of last + set values; decide and document). +2. ⬜ Alarm `name:` config field + `SetAlarm`/`ClearAlarm` RPCs + tests. +3. ⬜ `RequestControlState` (operator online/offline) + control-state atomic + mirror (fixes the known race) + `WatchHealth` stream (link state from the + selected/closed handlers, spool depth, control state). +4. ⬜ Extend `test_daemon_service.cpp` + `daemon_interop.py` for all of the above. + +### Phase B — the command stream (the big one) +5. ⬜ Implement `Subscribe`/`CompleteCommand` per the design above, including + the no-subscriber fallback and bounded buffering. In-process gRPC tests: + command arrives on stream; HCACK 4 on the wire; declarative fallback when + unsubscribed. +6. ⬜ Extend `daemon_interop.py`: secsgem-py host sends `S2F41 START` → gRPC + tool receives it on the stream → tool fires completion event → host sees + `S6F11`. (The full conformant loop against the reference implementation.) +7. ⬜ Java interop: `secs4j` host variant of the same scenario. + +### Phase C — the beautiful Python client +8. ⬜ `clients/python/` package (`pip install secsgem-client`): wraps generated + stubs in the agreed API — `eq.set(chamber_pressure=2.5)`, `eq.fire("wafer_complete", thickness=1.2)`, + `eq.alarm("pressure_high")`, `@eq.on("START")` consuming the stream, + `eq.health()`. Pure Python (no compiled ext). Ship stubs pre-generated. +9. ⬜ Example: rewrite a minimal `pvd_tool`-equivalent in ~40 lines of Python + against the daemon; also migrate the C++ `pvd_tool` to `set_handler`. + +### Phase D — GEM300 in-the-loop (process/carrier tools) +10. ⬜ Settle job/carrier semantics (who acks S16F5/S3F17, gate vs observe — + see proto comments), then wire `ProcessJob`/`CarrierAction` onto the + stream + `ReportProcessJob`/`ReportCarrier` into the PJ/CJ/carrier stores. +11. ⬜ Recipe download (`ProcessProgram` on the stream when S7F3 lands) and + EC-change notification (`ConstantChange` when S2F15 lands). +12. ⬜ Interop scenarios for jobs/carriers vs secsgem-py + secs4j. + +### Phase E — hardening & operations +13. ⬜ gRPC exposure: default to localhost + document UDS; optional TLS creds. +14. ⬜ `tools/run_interop.sh` + CI lanes: all interop harnesses + TSan daemon lane. +15. ⬜ Daemon Prometheus metrics + supervised deployment recipe (systemd unit). +16. ⬜ Remaining Layer-1 API: traces, limits, substrates/modules, terminal + services, spool depth/flush, `Describe` RPC. + +### Phase F — fab acceptance (parallel track; the hard gate) +- ⚠️ **Standards correctness remains unverified against SEMI texts** (behaviour + reconstructed without the standards; interop with secsgem-py/secs4j/Wireshark + mitigates but does not prove). The #1 fab-readiness risk; needs real + standards access and/or a fab's MES qualification run (`docs/MES_INTEROP.md`). +- ⬜ GEM compliance statement + manual matching the tool's data dictionary. +- ⬜ SECS-I serial driver (asio `serial_port` adapter; FSM done) — only if a + target tool uses RS-232.