raphael
8686654b15
clients/python: pip-installable "secsgem-client", pure Python (stubs
pre-generated from equipment.proto, import made package-relative; no
compiled extension, no SEMI knowledge, no C++ toolchain). The API the whole
effort aimed at:
eq = Equipment("localhost:50051")
eq.set(ChamberPressure=2.5); eq["WaferCounter"] = 7
eq.fire("ProcessStarted", ChamberPressure=2.75)
eq.alarm("chiller_temp_high"); eq.clear("chiller_temp_high")
@eq.on("START")
def start(cmd): ... # auto-CompleteCommand after return
eq.listen(background=True)
eq.control_state; eq.request_control_state("HOST_OFFLINE"); eq.health()
Errors raise SecsGemError carrying the daemon's message ("no variable named
..."). bool checked before int in conversion (isinstance(True, int)).
examples/mini_tool.py is a complete GEM tool in ~25 lines.
PROOF — interop/pyclient_interop.py drives the PUBLISHED package (not raw
stubs) against a live secs_gemd with secsgem-py as the fab host: 13 checks
all green on first run — set/get round-trips, item syntax, SecsGemError on
unknown names, control state, health, fire->S6F11 on the host's wire,
alarm/clear->S5F1 with correct set bit, the full command loop (host S2F41 ->
HCACK=4 -> @eq.on handler -> completion event back at the host), operator
offline. Conversion layer unit-tested standalone; both wired into
tools/run_interop.sh as the pyclient step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
215 lines
14 KiB
Markdown
215 lines
14 KiB
Markdown
# Vendor Daemon & gRPC API — Status, Known Issues, and Plan to Fab-Readiness
|
|
|
|
> **This is a forward-looking roadmap, not a description of shipped behaviour.**
|
|
> Every item carries a status marker. Do not read an item as "done" unless it
|
|
> says ✅. (Last full audit: 2026-06-10.)
|
|
>
|
|
> Status legend: ✅ done · 🚧 in progress · ⬜ planned · ⚠️ risk/unknown
|
|
|
|
## What this is
|
|
|
|
A vendor-facing **daemon** (`secs_gemd`) that runs the SECS/GEM engine as its
|
|
own process and exposes a small, name-based, language-agnostic API over gRPC,
|
|
so a tool's control software (in any language) can drive the equipment without
|
|
linking C++ or knowing SEMI. See `proto/secsgem/v1/equipment.proto`.
|
|
|
|
The point of the daemon model: it owns the durable HSMS relationship with the
|
|
host and stays conformant while the tool software restarts/upgrades/crashes.
|
|
|
|
## Current status (2026-06-10, end of day)
|
|
|
|
| Piece | Status | Notes |
|
|
|---|---|---|
|
|
| `proto/secsgem/v1/equipment.proto` | ✅ | v1 surface designed: universal + carrier/recipe/job tiers, `Subscribe` stream, health |
|
|
| `HostCommandRegistry::set_handler` behaviour hook | ✅ | the engine seam for command behaviour; tested |
|
|
| `EquipmentRuntime` (engine owner) | ✅ | tested (`test_runtime.cpp`); `secs_server` runs entirely on it (live GEM300 demo passes) |
|
|
| `register_default_handlers` (the 56 GEM handlers as a library fn) | ✅ | `src/gem/default_handlers.cpp`; tested (`test_default_handlers.cpp`) |
|
|
| gRPC/protobuf toolchain (Dockerfile + CMake codegen) | ✅ | grpc++ 1.51 / protoc 3.21; opt-in `SECSGEM_DAEMON`, graceful skip without grpc |
|
|
| `secs_gemd`: `SetVariables` / `FireEvent` / `GetControlState` / `GetVariables` | ✅ | **format-aware** both directions (declared SECS-II formats on write, `from_item` on read) and thread-safe (snapshot maps + posted writes + `read_sync` reads). In-process gRPC tests incl. run_async production mode (`test_daemon_service.cpp`, 61 assertions) |
|
|
| Daemon interop vs **secsgem-py** reference host | ✅ | `interop/daemon_interop.py` (via `gemd` compose service): gRPC `SetVariables(ChamberPressure=2.5)` + `FireEvent` → host receives `S6F11 CEID 300` carrying `<F4 2.5>` — value *and declared format* flow gRPC→engine→HSMS→host |
|
|
| Daemon interop vs **secs4j** (Java) | ⬜ | mirror the secsgem-py harness against `interop/secs4j` |
|
|
| `Subscribe` host→tool command stream + `CompleteCommand` | ✅ | HCACK-4 contract implemented + tested in-process AND live vs secsgem-py (full loop: S2F41 → stream → complete → S6F11) |
|
|
| Universal RPC surface complete (vars/events/alarms/control-state/health) | ✅ | Phase A done; daemon tests 101 assertions, interop 15 checks |
|
|
| Python client package (the "beautiful API") | ✅ | `clients/python` (`secsgem-client`); 13-check interop green via the published API |
|
|
|
|
## Known issues (found in the 2026-06-10 audit; honest list)
|
|
|
|
- ✅ ~~**`GetControlState` cross-thread read.**~~ Fixed 2026-06-10: the runtime
|
|
keeps an atomic control-state mirror updated via an `add_state_change_handler`
|
|
observer (`HandlerSlot` primary+observers pattern), so the mirror survives
|
|
`register_default_handlers` claiming the primary slot. `control_state()` is
|
|
now safe from any thread.
|
|
- ✅ ~~**Alarms have no name key.**~~ Optional `name:` added to the alarm
|
|
config (loader + validator + shipped equipment.yaml); daemon RPCs accept
|
|
the name or the stringified ALID.
|
|
- ⬜ **`pvd_tool` predates the behaviour hook AND the runtime.** It still
|
|
hard-codes START behaviour in a router handler and hand-wires its own
|
|
main(). Migrate it to EquipmentRuntime + per-capability registration +
|
|
`commands.set_handler` so the flagship example showcases the intended
|
|
integration shape. (Phase C item 9.)
|
|
- ✅ ~~**Interop harnesses are manual.**~~ `tools/run_interop.sh` runs all nine
|
|
validation steps with one command (verified green); CI lanes added, pending
|
|
first-push verification (Phase 0 item 2).
|
|
- ✅ ~~**TSan lane doesn't cover the daemon.**~~ Covered locally + in CI
|
|
with `tools/tsan.supp` (third-party-only suppressions). Caught + fixed a
|
|
real test-side contract violation on its first run.
|
|
- ⚠️ **macOS bind-mount staleness can break Docker builds mid-edit** (a build
|
|
reading a half-synced source file). Not a product bug; re-run the build.
|
|
|
|
## The `Subscribe` design (settled — implement to this)
|
|
|
|
`S2F42` is an *acknowledgement*, not a completion: SEMI separates "I accept
|
|
your command" from "the work finished". The conformant, non-blocking flow:
|
|
|
|
1. Host sends `S2F41 START`. The engine's `on_command` handler (registered by
|
|
the daemon) runs on the io thread.
|
|
2. If no tool client is subscribed → fall back to the YAML declarative ack.
|
|
If a tool is subscribed → push the command onto its `Subscribe` stream and
|
|
**return `HCACK=4` (AcceptedWillFinishLater) immediately** — never block
|
|
the io thread or the T3 window on the tool.
|
|
3. The tool does the work and reports the outcome via `FireEvent` (success
|
|
event) / `SetAlarm` (failure) — exactly how secsgem-py applications and
|
|
commercial gateways do it.
|
|
4. `CompleteCommand` therefore only correlates/audits the command lifecycle in
|
|
v1. A *synchronous gating* mode (tool decides HCACK 0/2 before the S2F42
|
|
goes out) requires a deferred-reply mechanism in the engine — explicitly a
|
|
v2 refinement, not needed for conformance.
|
|
|
|
Sub-decisions (settled 2026-06-10, implemented + tested):
|
|
- v1 is a firehose: every subscriber receives every host request.
|
|
- NO buffering: with no subscriber a command takes its declarative YAML ack
|
|
and is not replayed on reconnect — never "will finish later" for work no
|
|
tool will do. Documented in the proto's Subscribe contract.
|
|
|
|
## Plan — ordered next steps
|
|
|
|
### Phase 0 — structural debts (from the 2026-06-10 design review; pay before sprinting)
|
|
The review's verdict: architecture and API bets are sound, but two structural
|
|
debts tax every later phase, and the most valuable tests aren't automated.
|
|
|
|
1. 🚧 **Multi-observer callbacks** (THE structural blocker — hit twice already).
|
|
`HandlerSlot` (primary slot keeps legacy set_ semantics; append-only add_
|
|
observers survive it) — done for `ControlStateMachine` + PJ/CJ stores, plus
|
|
runtime atomic control-state mirror (race retired) and `add_link_observer`
|
|
(WatchHealth foundation). ⬜ Remaining: roll the same 3-line pattern onto the
|
|
other single-slot classes (comm-state, EPT, exceptions, substrates, modules,
|
|
carriers, E84) as each phase needs them — mechanical now that the type exists.
|
|
2. 🚧 **CI the interop + conformance harnesses.** `tools/run_interop.sh` ✅ —
|
|
one command runs ALL nine validation steps (build, unit, daemon-unit,
|
|
py-host 31 checks, conformance 47, daemon bridge, spool restart, tshark,
|
|
secs4j 55) with a PASS/FAIL summary; verified green end-to-end 2026-06-10.
|
|
CI ✅ added but UNVERIFIED until pushed: grpc deps + `secs_gemd_tests` in
|
|
the build job (fails loudly if the daemon silently drops out), and a new
|
|
`python-interop` lane (py-host + conformance + daemon harness against
|
|
localhost, no docker-in-docker). ⬜ Verify the lanes on the first push.
|
|
3. ✅ **Fix `CompleteCommand` proto comment** — it described the rejected
|
|
blocking model; now states the HCACK-4 contract.
|
|
4. ✅ **Table-driven handler conformance test** — one ordered scenario
|
|
drives 53 of the 56 handlers through `router.dispatch` (236 assertions).
|
|
Golden frames: S1F13, S5F1, and a composed S6F11, all hand-computed from
|
|
E5 rules (external pins, not codec-derived).
|
|
5. ✅ **Decomposed `register_default_handlers` into 15 per-capability
|
|
functions** (identification, ECs, clock, event reports, remote commands,
|
|
trace/limits, spooling, alarms, exceptions, material tracking, carriers,
|
|
recipes, object services, jobs, terminal) — vendors register only what
|
|
their equipment is; `register_default_handlers` = all 15. Magic constants
|
|
replaced by YAML **role bindings** (`roles:` block — control_state_svid,
|
|
clock_svid, cj_executing_ceid, cj_completed_ceid) parsed into the
|
|
descriptor with historical defaults, validated (CEID roles must be
|
|
declared). Tested: subset registration, role-driven SVID refresh, roles
|
|
loader (present/custom/absent); full battery green (473/3087 core incl.
|
|
the 53-handler sweep, live GEM300 demo, 20-check daemon interop).
|
|
6. ✅ **Standardize the mutable-read pattern** — `EquipmentRuntime::read_sync`
|
|
(post-to-io + future with deadline; nullopt => UNAVAILABLE at the RPC edge).
|
|
Precedent set by `GetVariables`; every future mutable read copies it.
|
|
7. ✅ `equipment_service.hpp` moved to `include/secsgem/daemon/` (apps/
|
|
include-path hack removed). TSan daemon lane added locally + in CI
|
|
(`tools/tsan.supp` suppresses UNinstrumented system libgrpc/libabsl
|
|
internals only — our frames stay checked). The lane caught a real
|
|
contract violation on its first run (a test reading the model from the
|
|
test thread under run_async — fixed to read_sync); now TSan-clean with
|
|
halt_on_error=1.
|
|
8. ✅ Identifier-safe name validation: `ConfigValidator` warns (not errors)
|
|
on non-identifier variable/event/alarm/command names — bindings expose
|
|
names as kwargs/attributes. Format-compliance property test ✅; unset-
|
|
`Value` guard ✅.
|
|
|
|
### Phase A — finish the universal daemon surface (small, unblock vendors)
|
|
1. ✅ `GetVariables` — `from_item` reverse conversion (scalar for 1-element
|
|
arrays, List otherwise; C2-as-text and U8>2^63 noted as TODOs) + reads via
|
|
`read_sync`. Tested under **run_async (production threading)** — write
|
|
through the API, read back through the API — plus empty-query-returns-all,
|
|
INVALID_ARGUMENT on unknown names, and a live round-trip check in
|
|
`daemon_interop.py`.
|
|
2. ✅ Alarm `name:` config field (optional local key; `name` appended LAST on
|
|
the Alarm struct so existing brace-inits compile unchanged) + `SetAlarm`/
|
|
`ClearAlarm` RPCs (addressable by config name AND stringified ALID).
|
|
Validated end-to-end: gRPC `SetAlarm(chiller_temp_high)` -> secsgem-py host
|
|
receives `S5F1 ALCD=0x84 ALID=1`.
|
|
3. ✅ `RequestControlState` — fires operator events on the io thread and
|
|
reports what the E30 table actually did (ACCEPT iff landed in the requested
|
|
state; the shipped table has NO operator path to EquipmentOffline and the
|
|
test pins that honesty). ✅ `WatchHealth` — initial snapshot + push on
|
|
link/control-state change (+ spool depth sampled at 500ms); unit-tested
|
|
incl. the change push; link state still SELECTED/DISCONNECTED only
|
|
(CONNECTED reserved, TODO in code). Interop covers RequestControlState;
|
|
WatchHealth external check rides with Phase B.
|
|
4. ✅ Done per-item above (daemon suite at 101 assertions; interop at 15 checks).
|
|
|
|
### Phase B — the command stream (the big one)
|
|
5. ✅ `Subscribe`/`CompleteCommand` implemented per the HCACK-4 design.
|
|
Reconnect decision settled and documented in the proto: **no buffering** —
|
|
a command with no subscriber takes its declarative YAML ack (the honest
|
|
pre-daemon behaviour) and is not replayed. Firehose fan-out; per-command
|
|
forwarding handlers registered from the registry (new `names()`/`spec()`
|
|
accessors); pending-id audit map. In-process tests drive a REAL S2F41
|
|
through the default-handler router on the io thread: HCACK 4 with a
|
|
subscriber (params arrive on the stream), declarative Accept without,
|
|
CompleteCommand known/unknown ids, fallback restored after unsubscribe.
|
|
6. ✅ The full conformant loop runs against secsgem-py live: host `S2F41
|
|
START` → `S2F42 HCACK=4` → tool receives Command(name=START, id) on the
|
|
stream → `CompleteCommand` → tool fires the event → host receives `S6F11`.
|
|
(interop now 20 checks.)
|
|
7. ⬜ Java interop: `secs4j` host variant of the same scenario.
|
|
|
|
### Phase C — the beautiful Python client
|
|
8. ✅ `clients/python/` — pip-installable `secsgem-client`, pure Python,
|
|
stubs pre-generated (relative-import fixed). The full agreed API:
|
|
`eq.set(ChamberPressure=2.5)` kwargs + `eq["..."]` item syntax, `eq.get`,
|
|
`eq.fire(event, **data)`, `eq.alarm`/`eq.clear`, `eq.control_state`,
|
|
`eq.request_control_state`, `eq.health()`/`watch_health()`, and
|
|
`@eq.on("START")` + `eq.listen(background=...)` with auto-CompleteCommand.
|
|
Errors raise `SecsGemError` carrying the daemon's explanation.
|
|
PROOF: `interop/pyclient_interop.py` drives the PUBLISHED package against
|
|
a live daemon with secsgem-py as the host — 13 checks all green (S6F11/
|
|
S5F1 set+clear on the wire, HCACK-4 command loop through the decorator,
|
|
operator offline). Conversion layer unit-tested (bool-before-int etc).
|
|
Wired into tools/run_interop.sh as the `pyclient` step.
|
|
9. 🚧 `clients/python/examples/mini_tool.py` — a complete GEM tool in ~25
|
|
lines ✅. Migrating the C++ `pvd_tool` to EquipmentRuntime + capability
|
|
registration + `set_handler` ⬜.
|
|
|
|
### Phase D — GEM300 in-the-loop (process/carrier tools)
|
|
10. ⬜ Settle job/carrier semantics (who acks S16F5/S3F17, gate vs observe —
|
|
see proto comments), then wire `ProcessJob`/`CarrierAction` onto the
|
|
stream + `ReportProcessJob`/`ReportCarrier` into the PJ/CJ/carrier stores.
|
|
11. ⬜ Recipe download (`ProcessProgram` on the stream when S7F3 lands) and
|
|
EC-change notification (`ConstantChange` when S2F15 lands).
|
|
12. ⬜ Interop scenarios for jobs/carriers vs secsgem-py + secs4j.
|
|
|
|
### Phase E — hardening & operations
|
|
13. ⬜ gRPC exposure: default to localhost + document UDS; optional TLS creds.
|
|
14. ⬜ `tools/run_interop.sh` + CI lanes: all interop harnesses + TSan daemon lane.
|
|
15. ⬜ Daemon Prometheus metrics + supervised deployment recipe (systemd unit).
|
|
16. ⬜ Remaining Layer-1 API: traces, limits, substrates/modules, terminal
|
|
services, spool depth/flush, `Describe` RPC.
|
|
|
|
### Phase F — fab acceptance (parallel track; the hard gate)
|
|
- ⚠️ **Standards correctness remains unverified against SEMI texts** (behaviour
|
|
reconstructed without the standards; interop with secsgem-py/secs4j/Wireshark
|
|
mitigates but does not prove). The #1 fab-readiness risk; needs real
|
|
standards access and/or a fab's MES qualification run (`docs/MES_INTEROP.md`).
|
|
- ⬜ GEM compliance statement + manual matching the tool's data dictionary.
|
|
- ⬜ SECS-I serial driver (asio `serial_port` adapter; FSM done) — only if a
|
|
target tool uses RS-232.
|