raphael
8a55137e57
Interface cleanup so the report_* family matches the typo-safe ethos of eq.names instead of leaking raw protobuf errors on a misspelled value. - Milestone / ModuleState / JobState: importable str-enums (member == its wire name, so plain strings still work) — autocomplete + a typo-checked happy path. The clean rule: equipment-specific *names* live on eq.names; fixed protocol *value-sets* are enums. - _enum_value(): resolves an enum-or-string arg client-side and, on a bad value, raises ValueError with a close-match hint *before* the wire. Wired into report_job / report_substrate / report_module / request_control_state (all previously raised a raw protobuf ValueError). - Equipment is now a context manager (with Equipment(...) as eq: ...). - examples/wafer_tool.py: a cluster tool tracking one wafer through one module end-to-end (E90 + E157), showing the enums + context manager. - tests/test_enums.py: asserts the enums stay in lockstep with the proto and that the typo path is helpful. Wired into run_interop.sh (pyclient step). - Interop drives both the enum and string forms on the wire + the ValueError typo path. Docs (ch16/ch42) updated; names-vs-enums rule documented. All Python unit tests + 25 pyclient interop checks pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
298 lines
14 KiB
Markdown
298 lines
14 KiB
Markdown
# 42 — The vendor daemon and language clients
|
||
|
||
Chapters 30–41 teach the embedded C++ path: your `main()` owns the engine.
|
||
This chapter teaches the **daemon path** — the engine as its own process,
|
||
your tool software in any language on the other side of a socket — and the
|
||
runtime/capability API both paths share. If you are integrating a tool and
|
||
your controller is not C++, start here.
|
||
|
||
Everything in this chapter is exercised by `tools/run_interop.sh` (the
|
||
`daemon`, `pyclient`, and `daemon-unit` steps) against the secsgem-py
|
||
reference host. Status and remaining work: [DAEMON_ROADMAP.md](DAEMON_ROADMAP.md).
|
||
|
||
---
|
||
|
||
## 1. Why a daemon at all
|
||
|
||
A fab host enforces timers: T3 reply timeouts, T6 control transactions,
|
||
linktest heartbeats. If the GEM stack lives inside your tool application,
|
||
every crash, upgrade, GC pause, or hung hardware call of that application
|
||
is a **communication failure the fab can see** — and in production that
|
||
can mean the tool gets taken offline and lots get held.
|
||
|
||
`secs_gemd` inverts the deployment: the daemon owns the durable HSMS
|
||
relationship and answers the host from its own process, around the clock.
|
||
Your tool software connects over gRPC, pushes values and events in, and
|
||
receives host commands out. It can restart any time; the host never
|
||
notices. Spooling (chapter 13 §spool) covers the gap if the *host* link
|
||
drops; the daemon model covers the gap if *your software* drops.
|
||
|
||
```
|
||
tool software (any language) secs_gemd fab host / MES
|
||
┌──────────────────────────┐ gRPC ┌──────────────────────────┐ HSMS ┌────────┐
|
||
│ set / fire / alarm │◄─────►│ EquipmentRuntime │◄────►│ MES │
|
||
│ @command / @on handlers │ :50051│ + register_default_* │SECS-II└────────┘
|
||
│ (restartable, crashable) │ │ + spool, timers, FSMs │
|
||
└──────────────────────────┘ └──────────────────────────┘
|
||
```
|
||
|
||
Run it:
|
||
|
||
```sh
|
||
build/secs_gemd --port 5000 --config-dir data # gRPC on 127.0.0.1:50051
|
||
build/secs_gemd --grpc unix:///run/secs_gemd/api.sock … # production: no TCP at all
|
||
```
|
||
|
||
One process, two faces: passive HSMS equipment on `--port`, the gRPC tool
|
||
API on `--grpc` (localhost by default — see §5 before exposing anything).
|
||
|
||
---
|
||
|
||
## 2. The API contract (`proto/secsgem/v1/equipment.proto`)
|
||
|
||
The proto is the source of truth; read it — it is heavily commented. The
|
||
shape in one breath: everything is **name-based** (the names from your
|
||
`equipment.yaml`; never numeric SVIDs/CEIDs/ALIDs) and **plain-typed**
|
||
(a `Value` oneof of text/integer/real/boolean/binary/list; the daemon
|
||
converts to each variable's declared SECS-II format, so an F4 variable
|
||
stays F4 on the wire no matter what you send).
|
||
|
||
| RPC | What it does |
|
||
|---|---|
|
||
| `SetVariables` / `GetVariables` | write/read variables by name |
|
||
| `FireEvent` | trigger a collection event; daemon assembles the configured report → S6F11 |
|
||
| `SetAlarm` / `ClearAlarm` | S5F1 set/clear, by alarm `name:` (or stringified ALID) |
|
||
| `GetControlState` / `RequestControlState` | read the E30 control state / operator transitions |
|
||
| `ReportCarrier` | E87 carrier state transitions (WAITING / IN_ACCESS / COMPLETE) |
|
||
| `ReportSubstrate` | E90 wafer tracking (ARRIVED / AT_WORK / PROCESSING / PROCESSED / AT_DESTINATION) |
|
||
| `ReportModule` | E157 module tracking (NOT_EXECUTING / GENERAL_EXECUTING / STEP_EXECUTING / STEP_COMPLETED) |
|
||
| `WatchHealth` | server stream: link state, control state, spool depth |
|
||
| `Subscribe` | server stream: everything the host asks of the tool |
|
||
| `CompleteCommand` | close a streamed command's audit entry |
|
||
| `Describe` | all names this equipment exposes (variables, events, alarms, commands, constants) |
|
||
| `FlushSpool` | drain or discard spooled messages |
|
||
| `SendTerminalMessage` | S10F1 operator message to the host |
|
||
|
||
### The HCACK-4 command contract
|
||
|
||
The one piece of SEMI behaviour you must understand: when the host sends a
|
||
remote command (S2F41 START), the daemon does **not** wait for your tool.
|
||
It answers the host immediately:
|
||
|
||
- **No tool subscribed** → the command's declarative ack from
|
||
`equipment.yaml` (exactly the pre-daemon behaviour; nothing buffered,
|
||
nothing replayed later).
|
||
- **Tool subscribed** → `HCACK=4` ("accepted, will finish later"), and the
|
||
command appears on your `Subscribe` stream with its parameters.
|
||
|
||
The S2F42 transaction is already closed by the time you see the command.
|
||
The host learns the *real* outcome the way E30 intends: from the
|
||
**collection event you fire on success** (or the alarm you raise on
|
||
failure). `CompleteCommand` only feeds the daemon's audit log. This is the
|
||
same pattern secsgem-py applications and commercial GEM gateways use — the
|
||
protocol was designed for it.
|
||
|
||
---
|
||
|
||
## 3. The Python client (`clients/python`)
|
||
|
||
`pip install` the package (pure Python — pre-generated stubs, no compiled
|
||
extension) and the entire integration is:
|
||
|
||
```python
|
||
from secsgem_client import Equipment, Milestone, ModuleState
|
||
|
||
with Equipment("localhost:50051") as eq: # context manager closes the channel
|
||
|
||
eq.set(ChamberPressure=2.5) # host sees it on its next S1F3
|
||
eq["WaferCounter"] = 7 # item syntax, same thing
|
||
print(eq.get("ChamberPressure")) # read back through the daemon
|
||
|
||
# eq.names — autocomplete-able, typo-safe name lookup (fetched from Describe)
|
||
eq.fire(eq.names.event.ProcessStarted) # typo → AttributeError at the line it happened
|
||
eq.alarm(eq.names.alarm.chiller_temp_high)
|
||
eq.clear(eq.names.alarm.chiller_temp_high)
|
||
# Plain strings still work; names are a convenience, not a requirement.
|
||
eq.fire("ProcessStarted", ChamberPressure=2.75)
|
||
|
||
@eq.command # function name IS the command name;
|
||
def START(cmd): # validated against Describe at decoration time
|
||
run_recipe(cmd.params.get("PPID"))
|
||
eq.fire(eq.names.event.ProcessStarted)
|
||
|
||
# @eq.on("NAME") still works — use it when the name can't be a Python identifier
|
||
# or when you prefer explicit strings.
|
||
|
||
eq.listen(background=True) # consume the Subscribe stream
|
||
|
||
eq.control_state # "ONLINE_REMOTE"
|
||
eq.request_control_state("HOST_OFFLINE") # operator panel -> maintenance
|
||
eq.health() # link / control state / spool depth
|
||
|
||
# E90 / E157 material tracking. Milestone / ModuleState are importable
|
||
# enums (autocomplete + typo-checked); the equivalent plain strings work too.
|
||
eq.report_substrate("WFR-001", Milestone.ARRIVED, carrier_id="FOUP-7", slot=3)
|
||
eq.report_substrate("WFR-001", Milestone.AT_WORK)
|
||
eq.report_substrate("WFR-001", Milestone.PROCESSING)
|
||
eq.report_substrate("WFR-001", Milestone.PROCESSED)
|
||
eq.report_substrate("WFR-001", Milestone.AT_DESTINATION)
|
||
|
||
eq.report_module("CHAMBER-A", ModuleState.GENERAL_EXECUTING)
|
||
eq.report_module("CHAMBER-A", ModuleState.STEP_EXECUTING)
|
||
eq.report_module("CHAMBER-A", ModuleState.STEP_COMPLETED)
|
||
eq.report_module("CHAMBER-A", ModuleState.NOT_EXECUTING)
|
||
```
|
||
|
||
Two error channels, by design: a **bad value you control** (a misspelled
|
||
milestone, an unknown control state) raises a plain `ValueError`/`NameError`
|
||
*before* any round-trip, with a close-match hint; anything the **daemon**
|
||
declines (unknown variable name, illegal FSM transition) raises `SecsGemError`
|
||
with its explanation (`no variable named 'ChamberPresure'`). Runnable tools:
|
||
[mini_tool.py](../clients/python/examples/mini_tool.py) (~25-line quickstart)
|
||
and [wafer_tool.py](../clients/python/examples/wafer_tool.py) (E90/E157
|
||
material tracking). The package is validated end-to-end by
|
||
`interop/pyclient_interop.py` driving the published API while secsgem-py
|
||
judges the wire.
|
||
|
||
### `eq.names` — name autocomplete and typo safety
|
||
|
||
`eq.names` fetches `Describe` from the daemon once (lazy, cached), then
|
||
exposes five sub-namespaces:
|
||
|
||
```python
|
||
eq.names.event.ProcessStarted # → "ProcessStarted"
|
||
eq.names.alarm.chiller_temp_high # → "chiller_temp_high"
|
||
eq.names.command.START # → "START"
|
||
eq.names.var.ChamberPressure # → "ChamberPressure"
|
||
eq.names.constant.MaxPressure # → "MaxPressure"
|
||
|
||
# Typo → AttributeError with suggestions:
|
||
# AttributeError: no event named 'ProcessStated'. Did you mean ProcessStarted?
|
||
|
||
# Membership test — useful in @eq.on guards:
|
||
"START" in eq.names.command # → True
|
||
```
|
||
|
||
`dir(eq.names.event)` lists all event names — REPL and IDE autocomplete
|
||
work out of the box.
|
||
|
||
### Names vs. enums — one rule
|
||
|
||
There are two kinds of identifier in the API, split on whether *your tool* or
|
||
*the SEMI standard* owns the value:
|
||
|
||
- **Equipment-specific names** — events, alarms, commands, variables,
|
||
constants — come from *your* `equipment.yaml`, so they live on the instance
|
||
as `eq.names.*` (fetched from the live daemon).
|
||
- **Fixed protocol value-sets** — `Milestone`, `ModuleState`, `JobState` —
|
||
are defined by the standards, so they're importable enums
|
||
(`from secsgem_client import Milestone`). `Milestone.ARRIVED == "ARRIVED"`,
|
||
so an enum member and its plain string are interchangeable; the enum just
|
||
buys you autocomplete and a typo-checked happy path.
|
||
|
||
Either way a wrong value fails fast and helpfully — a `ValueError`/`NameError`
|
||
with a close-match suggestion, raised client-side before the wire.
|
||
|
||
Other languages: generate stubs from the proto (`protoc` supports 11+
|
||
languages) and wrap them the same way — the Python client is ~200 lines
|
||
and is the reference for what a thin wrapper should feel like.
|
||
|
||
---
|
||
|
||
## 4. The shared core: `EquipmentRuntime` + capability registration
|
||
|
||
Both `secs_server` and `secs_gemd` (and any future surface) are thin
|
||
fronts over the same two calls:
|
||
|
||
```cpp
|
||
#include "secsgem/gem/runtime.hpp"
|
||
#include "secsgem/gem/default_handlers.hpp"
|
||
|
||
gem::EquipmentRuntime::Config cfg;
|
||
cfg.equipment_yaml = "data/equipment.yaml";
|
||
cfg.control_state_yaml = "data/control_state.yaml";
|
||
cfg.process_job_yaml = "data/process_job_state.yaml";
|
||
cfg.control_job_yaml = "data/control_job_state.yaml";
|
||
cfg.port = 5000;
|
||
cfg.log = [](const std::string& m) { std::cout << m << std::endl; };
|
||
|
||
gem::EquipmentRuntime R(cfg);
|
||
gem::register_default_handlers(R); // all 56 GEM handlers + emitters
|
||
R.run(); // accept + io_context (blocks)
|
||
```
|
||
|
||
`register_default_handlers` is the composition of **15 per-capability
|
||
functions** (`register_identification`, `register_alarms`,
|
||
`register_carriers`, `register_jobs`, …) mirroring how E30 itself lists
|
||
capabilities (S1F19). A sensor-class tool with no carriers or jobs
|
||
registers only what it is — the unregistered messages get the Router's
|
||
SxF0/S9 default treatment, which is exactly what "I don't implement that
|
||
capability" should look like on the wire.
|
||
|
||
The ids the built-ins touch (the control-state/clock SVIDs the engine
|
||
refreshes, the CEIDs fired on CJ state changes) come from the `roles:`
|
||
block in `equipment.yaml` — visible coupling, no magic constants.
|
||
|
||
### The threading contract (the one rule)
|
||
|
||
One io_context thread owns the model. From any other thread:
|
||
|
||
- **writes** go through the runtime's posting API
|
||
(`set_variable`, `emit_event`, `set_alarm`, `clear_alarm`);
|
||
- **reads of mutable state** go through `read_sync` (post + future with a
|
||
deadline) — `control_state()` alone is lock-free (atomic mirror);
|
||
- **behaviour hooks** (`commands.set_handler`, state-change observers) run
|
||
*on* the io thread: return promptly, post long work elsewhere.
|
||
|
||
This is TSan-enforced in CI, daemon included. The first violation ever
|
||
caught was in our own test — the lane works.
|
||
|
||
### Observers vs. the primary slot
|
||
|
||
State machines expose `set_state_change_handler` (the primary slot —
|
||
yours, replaceable) **and** `add_state_change_handler` (append-only
|
||
observers that survive the primary being set). The runtime and daemon use
|
||
observers for the control-state mirror, `WatchHealth`, and the command
|
||
stream, so they never fight your application over the slot.
|
||
|
||
---
|
||
|
||
## 5. Running it in production
|
||
|
||
- **Exposure.** `--grpc` defaults to `127.0.0.1:50051`; the API is
|
||
unauthenticated by design (auth belongs to the transport), so it must
|
||
never face the equipment LAN. For same-host tool software use a Unix
|
||
domain socket — `--grpc unix:///run/secs_gemd/api.sock` — and there is
|
||
no network surface at all. The HSMS port faces the fab host; firewall
|
||
it to the host's address ([SECURITY.md](SECURITY.md) has the nftables
|
||
recipe).
|
||
- **Shutdown.** SIGTERM/SIGINT drain gracefully: open Subscribe/WatchHealth
|
||
streams are cancelled (2s deadline), the engine stops cleanly, the spool
|
||
journal is never cut mid-write, exit code 0. Safe under systemd and
|
||
`docker stop`.
|
||
- **Supervision.** [deploy/secs_gemd.service](../deploy/secs_gemd.service)
|
||
is a hardened systemd unit (DynamicUser, ProtectSystem, StateDirectory
|
||
for the spool, Restart=always). Pair with `--spool-dir` so host-bound
|
||
events survive daemon restarts too.
|
||
- **Metrics.** `--metrics 9091` serves Prometheus gauges:
|
||
`secsgem_link_selected`, `secsgem_control_state`, `secsgem_spool_depth`.
|
||
- **Sessions.** v1 runs one equipment identity per daemon (HSMS-SS). The
|
||
engine supports HSMS-GS multi-session, but the daemon doesn't surface it
|
||
yet — run one daemon per equipment identity.
|
||
- All of the above is enforced by `tools/check_daemon_ops.sh` (the
|
||
`daemon-ops` step of `tools/run_interop.sh`, also in CI).
|
||
|
||
---
|
||
|
||
## 6. Which tier do I pick?
|
||
|
||
| Your situation | Tier |
|
||
|---|---|
|
||
| New tool, Python anywhere in the controller, fastest start | Python client |
|
||
| Existing controller in C#/Java/Go/…; multi-process architecture; tool app must be restartable without the host noticing | gRPC daemon + thin client |
|
||
| In-process integration, custom transports, hard real-time adjacency | Embedded C++ (`EquipmentRuntime`, chapter 41) |
|
||
|
||
They compose: a C++ tool can still run the daemon for the HSMS face and
|
||
talk gRPC locally — that is precisely the "tool software + separate
|
||
SECS server" deployment many fabs already run.
|