raphael
4f3031aeb9
C9 — the flagship vendor example now demonstrates the intended integration
shape. examples/pvd_tool/main.cpp: 1093 -> 570 lines. The 466-line
hand-registered handler section and the hand-wired Server/Router/emit
plumbing are gone, replaced by EquipmentRuntime + register_default_handlers
(the example now serves all 56 handlers, up from its hand-picked 51) +
commands.set_handler for the START-runs-the-recipe behaviour (was a
hard-coded S2F41 router override). All domain logic — sensor simulator,
recipe runner, alarm threshold monitor, EPT cycler, Prometheus gauges —
unchanged. pvd's SVIDs 1/2 and CEIDs 400/401 match the roles: defaults, so
the built-ins bind with no config change. Verified: builds clean, boots
("registered 56 handlers", config loaded, EPT cycling), HSMS :5000 accepts,
metrics :9090 answers HTTP 200. logfn flushes per line so docker/CI logs
are visible immediately.
Writing project — new tutorial chapter docs/42_vendor_daemon_and_clients.md:
why a daemon (the host-timer argument), the proto contract and the HCACK-4
command semantics, the Python client walkthrough, EquipmentRuntime +
capability registration + roles:, the threading contract (posting API /
read_sync / hooks-on-io-thread) and primary-vs-observer slots, and a
which-tier-do-I-pick table. Indexed in 00_index Part 4. Refreshed the three
spots that still described pvd_tool's old "51 handlers in ~460 lines" shape
(ch35, ch41, pvd README) — drift killed in the same commit that made it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
202 lines
8.9 KiB
Markdown
202 lines
8.9 KiB
Markdown
# 42 — The vendor daemon and language clients
|
||
|
||
Chapters 30–41 teach the embedded C++ path: your `main()` owns the engine.
|
||
This chapter teaches the **daemon path** — the engine as its own process,
|
||
your tool software in any language on the other side of a socket — and the
|
||
runtime/capability API both paths share. If you are integrating a tool and
|
||
your controller is not C++, start here.
|
||
|
||
Everything in this chapter is exercised by `tools/run_interop.sh` (the
|
||
`daemon`, `pyclient`, and `daemon-unit` steps) against the secsgem-py
|
||
reference host. Status and remaining work: [DAEMON_ROADMAP.md](DAEMON_ROADMAP.md).
|
||
|
||
---
|
||
|
||
## 1. Why a daemon at all
|
||
|
||
A fab host enforces timers: T3 reply timeouts, T6 control transactions,
|
||
linktest heartbeats. If the GEM stack lives inside your tool application,
|
||
every crash, upgrade, GC pause, or hung hardware call of that application
|
||
is a **communication failure the fab can see** — and in production that
|
||
can mean the tool gets taken offline and lots get held.
|
||
|
||
`secs_gemd` inverts the deployment: the daemon owns the durable HSMS
|
||
relationship and answers the host from its own process, around the clock.
|
||
Your tool software connects over gRPC, pushes values and events in, and
|
||
receives host commands out. It can restart any time; the host never
|
||
notices. Spooling (chapter 13 §spool) covers the gap if the *host* link
|
||
drops; the daemon model covers the gap if *your software* drops.
|
||
|
||
```
|
||
tool software (any language) secs_gemd fab host / MES
|
||
┌──────────────────────────┐ gRPC ┌──────────────────────────┐ HSMS ┌────────┐
|
||
│ set / fire / alarm │◄─────►│ EquipmentRuntime │◄────►│ MES │
|
||
│ @on("START") handlers │ :50051│ + register_default_* │SECS-II└────────┘
|
||
│ (restartable, crashable) │ │ + spool, timers, FSMs │
|
||
└──────────────────────────┘ └──────────────────────────┘
|
||
```
|
||
|
||
Run it:
|
||
|
||
```sh
|
||
build/secs_gemd --port 5000 --grpc 0.0.0.0:50051 --config-dir data
|
||
```
|
||
|
||
One process, two faces: passive HSMS equipment on `--port`, the gRPC tool
|
||
API on `--grpc`.
|
||
|
||
---
|
||
|
||
## 2. The API contract (`proto/secsgem/v1/equipment.proto`)
|
||
|
||
The proto is the source of truth; read it — it is heavily commented. The
|
||
shape in one breath: everything is **name-based** (the names from your
|
||
`equipment.yaml`; never numeric SVIDs/CEIDs/ALIDs) and **plain-typed**
|
||
(a `Value` oneof of text/integer/real/boolean/binary/list; the daemon
|
||
converts to each variable's declared SECS-II format, so an F4 variable
|
||
stays F4 on the wire no matter what you send).
|
||
|
||
| RPC | What it does |
|
||
|---|---|
|
||
| `SetVariables` / `GetVariables` | write/read variables by name |
|
||
| `FireEvent` | trigger a collection event; daemon assembles the configured report → S6F11 |
|
||
| `SetAlarm` / `ClearAlarm` | S5F1 set/clear, by alarm `name:` (or stringified ALID) |
|
||
| `GetControlState` / `RequestControlState` | read the E30 control state / operator transitions |
|
||
| `WatchHealth` | server stream: link state, control state, spool depth |
|
||
| `Subscribe` | server stream: everything the host asks of the tool |
|
||
| `CompleteCommand` | close a streamed command's audit entry |
|
||
|
||
### The HCACK-4 command contract
|
||
|
||
The one piece of SEMI behaviour you must understand: when the host sends a
|
||
remote command (S2F41 START), the daemon does **not** wait for your tool.
|
||
It answers the host immediately:
|
||
|
||
- **No tool subscribed** → the command's declarative ack from
|
||
`equipment.yaml` (exactly the pre-daemon behaviour; nothing buffered,
|
||
nothing replayed later).
|
||
- **Tool subscribed** → `HCACK=4` ("accepted, will finish later"), and the
|
||
command appears on your `Subscribe` stream with its parameters.
|
||
|
||
The S2F42 transaction is already closed by the time you see the command.
|
||
The host learns the *real* outcome the way E30 intends: from the
|
||
**collection event you fire on success** (or the alarm you raise on
|
||
failure). `CompleteCommand` only feeds the daemon's audit log. This is the
|
||
same pattern secsgem-py applications and commercial GEM gateways use — the
|
||
protocol was designed for it.
|
||
|
||
---
|
||
|
||
## 3. The Python client (`clients/python`)
|
||
|
||
`pip install` the package (pure Python — pre-generated stubs, no compiled
|
||
extension) and the entire integration is:
|
||
|
||
```python
|
||
from secsgem_client import Equipment
|
||
|
||
eq = Equipment("localhost:50051")
|
||
|
||
eq.set(ChamberPressure=2.5) # host sees it on its next S1F3
|
||
eq["WaferCounter"] = 7 # item syntax, same thing
|
||
print(eq.get("ChamberPressure")) # read back through the daemon
|
||
|
||
eq.fire("ProcessStarted", ChamberPressure=2.75) # values, then S6F11
|
||
eq.alarm("chiller_temp_high") # S5F1 set
|
||
eq.clear("chiller_temp_high") # S5F1 clear
|
||
|
||
@eq.on("START") # host S2F41 -> your function
|
||
def start(cmd):
|
||
run_recipe(cmd.params.get("PPID"))
|
||
eq.fire("ProcessStarted") # the host's completion signal
|
||
|
||
eq.listen(background=True) # consume the Subscribe stream
|
||
|
||
eq.control_state # "ONLINE_REMOTE"
|
||
eq.request_control_state("HOST_OFFLINE") # operator panel -> maintenance
|
||
eq.health() # link / control state / spool depth
|
||
```
|
||
|
||
Anything the daemon declines raises `SecsGemError` with its explanation
|
||
(`no variable named 'ChamberPresure'`). A complete runnable tool is
|
||
[clients/python/examples/mini_tool.py](../clients/python/examples/mini_tool.py)
|
||
(~25 lines). The package is validated end-to-end by
|
||
`interop/pyclient_interop.py`: 13 checks driving the published API while
|
||
secsgem-py judges the wire.
|
||
|
||
Other languages: generate stubs from the proto (`protoc` supports 11+
|
||
languages) and wrap them the same way — the Python client is ~200 lines
|
||
and is the reference for what a thin wrapper should feel like.
|
||
|
||
---
|
||
|
||
## 4. The shared core: `EquipmentRuntime` + capability registration
|
||
|
||
Both `secs_server` and `secs_gemd` (and any future surface) are thin
|
||
fronts over the same two calls:
|
||
|
||
```cpp
|
||
#include "secsgem/gem/runtime.hpp"
|
||
#include "secsgem/gem/default_handlers.hpp"
|
||
|
||
gem::EquipmentRuntime::Config cfg;
|
||
cfg.equipment_yaml = "data/equipment.yaml";
|
||
cfg.control_state_yaml = "data/control_state.yaml";
|
||
cfg.process_job_yaml = "data/process_job_state.yaml";
|
||
cfg.control_job_yaml = "data/control_job_state.yaml";
|
||
cfg.port = 5000;
|
||
cfg.log = [](const std::string& m) { std::cout << m << std::endl; };
|
||
|
||
gem::EquipmentRuntime R(cfg);
|
||
gem::register_default_handlers(R); // all 56 GEM handlers + emitters
|
||
R.run(); // accept + io_context (blocks)
|
||
```
|
||
|
||
`register_default_handlers` is the composition of **15 per-capability
|
||
functions** (`register_identification`, `register_alarms`,
|
||
`register_carriers`, `register_jobs`, …) mirroring how E30 itself lists
|
||
capabilities (S1F19). A sensor-class tool with no carriers or jobs
|
||
registers only what it is — the unregistered messages get the Router's
|
||
SxF0/S9 default treatment, which is exactly what "I don't implement that
|
||
capability" should look like on the wire.
|
||
|
||
The ids the built-ins touch (the control-state/clock SVIDs the engine
|
||
refreshes, the CEIDs fired on CJ state changes) come from the `roles:`
|
||
block in `equipment.yaml` — visible coupling, no magic constants.
|
||
|
||
### The threading contract (the one rule)
|
||
|
||
One io_context thread owns the model. From any other thread:
|
||
|
||
- **writes** go through the runtime's posting API
|
||
(`set_variable`, `emit_event`, `set_alarm`, `clear_alarm`);
|
||
- **reads of mutable state** go through `read_sync` (post + future with a
|
||
deadline) — `control_state()` alone is lock-free (atomic mirror);
|
||
- **behaviour hooks** (`commands.set_handler`, state-change observers) run
|
||
*on* the io thread: return promptly, post long work elsewhere.
|
||
|
||
This is TSan-enforced in CI, daemon included. The first violation ever
|
||
caught was in our own test — the lane works.
|
||
|
||
### Observers vs. the primary slot
|
||
|
||
State machines expose `set_state_change_handler` (the primary slot —
|
||
yours, replaceable) **and** `add_state_change_handler` (append-only
|
||
observers that survive the primary being set). The runtime and daemon use
|
||
observers for the control-state mirror, `WatchHealth`, and the command
|
||
stream, so they never fight your application over the slot.
|
||
|
||
---
|
||
|
||
## 5. Which tier do I pick?
|
||
|
||
| Your situation | Tier |
|
||
|---|---|
|
||
| New tool, Python anywhere in the controller, fastest start | Python client |
|
||
| Existing controller in C#/Java/Go/…; multi-process architecture; tool app must be restartable without the host noticing | gRPC daemon + thin client |
|
||
| In-process integration, custom transports, hard real-time adjacency | Embedded C++ (`EquipmentRuntime`, chapter 41) |
|
||
|
||
They compose: a C++ tool can still run the daemon for the HSMS face and
|
||
talk gRPC locally — that is precisely the "tool software + separate
|
||
SECS server" deployment many fabs already run.
|