secs-gem/docs/42_vendor_daemon_and_clients.md

# 42 — The vendor daemon and language clients

Chapters 30–41 teach the embedded C++ path: your `main()` owns the engine.
This chapter teaches the **daemon path** — the engine as its own process,
your tool software in any language on the other side of a socket — and the
runtime/capability API both paths share. If you are integrating a tool and
your controller is not C++, start here.

Everything in this chapter is exercised by `tools/run_interop.sh` (the
`daemon`, `pyclient`, and `daemon-unit` steps) against the secsgem-py
reference host. Status and remaining work: [DAEMON_ROADMAP.md](DAEMON_ROADMAP.md).

---

## 1. Why a daemon at all

A fab host enforces timers: T3 reply timeouts, T6 control transactions,
linktest heartbeats. If the GEM stack lives inside your tool application,
every crash, upgrade, GC pause, or hung hardware call of that application
is a **communication failure the fab can see** — and in production that
can mean the tool gets taken offline and lots get held.

`secs_gemd` inverts the deployment: the daemon owns the durable HSMS
relationship and answers the host from its own process, around the clock.
Your tool software connects over gRPC, pushes values and events in, and
receives host commands out. It can restart any time; the host never
notices. Spooling (chapter 13 §spool) covers the gap if the *host* link
drops; the daemon model covers the gap if *your software* drops.

```
 tool software (any language)        secs_gemd                    fab host / MES
 ┌──────────────────────────┐  gRPC ┌──────────────────────────┐  HSMS ┌────────┐
 │ set / fire / alarm       │◄─────►│ EquipmentRuntime          │◄────►│  MES   │
 │ @command / @on handlers  │ :50051│ + register_default_*      │SECS-II└────────┘
 │ (restartable, crashable) │       │ + spool, timers, FSMs     │
 └──────────────────────────┘       └──────────────────────────┘
```

Run it:

```sh
build/secs_gemd --port 5000 --config-dir data            # gRPC on 127.0.0.1:50051
build/secs_gemd --grpc unix:///run/secs_gemd/api.sock …  # production: no TCP at all
```

One process, two faces: passive HSMS equipment on `--port`, the gRPC tool
API on `--grpc` (localhost by default — see §5 before exposing anything).

---

## 2. The API contract (`proto/secsgem/v1/equipment.proto`)

The proto is the source of truth; read it — it is heavily commented. The
shape in one breath: everything is **name-based** (the names from your
`equipment.yaml`; never numeric SVIDs/CEIDs/ALIDs) and **plain-typed**
(a `Value` oneof of text/integer/real/boolean/binary/list; the daemon
converts to each variable's declared SECS-II format, so an F4 variable
stays F4 on the wire no matter what you send).

| RPC | What it does |
|---|---|
| `SetVariables` / `GetVariables` | write/read variables by name |
| `FireEvent` | trigger a collection event; daemon assembles the configured report → S6F11 |
| `SetAlarm` / `ClearAlarm` | S5F1 set/clear, by alarm `name:` (or stringified ALID) |
| `GetControlState` / `RequestControlState` | read the E30 control state / operator transitions |
| `ReportCarrier` | E87 carrier state transitions (WAITING / IN_ACCESS / COMPLETE) |
| `ReportSubstrate` | E90 wafer tracking (ARRIVED / AT_WORK / PROCESSING / PROCESSED / AT_DESTINATION) |
| `ReportModule` | E157 module tracking (NOT_EXECUTING / GENERAL_EXECUTING / STEP_EXECUTING / STEP_COMPLETED) |
| `WatchHealth` | server stream: link state, control state, spool depth |
| `Subscribe` | server stream: everything the host asks of the tool |
| `CompleteCommand` | close a streamed command's audit entry |
| `Describe` | all names this equipment exposes (variables, events, alarms, commands, constants) |
| `FlushSpool` | drain or discard spooled messages |
| `SendTerminalMessage` | S10F1 operator message to the host |

### The HCACK-4 command contract

The one piece of SEMI behaviour you must understand: when the host sends a
remote command (S2F41 START), the daemon does **not** wait for your tool.
It answers the host immediately:

- **No tool subscribed** → the command's declarative ack from
  `equipment.yaml` (exactly the pre-daemon behaviour; nothing buffered,
  nothing replayed later).
- **Tool subscribed** → `HCACK=4` ("accepted, will finish later"), and the
  command appears on your `Subscribe` stream with its parameters.

The S2F42 transaction is already closed by the time you see the command.
The host learns the *real* outcome the way E30 intends: from the
**collection event you fire on success** (or the alarm you raise on
failure). `CompleteCommand` only feeds the daemon's audit log. This is the
same pattern secsgem-py applications and commercial GEM gateways use — the
protocol was designed for it.

---

## 3. The Python client (`clients/python`)

`pip install` the package (pure Python — pre-generated stubs, no compiled
extension) and the entire integration is:

```python
from secsgem_client import Equipment

eq = Equipment("localhost:50051")

eq.set(ChamberPressure=2.5)            # host sees it on its next S1F3
eq["WaferCounter"] = 7                 # item syntax, same thing
print(eq.get("ChamberPressure"))       # read back through the daemon

# eq.names — autocomplete-able, typo-safe name lookup (fetched from Describe)
eq.fire(eq.names.event.ProcessStarted)    # typo → AttributeError at the line it happened
eq.alarm(eq.names.alarm.chiller_temp_high)
eq.clear(eq.names.alarm.chiller_temp_high)
# Plain strings still work; names are a convenience, not a requirement.
eq.fire("ProcessStarted", ChamberPressure=2.75)

@eq.command                            # function name IS the command name;
def START(cmd):                        # validated against Describe at decoration time
    run_recipe(cmd.params.get("PPID"))
    eq.fire(eq.names.event.ProcessStarted)

# @eq.on("NAME") still works — use it when the name can't be a Python identifier
# or when you prefer explicit strings.

eq.listen(background=True)             # consume the Subscribe stream

eq.control_state                       # "ONLINE_REMOTE"
eq.request_control_state("HOST_OFFLINE")   # operator panel -> maintenance
eq.health()                            # link / control state / spool depth

# E90 / E157 material tracking
eq.report_substrate("WFR-001", "ARRIVED", carrier_id="FOUP-7", slot=3)
eq.report_substrate("WFR-001", "AT_WORK")
eq.report_substrate("WFR-001", "PROCESSING")
eq.report_substrate("WFR-001", "PROCESSED")
eq.report_substrate("WFR-001", "AT_DESTINATION")

eq.report_module("CHAMBER-A", "GENERAL_EXECUTING")
eq.report_module("CHAMBER-A", "STEP_EXECUTING")
eq.report_module("CHAMBER-A", "STEP_COMPLETED")
eq.report_module("CHAMBER-A", "NOT_EXECUTING")
```

Anything the daemon declines raises `SecsGemError` with its explanation
(`no variable named 'ChamberPresure'`). A complete runnable tool is
[clients/python/examples/mini_tool.py](../clients/python/examples/mini_tool.py)
(~25 lines). The package is validated end-to-end by
`interop/pyclient_interop.py` driving the published API while secsgem-py
judges the wire.

### `eq.names` — name autocomplete and typo safety

`eq.names` fetches `Describe` from the daemon once (lazy, cached), then
exposes five sub-namespaces:

```python
eq.names.event.ProcessStarted     # → "ProcessStarted"
eq.names.alarm.chiller_temp_high  # → "chiller_temp_high"
eq.names.command.START            # → "START"
eq.names.var.ChamberPressure      # → "ChamberPressure"
eq.names.constant.MaxPressure     # → "MaxPressure"

# Typo → AttributeError with suggestions:
# AttributeError: no event named 'ProcessStated'. Did you mean ProcessStarted?

# Membership test — useful in @eq.on guards:
"START" in eq.names.command       # → True
```

`dir(eq.names.event)` lists all event names — REPL and IDE autocomplete
work out of the box.

Other languages: generate stubs from the proto (`protoc` supports 11+
languages) and wrap them the same way — the Python client is ~200 lines
and is the reference for what a thin wrapper should feel like.

---

## 4. The shared core: `EquipmentRuntime` + capability registration

Both `secs_server` and `secs_gemd` (and any future surface) are thin
fronts over the same two calls:

```cpp
#include "secsgem/gem/runtime.hpp"
#include "secsgem/gem/default_handlers.hpp"

gem::EquipmentRuntime::Config cfg;
cfg.equipment_yaml     = "data/equipment.yaml";
cfg.control_state_yaml = "data/control_state.yaml";
cfg.process_job_yaml   = "data/process_job_state.yaml";
cfg.control_job_yaml   = "data/control_job_state.yaml";
cfg.port = 5000;
cfg.log  = [](const std::string& m) { std::cout << m << std::endl; };

gem::EquipmentRuntime R(cfg);
gem::register_default_handlers(R);   // all 56 GEM handlers + emitters
R.run();                             // accept + io_context (blocks)
```

`register_default_handlers` is the composition of **15 per-capability
functions** (`register_identification`, `register_alarms`,
`register_carriers`, `register_jobs`, …) mirroring how E30 itself lists
capabilities (S1F19). A sensor-class tool with no carriers or jobs
registers only what it is — the unregistered messages get the Router's
SxF0/S9 default treatment, which is exactly what "I don't implement that
capability" should look like on the wire.

The ids the built-ins touch (the control-state/clock SVIDs the engine
refreshes, the CEIDs fired on CJ state changes) come from the `roles:`
block in `equipment.yaml` — visible coupling, no magic constants.

### The threading contract (the one rule)

One io_context thread owns the model. From any other thread:

- **writes** go through the runtime's posting API
  (`set_variable`, `emit_event`, `set_alarm`, `clear_alarm`);
- **reads of mutable state** go through `read_sync` (post + future with a
  deadline) — `control_state()` alone is lock-free (atomic mirror);
- **behaviour hooks** (`commands.set_handler`, state-change observers) run
  *on* the io thread: return promptly, post long work elsewhere.

This is TSan-enforced in CI, daemon included. The first violation ever
caught was in our own test — the lane works.

### Observers vs. the primary slot

State machines expose `set_state_change_handler` (the primary slot —
yours, replaceable) **and** `add_state_change_handler` (append-only
observers that survive the primary being set). The runtime and daemon use
observers for the control-state mirror, `WatchHealth`, and the command
stream, so they never fight your application over the slot.

---

## 5. Running it in production

- **Exposure.** `--grpc` defaults to `127.0.0.1:50051`; the API is
  unauthenticated by design (auth belongs to the transport), so it must
  never face the equipment LAN. For same-host tool software use a Unix
  domain socket — `--grpc unix:///run/secs_gemd/api.sock` — and there is
  no network surface at all. The HSMS port faces the fab host; firewall
  it to the host's address ([SECURITY.md](SECURITY.md) has the nftables
  recipe).
- **Shutdown.** SIGTERM/SIGINT drain gracefully: open Subscribe/WatchHealth
  streams are cancelled (2s deadline), the engine stops cleanly, the spool
  journal is never cut mid-write, exit code 0. Safe under systemd and
  `docker stop`.
- **Supervision.** [deploy/secs_gemd.service](../deploy/secs_gemd.service)
  is a hardened systemd unit (DynamicUser, ProtectSystem, StateDirectory
  for the spool, Restart=always). Pair with `--spool-dir` so host-bound
  events survive daemon restarts too.
- **Metrics.** `--metrics 9091` serves Prometheus gauges:
  `secsgem_link_selected`, `secsgem_control_state`, `secsgem_spool_depth`.
- **Sessions.** v1 runs one equipment identity per daemon (HSMS-SS). The
  engine supports HSMS-GS multi-session, but the daemon doesn't surface it
  yet — run one daemon per equipment identity.
- All of the above is enforced by `tools/check_daemon_ops.sh` (the
  `daemon-ops` step of `tools/run_interop.sh`, also in CI).

---

## 6. Which tier do I pick?

| Your situation | Tier |
|---|---|
| New tool, Python anywhere in the controller, fastest start | Python client |
| Existing controller in C#/Java/Go/…; multi-process architecture; tool app must be restartable without the host noticing | gRPC daemon + thin client |
| In-process integration, custom transports, hard real-time adjacency | Embedded C++ (`EquipmentRuntime`, chapter 41) |

They compose: a C++ tool can still run the daemon for the HSMS face and
talk gRPC locally — that is precisely the "tool software + separate
SECS server" deployment many fabs already run.