secs-gem/docs/42_vendor_daemon_and_clients.md

42 — The vendor daemon and language clients

Chapters 30–41 teach the embedded C++ path: your main() owns the engine. This chapter teaches the daemon path — the engine as its own process, your tool software in any language on the other side of a socket — and the runtime/capability API both paths share. If you are integrating a tool and your controller is not C++, start here.

Everything in this chapter is exercised by tools/run_interop.sh (the daemon, pyclient, and daemon-unit steps) against the secsgem-py reference host. Status and remaining work: DAEMON_ROADMAP.md.

---

1. Why a daemon at all

A fab host enforces timers: T3 reply timeouts, T6 control transactions, linktest heartbeats. If the GEM stack lives inside your tool application, every crash, upgrade, GC pause, or hung hardware call of that application is a communication failure the fab can see — and in production that can mean the tool gets taken offline and lots get held.

secs_gemd inverts the deployment: the daemon owns the durable HSMS relationship and answers the host from its own process, around the clock. Your tool software connects over gRPC, pushes values and events in, and receives host commands out. It can restart any time; the host never notices. Spooling (chapter 13 §spool) covers the gap if the host link drops; the daemon model covers the gap if your software drops.

 tool software (any language)        secs_gemd                    fab host / MES
 ┌──────────────────────────┐  gRPC ┌──────────────────────────┐  HSMS ┌────────┐
 │ set / fire / alarm       │◄─────►│ EquipmentRuntime          │◄────►│  MES   │
 │ @on("START") handlers    │ :50051│ + register_default_*      │SECS-II└────────┘
 │ (restartable, crashable) │       │ + spool, timers, FSMs     │
 └──────────────────────────┘       └──────────────────────────┘

Run it:

build/secs_gemd --port 5000 --grpc 0.0.0.0:50051 --config-dir data

One process, two faces: passive HSMS equipment on --port, the gRPC tool API on --grpc.

---

2. The API contract (proto/secsgem/v1/equipment.proto)

The proto is the source of truth; read it — it is heavily commented. The shape in one breath: everything is name-based (the names from your equipment.yaml; never numeric SVIDs/CEIDs/ALIDs) and plain-typed (a Value oneof of text/integer/real/boolean/binary/list; the daemon converts to each variable's declared SECS-II format, so an F4 variable stays F4 on the wire no matter what you send).

RPC	What it does
SetVariables / GetVariables	write/read variables by name
FireEvent	trigger a collection event; daemon assembles the configured report → S6F11
SetAlarm / ClearAlarm	S5F1 set/clear, by alarm name: (or stringified ALID)
GetControlState / RequestControlState	read the E30 control state / operator transitions
WatchHealth	server stream: link state, control state, spool depth
Subscribe	server stream: everything the host asks of the tool
CompleteCommand	close a streamed command's audit entry

The HCACK-4 command contract

The one piece of SEMI behaviour you must understand: when the host sends a remote command (S2F41 START), the daemon does not wait for your tool. It answers the host immediately:

• No tool subscribed → the command's declarative ack from equipment.yaml (exactly the pre-daemon behaviour; nothing buffered, nothing replayed later).
• Tool subscribed → HCACK=4 ("accepted, will finish later"), and the command appears on your Subscribe stream with its parameters.

The S2F42 transaction is already closed by the time you see the command. The host learns the real outcome the way E30 intends: from the collection event you fire on success (or the alarm you raise on failure). CompleteCommand only feeds the daemon's audit log. This is the same pattern secsgem-py applications and commercial GEM gateways use — the protocol was designed for it.

---

3. The Python client (clients/python)

pip install the package (pure Python — pre-generated stubs, no compiled extension) and the entire integration is:

from secsgem_client import Equipment

eq = Equipment("localhost:50051")

eq.set(ChamberPressure=2.5)            # host sees it on its next S1F3
eq["WaferCounter"] = 7                 # item syntax, same thing
print(eq.get("ChamberPressure"))       # read back through the daemon

eq.fire("ProcessStarted", ChamberPressure=2.75)   # values, then S6F11
eq.alarm("chiller_temp_high")          # S5F1 set
eq.clear("chiller_temp_high")          # S5F1 clear

@eq.on("START")                        # host S2F41 -> your function
def start(cmd):
    run_recipe(cmd.params.get("PPID"))
    eq.fire("ProcessStarted")          # the host's completion signal

eq.listen(background=True)             # consume the Subscribe stream

eq.control_state                       # "ONLINE_REMOTE"
eq.request_control_state("HOST_OFFLINE")   # operator panel -> maintenance
eq.health()                            # link / control state / spool depth

Anything the daemon declines raises SecsGemError with its explanation (no variable named 'ChamberPresure'). A complete runnable tool is clients/python/examples/mini_tool.py (~25 lines). The package is validated end-to-end by interop/pyclient_interop.py: 13 checks driving the published API while secsgem-py judges the wire.

Other languages: generate stubs from the proto (protoc supports 11+ languages) and wrap them the same way — the Python client is ~200 lines and is the reference for what a thin wrapper should feel like.

---

4. The shared core: EquipmentRuntime + capability registration

Both secs_server and secs_gemd (and any future surface) are thin fronts over the same two calls:

#include "secsgem/gem/runtime.hpp"
#include "secsgem/gem/default_handlers.hpp"

gem::EquipmentRuntime::Config cfg;
cfg.equipment_yaml     = "data/equipment.yaml";
cfg.control_state_yaml = "data/control_state.yaml";
cfg.process_job_yaml   = "data/process_job_state.yaml";
cfg.control_job_yaml   = "data/control_job_state.yaml";
cfg.port = 5000;
cfg.log  = [](const std::string& m) { std::cout << m << std::endl; };

gem::EquipmentRuntime R(cfg);
gem::register_default_handlers(R);   // all 56 GEM handlers + emitters
R.run();                             // accept + io_context (blocks)

register_default_handlers is the composition of 15 per-capability functions (register_identification, register_alarms, register_carriers, register_jobs, …) mirroring how E30 itself lists capabilities (S1F19). A sensor-class tool with no carriers or jobs registers only what it is — the unregistered messages get the Router's SxF0/S9 default treatment, which is exactly what "I don't implement that capability" should look like on the wire.

The ids the built-ins touch (the control-state/clock SVIDs the engine refreshes, the CEIDs fired on CJ state changes) come from the roles: block in equipment.yaml — visible coupling, no magic constants.

The threading contract (the one rule)

One io_context thread owns the model. From any other thread:

• writes go through the runtime's posting API (set_variable, emit_event, set_alarm, clear_alarm);
• reads of mutable state go through read_sync (post + future with a deadline) — control_state() alone is lock-free (atomic mirror);
• behaviour hooks (commands.set_handler, state-change observers) run on the io thread: return promptly, post long work elsewhere.

This is TSan-enforced in CI, daemon included. The first violation ever caught was in our own test — the lane works.

Observers vs. the primary slot

State machines expose set_state_change_handler (the primary slot — yours, replaceable) and add_state_change_handler (append-only observers that survive the primary being set). The runtime and daemon use observers for the control-state mirror, WatchHealth, and the command stream, so they never fight your application over the slot.

---

5. Which tier do I pick?

Your situation	Tier
New tool, Python anywhere in the controller, fastest start	Python client
Existing controller in C#/Java/Go/…; multi-process architecture; tool app must be restartable without the host noticing	gRPC daemon + thin client
In-process integration, custom transports, hard real-time adjacency	Embedded C++ (EquipmentRuntime, chapter 41)

They compose: a C++ tool can still run the daemon for the HSMS face and talk gRPC locally — that is precisely the "tool software + separate SECS server" deployment many fabs already run.