secs-gem/docs/42_vendor_daemon_and_clients.md

42 — The vendor daemon and language clients

Chapters 30–41 teach the embedded C++ path: your main() owns the engine. This chapter teaches the daemon path — the engine as its own process, your tool software in any language on the other side of a socket — and the runtime/capability API both paths share. If you are integrating a tool and your controller is not C++, start here.

Everything in this chapter is exercised by tools/run_interop.sh (the daemon, pyclient, and daemon-unit steps) against the secsgem-py reference host. Status and remaining work: DAEMON_ROADMAP.md.

---

1. Why a daemon at all

A fab host enforces timers: T3 reply timeouts, T6 control transactions, linktest heartbeats. If the GEM stack lives inside your tool application, every crash, upgrade, GC pause, or hung hardware call of that application is a communication failure the fab can see — and in production that can mean the tool gets taken offline and lots get held.

secs_gemd inverts the deployment: the daemon owns the durable HSMS relationship and answers the host from its own process, around the clock. Your tool software connects over gRPC, pushes values and events in, and receives host commands out. It can restart any time; the host never notices. Spooling (chapter 13 §spool) covers the gap if the host link drops; the daemon model covers the gap if your software drops.

 tool software (any language)        secs_gemd                    fab host / MES
 ┌──────────────────────────┐  gRPC ┌──────────────────────────┐  HSMS ┌────────┐
 │ set / fire / alarm       │◄─────►│ EquipmentRuntime          │◄────►│  MES   │
 │ @command / @on handlers  │ :50051│ + register_default_*      │SECS-II└────────┘
 │ (restartable, crashable) │       │ + spool, timers, FSMs     │
 └──────────────────────────┘       └──────────────────────────┘

Run it:

build/secs_gemd --port 5000 --config-dir data            # gRPC on 127.0.0.1:50051
build/secs_gemd --grpc unix:///run/secs_gemd/api.sock …  # production: no TCP at all

One process, two faces: passive HSMS equipment on --port, the gRPC tool API on --grpc (localhost by default — see §5 before exposing anything).

---

2. The API contract (proto/secsgem/v1/equipment.proto)

The proto is the source of truth; read it — it is heavily commented. The shape in one breath: everything is name-based (the names from your equipment.yaml; never numeric SVIDs/CEIDs/ALIDs) and plain-typed (a Value oneof of text/integer/real/boolean/binary/list; the daemon converts to each variable's declared SECS-II format, so an F4 variable stays F4 on the wire no matter what you send).

RPC	What it does
SetVariables / GetVariables	write/read variables by name
FireEvent	trigger a collection event; daemon assembles the configured report → S6F11
SetAlarm / ClearAlarm	S5F1 set/clear, by alarm name: (or stringified ALID)
GetControlState / RequestControlState	read the E30 control state / operator transitions
ReportCarrier	E87 carrier state transitions (WAITING / IN_ACCESS / COMPLETE)
ReportSubstrate	E90 wafer tracking (ARRIVED / AT_WORK / PROCESSING / PROCESSED / AT_DESTINATION)
ReportModule	E157 module tracking (NOT_EXECUTING / GENERAL_EXECUTING / STEP_EXECUTING / STEP_COMPLETED)
WatchHealth	server stream: link state, control state, spool depth
Subscribe	server stream: everything the host asks of the tool
CompleteCommand	close a streamed command's audit entry
Describe	all names this equipment exposes (variables, events, alarms, commands, constants)
FlushSpool	drain or discard spooled messages
SendTerminalMessage	S10F1 operator message to the host

The HCACK-4 command contract

The one piece of SEMI behaviour you must understand: when the host sends a remote command (S2F41 START), the daemon does not wait for your tool. It answers the host immediately:

• No tool subscribed → the command's declarative ack from equipment.yaml (exactly the pre-daemon behaviour; nothing buffered, nothing replayed later).
• Tool subscribed → HCACK=4 ("accepted, will finish later"), and the command appears on your Subscribe stream with its parameters.

The S2F42 transaction is already closed by the time you see the command. The host learns the real outcome the way E30 intends: from the collection event you fire on success (or the alarm you raise on failure). CompleteCommand only feeds the daemon's audit log. This is the same pattern secsgem-py applications and commercial GEM gateways use — the protocol was designed for it.

---

3. The Python client (clients/python)

pip install the package (pure Python — pre-generated stubs, no compiled extension) and the entire integration is:

from secsgem_client import Equipment, Milestone, ModuleState

with Equipment("localhost:50051") as eq:   # context manager closes the channel

    eq.set(ChamberPressure=2.5)            # host sees it on its next S1F3
    eq["WaferCounter"] = 7                 # item syntax, same thing
    print(eq.get("ChamberPressure"))       # read back through the daemon

    # eq.names — autocomplete-able, typo-safe name lookup (fetched from Describe)
    eq.fire(eq.names.event.ProcessStarted)    # typo → AttributeError at the line it happened
    eq.alarm(eq.names.alarm.chiller_temp_high)
    eq.clear(eq.names.alarm.chiller_temp_high)
    # Plain strings still work; names are a convenience, not a requirement.
    eq.fire("ProcessStarted", ChamberPressure=2.75)

    @eq.command                            # function name IS the command name;
    def START(cmd):                        # validated against Describe at decoration time
        run_recipe(cmd.params.get("PPID"))
        eq.fire(eq.names.event.ProcessStarted)

    # @eq.on("NAME") still works — use it when the name can't be a Python identifier
    # or when you prefer explicit strings.

    eq.listen(background=True)             # consume the Subscribe stream

    eq.control_state                       # "ONLINE_REMOTE"
    eq.request_control_state("HOST_OFFLINE")   # operator panel -> maintenance
    eq.health()                            # link / control state / spool depth

    # E90 / E157 material tracking. Milestone / ModuleState are importable
    # enums (autocomplete + typo-checked); the equivalent plain strings work too.
    eq.report_substrate("WFR-001", Milestone.ARRIVED, carrier_id="FOUP-7", slot=3)
    eq.report_substrate("WFR-001", Milestone.AT_WORK)
    eq.report_substrate("WFR-001", Milestone.PROCESSING)
    eq.report_substrate("WFR-001", Milestone.PROCESSED)
    eq.report_substrate("WFR-001", Milestone.AT_DESTINATION)

    eq.report_module("CHAMBER-A", ModuleState.GENERAL_EXECUTING)
    eq.report_module("CHAMBER-A", ModuleState.STEP_EXECUTING)
    eq.report_module("CHAMBER-A", ModuleState.STEP_COMPLETED)
    eq.report_module("CHAMBER-A", ModuleState.NOT_EXECUTING)

Two error channels, by design: a bad value you control (a misspelled milestone, an unknown control state) raises a plain ValueError/NameError before any round-trip, with a close-match hint; anything the daemon declines (unknown variable name, illegal FSM transition) raises SecsGemError with its explanation (no variable named 'ChamberPresure'). Runnable tools: mini_tool.py (~25-line quickstart) and wafer_tool.py (E90/E157 material tracking). The package is validated end-to-end by interop/pyclient_interop.py driving the published API while secsgem-py judges the wire.

eq.names — name autocomplete and typo safety

eq.names fetches Describe from the daemon once (lazy, cached), then exposes five sub-namespaces:

eq.names.event.ProcessStarted     # → "ProcessStarted"
eq.names.alarm.chiller_temp_high  # → "chiller_temp_high"
eq.names.command.START            # → "START"
eq.names.var.ChamberPressure      # → "ChamberPressure"
eq.names.constant.MaxPressure     # → "MaxPressure"

# Typo → AttributeError with suggestions:
# AttributeError: no event named 'ProcessStated'. Did you mean ProcessStarted?

# Membership test — useful in @eq.on guards:
"START" in eq.names.command       # → True

dir(eq.names.event) lists all event names — REPL and IDE autocomplete work out of the box.

Names vs. enums — one rule

There are two kinds of identifier in the API, split on whether your tool or the SEMI standard owns the value:

• Equipment-specific names — events, alarms, commands, variables, constants — come from your equipment.yaml, so they live on the instance as eq.names.* (fetched from the live daemon).
• Fixed protocol value-sets — Milestone, ModuleState, JobState — are defined by the standards, so they're importable enums (from secsgem_client import Milestone). Milestone.ARRIVED == "ARRIVED", so an enum member and its plain string are interchangeable; the enum just buys you autocomplete and a typo-checked happy path.

Either way a wrong value fails fast and helpfully — a ValueError/NameError with a close-match suggestion, raised client-side before the wire.

Other languages: generate stubs from the proto (protoc supports 11+ languages) and wrap them the same way — the Python client is ~200 lines and is the reference for what a thin wrapper should feel like.

---

4. The shared core: EquipmentRuntime + capability registration

Both secs_server and secs_gemd (and any future surface) are thin fronts over the same two calls:

#include "secsgem/gem/runtime.hpp"
#include "secsgem/gem/default_handlers.hpp"

gem::EquipmentRuntime::Config cfg;
cfg.equipment_yaml     = "data/equipment.yaml";
cfg.control_state_yaml = "data/control_state.yaml";
cfg.process_job_yaml   = "data/process_job_state.yaml";
cfg.control_job_yaml   = "data/control_job_state.yaml";
cfg.port = 5000;
cfg.log  = [](const std::string& m) { std::cout << m << std::endl; };

gem::EquipmentRuntime R(cfg);
gem::register_default_handlers(R);   // all 56 GEM handlers + emitters
R.run();                             // accept + io_context (blocks)

register_default_handlers is the composition of 15 per-capability functions (register_identification, register_alarms, register_carriers, register_jobs, …) mirroring how E30 itself lists capabilities (S1F19). A sensor-class tool with no carriers or jobs registers only what it is — the unregistered messages get the Router's SxF0/S9 default treatment, which is exactly what "I don't implement that capability" should look like on the wire.

The ids the built-ins touch (the control-state/clock SVIDs the engine refreshes, the CEIDs fired on CJ state changes) come from the roles: block in equipment.yaml — visible coupling, no magic constants.

The threading contract (the one rule)

One io_context thread owns the model. From any other thread:

• writes go through the runtime's posting API (set_variable, emit_event, set_alarm, clear_alarm);
• reads of mutable state go through read_sync (post + future with a deadline) — control_state() alone is lock-free (atomic mirror);
• behaviour hooks (commands.set_handler, state-change observers) run on the io thread: return promptly, post long work elsewhere.

This is TSan-enforced in CI, daemon included. The first violation ever caught was in our own test — the lane works.

Observers vs. the primary slot

State machines expose set_state_change_handler (the primary slot — yours, replaceable) and add_state_change_handler (append-only observers that survive the primary being set). The runtime and daemon use observers for the control-state mirror, WatchHealth, and the command stream, so they never fight your application over the slot.

---

5. Running it in production

• Exposure. --grpc defaults to 127.0.0.1:50051; the API is unauthenticated by design (auth belongs to the transport), so it must never face the equipment LAN. For same-host tool software use a Unix domain socket — --grpc unix:///run/secs_gemd/api.sock — and there is no network surface at all. The HSMS port faces the fab host; firewall it to the host's address (SECURITY.md has the nftables recipe).
• Shutdown. SIGTERM/SIGINT drain gracefully: open Subscribe/WatchHealth streams are cancelled (2s deadline), the engine stops cleanly, the spool journal is never cut mid-write, exit code 0. Safe under systemd and docker stop.
• Supervision. deploy/secs_gemd.service is a hardened systemd unit (DynamicUser, ProtectSystem, StateDirectory for the spool, Restart=always). Pair with --spool-dir so host-bound events survive daemon restarts too.
• Metrics. --metrics 9091 serves Prometheus gauges: secsgem_link_selected, secsgem_control_state, secsgem_spool_depth.
• Sessions. v1 runs one equipment identity per daemon (HSMS-SS). The engine supports HSMS-GS multi-session, but the daemon doesn't surface it yet — run one daemon per equipment identity.
• All of the above is enforced by tools/check_daemon_ops.sh (the daemon-ops step of tools/run_interop.sh, also in CI).

---

6. Which tier do I pick?

Your situation	Tier
New tool, Python anywhere in the controller, fastest start	Python client
Existing controller in C#/Java/Go/…; multi-process architecture; tool app must be restartable without the host noticing	gRPC daemon + thin client
In-process integration, custom transports, hard real-time adjacency	Embedded C++ (EquipmentRuntime, chapter 41)

They compose: a C++ tool can still run the daemon for the HSMS face and talk gRPC locally — that is precisely the "tool software + separate SECS server" deployment many fabs already run.