Completes the daemon's GEM300 surface and adds two new test tiers.
D10 — E87 carriers: CarrierStore gains the HandlerSlot observer pattern
(add_id/slot_map/access_handler). The daemon's id-observer forwards host
S3F17 decisions onto the Subscribe stream as CarrierAction (PROCEED on a
Confirmed transition, CANCEL on CancelCarrier); ReportCarrier drives the
flow tool-side: WAITING creates the carrier + records the slot map,
IN_ACCESS/COMPLETE advance the access FSM (INVALID_OBJECT on unknown,
CANNOT_DO_NOW on an illegal transition).
E16 — operations RPCs: Describe (full name inventory: variables/events/
alarms/commands/constants + device header), FlushSpool (purge or drain),
SendTerminalMessage (S10F1 tool->host, honest CANNOT_DO_NOW when no host
and stream 10 isn't spoolable).
Stream responsiveness: Subscribe/WatchHealth poll at 100ms (was 500ms) so a
cancelled stream frees its sync-server worker thread promptly — this was
found by the new stress test, which hung under Subscribe churn at 500ms.
Tests:
- A randomized concurrent RPC stress case: 4 threads x 250 seeded ops
(set/get/fire/alarm/control-state/describe + Subscribe churn), asserts no
failed RPC and a still-responsive engine afterward; prints its seed; a
strong TSan target.
- A virtual fab (interop/virtual_fab.py + the `fab` compose service /
tools/spawn_fab.sh): N daemons, each with a secsgem-py host AND a
secsgem_client tool, driven by seeded random traffic with end-to-end
invariant checks (set/get round-trips, event->S6F11 and alarm->S5F1
delivery, command->tool->completion). Verified green at N=3 (~150 ops/eq,
all commands round-tripped, 0 violations). Wired into run_interop.sh
(now 13 steps).
Also fixes the CI break from the previous commit: the Python-client lane's
test_values.py step lacked PYTHONPATH=clients/python (now step-level env).
Two bugs found and fixed while building this, both mine from this batch:
1. carrier test hung on a CancelCarrier of a still-NotConfirmed carrier — a
self-transition the FSM doesn't signal, so the observer never fired and
the stream Read blocked forever. Fixed to cancel a Confirmed carrier;
the NotConfirmed edge is documented as a known E87 limitation.
2. the 500ms stream poll above.
Daemon suite 7 cases / 214 assertions; core 475 / 3097; virtual fab green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The HCACK-4 contract, implemented end to end. For every YAML-declared
command the service registers a forwarding handler (new HostCommandRegistry
names()/spec() accessors): with a subscribed tool client the command is
queued onto the Subscribe stream (id + name + params via from_item) and the
host is answered S2F42 HCACK=4 immediately — never blocking the io thread or
the T3 window; with NO subscriber the command takes its declarative YAML ack
(the honest pre-daemon behaviour). Settled + documented in the proto: v1 is
a firehose with no buffering/replay. CompleteCommand correlates the pending
id (audit; unknown id => PARAMETER_INVALID). Side effects stay suppressed on
HCACK-4 (router applies them only on Accept), so the completion event the
TOOL fires is the host's real signal — exactly E30's intent.
Tests (daemon suite 101 -> 124 assertions): a real S2F41 dispatched through
the full default-handler router ON the io thread under run_async — HCACK 4
with subscriber + params on the stream, declarative Accept without,
CompleteCommand known/unknown, fallback restored after unsubscribe.
Interop (now 20 checks, all green): the complete conformant loop against
the secsgem-py reference host — S2F41 START -> S2F42 HCACK=4 -> tool
receives Command(name=START, id=1) -> CompleteCommand -> FireEvent -> host
receives S6F11.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 0 captures the 2026-06-10 review: multi-observer callbacks (done for
the critical three), CI for the interop/conformance harnesses (the unit
suite is partly self-referential; the external validators are the real
oracle), table-driven handler conformance + message-level golden frames,
register_default_handlers decomposition per GEM capability + YAML role
bindings for today's magic constants, the post+future mutable-read pattern,
service relocation + TSan run_async daemon test, identifier-safe name
validation. CompleteCommand's proto comment described the rejected blocking
model; it now states the settled HCACK-4 contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extract the SECS/GEM engine wiring out of the secs_server app into a
reusable class, and stand up a language-agnostic gRPC daemon on top so a
tool's software (any language) can drive the equipment without linking C++
or knowing SEMI. Foundation for replacing a vendor's SECS/GEM server.
Engine reuse:
- EquipmentRuntime (include/secsgem/gem/runtime.hpp, src/gem/runtime.cpp):
owns io_context, passive Server, model, control-state machine, Router;
thread-safe outbound API (set_variable/emit_event/set_alarm/clear_alarm),
on_command hook, deliver_or_spool, run()/run_async()/poll()/stop().
- register_default_handlers (src/gem/default_handlers.cpp): the 56 GEM
handlers + domain emitters, relocated from secs_server so the app and the
daemon speak byte-identical GEM. secs_server.cpp reduced ~1270 -> 113 lines.
- name_index.hpp: resolve_variable(name) -> VID (the name->id binding layer).
Daemon (apps/secs_gemd.cpp, proto/secsgem/v1/equipment.proto):
- runs the engine + HSMS link on a background thread; serves the gRPC
Equipment service. Increment 1: SetVariables (name-resolved, plain
value->Item) and GetControlState. proto carries the full v1 surface
(universal + carrier/recipe/job tiers); remaining RPCs + the Subscribe
command stream are next (docs/DAEMON_ROADMAP.md).
- CMake: opt-in SECSGEM_DAEMON, protoc/grpc_cpp_plugin codegen, gracefully
skipped where protobuf/grpc++ are absent. Dockerfile gains the grpc deps.
Tests (proof): test_runtime, test_default_handlers (S1F1->S1F2, S2F41->hook),
test_name_index. Full suite 458/458, 2795 assertions; live server<->client
GEM300 demo still passes on the refactored server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
raphael