feat(daemon): D10 carriers + E16 ops RPCs + stress test + virtual fab
tests / build-and-test (push) Successful in 2m59s
tests / thread-sanitizer (push) Successful in 3m36s
tests / tshark-dissector (push) Successful in 2m25s
tests / secs4j-interop (push) Successful in 59s
tests / python-interop (push) Successful in 3m20s
tests / libfuzzer (push) Successful in 3m40s

Completes the daemon's GEM300 surface and adds two new test tiers.

D10 — E87 carriers: CarrierStore gains the HandlerSlot observer pattern
(add_id/slot_map/access_handler). The daemon's id-observer forwards host
S3F17 decisions onto the Subscribe stream as CarrierAction (PROCEED on a
Confirmed transition, CANCEL on CancelCarrier); ReportCarrier drives the
flow tool-side: WAITING creates the carrier + records the slot map,
IN_ACCESS/COMPLETE advance the access FSM (INVALID_OBJECT on unknown,
CANNOT_DO_NOW on an illegal transition).

E16 — operations RPCs: Describe (full name inventory: variables/events/
alarms/commands/constants + device header), FlushSpool (purge or drain),
SendTerminalMessage (S10F1 tool->host, honest CANNOT_DO_NOW when no host
and stream 10 isn't spoolable).

Stream responsiveness: Subscribe/WatchHealth poll at 100ms (was 500ms) so a
cancelled stream frees its sync-server worker thread promptly — this was
found by the new stress test, which hung under Subscribe churn at 500ms.

Tests:
- A randomized concurrent RPC stress case: 4 threads x 250 seeded ops
  (set/get/fire/alarm/control-state/describe + Subscribe churn), asserts no
  failed RPC and a still-responsive engine afterward; prints its seed; a
  strong TSan target.
- A virtual fab (interop/virtual_fab.py + the `fab` compose service /
  tools/spawn_fab.sh): N daemons, each with a secsgem-py host AND a
  secsgem_client tool, driven by seeded random traffic with end-to-end
  invariant checks (set/get round-trips, event->S6F11 and alarm->S5F1
  delivery, command->tool->completion). Verified green at N=3 (~150 ops/eq,
  all commands round-tripped, 0 violations). Wired into run_interop.sh
  (now 13 steps).

Also fixes the CI break from the previous commit: the Python-client lane's
test_values.py step lacked PYTHONPATH=clients/python (now step-level env).

Two bugs found and fixed while building this, both mine from this batch:
1. carrier test hung on a CancelCarrier of a still-NotConfirmed carrier — a
   self-transition the FSM doesn't signal, so the observer never fired and
   the stream Read blocked forever. Fixed to cancel a Confirmed carrier;
   the NotConfirmed edge is documented as a known E87 limitation.
2. the 500ms stream poll above.

Daemon suite 7 cases / 214 assertions; core 475 / 3097; virtual fab green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-26 15:05:13 +02:00
parent 54626ceb6a
commit 9876dd9b5a
10 changed files with 703 additions and 13 deletions
+24 -7
View File
@@ -211,10 +211,15 @@ debts tax every later phase, and the most valuable tests aren't automated.
→Aborting=ABORT; carries recipe + material bindings) +
`ReportProcessJob` (SETTING_UP→SetupComplete, COMPLETE→ProcessComplete,
ABORTED→AbortComplete; PROCESSING informational; INVALID_OBJECT /
CANNOT_DO_NOW on unknown job / illegal transition). Carriers deferred:
`CarrierStore` has no observer machinery yet — apply the HandlerSlot
pattern, then mirror the job wiring; `ReportCarrier` stays gRPC
UNIMPLEMENTED (the stub default) until then.
CANNOT_DO_NOW on unknown job / illegal transition). Carriers (E87):
`CarrierStore` gained the HandlerSlot observer pattern; the daemon's
id-observer forwards host S3F17 decisions as `CarrierAction` (PROCEED on
Confirmed, CANCEL on CancelCarrier) and `ReportCarrier` drives the
arrival/access flow (WAITING creates + slot map; IN_ACCESS/COMPLETE the
access FSM). KNOWN EDGE: a host CancelCarrier on a still-NotConfirmed
carrier is an FSM self-transition, so no observer fires and the tool
isn't notified — fix needs an event-level (not state-change) hook;
low priority (hosts cancel after a read, i.e. from Confirmed).
11. ✅ `ProcessProgram` on the stream when S7F3 lands (new RecipeStore
added-observer) and `ConstantChange` when S2F15 is ACCEPTED (new
EquipmentConstantStore changed-observer; rejected writes never fire).
@@ -242,9 +247,21 @@ debts tax every later phase, and the most valuable tests aren't automated.
DynamicUser/ProtectSystem/StateDirectory/TimeoutStopSec). All enforced
by `tools/check_daemon_ops.sh`. Single-session (HSMS-SS) assumption
documented in ch42 §5.
16. ⬜ Remaining Layer-1 API: traces, limits, substrates/modules, terminal
services, spool flush RPC, `Describe` RPC. (Engine-side all exists;
surface on demand.)
16. 🚧 Surfaced: `Describe` (full name inventory), `FlushSpool` (purge/
drain), `SendTerminalMessage` (S10F1 tool→host). ⬜ Still engine-only:
traces (S2F23), limits (S2F45), substrate/E90 + module/E157 tracking —
surface on demand.
17. ✅ Stream responsiveness: Subscribe/WatchHealth poll at 100ms (was
500ms) so a cancelled stream frees its sync-server worker thread
promptly — found via the randomized stress test below.
18. ✅ Test tiers added: a randomized concurrent RPC stress case (4 threads
× 250 seeded ops incl. Subscribe churn; TSan target; prints its seed)
and a **virtual fab** (`interop/virtual_fab.py` + the `fab` compose
service / `tools/spawn_fab.sh`): N daemons, each with a secsgem-py host
AND a secsgem_client tool, driven by seeded random traffic with
end-to-end invariant checks (round-trips, S6F11/S5F1 delivery, the
command loop). Wired into `run_interop.sh` (now 13 steps). Verified
green at N=3.
### Phase F — fab acceptance (parallel track; the hard gate)
- ⚠️ **Standards correctness remains unverified against SEMI texts** (behaviour