B7 — the daemon's HSMS face under the Java reference: Dockerfile.server now
bakes secs_gemd alongside secs_server (grpc deps in both stages), and
secs4j_validate.sh gains TARGET=gemd to point the 55-check secs4java8 suite
at the daemon instead. Result: 55/55 green. With secsgem-py already
validating both faces, byte-identical GEM between secs_server and secs_gemd
is now proven by both reference implementations, not inferred from shared
code. CI runs the daemon target as an extra step (image layers shared).
Second client — clients/cpp: a header-only C++ twin of the Python client
over the same proto. eq.set("ChamberPressure", 2.5) with bare literals
(integral/floating dispatch avoids variant ambiguity), get/fire/alarm/
clear, control_state/request_control_state/health, on("START", fn) +
listen()/listen_async()/stop() with auto-CompleteCommand, SecsGemError
carrying the daemon's message. cpp_mini_tool (~30 lines) mirrors the
Python mini_tool. Tested end-to-end over real loopback TCP against the
service inside secs_gemd_tests — now 4 cases / 141 assertions — including
set/get round-trips, error text, alarm-by-name into the model, health,
and the full HCACK-4 command loop with parameters.
(Build note: two grpc-heavy TUs at -O3 OOM even at -j2 on Docker Desktop;
built -j1. Known environment limitation, roadmap-documented.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
C9 — the flagship vendor example now demonstrates the intended integration
shape. examples/pvd_tool/main.cpp: 1093 -> 570 lines. The 466-line
hand-registered handler section and the hand-wired Server/Router/emit
plumbing are gone, replaced by EquipmentRuntime + register_default_handlers
(the example now serves all 56 handlers, up from its hand-picked 51) +
commands.set_handler for the START-runs-the-recipe behaviour (was a
hard-coded S2F41 router override). All domain logic — sensor simulator,
recipe runner, alarm threshold monitor, EPT cycler, Prometheus gauges —
unchanged. pvd's SVIDs 1/2 and CEIDs 400/401 match the roles: defaults, so
the built-ins bind with no config change. Verified: builds clean, boots
("registered 56 handlers", config loaded, EPT cycling), HSMS :5000 accepts,
metrics :9090 answers HTTP 200. logfn flushes per line so docker/CI logs
are visible immediately.
Writing project — new tutorial chapter docs/42_vendor_daemon_and_clients.md:
why a daemon (the host-timer argument), the proto contract and the HCACK-4
command semantics, the Python client walkthrough, EquipmentRuntime +
capability registration + roles:, the threading contract (posting API /
read_sync / hooks-on-io-thread) and primary-vs-observer slots, and a
which-tier-do-I-pick table. Indexed in 00_index Part 4. Refreshed the three
spots that still described pvd_tool's old "51 handlers in ~460 lines" shape
(ch35, ch41, pvd README) — drift killed in the same commit that made it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A large gap had opened between the docs and the code: the README and
INTEGRATION guide did not mention the gRPC daemon or the Python client at
all (the entire vendor surface), ARCHITECTURE still described secs_server
as the ~1200-line canonical wiring example (it is a ~110-line thin main
over EquipmentRuntime), and test counts across six files were stale
(445/2753 -> 473/3087 core + the separate 125-assertion daemon suite).
- README: new "Integrating your tool (pick a tier)" section — Python
client / any-language gRPC / embedded C++ — plus daemon tests and
tools/run_interop.sh in the Testing section.
- ARCHITECTURE: layer diagram gains the vendor-surface and
EquipmentRuntime/default_handlers tiers; stale wiring row fixed.
- INTEGRATION: three-tier chooser up front (this guide = the C++ tier).
- ch30 tour: secs_gemd + secs_gemd_tests in the binaries table.
- ch31: example alarm used a nonexistent `alcd:` field with bit 7 set
(which the validator forbids) -> real `category:`/`name:` fields, and
the roles: block documented.
- ch35: handler-location note now points at default_handlers.cpp's 15
per-capability register_* functions.
- ch40: built-artifacts list + sample output counts.
- ch50: secsgem::gem runtime/default_handlers/handler_slot/name_index
includes + new secsgem::daemon namespace section.
- PROOFS: test-count table gains the runtime/handlers/daemon row so the
tally adds up; daemon suite noted. VERIFICATION/COMPLIANCE counts.
- interop/README: the one-command runner + the two daemon-track harnesses
(daemon_interop, pyclient_interop).
Audited via a docs-vs-code sweep (the audit itself under-reported: it
validated counts textually; reality was 473/3087).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
clients/python: pip-installable "secsgem-client", pure Python (stubs
pre-generated from equipment.proto, import made package-relative; no
compiled extension, no SEMI knowledge, no C++ toolchain). The API the whole
effort aimed at:
eq = Equipment("localhost:50051")
eq.set(ChamberPressure=2.5); eq["WaferCounter"] = 7
eq.fire("ProcessStarted", ChamberPressure=2.75)
eq.alarm("chiller_temp_high"); eq.clear("chiller_temp_high")
@eq.on("START")
def start(cmd): ... # auto-CompleteCommand after return
eq.listen(background=True)
eq.control_state; eq.request_control_state("HOST_OFFLINE"); eq.health()
Errors raise SecsGemError carrying the daemon's message ("no variable named
..."). bool checked before int in conversion (isinstance(True, int)).
examples/mini_tool.py is a complete GEM tool in ~25 lines.
PROOF — interop/pyclient_interop.py drives the PUBLISHED package (not raw
stubs) against a live secs_gemd with secsgem-py as the fab host: 13 checks
all green on first run — set/get round-trips, item syntax, SecsGemError on
unknown names, control state, health, fire->S6F11 on the host's wire,
alarm/clear->S5F1 with correct set bit, the full command loop (host S2F41 ->
HCACK=4 -> @eq.on handler -> completion event back at the host), operator
offline. Conversion layer unit-tested standalone; both wired into
tools/run_interop.sh as the pyclient step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
register_default_handlers was a relocated app main(): one 1086-line function,
all-or-nothing. It is now 15 per-capability registration functions along the
lines GEM itself defines (S1F19): identification, equipment constants, clock,
event reports, remote commands, trace/limits, spooling, alarms, exceptions,
material tracking (E90/E116/E157), carriers (E87), recipes, object services
(E39), jobs (E40/E94), terminal services. A sensor-class tool registers three
functions instead of carrying carrier/job handlers it doesn't have;
register_default_handlers composes all 15. Each function derives exactly the
runtime aliases its handlers use (generated programmatically from the moved
bodies with boundary/substitution guards — zero hand-retyping).
Magic constants are gone: the control-state/clock SVIDs (were hardcoded 1/2)
and the CJ Executing/Completed CEIDs (were 400/401) now come from a "roles:"
block in equipment.yaml via EquipmentDescriptor, with historical defaults
when absent, loader parsing, and validation (CEID roles must name declared
events). The coupling is now visible in ONE file instead of silently split
between YAML and C++ — the exact drift class this repo's spec-as-data
philosophy exists to kill.
Tests: capability subsetting, role-driven SVID refresh via S1F3, roles
loader (shipped/custom/absent). Battery: core 473/3087 incl. the 53-handler
conformance sweep, daemon 125/125, live GEM300 demo (client exit 0), daemon
interop 20/20 vs secsgem-py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Item 8a — ConfigValidator warns on non-identifier variable/event/alarm/
command names ([A-Za-z_][A-Za-z0-9_]*): language bindings expose names as
kwargs/attributes, so 'Chamber Pressure' would be unusable in the planned
Python client. Warning not error — the wire doesn't care. Tested (4 warning
sites + good-name negative).
Item 4 tail — golden frames for S5F1 (Binary ALCD / U4 ALID / ASCII ALTX)
and a composed S6F11 (the production-critical report shape), bytes hand-
computed from E5 encoding rules: external pins on message composition.
Item 7 — equipment_service.hpp moved to include/secsgem/daemon/ (apps/
include-path hack removed) and a TSan daemon lane added locally + in CI.
tools/tsan.supp suppresses races whose accesses sit entirely inside the
UNinstrumented system libgrpc/libgpr/libabsl (epoll wakeups, absl Mutex
GraphCycles bookkeeping); our frames stay fully checked. The lane earned its
keep on first run: it caught a REAL threading-contract violation — a daemon
test reading model stores from the test thread while the io thread serviced
posted writes — fixed to use read_sync, exactly per the documented contract.
Now TSan-clean under halt_on_error=1 in the full production threading shape.
Suites: core 470/3068, daemon Release+TSan 125/125 each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The HCACK-4 contract, implemented end to end. For every YAML-declared
command the service registers a forwarding handler (new HostCommandRegistry
names()/spec() accessors): with a subscribed tool client the command is
queued onto the Subscribe stream (id + name + params via from_item) and the
host is answered S2F42 HCACK=4 immediately — never blocking the io thread or
the T3 window; with NO subscriber the command takes its declarative YAML ack
(the honest pre-daemon behaviour). Settled + documented in the proto: v1 is
a firehose with no buffering/replay. CompleteCommand correlates the pending
id (audit; unknown id => PARAMETER_INVALID). Side effects stay suppressed on
HCACK-4 (router applies them only on Accept), so the completion event the
TOOL fires is the host's real signal — exactly E30's intent.
Tests (daemon suite 101 -> 124 assertions): a real S2F41 dispatched through
the full default-handler router ON the io thread under run_async — HCACK 4
with subscriber + params on the stream, declarative Accept without,
CompleteCommand known/unknown, fallback restored after unsubscribe.
Interop (now 20 checks, all green): the complete conformant loop against
the secsgem-py reference host — S2F41 START -> S2F42 HCACK=4 -> tool
receives Command(name=START, id=1) -> CompleteCommand -> FireEvent -> host
receives S6F11.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A2 — alarms: optional 'name:' on alarm config (a LOCAL key — SEMI only
defines numeric ALID + freetext ALTX; field appended last so existing
{id, text, category} brace-inits compile unchanged), parsed by the loader,
checked by the validator, shipped in equipment.yaml. SetAlarm/ClearAlarm
RPCs resolve config name OR stringified ALID via a constructor snapshot.
A3 — control state + health: RequestControlState fires operator events on
the io thread (read_sync) and reports what the E30 table actually did —
ACCEPT iff the equipment landed in the requested state, CANNOT_DO_NOW naming
the actual state otherwise (the shipped table has no operator path to
EquipmentOffline; the test pins that honesty). ATTEMPT_ONLINE is rejected as
transient. WatchHealth streams an immediate snapshot then pushes on link/
control-state changes via service observers (add_link_observer +
add_control_state_observer — the HandlerSlot work paying off), spool depth
sampled at the 500ms poll; ends on cancel or engine stop.
Tests: daemon suite 61 -> 101 assertions (alarm lifecycle by name/id/unknown,
WatchHealth initial + change push, all four RequestControlState semantics);
loader test for the alarm name (present + absent fallback); core 467/3055.
Interop now 15 checks incl. gRPC SetAlarm -> host receives S5F1 ALCD=0x84
ALID=1, and RequestControlState(HOST_OFFLINE) -> GetControlState confirms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
EquipmentRuntime::read_sync establishes THE pattern for reading mutable
engine state from gRPC/binding threads (Phase 0 item 6): post the read onto
the io thread (the model's single owner), wait on a future with a deadline,
nullopt => UNAVAILABLE at the RPC edge. Always truthful, no cache to
invalidate; milliseconds are irrelevant at SECS rates.
GetVariables: name resolution against the service snapshot (empty query =
all; unknown name => INVALID_ARGUMENT naming the offender), values read via
read_sync, converted by the new from_item reverse conversion (single-element
numeric arrays => scalars, multi-element => List; Boolean/Binary/text per
format; C2-as-integer and U8>2^63 wrap documented as TODOs).
Tests run the engine in run_async — the daemon's PRODUCTION threading mode,
previously untested — and round-trip through both conversions: SetVariables
(declared-format write) then GetVariables (read) over a real in-process
channel. Daemon suite 41 -> 61 assertions. daemon_interop.py gains a live
GetVariables round-trip check vs the running daemon (verified green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One ordered in-process scenario drives 53 of the 56 registered handlers
through Router::dispatch — S1 identification/comms/control, S2 ECs/clock/
event-config/commands/trace/limits/spool, S5 alarms+exceptions, S6 reports,
S7 recipes, S10 terminal, S14/S16 E39+E40/E94 jobs, S3 carriers — asserting
every reply is the paired (stream, function+1) with a body, plus targeted
state checks (OnlineRemote after S1F17, PJ exists after S16F11, HostOffline
after S1F15) and the Router's SxF0 abort fallback for unregistered W=1
primaries. Same flow secs_conformance runs over a live socket, but cheap
enough for every build; closes the '56 handlers, 4 direct tests' gap from
the design review.
Also seeds message-level golden frames: S1F13's body pinned to bytes
hand-computed from the E5 encoding rules — an external check on message
composition, not our codec validating itself (TODO: S5F1, composed S6F11).
Suite: 466 cases / 3052 assertions (+236), all green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tools/run_interop.sh runs ALL nine validation steps with a PASS/FAIL summary:
build, unit (464), daemon-unit (41), secsgem-py host vs server (31 checks),
secs_conformance (47), gRPC+secsgem-py daemon bridge, spool persistence
across restart, tshark HSMS dissector, secs4java8 (55 checks). Verified green
end-to-end. The unit suite is partly self-referential (our parsers validate
our builders); these external validators are the real oracle — now they run
with one command instead of by hand. Two bugs found by running it: unbounded
ninja at -O3 OOM-kills cc1plus in memory-constrained Docker VMs (build with
-j 2) and bash-3.2 lacks negative array subscripts.
CI: grpc deps added to the build job so secs_gemd + secs_gemd_tests build and
RUN in CI (previously the daemon silently dropped out — now fails loudly if
missing), plus a python-interop lane running py-host/conformance/daemon
harnesses against localhost in one container (no docker-in-docker).
Service hardening while in there: reject proto Values with no kind set at
the RPC edge (previously silently became ASCII ""), TODO markers for list
element formats and daemon graceful shutdown. New tests: unset-Value guard
+ a property test iterating ALL configured variables via gRPC asserting each
keeps its declared SECS-II format (daemon tests 16 -> 41 assertions).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 0 captures the 2026-06-10 review: multi-observer callbacks (done for
the critical three), CI for the interop/conformance harnesses (the unit
suite is partly self-referential; the external validators are the real
oracle), table-driven handler conformance + message-level golden frames,
register_default_handlers decomposition per GEM capability + YAML role
bindings for today's magic constants, the post+future mutable-read pattern,
service relocation + TSan run_async daemon test, identifier-safe name
validation. CompleteCommand's proto comment described the rejected blocking
model; it now states the settled HCACK-4 contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Status table brought current (format-aware daemon, secsgem-py interop), the
stale Layer-0 section replaced, and the path to an excellent GEM300 repo laid
out as ordered phases A–F: finish universal RPCs, the Subscribe command
stream (HCACK-4 design written down as the implementation contract), the
Python client package, GEM300 job/carrier in-the-loop, hardening/CI, and the
fab-acceptance track. Known-issues section records what the audit found
(GetControlState enum race + why the state-change-handler slot can't be
reused, missing alarm name key, pvd_tool predating set_handler, manual
interop harnesses, TSan gap).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extract the SECS/GEM engine wiring out of the secs_server app into a
reusable class, and stand up a language-agnostic gRPC daemon on top so a
tool's software (any language) can drive the equipment without linking C++
or knowing SEMI. Foundation for replacing a vendor's SECS/GEM server.
Engine reuse:
- EquipmentRuntime (include/secsgem/gem/runtime.hpp, src/gem/runtime.cpp):
owns io_context, passive Server, model, control-state machine, Router;
thread-safe outbound API (set_variable/emit_event/set_alarm/clear_alarm),
on_command hook, deliver_or_spool, run()/run_async()/poll()/stop().
- register_default_handlers (src/gem/default_handlers.cpp): the 56 GEM
handlers + domain emitters, relocated from secs_server so the app and the
daemon speak byte-identical GEM. secs_server.cpp reduced ~1270 -> 113 lines.
- name_index.hpp: resolve_variable(name) -> VID (the name->id binding layer).
Daemon (apps/secs_gemd.cpp, proto/secsgem/v1/equipment.proto):
- runs the engine + HSMS link on a background thread; serves the gRPC
Equipment service. Increment 1: SetVariables (name-resolved, plain
value->Item) and GetControlState. proto carries the full v1 surface
(universal + carrier/recipe/job tiers); remaining RPCs + the Subscribe
command stream are next (docs/DAEMON_ROADMAP.md).
- CMake: opt-in SECSGEM_DAEMON, protoc/grpc_cpp_plugin codegen, gracefully
skipped where protobuf/grpc++ are absent. Dockerfile gains the grpc deps.
Tests (proof): test_runtime, test_default_handlers (S1F1->S1F2, S2F41->hook),
test_name_index. Full suite 458/458, 2795 assertions; live server<->client
GEM300 demo still passes on the refactored server.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An audit of doc code blocks against the real headers found APIs that do
not exist in the codebase, presented as authoritative walkthroughs:
- ch35 (dispatch): an entirely fabricated callback architecture —
HostCommandRegistry::set_emit_ceid_handler, CommandOutcome, emit_ceids.
Rewritten to the real Spec/Result/dispatch + the new set_handler hook.
- ch13 (E30): wrong store names — EventStore/ReportStore -> EventReportSubscriptions,
SvidStore -> StatusVariableStore, AlarmStore/AlarmDispatcher -> AlarmRegistry,
ClockStore -> Clock, TerminalServiceStore -> (no store), in both the
capability tables and the worked S2F33 example.
- ch17 (E116): EptStore/seconds/bucket_ -> EptStateMachine/milliseconds/buckets_.
- ch51 (extending): stale host-command handler -> the real set_handler signature.
Verified clean by grep: no fabricated symbols remain in docs/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tone pass across the non-tutorial markdown — README, PROOFS,
ARCHITECTURE, BENCHMARKS, COMPLIANCE, FAQ, MES_INTEROP, SECURITY,
and interop/README. Three patterns came out:
- Bug-history war stories ("Past interop sweeps surfaced…",
"What these harnesses caught: 1. Strict U-width parsing…").
- Chat-with-reader framing ("Don't skip TLS unless…", "Treat as a
punch list", "If you're running in a pod…", "Misconfiguration
incidents drop dramatically").
- Self-referential narration ("we ship", "our codec", "the
codebase's most-tested layer", "three orders of magnitude above
fab load", "the gift that keeps giving").
README also drops the standalone ThreadSanitizer subsection under
Build details (now a single line under the new Testing section).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously written as a forward-looking plan ("Plan: (1) KAT → (2)
tshark → (3) secs4j → (4) libFuzzer", "Effort: ~3 hours", "Survey
step (do this first)"). All four validators have shipped —
test_e5_kat.cpp, interop/secs4j/Secs4jHostHarness.java,
interop/tshark_validate.sh, apps/fuzz_*.cpp. Rewritten as
documentation of what's there: file paths, CI job names, actual
result numbers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
generate_messages.py → gen_messages.py and several gem/ headers moved
under gem/store/ (carrier_store.hpp → store/carriers.hpp, etc.);
e84.hpp split into e84_state.hpp. The guided-tour chapters still
pointed at the old paths — relink them so the deep-link footnotes
resolve.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Last four chapters of the guided tour.
40 — Building, running, the demo. Docker prerequisites, the build
flow, what each binary is for, running the 24-transaction demo
flow annotated step by step. Running the 4 external-validator
sweeps + the libFuzzer pass. Inspecting the demo with tcpdump and
tshark. Reading source while running as the recommended learning
workflow.
41 — Integration: hardware, MES, production. Four-phase tour:
wiring sensors / recipe engine / alarms / E84 GPIO; talking to a
real MES with the day-1 punch list + commercial-MES quirks (Wonderware
S2F21, Camstar Linktest cadence, etc.); production hardening
(nftables / stunnel / minisign / persistence layout / monitoring /
runbook); performance envelope + memory footprint + capacity
planning. Pointers to the long-form INTEGRATION.md / MES_INTEROP.md /
SECURITY.md / BENCHMARKS.md.
50 — API + message catalog + YAML schemas reference. Namespace-by-
namespace table of public symbols (secs2, hsms, secsi, gem, config,
metrics) with brief descriptions. Stream-by-stream message catalog
reference (S1, S2, S3, S5, S6, S7, S9, S10, S12, S14, S16). YAML
schema reference for messages.yaml + the three state-table files +
equipment.yaml.
51 — Extending the codebase. Seven recipes ordered from no-code to
substantial: new SVID/DVID/ECID (YAML only), new CEID with reports
(YAML only), new host command (YAML + optional handler), new control-
state transition (YAML only), new SECS-II message (YAML + handler),
new store (header + tests), new persistence backend (drop-in vs
pluggable trade-off). Each recipe has the actual mechanical steps,
the test pattern, and pointers to the chapter that explains why it
works.
Index updated to mark all 24 chapters published.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Seven chapters walking the implementation top-to-bottom.
30 — Repository tour. Top-level layout, directory by directory.
The eight built binaries. The dependency graph from TCP socket
up through EquipmentDataModel. CMake's role. Test layout.
31 — Spec-as-data and codegen. Why the design choice fits SECS/
GEM specifically. The five YAML files: messages catalog,
control/PJ/CJ transition tables, equipment dictionary. How
tools/gen_messages.py turns messages.yaml into typed C++ at build
time. The --validate-config multi-error validator. How to add a
new SVID / CEID / host command / state / message without C++.
32 — Stores and the data model. What a store IS (records + API +
change handler + optional persistence). Every store in the
codebase mapped to the SEMI standard it serves (table of 21).
EquipmentDataModel as plain composition + cross-store convenience
methods (vid_value, compose_reports_for). The no-locks single-
threaded contract. How to add a new store.
33 — Transport. hsms::Connection read path (length+payload async
chain), write path (queue + one outstanding write), timer model
(5 steady_timers + per-request T3). The asio executor / strand
model and why it's the right shape. secsi::Protocol as the IO-
free FSM with Action / Event variants; secsi::TcpTransport as the
asio adapter. Pattern repeats for E84 + GEM comm-state.
34 — Codec and SML. The four files (170 + 30 + 52 + 32 lines of
header, 229 + 220 lines of impl). Item variant storage layout
(11 alternatives, 16 formats, shared storage where E5 permits).
encode_into recursion; decode_at with bounds checks throwing
CodecError. Message wrapper. SML printer + try_parse_sml +
why SML round-trips Items but not necessarily bytes.
35 — State machines and dispatch. gem::Router as a typed
(stream, function) dispatch table. How an S2F41 round-trip walks
through parser → store dispatch → side-effect → CEID emission →
S6F11 build → spool-aware deliver. The 11 FSMs all sharing the
same three-property shape (pure data table + pure FSM + observer
pattern). CEID cascading from FSM transitions to wire bytes.
36 — Persistence, validation, metrics. Which 7 stores have file
journals + why the others don't. Per-record file pattern (atomic
rename, partial-write safe). Schema versioning + multi-version
read. Multi-error YAML validator (--validate-config) + cross-file
reference checks. Prometheus registry + HTTP exporter + worked
metric patterns from the PVD example.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Six more chapters finishing Part 2. Together with chapters 10–13
they document every SEMI standard this codebase implements.
14 — E40 + E94: process jobs (8-state lifecycle, S16F11/F5/F7/F9
on the wire) and control jobs (CJ wraps PJs with batch policy,
S14F9/S16F27 messages). Worked cascade showing how CJSTART
propagates through the PJ FSM and triggers S6F11 CEIDs at each
transition.
15 — E87 carriers: three orthogonal sub-machines (CarrierID,
SlotMap, CarrierAccess) per carrier and three more (Transfer,
Reservation, Association) per load port. S3F17 CarrierAction
strings + CAACK codes, S3F19 SlotMap verify, the 5-state slot
encoding, multi-port concurrency.
16 — E90 + E157: substrate tracking via three orthogonal axes
(STS / SPS / SubstrateIDStatus) and module process tracking
(NotExecuting / GeneralExecuting / StepExecuting / StepCompleted).
End-to-end PVD example showing E40 + E157 + E90 transitions
cascading into CEIDs.
17 — E116 + E120 + E39: equipment performance time-buckets across
six states, common equipment model object hierarchy, S14F1/F3
GetAttr/SetAttr as the uniform wire access for any object type
across multiple standards.
18 — E84 parallel I/O: ten signal lines, the 9-state handshake
FSM, the three TA1/TA2/TA3 timing-critical timers, why a physical
handshake gets modeled in software (testability, timer enforcement,
CEID emission, multi-port concurrency), the pure-FSM + asio-adapter
split.
19 — E42 + E148 + S5F9–F18: formatted recipes (S7F23/F25 typed
PPBODY), time synchronization with 16-char + 14-char accepted on
set, exception recovery as a persistent multi-step host-supervised
FSM (Posted → Recovering → Cleared with abort/retry). Revisits
the auto-S9 family and contrasts S9 (transport) vs S5F9
(application).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three more chapters of Part 2:
11 — E37 HSMS. 4-byte length prefix + 10-byte header (R-bit + session
id + W-bit + stream + function + PType + SType + system_bytes), the
9 SType control messages, the NOT-SELECTED → SELECTED state machine,
T3/T5/T6/T7/T8 with what each one bounds, the auto-S9 paths
(S9F1/F3/F5/F7/F9/F11), HSMS-SS vs HSMS-GS, the asio
single-threaded contract.
12 — E4 SECS-I. Half-duplex line turnaround (ENQ/EOT/ACK/NAK), the
10-byte block header bit-packing (R-bit / W-bit / E-bit / system
bytes), the 244-byte block cap and multi-block split/assemble, the
event-driven IO-free FSM with its Action / Event variants, T1/T2/T3/T4
with semantics + defaults, master/slave contention. Notes the
deferred asio serial_port adapter; explains why this chapter
matters even for HSMS-only readers.
13 — E30 GEM. Disambiguates the three state machines (HSMS transport
vs GEM communication vs GEM control), walks the comm-state FSM
(DISABLED → WAIT-CRA → COMMUNICATING with T_CRA / T_DELAY) and the
control-state FSM (5 states + the YAML transition table). Lists
every Fundamental and Additional capability with its messages, code
locations, and store assignments. One worked Event-Notification
scenario tracing seven on-wire steps to their EquipmentDataModel
internals.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Opens Part 2 (the standards in detail). Walks the entire SECS-II
encoding from first principles: the mental model (every value is one
Item; a List is a recursive Item), the format-byte arithmetic
(6-bit format code, 2-bit length-byte-count), the 14 format codes,
length bytes 1/2/3 (with the 16 MiB cap), big-endian everywhere,
the difference between byte-count (scalars) and child-count (lists).
Then walks every format with worked hexdumps: empty list, nested
list, ASCII with length-byte boundary crossing, Binary vs Boolean,
U1/U2/U4/U8, signed integers with two's-complement edges, F4 / F8
with NaN / ±Inf / −0.0, JIS-8, C2 Unicode.
Then the codebase mapping: Format enum, Item variant storage layout,
encode_into / decode_at recursion, SML printer/parser, the
identifier-wildcard rule (SEMI allows U1/U2/U4/U8 interchangeably
for ID fields) with the messages_helpers::any_unsigned_first<Out>
helper that closes the leniency contract.
Closes with the well-defined CodecError conditions, what the codec
deliberately doesn't reject (unknown format codes), and pointers to
chapter 31 (codegen) and chapter 11 (HSMS) as the next dependencies
above the codec.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Picks up the file renames that landed alongside the previous commit
and fixes everything that pointed at the old root locations:
- README.md doc-map updated: every entry now points at docs/X.md,
with a new "docs/" lead entry pointing at the guided-tour index.
- README inline cross-refs (ARCHITECTURE / INTEGRATION / SECURITY /
BENCHMARKS / MES_INTEROP / PROOFS) repointed to docs/.
- README "Interop" section rewritten — used to mention only
secsgem-py; now covers all four external validators (secsgem-py
31 / secs4java8 55 / tshark 69 frames / libFuzzer 200 k+ runs)
with a one-line summary each, plus pointers to interop/README.md
and docs/VERIFICATION.md.
- README "Deferred follow-ups" cleaned: dropped the explanatory
"Listed here so reviewers don't go looking for them in
COMPLIANCE.md and find an 'out of scope' entry that sounds
defensive" sentence — the section header speaks for itself.
- docs/00_index.md "Where the rest of the docs live" table: dropped
every `../` prefix since the docs are now siblings.
- docs/01_what_is_secs_gem.md PROOFS reference updated to sibling.
- docs/02_the_cast.md INTEGRATION + MES_INTEROP refs updated to
siblings; dropped the stale "at the repo root" wording.
- interop/README.md: VERIFICATION + PROOFS refs updated to
../docs/X.md; stale "~24 + 4 checks" updated to 31 (matches
PROOFS.md and README).
- examples/pvd_tool/README.md: every doc cross-ref now points at
../../docs/X.md.
- Source / data / CI comments mentioning doc names (e.g.
"INTEGRATION.md §3", "COMPLIANCE.md gap") rewritten to
"docs/INTEGRATION.md §3" etc. — affects 9 files across
include/, apps/, tests/, data/, examples/, .gitea/workflows/.
Verified: full build under docker passes, 445/445 test cases pass,
2 753/2 753 assertions pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
02 — The cast of characters: equipment, EAP, MES, fab planner, AMHS,
operator. Who initiates which conversation, why the equipment is
the passive side of HSMS by convention, how the AMHS handshake is
out-of-band relative to SECS. Cross-references the relevant
namespace and test files for each actor.
03 — Vocabulary + a wafer's journey: follows one 300 mm wafer
end-to-end through a fab and labels every SECS message and acronym
that fires. Introduces SVID / DVID / ECID / CEID / RPTID / ALID /
PPID / MDLN / SOFTREV / HCACK / ALCD / OFLACK / CAACK / SMACK / etc.
in context rather than as a list. Includes one-screen reference
tables for the remaining acknowledge codes, T-timers in all four
contexts (HSMS / SECS-I / E84 / E30 communication state), and a
stream-by-stream summary.
Part 1 (Foundations) of the guided tour is now complete — a reader
who reads chapters 01–03 can describe the protocol stack, identify
the actors, and recognise every acronym they'll meet in Part 2.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A linear teach-from-zero tutorial that walks both SECS/GEM as a
protocol family and this codebase as an implementation. Each
chapter explains a SEMI concept and shows where it lives in code,
so a reader builds a mental model of the standards and the
repository simultaneously.
Structure (24 chapters across 5 parts):
- Part 1 (3 ch) — Foundations: what SECS/GEM is, the cast of
characters, vocabulary + a wafer's end-to-end journey
- Part 2 (10 ch) — Standards in detail: E5, E37, E4, E30,
E40+E94, E87, E90+E157, E116+E120+E39, E84, E42+E148+S9
- Part 3 (7 ch) — Codebase: repository tour, spec-as-data + codegen,
stores, transport, codec, state machines, persistence
- Part 4 (2 ch) — Operations: build/run/demo, integration
- Part 5 (2 ch) — Reference: API + messages + YAML, extension guide
Published in this commit:
- 00_index.md — guide layout, audience map, reading paths,
conventions, status table
- 01_what_is_secs_gem.md — the N×M integration problem, what SECS
vs. HSMS vs. GEM each actually refer to, the GEM 300 suite, the
transport→message→behaviour layering, where each layer lives in
this codebase, an end-to-end S2F17/F18 example
Chapters publish iteratively from here.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raphael