Initial commit: C++20 SECS-II / HSMS / GEM client + server

A fully containerised SECS/GEM toolchain. Single docker compose project,
no host build tools. 63 unit-test cases / 278 assertions, two demo
executables, end-to-end two-container demo exercising every implemented
capability.

Architecture (bottom-up):

  secs2/   E5 SECS-II codec
    Item        variant over L/A/B/BOOLEAN/I1-8/U1-8/F4/F8
    encode/decode  big-endian, 1/2/3-byte length encoding
    Message     SxFy + W-bit + optional root item
    to_sml      human-readable text rendering

  hsms/    E37 HSMS transport (TCP)
    Header      10-byte header + SType enum (Data/Select/Deselect/
                Linktest/Reject/Separate)
    Frame       4-byte length prefix + payload encode/decode
    Connection  async Asio TCP, NOT-SELECTED -> SELECTED state machine,
                T3/T5/T6/T7/T8 timers, system-bytes reply correlation,
                graceful close-after-flush separation

  endpoint  active Client (connect with T5 retry) and passive Server
            (accept loop) wrappers over Connection

  gem/     E30 GEM logic
    ControlStateMachine  5-state E30 control model with operator
                         actions, host requests, SEMI-mandated ack
                         codes (OnlineAck, OfflineAck, CommAck), and
                         a state-change handler
    EquipmentDataModel   in-memory dictionary: SVIDs, DVIDs, ECIDs
                         (with EAC), CEIDs, report defs, CEID->report
                         links, enabled-events set, alarm table
                         (ALCD, enabled, active), process programs,
                         host command registry, clock (16-char
                         YYYYMMDDhhmmsscc with offset)
    messages.hpp         builders + parsers for every SxFy below

GEM message coverage (full list):

  S1F1/F2    Are You There / On Line Data
  S1F3/F4    Selected Equipment Status Request / Data
  S1F11/F12  Status Variable Namelist Request / Data
  S1F13/F14  Establish Communications (+ CommAck)
  S1F15/F16  Request OFFLINE (+ OfflineAck)
  S1F17/F18  Request ONLINE (+ OnlineAck)
  S2F13/F14  Equipment Constant Request / Data
  S2F15/F16  EC Send + EquipmentAck (Accept/UnknownEcid/Busy/OutOfRange)
  S2F17/F18  Date and Time Request / Data
  S2F29/F30  Equipment Constant Namelist Request / Data
  S2F31/F32  Date and Time Set Request / TimeAck
  S2F33/F34  Define Report + DefineReportAck (5 enum values)
  S2F35/F36  Link Event Report + LinkEventAck
  S2F37/F38  Enable / Disable Event Report + EnableEventAck
  S2F41/F42  Host Command + HostCmdAck (7 values) + per-param CPACKs
  S5F1/F2    Alarm Report Send + AlarmAck (ALCD bit-7 set/cleared
             + lower-7 category)
  S5F3/F4    Enable/Disable Alarm Send + AlarmAck
  S5F5/F6    List Alarms Request / Data (active alarms tagged in ALCD)
  S6F11/F12  Event Report Send (equipment-initiated CEID emission
             with full report data) + EventReportAck
  S7F3/F4    Process Program Send + ProcessProgramAck (7 values)
  S7F5/F6    Process Program Request / Data
  S7F19/F20  Current EPPD List Request / Data
  S10F1/F2   Terminal Display Single (host->equipment) + TerminalAck
  S10F3/F4   Terminal Display Single (equipment->host)

Demo apps:

  apps/secs_server.cpp   passive equipment. Populates the data model
                         with 3 SVIDs (ControlState, Clock,
                         EventsEnabled), 2 ECIDs, 3 CEIDs
                         (ControlStateChanged, AlarmSetEvent,
                         ProcessStarted), 2 alarms (Chiller Temp High
                         cat 4, Door Open cat 1), 2 recipes
                         (RECIPE-A, RECIPE-B), and 4 host commands
                         (START, STOP, PAUSE, FAULT). Emits S6F11 on
                         every control state transition + on START;
                         emits S5F1 + the AlarmSetEvent CEID on FAULT.
                         Pushes an S10F3 welcome message when the host
                         comes online.

  apps/secs_client.cpp   active host. Walks 17 steps: Establish ->
                         Online -> S1F11 SVID namelist -> S1F3 read ->
                         S2F29 EC namelist -> S2F13 read ->
                         S2F17 clock -> S2F33/S2F35/S2F37 dynamic
                         event subscription -> S2F41 START
                         (-> receives S6F11) -> S5F5 alarm list ->
                         S5F3 enable alarm 1 -> S2F41 FAULT
                         (-> receives S5F1 + S6F11) -> S7F19/S7F5
                         recipe list + body -> S10F1 terminal ->
                         S1F15 Offline -> Separate. Handles inbound
                         S6F11, S5F1, S10F3 primaries.

Testing:

  tests/test_secs2.cpp         codec round-trip for every format,
                               byte-layout assertions for known values,
                               truncation/trailing-byte rejection,
                               nested list round-trip, SML rendering
  tests/test_hsms.cpp          header byte layout, data + control
                               header round-trip, full frame round-
                               trip with length prefix, short-payload
                               rejection
  tests/test_control_state.cpp every (state, event) pair in the E30
                               control state machine, including
                               AlreadyOnline / NotAccept rejections
                               and idempotent offline-while-offline
  tests/test_data_model.cpp    SVID/ECID/Alarm/Recipe CRUD, clock
                               format + parse, host command registry,
                               full event-report pipeline (define ->
                               link -> enable -> compose) with
                               every error path (InvalidVid,
                               UnknownCeid, UnknownRptid), alarm
                               set/clear with ALCD bit-7 semantics
  tests/test_messages.cpp      round-trip + byte-layout for every
                               builder/parser pair, including S6F11
                               event reports with mixed item types

Toolchain:

  Dockerfile          Ubuntu 24.04, g++-13, CMake, Ninja, libasio-dev
  docker-compose.yml  builder / tests / server / client services,
                      source bind-mounted, build artifacts in a
                      named volume so the host tree stays clean
  CMakeLists.txt      C++20, -Wall -Wextra -Wpedantic, standalone
                      Asio (ASIO_STANDALONE), doctest via FetchContent

Documentation:

  README.md           architecture, quick start, demo log
  COMPLIANCE.md       honest per-capability E5/E30/E37 audit with
                      spec section refs. Calls out what's implemented,
                      what's partial (Reject.req, Alarms missing F7/F8,
                      EC range validation, PP without verify, terminal
                      single-line only), and what's intentionally not
                      yet implemented (spooling, S9 error stream,
                      Documentation S1F19/F20+F21/F22, limits monitoring,
                      trace data collection, multi-block, material
                      movement). Does NOT claim "100% GEM-compliant" and
                      lists the work required to honestly make that claim.

This is Layer 0 + the start of Layer 1 from implementation_plan.md.
The transition-table-driven "spec-as-data" architecture (Layer 1
proper) is not yet implemented; the current code uses imperative
state machines that are structurally ready to be refactored onto
tables.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

implementation_plan.md

@@ -0,0 +1,211 @@
+# SECS/GEM Spec-as-Data Project — Implementation Plan
+
+A layered plan for building a SECS/GEM toolchain whose core asset is the SEMI behavioral spec encoded as machine-readable transition tables. From that one artifact you derive a runtime stack, a passive wire analyzer, a conformance test generator, and a simulator. The plan is sequenced so each layer is independently useful and ships value on its own — you can stop after Layer 2 and still have a shipped product.
+
+## Guiding principles
+
+A few discipline rules that hold across every layer:
+
+- **One source of truth.** The spec tables are the only place behavioral rules live. Runtime, tests, analyzer, simulator, and docs all *read* from them; none of them re-encode the spec.
+- **Data, not code.** Tables are versioned data files (YAML/TOML/JSON), not classes. Adding GEM300 standards or new spec revisions is additive — drop in another table file.
+- **One mutation point.** Every state change in every runtime/simulator goes through one validated apply-step that emits the collection event as a side effect. No alternate paths.
+- **Exhaustive coverage, including rejections.** Every `(machine, state, event)` pair has an explicit entry. No defaults that silently swallow.
+- **Trust ladder.** Ship the lowest-trust-barrier artifact first (the passive analyzer touches no wafer), build credibility, then ascend to higher-stakes artifacts (simulator, runtime).
+- **Validate the pain before building each layer.** Talk to integration engineers between layers. If they don't unprompted complain about what the next layer solves, don't build it.
+
+## Layer 0 — Foundations (Weeks 1–4)
+
+Goal: establish the data model and conventions everything else depends on. No user-facing artifact yet.
+
+### Deliverables
+
+- **Transition table schema.** A formal schema (JSON Schema or similar) for rows of the form:
+  ```
+  machine, state, event, guard, result, new_state, emitted_messages, emitted_ceids, ack_code, notes, spec_ref
+  ```
+  Include `spec_ref` (e.g. `E30 §6.5.2`) on every row so the table is auditable line-by-line against the standard.
+- **SECS-II message codec.** The byte-level encoder/decoder for SECS-II data items (L, A, B, U1/U2/U4, I1/I2/I4, F4/F8, etc.) and the SxFy header. This is mechanical, well-specified, and reusable by every layer.
+- **HSMS framer.** TCP framing, SELECT/DESELECT/SEPARATE control messages, T-timer constants. Codec layer only — no state machine yet.
+- **Project skeleton.** Language choice (Rust is the recommendation — memory-safe, good for the long-lived stack, easy FFI later; Python a viable alternative for faster iteration). CI, lint, test harness, fuzz harness for the codec.
+
+### Acceptance for this layer
+
+You can round-trip arbitrary SECS-II messages to bytes and back, you can read/write HSMS frames against a loopback socket, and you can load a tiny example transition table and query it.
+
+## Layer 1 — Spec encoding (Weeks 5–12, ongoing thereafter)
+
+Goal: encode the E30 GEM behavioral spec as transition tables. This is the real intellectual work and the moat.
+
+### Approach
+
+- Start with the **communication state machine** (HSMS): SELECT/DESELECT/SEPARATE handshake, T3/T5/T6/T7/T8 timers, NOT_CONNECTED ↔ CONNECTED ↔ SELECTED.
+- Then the **control state machine**: the OFFLINE sub-states, ONLINE LOCAL/REMOTE, all transitions with their triggers and rejections (HCACK codes).
+- Then the **processing/equipment state machine**: IDLE → SETUP → READY → EXECUTING → PAUSE.
+- Then the **event/report configuration**: S2F33 (define report), S2F35 (link to CEID), S2F37 (enable), and the S6F11 emission rule.
+- Then **alarm management** (S5F1/S5F3) and **remote commands** (S2F41/S2F42 with the full HCACK enumeration).
+- Then **spooling**: the SPOOL state machine, queue policy, transmit-on-reconnect ordering.
+
+For each subsection: write rows for every legal transition *and* every illegal one (with the spec-mandated rejection/ACK code). Cite the spec section in `spec_ref`. Add a `notes` field whenever the spec is ambiguous, so the choice is recorded.
+
+### Tooling for this layer
+
+- A **table linter**: every `(machine, state, event)` pair must have either a transition row or an explicit "ignore"/"reject" row. The linter fails CI on missing pairs. This is what guarantees exhaustive coverage.
+- A **table-to-Markdown renderer**: generate human-readable state tables and diagrams from the data, so reviewing the encoding against the SEMI document is tractable.
+
+### Acceptance
+
+The base E30 tables are complete, lint-clean, and human-reviewable against the SEMI document. You haven't built any runtime yet, but you have the asset everything else generates from.
+
+## Layer 2 — Passive wire analyzer (Weeks 9–16, overlaps Layer 1)
+
+Goal: ship the first user-facing artifact. The "Wireshark for SECS, but understands the state machines." Zero trust barrier, vendor-neutral, immediately useful.
+
+### Architecture
+
+```
+pcap file / live capture / log file
+        │
+        ▼
+   HSMS framer  →  SECS-II decoder  →  message stream
+                                            │
+                                            ▼
+                      ┌──────────────────────────────────┐
+                      │   passive state reconstructor    │
+                      │   (runs the transition tables    │
+                      │    on observed messages, in      │
+                      │    "observer" mode — never       │
+                      │    sends anything)               │
+                      └──────────┬───────────────────────┘
+                                 ▼
+              ┌──────────────────────────────────────────┐
+              │   diagnostic engine                       │
+              │   - reports protocol violations           │
+              │   - explains rejections ("S2F41 rejected │
+              │     because control was LOCAL")          │
+              │   - explains silences ("CEID 12 fired   │
+              │     but report 5 was never enabled")    │
+              │   - flags timeout misses                 │
+              └──────────┬───────────────────────────────┘
+                         ▼
+                    UI (web or TUI):
+                    - timeline of messages
+                    - live state-machine view
+                    - time-travel scrubbing
+                    - golden-trace diff
+```
+
+### Deliverables
+
+- **Passive reconstructor.** Consumes a SECS message stream and runs both endpoints' state machines in parallel (one for host, one for equipment), inferring state from observed messages.
+- **Diagnostic engine.** Knows every legal transition (from the tables) and flags violations with human-readable explanations citing the spec reference.
+- **UI.** Web UI is the right call (cross-platform, easy to share, screenshot-able for support tickets). Timeline + state panels + scrubber + filterable message inspector.
+- **Capture options:** read pcap, attach to a live HSMS connection in tap mode (e.g. port mirror or local proxy that forwards and tees), import vendor log files (start with the secsgem Python format and add formats as users request).
+
+### Acceptance
+
+An integration engineer can drop a captured session in, see exactly why the bring-up failed, and point a senior at the rendered explanation rather than the raw log. This is the credibility-building artifact — open-source it.
+
+### Suggested name worth squatting
+
+Something like `secscope` or `gemtrace`. Pick early; matters for adoption.
+
+## Layer 3 — Simulator (Weeks 16–24)
+
+Goal: an active GEM-compliant simulator usable as (a) a virtual equipment for host developers, (b) a virtual host for equipment developers, and (c) the engine that drives Layer 4's test generator.
+
+### Approach
+
+The simulator is the transition tables, run in *active* mode rather than observer mode. The same dispatcher/event-queue architecture, but now it originates messages instead of just observing them. Key additions over the analyzer:
+
+- **Scriptable scenarios.** A small DSL or just YAML for "carrier arrives at LP2, host issues START, after 30s a wafer-complete event fires." Lets users script reproducible test situations.
+- **Equipment-specific data dictionary.** SVID/ECID/CEID/DVID definitions loaded from a config file per simulated tool. Default to a generic minimal tool; allow users to load richer ones.
+- **Fault injection.** Drop the link mid-transaction, delay a reply past T3, return malformed messages, send out-of-spec ACK codes. This is what makes the simulator valuable for hardening host implementations.
+- **Replay mode.** Take a captured session and replay it as either side. Enables "develop your host against last week's tool session."
+
+### Acceptance
+
+A host developer can run the simulator locally and develop against it without owning a physical tool, and an equipment developer can point their tool at it as a fake host for bring-up testing.
+
+## Layer 4 — Conformance test generator (Weeks 20–28)
+
+Goal: from the tables, mechanically generate the exhaustive conformance test suite (especially the negative cases) and run it against any implementation.
+
+### How it works
+
+For each row in the transition table, the generator emits a test case consisting of:
+
+1. **Setup:** the sequence of messages needed to drive the system into `state`. Computed by graph-search over the table — find a shortest path from the initial state to the target state, using only known transitions.
+2. **Stimulus:** the `event` (a message to send, or an internal trigger the simulator can fake).
+3. **Assertions:** the expected `result`, `emitted_messages`, `ack_code`, and resulting state — all read directly from the row.
+
+The generator runs as a host (or equipment) using the Layer 3 simulator engine, connected over HSMS to the system under test, and produces a structured report: pass/fail per row, with the spec reference cited on every failure.
+
+### Negative-case coverage
+
+The win is that the table contains every illegal `(state, event)` pair with its mandated rejection. The generator emits a test for each. This is the coverage humans skip out of tedium — sending S2F41 in every non-REMOTE state and asserting the correct HCACK comes back, for instance.
+
+### Equipment-specific discovery
+
+For the parts of the test suite that need tool-specific knowledge (SVID list, CEID list, recipe names), the generator first runs a discovery phase: S1F11 for SVID names, S1F13 for capabilities, S7F19 for recipe names, and so on. The discovered dictionary is merged with the generic tables to produce the full test plan.
+
+### Acceptance
+
+Point the generator at an implementation; get a pass/fail report against the full E30 (and later GEM300) behavioral spec, including all negative cases, with spec references on every failure.
+
+## Layer 5 — GEM300 extensions (Months 6–12)
+
+Goal: add E87 (carrier management), E90 (substrate tracking), E40 (process jobs), E94 (control jobs), E116 (EPT) as additional table files.
+
+Because the architecture is additive — each standard is another set of transition rows plus possibly another machine in the dispatcher — adding GEM300 is wiring, not surgery. Sequence:
+
+1. **E90 first** (substrate tracking) — applies to almost every tool, smallest dependency surface.
+2. **E87** (carrier management) — only for tools that handle carriers, but the most asked-for after E90.
+3. **E40/E94** (jobs) — adds the largest new state machines; do them as a pair since E94 references E40.
+4. **E116** (EPT) — comparatively simple state model, valuable for fab metrics customers.
+
+Each addition automatically gets analyzer support (new states render in the UI), simulator support (new scenarios scriptable), and conformance tests (new rows → new tests).
+
+## Layer 6 — Spec revision diff and impact reporter (Month 9+)
+
+Goal: when a new SEMI revision lands, produce a machine-generated report of exactly what changes for any given implementation.
+
+### Mechanism
+
+- Maintain one table file per spec revision (`e30-0307.yaml`, `e30-0712.yaml`, etc.).
+- A **table diff tool** computes structural diffs: which rows added/removed/modified, which ACK codes changed, which transitions newly legal/illegal.
+- An **impact reporter** re-runs the Layer 4 conformance generator against a customer's implementation using the new tables and produces a focused report: "these N behaviors must change to remain compliant under revision X; here are the spec references and the exact transitions affected."
+
+### Honest framing
+
+This is not "live" in the sense of auto-detecting that SEMI published something — a human still encodes each new revision into the table. The value is converting "your implementation drifted out of compliance, found out at a fab acceptance test" into "here is a precise diff the day you load the new table." That conversion is the product.
+
+## Layer 7 (optional) — Runtime stack for equipment makers
+
+Goal: a production-grade SECS/GEM runtime that equipment vendors embed in their tools. This is the high-trust, slow-cycle, sales-heavy artifact — only worth attempting after Layers 2–4 have built credibility and identified the right customer segment.
+
+The runtime is the same dispatcher/event-queue executing the same tables, but in production mode: persistent spool, real timers, OS integration, supported APIs (Rust/C/C++/Python/.NET bindings via FFI). Skip this layer entirely if the project stays a side project; it's where commercial competition is hardest.
+
+## Sequencing summary
+
+| Months | Focus | User-visible artifact |
+|---|---|---|
+| 1 | Foundations (codec, framer, table schema) | none |
+| 2–3 | Encode E30 base tables | none (internal asset) |
+| 3–4 | Passive analyzer MVP | **Open-source release: SECS analyzer** |
+| 5–6 | Simulator | simulator alongside analyzer |
+| 6–7 | Conformance test generator | conformance reports |
+| 7–9 | GEM300 (E90, E87) | analyzer + tests cover GEM300 |
+| 9–12 | E40/E94/E116 + spec-revision diff | full GEM300 coverage + impact reports |
+| 12+ | Optional: commercial runtime | (if pursued) |
+
+## Risks and how each layer mitigates them
+
+- **Wrong-table risk** (you encode the spec wrong, confidently emit wrong tests): mitigated by the `spec_ref` requirement and table-to-Markdown review, plus community review once open-sourced.
+- **Adoption risk** (nobody uses it): mitigated by leading with the analyzer — low trust barrier, immediately useful, no commitment.
+- **Scope risk** (GEM300 is huge): mitigated by additive architecture and only adding standards customers actually request.
+- **Incumbent response risk** (a Cimetrix/PEER builds the analyzer themselves): mitigated by being open-source and vendor-neutral, which they structurally won't match.
+- **Solo-bandwidth risk**: every layer is independently shippable. If life happens after Layer 2, the analyzer is still a real contribution.
+
+## First concrete next step
+
+Before any code: pick three integration engineers at equipment vendors (your IMS network has adjacencies) and ask them, unprompted, *"what's the worst part of bringing up a SECS/GEM interface?"* If "blind debugging" and "compliance test maintenance" come back without you naming them, the wedge is validated. Then start Layer 0.