# secs-gem A C++20 SECS-II / HSMS / SECS-I / GEM / GEM 300 runtime, fully containerized. Every behavioural rule lives in YAML; the C++ is the engine that reads them. Implements **all of E4, E5, E30, E37 (SS + GS), E39, E40, E42, E84, E87, E90, E94, E116, E120, E148, E157**. > **License: proprietary — see [LICENSE](LICENSE).** No use, copy, > compile, evaluate, benchmark, or deploy without a written license > from the copyright holder. Contact `raphael@maenle.net` for > commercial licensing, evaluation terms, or fab deployment. --- ## Proof of feature-completeness "Feature-complete" is a claim that the code must prove, not the README. These five commands are the proof. If they all exit zero on a fresh clone, the codebase implements what [COMPLIANCE.md](COMPLIANCE.md) claims. | # | Command | What it proves | |---|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------| | 1 | `docker compose run --rm tests` | **426 test cases / 2 557 assertions** pass: every store, FSM, codec, parser, persistence path | | 2 | `docker compose run --rm builder /app/build/secs_conformance --host server --port 5000` | **47 wire-level conformance checks** PASS against a live passive equipment | | 3 | `docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py --host server` | **24 interop checks** PASS against secsgem-py 0.3.0 (the Python reference impl) | | 4 | `SECSGEM_ROBUSTNESS_SOAK=1 docker compose run --rm builder /app/build/secsgem_tests -tc='*soak*'` | **100 000 random tool operations** execute with all invariants and persistence round-trips holding | | 5 | `docker compose run --rm builder /app/build/secs_server --validate-config --config /app/data/equipment.yaml --state-table /app/data/control_state.yaml --pj-state-table /app/data/process_job_state.yaml --cj-state-table /app/data/control_job_state.yaml` | Every shipped YAML config passes structural + referential validation | | 6 | `docker compose run --rm builder bash /app/interop/tshark_validate.sh` | **69 HSMS frames** dissected by Wireshark's HSMS dissector (independent third codec) with no malformed packets | | 7 | `bash interop/secs4j_validate.sh` | **20 cross-validation checks** PASS against [secs4java8](https://github.com/kenta-shimizu/secs4java8) (independent Java implementation), covering full GEM 300 body shapes (S3, S14, S16) plus S2F49 / S5F13 / S2F23 | Plus, on every push to `main`, [Gitea Actions](.gitea/workflows/ci.yml) runs both a **Release build + full test suite** and a separate **ThreadSanitizer lane** that builds with `-fsanitize=thread` and fails on any race. All 426 cases / 2 557 assertions pass under TSan clean. ### Per-standard test coverage Every claimed standard has dedicated tests. Counts are `grep -c TEST_CASE`; cross-cutting tests (e.g. `test_robustness_fuzz`, `test_gem300_scenario`) exercise multiple standards in concert. | Standard | Test files | Cases | |-----------------------------------|-------------------------------------------------------------------------------------------|------:| | **E5** — SECS-II encoding | `test_secs2`, `test_sml`, `test_messages`, `test_identifier_wildcards`, `test_fuzz` | 120 | | **E5 §13** — exceptions | `test_exceptions`, `test_exception_persistence` | 16 | | **E4** — SECS-I transport | `test_secsi`, `test_secsi_timers`, `test_secsi_tcp` | 27 | | **E37** — HSMS (SS + GS) | `test_hsms`, `test_hsms_connection`, `test_hsms_timers`, `test_hsms_s9`, `test_hsms_gs`, `test_hsms_gs_integration`, `test_s9_fallback`, `test_concurrency` | 34 | | **E30** — GEM core | `test_control_state`, `test_communication_state`, `test_host_handler`, `test_data_model`, `test_loader`, `test_config_validate` | 71 | | **E40** — process jobs | `test_process_jobs` | 21 | | **E94** — control jobs | `test_control_jobs` | 9 | | **E42** — formatted PP | `test_e42_formatted_pp` | 6 | | **E87** — carriers + load ports | `test_carriers`, `test_carrier_state`, `test_carrier_persistence`, `test_e87_wire_scenarios` | 27 | | **E90** — substrate tracking | `test_substrates`, `test_substrate_persistence` | 21 | | **E116** — EPT | `test_ept` | 7 | | **E120 / E39** — common equip / object service | `test_cem_objects` | 3 | | **E157** — module process tracking | `test_modules` | 5 | | **E84** — parallel I/O + timers | `test_e84`, `test_e84_ports`, `test_e84_timers`, `test_e84_asio_timers` | 27 | | Persistence + cross-cutting | `test_job_persistence`, `test_persistence_upgrade`, `test_wire_ceid_emission`, `test_gem300_scenario`, `test_live_gem300`, `test_thread_safety`, `test_metrics_prometheus`, `test_robustness_fuzz` | 32 | | **Total** | | **426** | A single command to see this live: `docker compose run --rm builder /app/build/secsgem_tests --list-test-cases | wc -l` (currently 426). --- ## Quick start Everything runs in Docker — no compiler or build tools on the host. ```bash docker compose run --rm builder # configure + compile docker compose run --rm tests # 426 cases / 2 557 assertions docker compose up --no-deps server client # live two-container demo ``` The two-container demo walks ~24 SECS transactions end-to-end through the data model. Watch the logs interleave. --- ## Documentation map | File | What it covers | |-----------------------------------------------|-------------------------------------------------------------------------| | [COMPLIANCE.md](COMPLIANCE.md) | Per-capability audit against every SEMI standard implemented | | [INTEGRATION.md](INTEGRATION.md) | Vendor-side tutorial: YAML → callbacks → production deploy | | [BENCHMARKS.md](BENCHMARKS.md) | Performance envelope (throughput, latency, memory) + how to re-run | | [MES_INTEROP.md](MES_INTEROP.md) | Day-1 punch list to run against your commercial MES (60+ test IDs) | | [SECURITY.md](SECURITY.md) | Concrete configs: nftables, stunnel, minisign, SIEM audit-log schema | | [LICENSE](LICENSE) | Proprietary license terms | --- ## Architecture The project is **spec-as-data**: the SEMI behavioural rules live in YAML; the C++ is the engine that reads them. ``` ┌──────────────────────────────────────────────────────────────┐ │ data/ │ │ messages.yaml SECS-II message catalog (164 msgs) │ │ control_state.yaml E30 §6.2 control transition table │ │ process_job_state.yaml E40 §6 PJ transition table │ │ control_job_state.yaml E94 §6 CJ transition table │ │ equipment.yaml SVIDs / DVIDs / ECIDs / CEIDs / │ │ alarms / recipes / commands │ └──────────────────────┬───────────────────────────────────────┘ │ (codegen at build, YAML loaded at startup) ▼ ┌──────────────────────────────────────────────────────────────┐ │ apps/ │ │ secs_server passive equipment secs_bench perf │ │ secs_client active host secs_conformance │ │ secs_interop_probe │ └──────────────────────────────────────────────────────────────┘ secsgem::config loader.hpp + validate.hpp: YAML -> data model, with multi-error validator surfacing every issue at once (`--validate-config`) secsgem::gem per-standard FSM + per-store persistence (every store accepts v ∈ [1, kVersion] for forward-compatible schema migrations). EquipmentDataModel composes all stores. Router (stream, function) -> handler. Generated messages.hpp covers 164 SxFy. secsgem::hsms Connection (Asio): HSMS-SS + HSMS-GS, all T-timers enforced, auto S9F3/F5/F7/F9/F11. secsgem::secsi SECS-I Protocol FSM (E4): T1/T2/T3/T4 enforced in-FSM, TCP transport for tunnel testing. secsgem::secs2 Item (variant), encode/decode, Message, SML parser/printer. secsgem::metrics Prometheus exporter (Registry + HTTP server). ``` --- ## Adding a capability The point of "spec-as-data" is that adding behaviour almost never requires a C++ change. ### New SVID ```yaml # data/equipment.yaml svids: - {id: 4, name: ChamberTemp, units: "C", type: U4, value: 25} ``` ### New host command with side effects ```yaml host_commands: - {name: VENT, ack: Accept, emit_ceid: 400, set_alarm: 2} ``` ### New state transition ```yaml # data/control_state.yaml transitions: - {from: OnlineRemote, on: host_request_offline, to: EquipmentOffline, ack: Accept} ``` ### New SECS-II message ```yaml # data/messages.yaml - id: S6F30 stream: 6 function: 30 w: true builder: s6f30_something parser: parse_s6f30 body: kind: list struct_name: Something fields: - {name: field_a, shape: {kind: scalar, item_type: U4}} - {name: field_b, shape: {kind: scalar, item_type: ASCII}} ``` `docker compose run --rm builder` regenerates `messages.hpp`. The typed builder, parser, and struct definition appear automatically. Run `--validate-config` after every YAML edit. --- ## Production deployment See [INTEGRATION.md](INTEGRATION.md) for the full vendor-side tutorial — wiring sensors, plugging FSMs into the tool, persistence layout, monitoring/observability, HSMS-GS multi-MES setup. See [SECURITY.md](SECURITY.md) for concrete nftables / stunnel / minisign / SIEM configs. See [BENCHMARKS.md](BENCHMARKS.md) for the performance envelope — roughly **140 k req/s S1F1**, **79 k req/s S1F3 (32 SVIDs)**, **572 k S6F11/s push**, **~450 bytes per PJ+CJ pair**. Three orders of magnitude above typical fab tool load. See [MES_INTEROP.md](MES_INTEROP.md) for the day-1 punch list to run against your commercial MES before promoting from staging to a real tool. ### Operational runbook (starter) | Incident | First check | Mitigation | |-------------------------------------|--------------------------------------|-------------------------------------------| | HSMS connection flapping | T7 / T6 timer fires in logs | check MES reachability, network MTU | | Spool depth growing | host MES connectivity / ACK rate | force-drain via S6F23, escalate to MES | | State machine "stuck" | last state-change handler log line | host-issued offline + re-establish | | Alarm storm | `AlarmRegistry::all()` snapshot | check upstream sensor; quench via S5F3 | | Persistence dir growing unbounded | `du -s` + file count | sweep terminal-state records | | Cross-tool inconsistency | `secsgem_tests` on canary tool | compare wire trace vs validator | --- ## Build details The toolchain image (`Dockerfile`) is Ubuntu 24.04 with `g++-13`, CMake, Ninja, `libasio-dev`, `libyaml-cpp-dev`, and Python 3 for the codegen. doctest is fetched via CMake FetchContent. Build artifacts live in a named Docker volume so the host filesystem stays clean. Standalone Asio is used in header-only mode (`ASIO_STANDALONE`). No Boost dependency. ### ThreadSanitizer ```bash cmake -S . -B build-tsan -G Ninja -DCMAKE_BUILD_TYPE=Debug -DSECSGEM_TSAN=ON cmake --build build-tsan TSAN_OPTIONS=halt_on_error=1 build-tsan/secsgem_tests ``` Runs as a separate lane in CI. Catches data races in the io_context strand contract documented in INTEGRATION.md §3. --- ## Interop `interop/` contains the secsgem-py 0.3.0 cross-validation harness — secsgem-py active host driving our C++ passive server, our C++ active host probing secsgem-py's passive equipment, and a raw GEM-300 harness that round-trips S3 (E87), S14 (E94), S16 (E40), S12 (wafer maps) through hand-crafted `SecsStreamFunction` subclasses. See [`interop/README.md`](interop/README.md).