raphael
dae6bfd747
Tone pass across the non-tutorial markdown — README, PROOFS,
ARCHITECTURE, BENCHMARKS, COMPLIANCE, FAQ, MES_INTEROP, SECURITY,
and interop/README. Three patterns came out:
- Bug-history war stories ("Past interop sweeps surfaced…",
"What these harnesses caught: 1. Strict U-width parsing…").
- Chat-with-reader framing ("Don't skip TLS unless…", "Treat as a
punch list", "If you're running in a pod…", "Misconfiguration
incidents drop dramatically").
- Self-referential narration ("we ship", "our codec", "the
codebase's most-tested layer", "three orders of magnitude above
fab load", "the gift that keeps giving").
README also drops the standalone ThreadSanitizer subsection under
Build details (now a single line under the new Testing section).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
244 lines
11 KiB
Markdown
244 lines
11 KiB
Markdown
# secs-gem
|
||
|
||
A C++20 SECS-II / HSMS / SECS-I / GEM / GEM 300 runtime, fully
|
||
containerized. Every behavioural rule lives in YAML; the C++ is the
|
||
engine that reads them. Implements **all of E4, E5, E30, E37
|
||
(SS + GS), E39, E40, E42, E84, E87, E90, E94, E116, E120, E148,
|
||
E157**.
|
||
|
||
> **License: proprietary — see [LICENSE](LICENSE).** No use, copy,
|
||
> compile, evaluate, benchmark, or deploy without a written license
|
||
> from the copyright holder. Contact `raphael@maenle.net` for
|
||
> commercial licensing, evaluation terms, or fab deployment.
|
||
|
||
---
|
||
|
||
## Quick start
|
||
|
||
Everything runs in Docker — no compiler or build tools on the host.
|
||
|
||
```bash
|
||
docker compose run --rm builder # configure + compile
|
||
docker compose run --rm tests # 445 cases / 2 753 assertions
|
||
docker compose up --no-deps server client # live two-container demo
|
||
```
|
||
|
||
The two-container demo walks ~24 SECS transactions end-to-end
|
||
through the data model. Watch the logs interleave.
|
||
|
||
---
|
||
|
||
## Documentation map
|
||
|
||
| File | What it covers |
|
||
|------------------------------------------------------------|-------------------------------------------------------------------------|
|
||
| [docs/](docs/00_index.md) | Guided-tour tutorial series — teach-from-zero across the protocol and the codebase |
|
||
| [docs/COMPLIANCE.md](docs/COMPLIANCE.md) | Per-capability audit against every SEMI standard implemented |
|
||
| [docs/INTEGRATION.md](docs/INTEGRATION.md) | Vendor-side tutorial: YAML → callbacks → production deploy |
|
||
| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | How the pieces fit + how to extend (new store / FSM / message) |
|
||
| [docs/PROOFS.md](docs/PROOFS.md) | The eight commands that prove the feature-completeness claim |
|
||
| [docs/VERIFICATION.md](docs/VERIFICATION.md) | Test plan for the external validators behind the proof table |
|
||
| [docs/BENCHMARKS.md](docs/BENCHMARKS.md) | Performance envelope (throughput, latency, memory) + how to re-run |
|
||
| [docs/MES_INTEROP.md](docs/MES_INTEROP.md) | Day-1 punch list to run against your commercial MES (59 test IDs) |
|
||
| [docs/SECURITY.md](docs/SECURITY.md) | Concrete configs: nftables, stunnel, minisign, SIEM audit-log schema |
|
||
| [docs/GLOSSARY.md](docs/GLOSSARY.md) | SEMI vocabulary: SVID, CEID, PPID, ALCD, HCACK, T-timers, … |
|
||
| [docs/FAQ.md](docs/FAQ.md) | Common questions and their canonical answers |
|
||
| [examples/pvd_tool/](examples/pvd_tool/) | Worked example: a realistic fictional PVD tool, YAML + C++ wiring |
|
||
| [LICENSE](LICENSE) | Proprietary license terms |
|
||
|
||
---
|
||
|
||
## Testing
|
||
|
||
- **Unit + integration** — `docker compose run --rm tests` runs 445
|
||
cases / 2 753 assertions across every store, FSM, codec, parser, and
|
||
persistence path.
|
||
- **Live conformance harness** — 47 wire-level checks against the
|
||
passive server.
|
||
- **Interop** — round-trips against secsgem-py (31 checks), secs4java8
|
||
(55 checks), and Wireshark's HSMS dissector (69 frames, 0 malformed).
|
||
- **Soak + fuzz** — 100 000-op property test; libFuzzer with ASan +
|
||
UBSan over `secs2::decode` and the SML parser, 0 crashes.
|
||
- **Config validation** — `secs_server --validate-config` rejects
|
||
malformed YAML before startup.
|
||
- **CI** — [Gitea Actions](.gitea/workflows/ci.yml) runs the full
|
||
suite plus a `-fsanitize=thread` lane on every push to `main`; all
|
||
445 cases pass clean under TSan.
|
||
|
||
Exact commands, exit codes, and per-standard test counts are in
|
||
[docs/PROOFS.md](docs/PROOFS.md); the rationale behind the external
|
||
validators is in [docs/VERIFICATION.md](docs/VERIFICATION.md).
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
The project is **spec-as-data**: the SEMI behavioural rules live in
|
||
YAML; the C++ is the engine that reads them.
|
||
|
||
```
|
||
┌──────────────────────────────────────────────────────────────┐
|
||
│ data/ │
|
||
│ messages.yaml SECS-II message catalog (164 msgs) │
|
||
│ control_state.yaml E30 §6.2 control transition table │
|
||
│ process_job_state.yaml E40 §6 PJ transition table │
|
||
│ control_job_state.yaml E94 §6 CJ transition table │
|
||
│ equipment.yaml SVIDs / DVIDs / ECIDs / CEIDs / │
|
||
│ alarms / recipes / commands │
|
||
└──────────────────────┬───────────────────────────────────────┘
|
||
│ (codegen at build, YAML loaded at startup)
|
||
▼
|
||
┌──────────────────────────────────────────────────────────────┐
|
||
│ apps/ │
|
||
│ secs_server passive equipment secs_bench perf │
|
||
│ secs_client active host secs_conformance │
|
||
│ secs_interop_probe │
|
||
└──────────────────────────────────────────────────────────────┘
|
||
|
||
secsgem::config loader.hpp + validate.hpp:
|
||
YAML -> data model, with multi-error validator
|
||
surfacing every issue at once (`--validate-config`)
|
||
secsgem::gem per-standard FSM + per-store persistence
|
||
(every store accepts v ∈ [1, kVersion] for
|
||
forward-compatible schema migrations).
|
||
EquipmentDataModel composes all stores.
|
||
Router (stream, function) -> handler.
|
||
Generated messages.hpp covers 164 SxFy.
|
||
secsgem::hsms Connection (Asio): HSMS-SS + HSMS-GS, all
|
||
T-timers enforced, auto S9F3/F5/F7/F9/F11.
|
||
secsgem::secsi SECS-I Protocol FSM (E4): T1/T2/T3/T4 enforced
|
||
in-FSM, TCP transport for tunnel testing.
|
||
secsgem::secs2 Item (variant), encode/decode, Message,
|
||
SML parser/printer.
|
||
secsgem::metrics Prometheus exporter (Registry + HTTP server).
|
||
```
|
||
|
||
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for how to extend
|
||
it (new store / FSM / message).
|
||
|
||
---
|
||
|
||
## Adding a capability
|
||
|
||
The point of "spec-as-data" is that adding behaviour almost never
|
||
requires a C++ change.
|
||
|
||
### New SVID
|
||
```yaml
|
||
# data/equipment.yaml
|
||
svids:
|
||
- {id: 4, name: ChamberTemp, units: "C", type: U4, value: 25}
|
||
```
|
||
|
||
### New host command with side effects
|
||
```yaml
|
||
host_commands:
|
||
- {name: VENT, ack: Accept, emit_ceid: 400, set_alarm: 2}
|
||
```
|
||
|
||
### New state transition
|
||
```yaml
|
||
# data/control_state.yaml
|
||
transitions:
|
||
- {from: OnlineRemote, on: host_request_offline, to: EquipmentOffline, ack: Accept}
|
||
```
|
||
|
||
### New SECS-II message
|
||
```yaml
|
||
# data/messages.yaml
|
||
- id: S6F30
|
||
stream: 6
|
||
function: 30
|
||
w: true
|
||
builder: s6f30_something
|
||
parser: parse_s6f30
|
||
body:
|
||
kind: list
|
||
struct_name: Something
|
||
fields:
|
||
- {name: field_a, shape: {kind: scalar, item_type: U4}}
|
||
- {name: field_b, shape: {kind: scalar, item_type: ASCII}}
|
||
```
|
||
|
||
`docker compose run --rm builder` regenerates `messages.hpp`. The
|
||
typed builder, parser, and struct definition appear automatically.
|
||
Run `--validate-config` after every YAML edit.
|
||
|
||
---
|
||
|
||
## Production deployment
|
||
|
||
See [docs/INTEGRATION.md](docs/INTEGRATION.md) for the full
|
||
vendor-side tutorial — wiring sensors, plugging FSMs into the tool,
|
||
persistence layout, monitoring/observability, HSMS-GS multi-MES
|
||
setup.
|
||
|
||
See [docs/SECURITY.md](docs/SECURITY.md) for concrete nftables /
|
||
stunnel / minisign / SIEM configs.
|
||
|
||
See [docs/BENCHMARKS.md](docs/BENCHMARKS.md) for the performance
|
||
envelope — roughly **140 k req/s S1F1**, **79 k req/s S1F3 (32
|
||
SVIDs)**, **572 k S6F11/s push**, **~450 bytes per PJ+CJ pair**.
|
||
Three orders of magnitude above typical fab tool load.
|
||
|
||
See [docs/MES_INTEROP.md](docs/MES_INTEROP.md) for the day-1 punch
|
||
list to run against your commercial MES before promoting from
|
||
staging to a real tool.
|
||
|
||
### Operational runbook (starter)
|
||
|
||
| Incident | First check | Mitigation |
|
||
|-------------------------------------|--------------------------------------|-------------------------------------------|
|
||
| HSMS connection flapping | T7 / T6 timer fires in logs | check MES reachability, network MTU |
|
||
| Spool depth growing | host MES connectivity / ACK rate | force-drain via S6F23, escalate to MES |
|
||
| State machine "stuck" | last state-change handler log line | host-issued offline + re-establish |
|
||
| Alarm storm | `AlarmRegistry::all()` snapshot | check upstream sensor; quench via S5F3 |
|
||
| Persistence dir growing unbounded | `du -s` + file count | sweep terminal-state records |
|
||
| Cross-tool inconsistency | `secsgem_tests` on canary tool | compare wire trace vs validator |
|
||
|
||
---
|
||
|
||
## Deferred follow-ups
|
||
|
||
- **asio `serial_port` adapter for SECS-I.** `secsi::Protocol` is
|
||
tested end-to-end over `secsi::TcpTransport`; the matching serial
|
||
driver isn't written yet. Mirror `TcpTransport` to add it.
|
||
|
||
---
|
||
|
||
## Build details
|
||
|
||
The toolchain image (`Dockerfile`) is Ubuntu 24.04 with `g++-13`,
|
||
CMake, Ninja, `libasio-dev`, `libyaml-cpp-dev`, and Python 3 for the
|
||
codegen. doctest is fetched via CMake FetchContent. Build artifacts
|
||
live in a named Docker volume so the host filesystem stays clean.
|
||
|
||
Standalone Asio is used in header-only mode (`ASIO_STANDALONE`). No
|
||
Boost dependency.
|
||
|
||
---
|
||
|
||
## Interop
|
||
|
||
Four independent external validators cross-check the codebase:
|
||
|
||
- **secsgem-py 0.3.0** (Python reference impl) — three harnesses
|
||
under `interop/`: secsgem-py active host driving the C++ passive
|
||
server (31 checks), C++ active host probing secsgem-py's passive
|
||
equipment, and a raw GEM 300 harness round-tripping S3 / S14 /
|
||
S16 / S12 through hand-crafted `SecsStreamFunction` subclasses.
|
||
- **secs4java8** (independent Java SECS implementation) — 55
|
||
cross-validation checks covering S1/S2/S3/S5/S6/S7/S10/S14/S16,
|
||
full-body GEM 300 shapes, S2F49 enhanced commands, S5F13–F18
|
||
exception recovery.
|
||
- **Wireshark / tshark HSMS dissector** (independent network-protocol
|
||
authors) — 69 HSMS frames dissected on a recorded pcap, no
|
||
malformed-packet warnings.
|
||
- **libFuzzer + ASan + UBSan** — 200 000+ inputs through
|
||
`secs2::decode` and 1.4 M+ through `try_parse_sml` per 60 s lane,
|
||
0 crashes.
|
||
|
||
See [`interop/README.md`](interop/README.md) for harness-by-harness
|
||
detail and [docs/VERIFICATION.md](docs/VERIFICATION.md) for the test
|
||
plan rationale.
|