raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: README restructure + proof-of-feature-completeness section

Browse Source 
The old README mixed intro, quickstart, architecture, and 10 sections
of production deployment over 419 lines, with significant overlap
with INTEGRATION.md.  It claimed "implements every standard" without
making the claim concrete.

Restructured to ~250 lines with the proof front and center.

New top-of-README "Proof of feature-completeness" section: five
commands that, when they all exit zero on a fresh clone, prove the
COMPLIANCE.md claims.  Each command verified end-to-end before
landing in this commit:

  1. docker compose run --rm tests
       → 426 cases / 2557 assertions PASS
  2. secs_conformance --host server --port 5000
       → 47 / 47 wire-level checks PASS
  3. host_vs_cpp_server.py --host server
       → 24 secsgem-py interop checks PASS
  4. SECSGEM_ROBUSTNESS_SOAK=1 secsgem_tests -tc='*soak*'
       → 100 000 random tool operations, all invariants hold
  5. secs_server --validate-config <all four YAMLs>
       → 0 errors, 0 warnings across the shipped configs

Plus a per-standard test-coverage table mapping every claimed SEMI
standard (E5, E5 §13, E4, E37, E30, E40, E94, E42, E87, E90, E116,
E120/E39, E157, E84) to its test files and case count, summing to
426 to match the doctest totals.  Counts verified by
`grep -c TEST_CASE` per file.

CI also runs the TSan lane (separate job in
.gitea/workflows/ci.yml); README documents it under Build details.

Content moved out of README into specialized docs (eliminates
duplication):

- Security configs → SECURITY.md (was 14-line bullet list; now a
  365-line file with nftables, stunnel, minisign, SIEM schema)
- Persistence layout + monitoring + HA + deployment patterns +
  upgrade discipline + fab-stack integration → INTEGRATION.md
- Performance envelope → BENCHMARKS.md
- MES interop punch list → MES_INTEROP.md

README now reads top-to-bottom: what this is → license → proof →
quickstart → doc map → architecture → adding capabilities →
production (1-line pointers to the deep docs) → build details →
interop.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-09 15:37:33 +02:00
parent 0df229905d
commit e82f67ecad
1 changed files with 133 additions and 303 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+133 -303
View File
@@ -1,68 +1,127 @@
# secs-gem # secs-gem
A C++20 SECS-II / HSMS / SECS-I / GEM / GEM 300 runtime, fully containerized, A C++20 SECS-II / HSMS / SECS-I / GEM / GEM 300 runtime, fully
with every behavioural rule encoded as YAML data (control state, equipment containerized. Every behavioural rule lives in YAML; the C++ is the
data dictionary, E40 process-job state machine, E94 control-job state engine that reads them. Implements **all of E4, E5, E30, E37
machine, SECS-II message shapes). (SS + GS), E39, E40, E42, E84, E87, E90, E94, E116, E120, E148,
E157**.
Implements **all of E5, E30, E37 (SS + GS), E4 SECS-I, E40, E42, E84, E87,
E90, E94, E116, E120, E148, E157, E39**. Per-store persistence on every
mutable in-memory entity (spool, carriers, load-ports, substrates,
process-jobs, control-jobs, exceptions). See **[COMPLIANCE.md](COMPLIANCE.md)**
for the per-capability audit, **[INTEGRATION.md](INTEGRATION.md)** for
the vendor-side tutorial, **[BENCHMARKS.md](BENCHMARKS.md)** for the
performance envelope, and **[MES_INTEROP.md](MES_INTEROP.md)** for the
day-1 punch list to run against your commercial MES.
> **License: proprietary — see [LICENSE](LICENSE).** No use, copy, > **License: proprietary — see [LICENSE](LICENSE).** No use, copy,
> compile, evaluate, benchmark, or deploy without a written license > compile, evaluate, benchmark, or deploy without a written license
> from the copyright holder. Contact `raphael@maenle.net` for > from the copyright holder. Contact `raphael@maenle.net` for
> commercial licensing, evaluation terms, or fab deployment. > commercial licensing, evaluation terms, or fab deployment.
---
## Proof of feature-completeness
"Feature-complete" is a claim that the code must prove, not the
README. These five commands are the proof. If they all exit zero on
a fresh clone, the codebase implements what
[COMPLIANCE.md](COMPLIANCE.md) claims.
| # | Command | What it proves |
|---|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|
| 1 | `docker compose run --rm tests` | **426 test cases / 2 557 assertions** pass: every store, FSM, codec, parser, persistence path |
| 2 | `docker compose run --rm builder /app/build/secs_conformance --host server --port 5000` | **47 wire-level conformance checks** PASS against a live passive equipment |
| 3 | `docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py --host server` | **24 interop checks** PASS against secsgem-py 0.3.0 (the Python reference impl) |
| 4 | `SECSGEM_ROBUSTNESS_SOAK=1 docker compose run --rm builder /app/build/secsgem_tests -tc='*soak*'` | **100 000 random tool operations** execute with all invariants and persistence round-trips holding |
| 5 | `docker compose run --rm builder /app/build/secs_server --validate-config --config /app/data/equipment.yaml --state-table /app/data/control_state.yaml --pj-state-table /app/data/process_job_state.yaml --cj-state-table /app/data/control_job_state.yaml` | Every shipped YAML config passes structural + referential validation |
Plus, on every push to `main`, [Gitea Actions](.gitea/workflows/ci.yml)
runs both a **Release build + full test suite** and a separate
**ThreadSanitizer lane** that builds with `-fsanitize=thread` and
fails on any race. All 426 cases / 2 557 assertions pass under TSan
clean.
### Per-standard test coverage
Every claimed standard has dedicated tests. Counts are
`grep -c TEST_CASE`; cross-cutting tests (e.g. `test_robustness_fuzz`,
`test_gem300_scenario`) exercise multiple standards in concert.
| Standard | Test files | Cases |
|-----------------------------------|-------------------------------------------------------------------------------------------|------:|
| **E5** — SECS-II encoding | `test_secs2`, `test_sml`, `test_messages`, `test_identifier_wildcards`, `test_fuzz` | 120 |
| **E5 §13** — exceptions | `test_exceptions`, `test_exception_persistence` | 16 |
| **E4** — SECS-I transport | `test_secsi`, `test_secsi_timers`, `test_secsi_tcp` | 27 |
| **E37** — HSMS (SS + GS) | `test_hsms`, `test_hsms_connection`, `test_hsms_timers`, `test_hsms_s9`, `test_hsms_gs`, `test_hsms_gs_integration`, `test_s9_fallback`, `test_concurrency` | 34 |
| **E30** — GEM core | `test_control_state`, `test_communication_state`, `test_host_handler`, `test_data_model`, `test_loader`, `test_config_validate` | 71 |
| **E40** — process jobs | `test_process_jobs` | 21 |
| **E94** — control jobs | `test_control_jobs` | 9 |
| **E42** — formatted PP | `test_e42_formatted_pp` | 6 |
| **E87** — carriers + load ports | `test_carriers`, `test_carrier_state`, `test_carrier_persistence`, `test_e87_wire_scenarios` | 27 |
| **E90** — substrate tracking | `test_substrates`, `test_substrate_persistence` | 21 |
| **E116** — EPT | `test_ept` | 7 |
| **E120 / E39** — common equip / object service | `test_cem_objects` | 3 |
| **E157** — module process tracking | `test_modules` | 5 |
| **E84** — parallel I/O + timers | `test_e84`, `test_e84_ports`, `test_e84_timers`, `test_e84_asio_timers` | 27 |
| Persistence + cross-cutting | `test_job_persistence`, `test_persistence_upgrade`, `test_wire_ceid_emission`, `test_gem300_scenario`, `test_live_gem300`, `test_thread_safety`, `test_metrics_prometheus`, `test_robustness_fuzz` | 32 |
| **Total** | | **426** |
A single command to see this live: `docker compose run --rm builder
/app/build/secsgem_tests --list-test-cases | wc -l` (currently 426).
---
## Quick start ## Quick start
Everything runs in Docker — no compiler or build tools on the host. Everything runs in Docker — no compiler or build tools on the host.
```bash ```bash
docker compose run --rm builder # configure + compile docker compose run --rm builder # configure + compile
docker compose run --rm tests # 384 cases / 2390 assertions docker compose run --rm tests # 426 cases / 2 557 assertions
docker compose up --no-deps server client # live two-container demo docker compose up --no-deps server client # live two-container demo
``` ```
The two-container demo walks ~24 SECS transactions end-to-end
through the data model. Watch the logs interleave.
---
## Documentation map
| File | What it covers |
|-----------------------------------------------|-------------------------------------------------------------------------|
| [COMPLIANCE.md](COMPLIANCE.md) | Per-capability audit against every SEMI standard implemented |
| [INTEGRATION.md](INTEGRATION.md) | Vendor-side tutorial: YAML → callbacks → production deploy |
| [BENCHMARKS.md](BENCHMARKS.md) | Performance envelope (throughput, latency, memory) + how to re-run |
| [MES_INTEROP.md](MES_INTEROP.md) | Day-1 punch list to run against your commercial MES (60+ test IDs) |
| [SECURITY.md](SECURITY.md) | Concrete configs: nftables, stunnel, minisign, SIEM audit-log schema |
| [LICENSE](LICENSE) | Proprietary license terms |
---
## Architecture ## Architecture
The project is "spec-as-data": the SEMI behavioural rules live in YAML; The project is **spec-as-data**: the SEMI behavioural rules live in
the C++ is the engine that reads them. YAML; the C++ is the engine that reads them.
``` ```
┌──────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────┐
│ data/ │ │ data/ │
│ messages.yaml SECS-II message catalog │ │ messages.yaml SECS-II message catalog (164 msgs) │
│ control_state.yaml E30 §6.2 control transition table │ │ control_state.yaml E30 §6.2 control transition table │
│ process_job_state.yaml E40 §6 PJ transition table │ │ process_job_state.yaml E40 §6 PJ transition table │
│ control_job_state.yaml E94 §6 CJ transition table │ │ control_job_state.yaml E94 §6 CJ transition table │
│ equipment.yaml SVIDs / DVIDs / ECIDs / CEIDs / │ │ equipment.yaml SVIDs / DVIDs / ECIDs / CEIDs / │
│ alarms / recipes / commands │ │ alarms / recipes / commands │
└──────────────────────┬───────────────────────────────────────┘ └──────────────────────┬───────────────────────────────────────┘
│ (loaded at startup, codegen at build) │ (codegen at build, YAML loaded at startup)
▼
┌──────────────────────────────────────────────────────────────┐
│ tools/gen_messages.py │
│ reads messages.yaml -> emits generated/secsgem/gem/messages.hpp
└──────────────────────┬───────────────────────────────────────┘
│
▼ ▼
┌──────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────┐
│ apps/ │ │ apps/ │
│ secs_server.cpp passive equipment │ │ secs_server passive equipment secs_bench perf │
│ secs_client.cpp active host │ │ secs_client active host secs_conformance │
│ (both use gem::Router for dispatch) │ │ secs_interop_probe │
└──────────────────────────────────────────────────────────────┘ └──────────────────────────────────────────────────────────────┘
secsgem::config loader.hpp: YAML -> tables + data model secsgem::config loader.hpp + validate.hpp:
secsgem::gem every per-standard FSM (E30, E40, E84, E87, YAML -> data model, with multi-error validator
E90, E94, E116, E120, E148, E157, E39, E5 surfacing every issue at once (`--validate-config`)
exceptions), each per-store-persistable. secsgem::gem per-standard FSM + per-store persistence
(every store accepts v ∈ [1, kVersion] for
forward-compatible schema migrations).
EquipmentDataModel composes all stores. EquipmentDataModel composes all stores.
Router (stream, function) -> handler. Router (stream, function) -> handler.
Generated messages.hpp covers 164 SxFy. Generated messages.hpp covers 164 SxFy.
@@ -72,41 +131,10 @@ the C++ is the engine that reads them.
in-FSM, TCP transport for tunnel testing. in-FSM, TCP transport for tunnel testing.
secsgem::secs2 Item (variant), encode/decode, Message, secsgem::secs2 Item (variant), encode/decode, Message,
SML parser/printer. SML parser/printer.
secsgem::metrics Prometheus exporter (Registry + HTTP server).
``` ```
### Tree ---
```
secs-gem/
├── Dockerfile, docker-compose.yml # toolchain + demo
├── CMakeLists.txt
├── README.md
├── COMPLIANCE.md # per-capability audit
├── INTEGRATION.md # vendor integration tutorial
├── data/
│ ├── messages.yaml # SECS-II message catalog (164 msgs)
│ ├── control_state.yaml # E30 control state transitions
│ ├── process_job_state.yaml # E40 PJ transitions
│ ├── control_job_state.yaml # E94 CJ transitions
│ └── equipment.yaml # equipment data dictionary
├── tools/
│ └── gen_messages.py # codegen (messages.yaml -> .hpp)
├── include/secsgem/
│ ├── secs2/{item,codec,sml,message}.hpp
│ ├── hsms/{header,connection}.hpp
│ ├── secsi/{header,block,protocol,tcp_transport}.hpp
│ ├── gem/ # FSMs per SEMI standard
│ ├── gem/store/ # one file per focused store
│ ├── config/loader.hpp
│ └── endpoint.hpp
├── src/{secs2,hsms,secsi,gem,config}/*.cpp
├── apps/
│ ├── secs_server.cpp # passive equipment demo
│ ├── secs_client.cpp # active host demo
│ └── secs_interop_probe.cpp # cross-test against secsgem-py
├── interop/ # secsgem-py 0.3.0 cross-validation
└── tests/test_*.cpp # 384 cases / 2390 assertions
```
## Adding a capability ## Adding a capability
@@ -150,269 +178,71 @@ transitions:
- {name: field_b, shape: {kind: scalar, item_type: ASCII}} - {name: field_b, shape: {kind: scalar, item_type: ASCII}}
``` ```
`docker compose run --rm builder` regenerates `messages.hpp`. The typed `docker compose run --rm builder` regenerates `messages.hpp`. The
builder, parser, and struct definition appear automatically. typed builder, parser, and struct definition appear automatically.
Run `--validate-config` after every YAML edit.
--- ---
# Production / fab deployment ## Production deployment
The library is a runtime stack. Shipping it on a real tool involves See [INTEGRATION.md](INTEGRATION.md) for the full vendor-side
more than building the binary. This section enumerates the work tutorial — wiring sensors, plugging FSMs into the tool, persistence
that sits between "tests pass" and "this is running on the fab floor." layout, monitoring/observability, HSMS-GS multi-MES setup.
## 1. Persistence directory layout See [SECURITY.md](SECURITY.md) for concrete nftables / stunnel /
minisign / SIEM configs.
Enable persistence per store at startup, before the connection comes up. See [BENCHMARKS.md](BENCHMARKS.md) for the performance envelope —
Pattern (the call sites are equivalent on every store): roughly **140 k req/s S1F1**, **79 k req/s S1F3 (32 SVIDs)**, **572
k S6F11/s push**, **~450 bytes per PJ+CJ pair**. Three orders of
magnitude above typical fab tool load.
```cpp See [MES_INTEROP.md](MES_INTEROP.md) for the day-1 punch list to run
auto base = std::filesystem::path("/var/lib/acme-secsgem"); against your commercial MES before promoting from staging to a real
model->spool.enable_persistence(base / "spool"); tool.
model->carriers.enable_persistence(base / "carriers");
model->load_ports.enable_persistence(base / "loadports");
model->substrates.enable_persistence(base / "substrates");
model->process_jobs.enable_persistence(base / "pjobs");
model->control_jobs.enable_persistence(base / "cjobs");
model->exceptions.enable_persistence(base / "exceptions");
```
Storage rules: ### Operational runbook (starter)
- **Mount this volume on the same physical disk as the binary** —
network filesystems (NFS) can introduce latency that interferes
with the rename-based atomic write pattern.
- **Back this volume up daily**. Journal files are small (a few
hundred bytes each) and rsync-friendly.
- **Set sane retention**. Cleared exceptions and dequeued PJs are
removed automatically; complete carriers / substrates / CJs are
the application's responsibility to sweep. Cap by file count
(a million files in one directory is fine on ext4 / xfs; less
on others).
- **Disk space**: budget 100 MB for a busy fab tool over a year
(~500 K transitions, ~200 bytes each). In practice it's far
less because terminal-state records are removed.
After a crash, the next process start replays every store and is
back to the prior in-memory state before the HSMS port opens.
## 2. Security
HSMS over plain TCP is the spec's wire protocol. The library
ships unencrypted by design — that's what equipment manufacturers
expect. In a real fab:
- **Network isolation**: HSMS must run on a control LAN, never
exposed to engineering / corporate networks. Default the
`--port` to 5000 / 5005 on a dedicated VLAN behind firewall ACLs
that whitelist your MES host's IP.
- **TLS tunnel**: for cross-site HSMS (rare but real for multi-fab
shared hosts), tunnel the TCP through stunnel or a sidecar
proxy. Don't modify the HSMS wire protocol; wrap the socket.
- **Authentication**: HSMS doesn't include peer auth. Rely on
network-layer mTLS (sidecar proxy) and per-tool firewall rules.
- **Audit logging**: enable `Connection::set_log_handler` and
ship to a SIEM. Every SECS-II message in/out should be
retrievable for a configurable retention window — many fabs
require 90 days.
- **YAML config integrity**: sign your config bundles
(`equipment.yaml`, `control_state.yaml`, etc.) and verify the
signature on load. Misconfiguration is one of the top
root-causes of GEM-related fab incidents.
## 3. Monitoring and observability
The library exposes hooks at every layer. Wire them to whatever
your fab already runs.
| Signal | Hook | Why it matters |
| ---------------------------- | ------------------------------------------ | -------------------------------------------------- |
| HSMS connection lifecycle | `Connection::set_log_handler`, `set_selected_handler`, `set_closed_handler` | reconnect storms, unexpected separates |
| T3 / T6 / T7 / T8 timer fires | `set_closed_handler` reason starts with "T*" | host MES unreachable, fab network event |
| Auto S9F* emission | `set_log_handler` line containing "-> S9F" | malformed peer traffic, schema drift |
| Spool depth | `model->spool.size()` | host MES backpressure / outage |
| FSM transitions (every store) | `set_*_change_handler` | tool state, throughput, anomaly detection |
| Persistence directory size | `du -s var/lib/acme-secsgem` | journal growth, untracked terminal-state records |
Recommended metrics export pattern: aggregate into Prometheus
via a sidecar that polls the data model. Per-CEID emission rates,
alarm set/clear rates, T-timer expiry counts, and spool depth
form a reasonable starter dashboard.
**Hooks fire on the io_context thread.** Every `set_*_change_handler`
callback the library invokes runs on the connection's io_context
(there are no locks anywhere in `EquipmentDataModel`). Metrics
exporters and log shippers wired into those callbacks must either be
thread-safe themselves or hand the work off (a lock-free queue, a
separate exporter thread polling published counters, `asio::post`
onto another executor). Doing blocking I/O from inside a handler
stalls the dispatcher — keep handlers cheap. See INTEGRATION.md §3
for the cross-thread update pattern.
## 4. High availability
The library is single-threaded per HSMS connection — that's how
HSMS works. For HA:
- **Run two equipment processes** in active/standby on the same
tool, sharing the persistence volume. Only the active accepts
the HSMS port; the standby tails the journal. Failover is
filesystem-locked.
- **Reconnect on the host side**: an MES-side disconnect should
trigger T5-based reconnect. Configure `Timers::t5` to your
MES's policy (default 10s).
- **Graceful shutdown**: SIGTERM should flush the write queue,
call `conn->separate()`, and exit cleanly so the journal is
point-consistent. The provided `apps/secs_server.cpp` shows
the pattern.
## 5. Deployment patterns
Three common shapes:
### Docker / podman on a tool PC
```dockerfile
FROM ubuntu:24.04
COPY build/secs_server /usr/local/bin/
COPY etc/ /etc/acme-secsgem/
VOLUME /var/lib/acme-secsgem
EXPOSE 5000
ENTRYPOINT ["/usr/local/bin/secs_server", \
"--port", "5000", \
"--config", "/etc/acme-secsgem/equipment.yaml", \
"--state-table", "/etc/acme-secsgem/control_state.yaml", \
"--spool-dir", "/var/lib/acme-secsgem/spool"]
```
### systemd unit
```ini
[Unit]
Description=ACME SECS/GEM equipment
After=network.target
[Service]
Type=simple
User=secsgem
Group=secsgem
ExecStart=/usr/local/bin/secs_server --port 5000 \
--config /etc/acme-secsgem/equipment.yaml \
--state-table /etc/acme-secsgem/control_state.yaml \
--spool-dir /var/lib/acme-secsgem/spool
Restart=always
RestartSec=5
LimitNOFILE=8192
[Install]
WantedBy=multi-user.target
```
### Kubernetes (multi-tool cell controller)
Run one Pod per tool with the persistence volume mounted from
local-storage (not NFS). The Service exposes the HSMS port on the
control LAN. Use a PodDisruptionBudget to ensure the standby is
available during rolling updates.
## 6. Upgrade path
YAML edits don't require a rebuild — restart the process and the
new dictionary loads. Code changes do require rebuild + restart.
- **Zero-downtime for YAML**: if you're using the active/standby
HA pattern, edit YAML on the standby, restart the standby,
promote.
- **Code upgrades**: deploy to a canary tool first; bake-test for
at least a full wafer batch before fleet-wide rollout.
- **Schema migrations**: persistence records carry a 1-byte version
stamp after the magic byte. Every store (`ProcessJobStore`,
`SubstrateStore`, `ControlJobStore`, `CarrierStore`, `LoadPortStore`,
`ExceptionStore`, `SpoolStore`) accepts any version in
`[1, kVersion]`: code at kVersion=2 loads both v1 and v2 records
(v1 trailer fields default to empty). Future versions beyond
`kVersion` are rejected so a downgrade can't silently corrupt
data. Upgrade discipline: when adding fields, bump `kVersion` and
gate the new trailer behind `if (version >= N)` in the loader.
Tests in `tests/test_persistence_upgrade.cpp` lock down the
contract and act as a tripwire if a writer bumps `kVersion`
without teaching the loader to handle prior versions. Always
test the upgrade with a real on-disk journal before fleet rollout.
## 7. Integration with the fab stack
| Other system | How this library talks to it |
| ------------------- | --------------------------------------------------------------------- |
| MES (Camstar, Mozaic, Camstar) | HSMS-SS over TCP (`secs_server` listens on a port the MES is configured to connect to) |
| Multi-MES (HSMS-GS) | `Connection::add_session(device_id)` registers extra sessions on one TCP socket |
| AMHS / OHT | E84 per-port FSMs (`E84PortStore::on_signal_change(port, signal, value)`); wire to your I/O bridge |
| Recipe engine | RecipeStore.add (opaque) + RecipeStore.add_formatted (E42 structured) |
| Alarm sources | `AlarmRegistry::set(alid, active)` from your sensor poll |
| Carrier scanner | `CarrierStore::create / fire_id_event / set_slot_state` |
| Wafer tracker | `SubstrateStore::create / fire_*_event` |
| EPT / shift report | `EptStateMachine::accumulated(state)` reads the time-bucket counters |
## 8. Compliance and certification
- Fork `COMPLIANCE.md` and prune it to *your* tool's claimed
coverage; ship that copy with the tool.
- Run the in-repo conformance harness against your tool:
```
build/secs_conformance --host <tool-ip> --port 5000 --device 0
```
Exits 0 with a per-check PASS / FAIL summary covering every E30
fundamental capability (establish comms, on-line ID, status data,
equipment constants, clock, alarms, PP management, documentation).
Adapt `apps/secs_conformance.cpp` to add your tool's
capability-specific checks.
- Run an independent third-party validator (GEM RTS or equivalent)
against your specific tool — a passing library + in-repo harness
is necessary but not sufficient for certification.
- Capture wire traces from every validator run; archive for audit.
## 9. Testing in production
- **Canary**: deploy to one or two tools per fab before fleet
rollout.
- **Synthetic transactions**: a heartbeat that issues S1F1 every
60s and alerts on T3 timeout. Catches MES-side outages before
a real recipe does.
- **Shadow traffic**: for upgrades, run the new version listening
on a second port; have MES dual-connect; diff the responses.
## 10. Operational runbook (starting point)
Common production incidents and the right response:
| Incident | First check | Mitigation | | Incident | First check | Mitigation |
| ----------------------------------- | ------------------------------------ | ----------------------------------------- | |-------------------------------------|--------------------------------------|-------------------------------------------|
| HSMS connection flapping | T7 / T6 timer fires in logs | check MES reachability, network MTU | | HSMS connection flapping | T7 / T6 timer fires in logs | check MES reachability, network MTU |
| Spool depth growing | host MES connectivity / ACK rate | force-drain via S6F23, escalate to MES | | Spool depth growing | host MES connectivity / ACK rate | force-drain via S6F23, escalate to MES |
| State machine "stuck" | last state-change handler log line | host-issued offline + re-establish | | State machine "stuck" | last state-change handler log line | host-issued offline + re-establish |
| Alarm storm | AlarmRegistry `all()` snapshot | check upstream sensor; quench via S5F3 | | Alarm storm | `AlarmRegistry::all()` snapshot | check upstream sensor; quench via S5F3 |
| Persistence dir growing unbounded | `du -s` + file count | sweep terminal-state records | | Persistence dir growing unbounded | `du -s` + file count | sweep terminal-state records |
| Cross-tool inconsistency | `secsgem_tests` on canary tool | compare wire trace vs validator | | Cross-tool inconsistency | `secsgem_tests` on canary tool | compare wire trace vs validator |
--- ---
## Demo
The two-container demo walks ~24 SECS transactions end-to-end
through the data model. Run `docker compose up --no-deps server client`
and watch the logs interleave.
## Build details ## Build details
The toolchain image (`Dockerfile`) is Ubuntu 24.04 with `g++-13`, CMake, The toolchain image (`Dockerfile`) is Ubuntu 24.04 with `g++-13`,
Ninja, `libasio-dev`, `libyaml-cpp-dev`, and Python 3 for the codegen. CMake, Ninja, `libasio-dev`, `libyaml-cpp-dev`, and Python 3 for the
doctest is fetched via CMake FetchContent. Build artifacts live in a codegen. doctest is fetched via CMake FetchContent. Build artifacts
named Docker volume so the host filesystem stays clean. live in a named Docker volume so the host filesystem stays clean.
Standalone Asio is used in header-only mode (`ASIO_STANDALONE`). No Standalone Asio is used in header-only mode (`ASIO_STANDALONE`). No
Boost dependency. Boost dependency.
### ThreadSanitizer
```bash
cmake -S . -B build-tsan -G Ninja -DCMAKE_BUILD_TYPE=Debug -DSECSGEM_TSAN=ON
cmake --build build-tsan
TSAN_OPTIONS=halt_on_error=1 build-tsan/secsgem_tests
```
Runs as a separate lane in CI. Catches data races in the io_context
strand contract documented in INTEGRATION.md §3.
---
## Interop ## Interop
`interop/` contains the secsgem-py 0.3.0 cross-validation harness — `interop/` contains the secsgem-py 0.3.0 cross-validation harness —
secsgem-py active host driving our C++ passive server, our C++ active secsgem-py active host driving our C++ passive server, our C++
host probing secsgem-py's passive equipment, and a raw GEM-300 harness active host probing secsgem-py's passive equipment, and a raw GEM-300
that round-trips S3 (E87), S14 (E94), S16 (E40), S12 (wafer maps) harness that round-trips S3 (E87), S14 (E94), S16 (E40), S12 (wafer
through hand-crafted `SecsStreamFunction` subclasses. See maps) through hand-crafted `SecsStreamFunction` subclasses. See
`interop/README.md`. [`interop/README.md`](interop/README.md).