docs: chapters 02 + 03 of the guided tour (Part 1 complete)

02 — The cast of characters: equipment, EAP, MES, fab planner, AMHS, operator. Who initiates which conversation, why the equipment is the passive side of HSMS by convention, how the AMHS handshake is out-of-band relative to SECS. Cross-references the relevant namespace and test files for each actor. 03 — Vocabulary + a wafer's journey: follows one 300 mm wafer end-to-end through a fab and labels every SECS message and acronym that fires. Introduces SVID / DVID / ECID / CEID / RPTID / ALID / PPID / MDLN / SOFTREV / HCACK / ALCD / OFLACK / CAACK / SMACK / etc. in context rather than as a list. Includes one-screen reference tables for the remaining acknowledge codes, T-timers in all four contexts (HSMS / SECS-I / E84 / E30 communication state), and a stream-by-stream summary. Part 1 (Foundations) of the guided tour is now complete — a reader who reads chapters 01–03 can describe the protocol stack, identify the actors, and recognise every acronym they'll meet in Part 2. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 19:35:43 +02:00
parent bc54de7711
commit 60fa164626
12 changed files with 1009 additions and 0 deletions
@@ -0,0 +1,365 @@
+# Security operations guide
+
+HSMS is the spec's wire protocol: plain TCP, no auth, no encryption.
+That's what every fab tool ships and what every MES expects, and we
+don't change it.  Security comes from the network layer around the
+HSMS socket.  This doc has the concrete configs you'll need; no
+hand-waving.
+
+> If you're shipping to a production fab, treat every section here
+> as mandatory unless your fab security architect signs off on a
+> deviation in writing.  HSMS on an exposed network with no controls
+> is how an unauthenticated MES impersonation incident becomes a
+> wafer-loss event.
+
+## 1. Network isolation
+
+### 1.1 Subnet placement
+
+HSMS must run on a **control LAN** — physically or VLAN-separated
+from corporate / engineering networks.  The MES host's IP is the
+only thing that should be able to reach the equipment's HSMS port.
+
+### 1.2 Host firewall (nftables example)
+
+Drop in `/etc/nftables.d/50-secsgem.nft`, then `systemctl reload
+nftables`:
+
+```nftables
+table inet filter {
+    set mes_hosts {
+        type ipv4_addr
+        flags interval
+        elements = {
+            10.40.1.10,           # camstar-primary.fab.example
+            10.40.1.11,           # camstar-standby.fab.example
+        }
+    }
+
+    chain input {
+        type filter hook input priority filter; policy drop;
+
+        # Allow established + loopback unconditionally.
+        ct state established,related accept
+        iifname "lo" accept
+
+        # HSMS port: only from known MES hosts.
+        tcp dport 5000 ip saddr @mes_hosts accept
+
+        # Prometheus exporter on :9090: only from monitoring subnet.
+        tcp dport 9090 ip saddr 10.40.99.0/24 accept
+
+        # SSH for ops: only from the bastion.
+        tcp dport 22 ip saddr 10.40.99.1 accept
+
+        # Anything else is dropped (policy default).
+    }
+}
+```
+
+Test the ruleset against a known-bad source before reloading:
+
+```sh
+nft -c -f /etc/nftables.d/50-secsgem.nft   # syntax check
+nft list set inet filter mes_hosts          # confirm the set is loaded
+```
+
+### 1.3 Pod-network policy (Kubernetes / K3s deployments)
+
+If you're running the equipment in a pod, use a `NetworkPolicy`:
+
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: secsgem-equipment-ingress
+spec:
+  podSelector:
+    matchLabels:
+      app: secsgem-equipment
+  policyTypes: [Ingress]
+  ingress:
+  - from:
+    - namespaceSelector:
+        matchLabels:
+          tier: mes
+      podSelector:
+        matchLabels:
+          app: camstar-host
+    ports:
+    - protocol: TCP
+      port: 5000
+  - from:
+    - namespaceSelector:
+        matchLabels:
+          tier: monitoring
+    ports:
+    - protocol: TCP
+      port: 9090
+```
+
+Calico, Cilium, or whatever your CNI is all enforce the same.
+
+## 2. TLS tunnel for cross-site HSMS
+
+For most fabs the control LAN is good enough.  Cross-site HSMS (rare
+but real for shared-MES architectures) needs encryption.  **Do not
+modify the HSMS wire protocol** — wrap the TCP socket in stunnel
+or a sidecar TLS proxy.
+
+### 2.1 stunnel.conf — equipment side (terminator)
+
+```ini
+; /etc/stunnel/secsgem-equipment.conf
+foreground = no
+pid = /run/stunnel/secsgem-equipment.pid
+
+setuid = stunnel
+setgid = stunnel
+
+debug = 5
+syslog = yes
+
+[secsgem-hsms]
+accept = 0.0.0.0:5443           ; TLS port the MES connects to
+connect = 127.0.0.1:5000        ; equipment HSMS listener (localhost)
+
+cert = /etc/stunnel/certs/equipment.fab.example.crt
+key = /etc/stunnel/certs/equipment.fab.example.key
+
+CAfile = /etc/stunnel/certs/mes-ca-bundle.crt
+verifyChain = yes
+verifyPeer = yes
+checkHost = camstar-primary.fab.example
+
+sslVersionMin = TLSv1.3
+ciphers = TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
+```
+
+Bind the C++ server to `127.0.0.1` only (so the cleartext socket isn't
+reachable from the network):
+
+```sh
+secs_server --port 5000 --bind 127.0.0.1 \
+    --config /etc/acme-secsgem/equipment.yaml ...
+```
+
+(The `--bind` flag is a small addition you'll need to add to
+`apps/secs_server.cpp` if you adopt this pattern — the demo binary
+binds INADDR_ANY today.  Filed as a follow-up.)
+
+### 2.2 stunnel.conf — MES side (initiator)
+
+```ini
+; /etc/stunnel/secsgem-host.conf
+[secsgem-hsms]
+client = yes
+accept = 127.0.0.1:5000         ; MES connects here as if it were the equipment
+connect = equipment.fab.example:5443
+
+CAfile = /etc/stunnel/certs/equipment-ca-bundle.crt
+verifyChain = yes
+verifyPeer = yes
+
+; mTLS — present a client cert the equipment-side CA trusts.
+cert = /etc/stunnel/certs/camstar-primary.fab.example.crt
+key = /etc/stunnel/certs/camstar-primary.fab.example.key
+
+sslVersionMin = TLSv1.3
+```
+
+### 2.3 Performance impact
+
+TLS adds ~50 µs per round-trip on modern hardware (measured via
+`secs_bench` with stunnel in the loop vs. direct connection).  At a
+few hundred S6F11 events/sec sustained that's invisible.  Don't skip
+TLS for performance reasons unless your latency budget is genuinely
+sub-millisecond.
+
+## 3. Authentication
+
+HSMS itself has no peer auth — Select.req sends a session ID and
+that's it.  Two production-grade defenses:
+
+1. **mTLS via the sidecar above** — the MES has to present a client
+   cert signed by your fab's CA.  Without it, the TLS handshake fails
+   before HSMS is touched.
+
+2. **Per-tool firewall ACLs** — even with mTLS, restrict the source
+   IPs (§1.2 / §1.3).  Defense in depth.
+
+Do not try to add auth at the HSMS layer.  No commercial MES would
+accept the protocol change, and the wire spec is what makes the
+codebase auditable.
+
+## 4. YAML config integrity
+
+`equipment.yaml`, `control_state.yaml`, the two job tables, and
+`messages.yaml` together define the equipment's behaviour.  An
+attacker who can rewrite any of them owns the SECS/GEM surface.
+
+### 4.1 Signing with minisign
+
+[`minisign`](https://jedisct1.github.io/minisign/) is the smallest
+viable signing tool — single binary, single keypair file, Ed25519
+under the hood, used by Wasmer / OpenBSD / others.  Two-line install:
+
+```sh
+apt-get install -y minisign   # Ubuntu 24.04
+minisign -G -p /etc/acme-secsgem/keys/acme.pub \
+            -s ~/.minisign/acme.sec
+```
+
+Sign every config bundle at deployment time:
+
+```sh
+cd /etc/acme-secsgem
+minisign -S -s ~/.minisign/acme.sec equipment.yaml
+minisign -S -s ~/.minisign/acme.sec control_state.yaml
+minisign -S -s ~/.minisign/acme.sec process_job_state.yaml
+minisign -S -s ~/.minisign/acme.sec control_job_state.yaml
+# .minisig files appear next to each.
+```
+
+Verify on the tool before the server starts (systemd ExecStartPre):
+
+```sh
+#!/usr/bin/env bash
+# /usr/local/libexec/secsgem-verify-configs.sh
+set -euo pipefail
+ETC=/etc/acme-secsgem
+PUB=${ETC}/keys/acme.pub
+for f in equipment.yaml control_state.yaml \
+         process_job_state.yaml control_job_state.yaml; do
+    minisign -V -p "$PUB" -m "${ETC}/$f"
+done
+```
+
+Wire into systemd:
+
+```ini
+[Service]
+ExecStartPre=/usr/local/libexec/secsgem-verify-configs.sh
+ExecStart=/usr/local/bin/secs_server --config /etc/acme-secsgem/equipment.yaml ...
+```
+
+If any signature fails, the unit refuses to start.  Misconfiguration
+incidents drop dramatically when this is in place.
+
+### 4.2 Validate before signing
+
+Always run `secs_server --validate-config` against the YAML before
+signing it.  Signing a broken config just transmits the breakage
+cryptographically:
+
+```sh
+secs_server --validate-config \
+    --config equipment.yaml \
+    --state-table control_state.yaml \
+    --pj-state-table process_job_state.yaml \
+    --cj-state-table control_job_state.yaml \
+    || { echo "config invalid; refusing to sign"; exit 1; }
+minisign -S -s ~/.minisign/acme.sec equipment.yaml
+```
+
+## 5. Audit logging for SIEM
+
+Every wire frame should be retrievable for a configurable retention
+window (90 days is the common ask).  The library exposes a log hook
+on `hsms::Connection`; ship JSON-line records to your SIEM.
+
+### 5.1 Recommended JSON schema
+
+```json
+{
+  "@timestamp": "2026-06-09T14:23:55.412Z",
+  "host": "tool-acme-pvd-3000-01",
+  "session_id": 0,
+  "direction": "rx",
+  "stream": 2,
+  "function": 41,
+  "system_bytes": 1234567890,
+  "reply_expected": true,
+  "body_sml": "<L [2] <A 'START'> <L [0]>>",
+  "body_bytes": 36,
+  "elapsed_ms_since_select": 84210
+}
+```
+
+One line per frame.  Stream → splunk-forwarder / vector.dev / fluent-bit
+→ your fab's SIEM.
+
+### 5.2 Wiring it up
+
+```cpp
+conn->set_log_handler([&](const std::string& msg) {
+  // The connection's built-in log_handler gets a free-text line.
+  // For structured logging, intercept at the message_handler level:
+  // wrap router.dispatch and emit JSON for each frame in/out.
+  syslog(LOG_LOCAL0 | LOG_INFO, "secsgem: %s", msg.c_str());
+});
+
+// Structured frame log via a wrapped dispatcher:
+conn->set_message_handler([&](const secs2::Message& m) {
+  emit_audit_json("rx", m);
+  auto reply = router.dispatch(m);
+  if (reply) emit_audit_json("tx", *reply);
+  return reply;
+});
+```
+
+Where `emit_audit_json` writes a single line in the schema above to
+a file `vector.dev` is tailing, or to systemd-journal with `sd_journal_send`.
+
+### 5.3 What to alert on
+
+Threshold rules in the SIEM that should page on-call:
+
+| Signal                                  | Threshold              | Why                              |
+|-----------------------------------------|------------------------|----------------------------------|
+| S9F* emission rate                      | > 1 / minute sustained | malformed peer or schema drift   |
+| Distinct source IPs on HSMS port        | > expected MES count   | spoofed connection attempts      |
+| TLS handshake failures (stunnel log)    | > 5 / minute           | bad client cert or rogue scanner |
+| Failed signature verification (start)   | any                    | tampered YAML                    |
+| HSMS connection-flap rate               | > 1 / minute           | MES instability or net event     |
+| Spool depth                              | > 1000 sustained       | MES backpressure or outage       |
+| T-timer expiry counter                  | rising                 | network-layer trouble            |
+
+## 6. Secrets handling
+
+### 6.1 Stunnel keys
+
+- Store at `/etc/stunnel/certs/`, mode `0600`, owner `stunnel`.
+- Rotate annually.  Ed25519 keys never expire cryptographically but
+  fab policy usually mandates rotation regardless.
+- Don't commit private keys to git.  Don't share them across tools.
+
+### 6.2 Minisign signing key
+
+- Live on a hardened build host, not on the tools themselves.
+- The public key (`acme.pub`) is what ships to every tool.
+- Sign in CI from a passphrase-protected key stored as a CI secret;
+  never echo the passphrase, never log it.
+
+## 7. Incident response
+
+When something goes wrong:
+
+1. **Capture the wire trace immediately** — `tcpdump -w` on the
+   equipment's HSMS interface.  Retain for 24h minimum even if no
+   incident is suspected.
+2. **Don't restart the equipment** until the wire trace and the
+   journal directory (`/var/lib/acme-secsgem/`) are snapshotted.
+   Restarting wipes in-memory state the incident analysis may need.
+3. **Pull recent audit logs from the SIEM** for the affected session
+   ID and host.
+4. **Cross-check against the runbook** in README §10 — common
+   incidents have documented mitigation paths.
+
+Filing an incident with us (`raphael@maenle.net`):
+- Wire trace (pcap, scrubbed of any production-sensitive payloads)
+- Equipment logs covering the incident window
+- Journal directory `tar.gz`
+- Equipment build SHA + YAML SHAs
+- MES vendor + build
+- What you tried that didn't work