secs-gem/docs/FAQ.md

FAQ

Questions we hear once per integration. Skim before you ask. If your question isn't here and isn't obvious from the other docs, ask once — your question probably belongs in this file and we'll add it.

Why is HSMS unencrypted?

Because SEMI E37 says so. HSMS is plain TCP with a 14-byte framing header — no TLS, no auth, no nonces. Every commercial MES on the market speaks exactly that wire, and changing it would make us incompatible with all of them. Encryption and authentication belong at the network layer: see SECURITY.md for the stunnel.conf + nftables setup that wraps the unencrypted TCP in mTLS without modifying the wire protocol.

What's the difference between SVID and DVID?

SVID is a status variable — equipment state the host queries (chamber pressure, current control state, wafer counter). DVID is a data variable — intermediate values, typically computed or sensor-derived, that aren't part of the equipment's state model.

In practice fab tools blur the line. The library treats them identically except for which message reports them: S1F3 / S1F11 for SVIDs, S1F21 / S1F22 for DVIDs. Variable lookups by VID span both (EquipmentDataModel::vid_value).

Do I really need all four YAML files?

Yes for production; no for a quick "does it compile":

• equipment.yaml — your tool's data dictionary. Required.
• control_state.yaml — the E30 control state machine (HostOffline, AttemptOnline, OnlineRemote, …). The default in data/ works as a starting point; you may customize transitions.
• process_job_state.yaml — the E40 PJ FSM. Default is spec-typical; customize only if your tool has unusual recipe semantics.
• control_job_state.yaml — the E94 CJ FSM. Same.

secs_server --validate-config checks all four in one pass and exits 0 / 1. Run it in CI on every config change.

PJ vs CJ — what's the difference?

A PJ (E40 Process Job) is "process this batch of material with this recipe." One PJ = one recipe run = one set of wafers. It has its own FSM (Queued → SettingUp → Processing → ProcessComplete).

A CJ (E94 Control Job) is "execute these PJs in order, as a unit, with start/pause/abort semantics." A CJ owns an ordered list of PRJOBIDs. When the host issues CJSTART, the CJ promotes its PJs through their lifecycles.

You typically need both: the MES creates a CJ containing N PJs, then starts the CJ. PJs without a CJ are legal — they just sit in Queued waiting for someone to select them — but most MES drives batches through CJs.

Who fires FSM transitions — the library or my code?

Your code. The library implements the FSMs (legal transitions, validation, persistence) but it doesn't know when a wafer was actually loaded or when a recipe step finished — those signals come from your tool. The pattern across every store is:

// You fire the event; the FSM validates + transitions + emits.
model->process_jobs.fire_internal("PJ-1", gem::ProcessJobEvent::SetupComplete);
model->carriers.fire_id_event("CAR-A1B2", gem::CarrierIDEvent::Read);

Host commands (S2F41 RCMD=START, S16F5 PRJSTART, S16F27 CJSTART) arrive via the wire and get dispatched into your registered handlers; the handler typically calls fire_internal or on_host_command on the relevant store.

See INTEGRATION.md §4 for the worked patterns.

What runs on which thread?

Everything that touches the data model runs on the io_context thread. There are no locks in EquipmentDataModel.

• The Router dispatch (incoming wire messages) — on the io_context.
• All set_*_change_handler callbacks — on the io_context.
• Periodic timers you register via asio — on the io_context.

If your code lives on another thread (typical for sensor polling), marshal updates via asio::post:

asio::post(io.get_executor(), [model, value] {
  model->svids.set_value(100, secs2::Item::f4(value));
});

INTEGRATION.md §3 has the full thread-safety contract.

How do I add a new SECS-II message?

Edit data/messages.yaml, add a row, rebuild. The codegen (tools/gen_messages.py) emits a typed builder + parser into messages.hpp. Then register a Router handler in your main.cpp for the new (stream, function) pair. See README "Adding a capability" or ARCHITECTURE.md for the full walkthrough.

What's the difference between Item::ascii("X") and Item::binary({'X'})?

The wire format byte differs — 0x41 01 58 for ASCII vs 0x21 01 58 for Binary. Some peers (notably secsgem-py) default PPBODY to ASCII; others use Binary. Our codec accepts either via the BINARY_OR_ASCII codegen type for fields the spec lists as ASCII | Binary | List (the PPBODY case in S7F3/F6).

For most fields it doesn't matter — pick the format that matches your data semantically.

My MES sends a message that worked in interop but fails in production. What's going on?

Three usual suspects:

1. U-width. Your MES is sending DATAID as U1 but our handler was strict for U4. We're lenient now via any_unsigned_first, but if you have custom handlers in your code, use that helper rather than as_u4_scalar for identifier fields.
2. PPBODY direction. Some MES send PPBODY as ASCII even when the spec says it can be Binary. Use as_text_or_binary not as_binary.
3. Trailing fields. Some MES add proprietary trailing fields to S2F41 / S16F11 / S3F17 bodies that aren't in the standard. Our parsers are tolerant of extras; check your handler's assumptions.

See MES_INTEROP.md §13 for the per-MES quirk register.

What if the spec is ambiguous on some detail?

Cross-check against the secsgem-py and secs4j wire output: VERIFICATION.md. If both peers agree on a shape, that's the working interpretation regardless of how you read the spec text. If they disagree, the secsgem-py output usually wins (it's the de-facto Python reference and most MES vendors test against it), but file the question — we may need a new test.

Can I run this without Docker?

In principle yes — you need g++-13 (or any C++20 compiler), CMake, Ninja, libasio-dev, libyaml-cpp-dev, python3. But every doc, every CI lane, every test command in the repo assumes Docker. Going off-piste means re-deriving the build on your host. We don't support it; we don't actively break it.

How does persistence survive a crash mid-write?

Every store uses a .tmp + atomic rename pattern: writes go to <file>.tmp, then rename(2)s into place. POSIX guarantees the rename is atomic on the same filesystem. A crash mid-write loses the .tmp (corrupt-drop on next replay) but leaves the prior record intact.

Every store's loader accepts versions in [1, kVersion] so future schema bumps don't nuke old records — see README §Production deployment "Schema migrations."

What does the "spool" actually do?

When the host MES disconnects, the equipment can't deliver unsolicited S5F1 alarms / S6F11 events. Without spool they'd be lost.

With spool enabled (SpoolStore::set_spoolable_streams({5, 6})), those frames queue to in-memory FIFO (and persistent disk if enable_persistence is set). On the host's next SELECT, the equipment emits S6F25 SpoolDataReady(count); the host issues S6F23(Transmit) to drain, or S6F23(Purge) to discard.

It's the GEM equivalent of an outbox. See E30 §6.22 and our SpoolStore source.

How is "robustness fuzz" different from "libFuzzer"?

• Robustness fuzz (tests/test_robustness_fuzz.cpp) is a model-level property test. It picks random tool operations (PJ create, alarm set, substrate move, …) respecting FSM legality, and checks invariants after each.
• libFuzzer (apps/fuzz_*.cpp) is a byte-level coverage- guided fuzzer. It feeds arbitrary bytes to the codec and asserts no crash / UB.

They cover different concerns: robustness fuzz catches semantic bugs (lost data, wrong state); libFuzzer catches parser bugs (crashes, UB, buffer overruns).

What's "conformance" vs "interop"?

• Conformance (build/secs_conformance) is us driving us through every claimed E30 capability and asserting the spec- mandated reply S/F. Catches our regressions against our own understanding of the spec.
• Interop (interop/*.py, interop/secs4j/*.java, interop/tshark_validate.sh) is third-party tools agreeing on the wire bytes our equipment produces. Catches "we got the spec wrong" — which conformance can't.

Both are necessary; neither replaces the other. See VERIFICATION.md.

How do I bring this to a customer site?

Run through the five external proofs in the eight commands in PROOFS.md at the customer's network. Then walk MES_INTEROP.md against their actual MES. Then deploy per SECURITY.md for the nftables / stunnel / signing setup. Then page on the metrics from INTEGRATION.md §6.4.

What's not implemented?

Every E30 Fundamental + Additional capability and every GEM 300 standard in scope is shipped. The two non-shipped pieces are:

1. The asio serial_port adapter for SECS-I — the FSM is implemented and tested end-to-end over TCP (secsi::TcpTransport); the serial-port driver is a deferred follow-up (most modern GEM equipment runs HSMS). Listed under "Deferred follow-ups" in README.md.
2. A GEM Reference Test System (RTS) run — paid third-party certification gate, not a code feature. See COMPLIANCE.md §8 for what "100% GEM-compliant" honestly means about a codebase vs. a certified tool.

Note: Equipment Processing States are tool-defined per E30 §6.3 — the engine ships, and vendors load their concrete states (IDLE / SETUP / READY / EXECUTING / …) the same way data/control_state.yaml is loaded. That isn't a gap, it's how the spec is designed.