docs: GLOSSARY + FAQ + interop README refresh + doc-map fixes
Fills four documentation gaps surfaced by the doc audit:
1. README "Documentation map" was missing VERIFICATION.md (the file
that backs the proof-of-feature-completeness claims) and is now
pointing at the new files added in this commit too — ARCHITECTURE,
GLOSSARY, FAQ, examples/pvd_tool/ (the last two land next).
2. interop/README.md only documented secsgem-py. Three of the five
external validators (tshark, secs4j, libFuzzer) plus the E5 KAT
were invisible from the directory's own README. Rewritten as a
complete index — what's external, what each catches, how to run,
what bugs they've already surfaced, when to add a new validator.
3. GLOSSARY.md is new. Every SEMI acronym used in the codebase or
the docs gets one row: SVID, DVID, CEID, RPTID, ALID, ECID, PPID,
MID, CARRIERID, PRJOBID, CTLJOBID, SUBSTID, OBJSPEC, OBJTYPE,
MDLN, SOFTREV, EQPTYP, DATAID + every ACK code (COMMACK, ONLACK,
OFLACK, HCACK, CMDA, ACKC5-7-10, DRACK, LRACK, ERACK, EAC, TIACK,
GRANT, ALCD, OBJACK) + stream/function shorthand + HSMS terms +
T-timers + E84 signals + the standards lineup + codebase shortcuts
("the model", "the router", "the proof", etc.). Cuts week-1
onboarding time.
4. FAQ.md is new. Canonical answers to the questions that come up
once per integration: why HSMS unencrypted, SVID vs DVID, PJ vs
CJ, who fires FSM transitions, what runs on which thread, how to
add a new SECS-II message, ASCII vs Binary, common MES quirks,
how spool works, robustness fuzz vs libFuzzer, conformance vs
interop, what's not implemented.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,225 @@
|
||||
# FAQ
|
||||
|
||||
Questions we hear once per integration. Skim before you ask. If
|
||||
your question isn't here and isn't obvious from the other docs,
|
||||
ask once — your question probably belongs in this file and we'll
|
||||
add it.
|
||||
|
||||
## Why is HSMS unencrypted?
|
||||
|
||||
Because SEMI E37 says so. HSMS is plain TCP with a 14-byte
|
||||
framing header — no TLS, no auth, no nonces. Every commercial MES
|
||||
on the market speaks exactly that wire, and changing it would make
|
||||
us incompatible with all of them. Encryption and authentication
|
||||
belong at the network layer: see [SECURITY.md](SECURITY.md) for
|
||||
the stunnel.conf + nftables setup that wraps the unencrypted TCP
|
||||
in mTLS without modifying the wire protocol.
|
||||
|
||||
## What's the difference between SVID and DVID?
|
||||
|
||||
**SVID** is a *status* variable — equipment state the host queries
|
||||
(chamber pressure, current control state, wafer counter).
|
||||
**DVID** is a *data* variable — intermediate values, typically
|
||||
computed or sensor-derived, that aren't part of the equipment's
|
||||
state model.
|
||||
|
||||
In practice fab tools blur the line. The library treats them
|
||||
identically except for which message reports them: `S1F3 / S1F11`
|
||||
for SVIDs, `S1F21 / S1F22` for DVIDs. Variable lookups by VID
|
||||
span both (`EquipmentDataModel::vid_value`).
|
||||
|
||||
## Do I really need all four YAML files?
|
||||
|
||||
Yes for production; no for a quick "does it compile":
|
||||
|
||||
- `equipment.yaml` — your tool's data dictionary. Required.
|
||||
- `control_state.yaml` — the E30 control state machine (HostOffline,
|
||||
AttemptOnline, OnlineRemote, …). The default in `data/` works as
|
||||
a starting point; you may customize transitions.
|
||||
- `process_job_state.yaml` — the E40 PJ FSM. Default is spec-typical;
|
||||
customize only if your tool has unusual recipe semantics.
|
||||
- `control_job_state.yaml` — the E94 CJ FSM. Same.
|
||||
|
||||
`secs_server --validate-config` checks all four in one pass and
|
||||
exits 0 / 1. Run it in CI on every config change.
|
||||
|
||||
## PJ vs CJ — what's the difference?
|
||||
|
||||
A **PJ** (E40 Process Job) is "process this batch of material with
|
||||
this recipe." One PJ = one recipe run = one set of wafers. It
|
||||
has its own FSM (Queued → SettingUp → Processing → ProcessComplete).
|
||||
|
||||
A **CJ** (E94 Control Job) is "execute these PJs in order, as a
|
||||
unit, with start/pause/abort semantics." A CJ owns an ordered list
|
||||
of PRJOBIDs. When the host issues `CJSTART`, the CJ promotes its
|
||||
PJs through their lifecycles.
|
||||
|
||||
You typically need both: the MES creates a CJ containing N PJs,
|
||||
then starts the CJ. PJs without a CJ are legal — they just sit in
|
||||
Queued waiting for someone to select them — but most MES drives
|
||||
batches through CJs.
|
||||
|
||||
## Who fires FSM transitions — the library or my code?
|
||||
|
||||
**Your code.** The library implements the FSMs (legal transitions,
|
||||
validation, persistence) but it doesn't know when a wafer was
|
||||
actually loaded or when a recipe step finished — those signals come
|
||||
from your tool. The pattern across every store is:
|
||||
|
||||
```cpp
|
||||
// You fire the event; the FSM validates + transitions + emits.
|
||||
model->process_jobs.fire_internal("PJ-1", gem::ProcessJobEvent::SetupComplete);
|
||||
model->carriers.fire_id_event("CAR-A1B2", gem::CarrierIDEvent::Read);
|
||||
```
|
||||
|
||||
Host commands (`S2F41` RCMD=START, `S16F5` PRJSTART, `S16F27` CJSTART)
|
||||
arrive via the wire and get dispatched into your registered
|
||||
handlers; the handler typically calls `fire_internal` or
|
||||
`on_host_command` on the relevant store.
|
||||
|
||||
See INTEGRATION.md §4 for the worked patterns.
|
||||
|
||||
## What runs on which thread?
|
||||
|
||||
**Everything that touches the data model runs on the io_context
|
||||
thread.** There are no locks in `EquipmentDataModel`.
|
||||
|
||||
- The Router dispatch (incoming wire messages) — on the io_context.
|
||||
- All `set_*_change_handler` callbacks — on the io_context.
|
||||
- Periodic timers you register via asio — on the io_context.
|
||||
|
||||
If your code lives on another thread (typical for sensor polling),
|
||||
marshal updates via `asio::post`:
|
||||
|
||||
```cpp
|
||||
asio::post(io.get_executor(), [model, value] {
|
||||
model->svids.set_value(100, secs2::Item::f4(value));
|
||||
});
|
||||
```
|
||||
|
||||
INTEGRATION.md §3 has the full thread-safety contract.
|
||||
|
||||
## How do I add a new SECS-II message?
|
||||
|
||||
Edit `data/messages.yaml`, add a row, rebuild. The codegen
|
||||
(`tools/gen_messages.py`) emits a typed builder + parser into
|
||||
`messages.hpp`. Then register a Router handler in your `main.cpp`
|
||||
for the new `(stream, function)` pair. See README "Adding a
|
||||
capability" or ARCHITECTURE.md for the full walkthrough.
|
||||
|
||||
## What's the difference between `Item::ascii("X")` and `Item::binary({'X'})`?
|
||||
|
||||
The wire format byte differs — `0x41 01 58` for ASCII vs
|
||||
`0x21 01 58` for Binary. Some peers (notably secsgem-py) default
|
||||
PPBODY to ASCII; others use Binary. Our codec accepts either via
|
||||
the `BINARY_OR_ASCII` codegen type for fields the spec lists as
|
||||
`ASCII | Binary | List` (the PPBODY case in S7F3/F6).
|
||||
|
||||
For most fields it doesn't matter — pick the format that matches
|
||||
your data semantically.
|
||||
|
||||
## My MES sends a message that worked in `interop` but fails in production. What's going on?
|
||||
|
||||
Three usual suspects:
|
||||
|
||||
1. **U-width.** Your MES is sending `DATAID` as U1 but our handler
|
||||
was strict for U4. We're lenient now via `any_unsigned_first`,
|
||||
but if you have custom handlers in your code, use that helper
|
||||
rather than `as_u4_scalar` for identifier fields.
|
||||
2. **PPBODY direction.** Some MES send PPBODY as ASCII even when
|
||||
the spec says it can be Binary. Use `as_text_or_binary` not
|
||||
`as_binary`.
|
||||
3. **Trailing fields.** Some MES add proprietary trailing fields
|
||||
to S2F41 / S16F11 / S3F17 bodies that aren't in the standard.
|
||||
Our parsers are tolerant of extras; check your handler's
|
||||
assumptions.
|
||||
|
||||
See MES_INTEROP.md §13 for the per-MES quirk register.
|
||||
|
||||
## What if the spec is ambiguous on some detail?
|
||||
|
||||
Cross-check against the secsgem-py and secs4j wire output:
|
||||
[VERIFICATION.md](VERIFICATION.md). If both peers agree on a
|
||||
shape, that's the working interpretation regardless of how you read
|
||||
the spec text. If they disagree, the secsgem-py output usually
|
||||
wins (it's the de-facto Python reference and most MES vendors test
|
||||
against it), but file the question — we may need a new test.
|
||||
|
||||
## Can I run this without Docker?
|
||||
|
||||
In principle yes — you need g++-13 (or any C++20 compiler), CMake,
|
||||
Ninja, libasio-dev, libyaml-cpp-dev, python3. But every doc, every
|
||||
CI lane, every test command in the repo assumes Docker. Going
|
||||
off-piste means re-deriving the build on your host. We don't
|
||||
support it; we don't actively break it.
|
||||
|
||||
## How does persistence survive a crash mid-write?
|
||||
|
||||
Every store uses a `.tmp + atomic rename` pattern: writes go to
|
||||
`<file>.tmp`, then `rename(2)`s into place. POSIX guarantees the
|
||||
rename is atomic on the same filesystem. A crash mid-write loses
|
||||
the `.tmp` (corrupt-drop on next replay) but leaves the prior
|
||||
record intact.
|
||||
|
||||
Every store's loader accepts versions in `[1, kVersion]` so future
|
||||
schema bumps don't nuke old records — see README §Production
|
||||
deployment "Schema migrations."
|
||||
|
||||
## What does the "spool" actually do?
|
||||
|
||||
When the host MES disconnects, the equipment can't deliver
|
||||
unsolicited S5F1 alarms / S6F11 events. Without spool they'd be
|
||||
lost.
|
||||
|
||||
With spool enabled (`SpoolStore::set_spoolable_streams({5, 6})`),
|
||||
those frames queue to in-memory FIFO (and persistent disk if
|
||||
`enable_persistence` is set). On the host's next SELECT, the
|
||||
equipment emits `S6F25 SpoolDataReady(count)`; the host issues
|
||||
`S6F23(Transmit)` to drain, or `S6F23(Purge)` to discard.
|
||||
|
||||
It's the GEM equivalent of an outbox. See E30 §6.22 and our
|
||||
SpoolStore source.
|
||||
|
||||
## How is "robustness fuzz" different from "libFuzzer"?
|
||||
|
||||
- **Robustness fuzz** (`tests/test_robustness_fuzz.cpp`) is a
|
||||
*model-level* property test. It picks random tool operations
|
||||
(PJ create, alarm set, substrate move, …) respecting FSM
|
||||
legality, and checks invariants after each.
|
||||
- **libFuzzer** (`apps/fuzz_*.cpp`) is a *byte-level* coverage-
|
||||
guided fuzzer. It feeds arbitrary bytes to the codec and
|
||||
asserts no crash / UB.
|
||||
|
||||
They cover different concerns: robustness fuzz catches *semantic*
|
||||
bugs (lost data, wrong state); libFuzzer catches *parser* bugs
|
||||
(crashes, UB, buffer overruns).
|
||||
|
||||
## What's "conformance" vs "interop"?
|
||||
|
||||
- **Conformance** (`build/secs_conformance`) is *us* driving *us*
|
||||
through every claimed E30 capability and asserting the spec-
|
||||
mandated reply S/F. Catches our regressions against our own
|
||||
understanding of the spec.
|
||||
- **Interop** (`interop/*.py`, `interop/secs4j/*.java`,
|
||||
`interop/tshark_validate.sh`) is third-party tools agreeing on
|
||||
the wire bytes our equipment produces. Catches "we got the
|
||||
spec wrong" — which conformance can't.
|
||||
|
||||
Both are necessary; neither replaces the other. See VERIFICATION.md.
|
||||
|
||||
## How do I bring this to a customer site?
|
||||
|
||||
Run through the five external proofs in
|
||||
[README's proof table](README.md#proof-of-feature-completeness) at
|
||||
the customer's network. Then walk MES_INTEROP.md against their
|
||||
actual MES. Then deploy per [SECURITY.md](SECURITY.md) for the
|
||||
nftables / stunnel / signing setup. Then page on the metrics from
|
||||
INTEGRATION.md §6.4.
|
||||
|
||||
## What's not implemented?
|
||||
|
||||
See [COMPLIANCE.md](COMPLIANCE.md) §8 ("Explicitly out of scope")
|
||||
for the honest list. The short version: multi-block SECS-I
|
||||
transfers (irrelevant on HSMS), tool-specific Equipment Processing
|
||||
States (the engine is there, vendors plug in their states), and
|
||||
GEM RTS certification (paid third-party gate, not a code feature).
|
||||
Reference in New Issue
Block a user