raphael
31f908e1bf
Last four chapters of the guided tour. 40 — Building, running, the demo. Docker prerequisites, the build flow, what each binary is for, running the 24-transaction demo flow annotated step by step. Running the 4 external-validator sweeps + the libFuzzer pass. Inspecting the demo with tcpdump and tshark. Reading source while running as the recommended learning workflow. 41 — Integration: hardware, MES, production. Four-phase tour: wiring sensors / recipe engine / alarms / E84 GPIO; talking to a real MES with the day-1 punch list + commercial-MES quirks (Wonderware S2F21, Camstar Linktest cadence, etc.); production hardening (nftables / stunnel / minisign / persistence layout / monitoring / runbook); performance envelope + memory footprint + capacity planning. Pointers to the long-form INTEGRATION.md / MES_INTEROP.md / SECURITY.md / BENCHMARKS.md. 50 — API + message catalog + YAML schemas reference. Namespace-by- namespace table of public symbols (secs2, hsms, secsi, gem, config, metrics) with brief descriptions. Stream-by-stream message catalog reference (S1, S2, S3, S5, S6, S7, S9, S10, S12, S14, S16). YAML schema reference for messages.yaml + the three state-table files + equipment.yaml. 51 — Extending the codebase. Seven recipes ordered from no-code to substantial: new SVID/DVID/ECID (YAML only), new CEID with reports (YAML only), new host command (YAML + optional handler), new control- state transition (YAML only), new SECS-II message (YAML + handler), new store (header + tests), new persistence backend (drop-in vs pluggable trade-off). Each recipe has the actual mechanical steps, the test pattern, and pointers to the chapter that explains why it works. Index updated to mark all 24 chapters published. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
346 lines
14 KiB
Markdown
346 lines
14 KiB
Markdown
# 50 — API + message catalog + YAML schemas
|
||
|
||
← [41 Integration: hardware, MES, production](41_integration_hardware_mes_production.md) | [Back to index](00_index.md) | Next: [51 Extending the codebase](51_extending_the_codebase.md) →
|
||
|
||
This chapter is **reference**, not tutorial. Look up the namespace
|
||
or YAML key you need; cross-reference the code.
|
||
|
||
The whole codebase is small enough that "go read the header" is
|
||
often the right answer — this chapter helps you find which header.
|
||
|
||
---
|
||
|
||
## Namespace reference
|
||
|
||
### `secsgem::secs2` — codec (chapter 34)
|
||
|
||
```cpp
|
||
#include "secsgem/secs2/item.hpp"
|
||
#include "secsgem/secs2/codec.hpp"
|
||
#include "secsgem/secs2/message.hpp"
|
||
#include "secsgem/secs2/sml.hpp"
|
||
```
|
||
|
||
| Symbol | What it is |
|
||
|-------------------------------------------------|----------------------------------------------------|
|
||
| `enum class Format` | 16 SECS-II format codes. |
|
||
| `class Item` | Variant-based value type. |
|
||
| `class Message` | Stream + function + W-bit + system_bytes + body. |
|
||
| `class CodecError` | Thrown on malformed input. |
|
||
| `std::vector<uint8_t> encode(const Item&)` | Serialize an Item to bytes. |
|
||
| `void encode_into(const Item&, std::vector<uint8_t>&)` | Append-encode into existing buffer. |
|
||
| `Item decode(const std::vector<uint8_t>&)` | Decode one Item from a complete buffer. |
|
||
| `Item decode_at(const uint8_t*, size_t, size_t&)`| Decode one Item from a position; advances cursor. |
|
||
| `std::string to_sml(const Item&)` | Render as SML. |
|
||
| `std::optional<Item> try_parse_sml(const std::string&)` | Parse SML; returns nullopt on error. |
|
||
|
||
### `secsgem::hsms` — TCP transport (chapter 11, 33)
|
||
|
||
```cpp
|
||
#include "secsgem/hsms/header.hpp"
|
||
#include "secsgem/hsms/connection.hpp"
|
||
```
|
||
|
||
| Symbol | What it is |
|
||
|-------------------------------------------------|----------------------------------------------------|
|
||
| `enum class SType` | 9 session types. |
|
||
| `enum class SelectStatus / DeselectStatus / RejectReason` | Reply codes. |
|
||
| `struct Header` | 10-byte HSMS header. |
|
||
| `struct Frame` | Header + body, length-prefixed on wire. |
|
||
| `class FrameError` | Thrown on framing errors. |
|
||
| `struct Timers` | T3/T5/T6/T7/T8 + linktest cadence. |
|
||
| `class Connection` | One-socket session manager. |
|
||
| `Connection::Mode { Active, Passive }` | TCP role. |
|
||
| `Connection::State { NotSelected, Selected }` | Transport state. |
|
||
|
||
### `secsgem::secsi` — SECS-I transport (chapter 12, 33)
|
||
|
||
```cpp
|
||
#include "secsgem/secsi/header.hpp"
|
||
#include "secsgem/secsi/block.hpp"
|
||
#include "secsgem/secsi/protocol.hpp"
|
||
#include "secsgem/secsi/tcp_transport.hpp"
|
||
```
|
||
|
||
| Symbol | What it is |
|
||
|-------------------------------------------------|----------------------------------------------------|
|
||
| `struct Header` | 10-byte SECS-I block header (R/W/E + system bytes).|
|
||
| `class Block` | One block (header + body + checksum). |
|
||
| `split_message(msg)` / `assemble_message(blocks)`| Multi-block split / assemble. |
|
||
| `class Protocol` | IO-free FSM. |
|
||
| `enum class Timer { T1, T2, T3, T4 }` | Timer IDs (raised via `EventTimeout`). |
|
||
| `Action / Event` variants | FSM IO. |
|
||
| `class TcpTransport` | asio adapter for testing tunnels. |
|
||
|
||
### `secsgem::gem` — behavioural layer (chapters 13–19, 32, 35)
|
||
|
||
```cpp
|
||
#include "secsgem/gem/data_model.hpp" // composite
|
||
#include "secsgem/gem/router.hpp" // dispatch table
|
||
#include "secsgem/gem/control_state.hpp" // E30 control FSM
|
||
#include "secsgem/gem/communication_state.hpp" // E30 comm FSM
|
||
#include "secsgem/gem/process_job_state.hpp" // E40
|
||
#include "secsgem/gem/control_job_state.hpp" // E94
|
||
#include "secsgem/gem/carrier_state.hpp" // E87
|
||
#include "secsgem/gem/load_port_state.hpp" // E87
|
||
#include "secsgem/gem/substrate_state.hpp" // E90
|
||
#include "secsgem/gem/module_state.hpp" // E157
|
||
#include "secsgem/gem/ept_state.hpp" // E116
|
||
#include "secsgem/gem/exception_state.hpp" // E5 §13
|
||
#include "secsgem/gem/e84_state.hpp" // E84 FSM
|
||
#include "secsgem/gem/e84_timers.hpp" // E84 TA1/TA2/TA3
|
||
#include "secsgem/gem/e84_asio_timers.hpp" // asio wrapper
|
||
#include "secsgem/gem/host_handler.hpp" // host-side analogue
|
||
#include "secsgem/gem/messages_helpers.hpp" // identifier wildcards
|
||
// Plus build/generated/secsgem/gem/messages.hpp (codegen).
|
||
```
|
||
|
||
`include/secsgem/gem/store/` — 18 per-domain stores. See
|
||
chapter [32](32_stores_and_the_data_model.md) for the full table.
|
||
|
||
### `secsgem::config` — YAML loader + validator (chapter 31, 36)
|
||
|
||
```cpp
|
||
#include "secsgem/config/loader.hpp"
|
||
#include "secsgem/config/validate.hpp"
|
||
```
|
||
|
||
| Symbol | What it loads |
|
||
|-------------------------------------------------|----------------------------------------------------|
|
||
| `load_equipment(path)` | `data/equipment.yaml` → `EquipmentDescriptor`. |
|
||
| `load_control_state_table(path)` | `data/control_state.yaml` → `ControlStateConfig`. |
|
||
| `load_process_job_state(path)` | `data/process_job_state.yaml`. |
|
||
| `load_control_job_state(path)` | `data/control_job_state.yaml`. |
|
||
| `class ConfigValidator` | Multi-error YAML validator. |
|
||
|
||
### `secsgem::metrics` — Prometheus exporter (chapter 36)
|
||
|
||
```cpp
|
||
#include "secsgem/metrics/prometheus.hpp"
|
||
```
|
||
|
||
| Symbol | What it is |
|
||
|-------------------------------------------------|----------------------------------------------------|
|
||
| `class Registry` | Holds Counter + Gauge series with labels. |
|
||
| `enum class MetricType { Counter, Gauge }` | |
|
||
| `class PrometheusServer` | HTTP server on a configurable port. |
|
||
|
||
---
|
||
|
||
## Message catalog reference
|
||
|
||
164 entries in [`data/messages.yaml`](../data/messages.yaml).
|
||
Grouped by stream:
|
||
|
||
### S1 — Identification + status
|
||
|
||
| S/F | W | Name | Body |
|
||
|-------|---|-------------------------------|--------------------------------------------|
|
||
| S1F1 | W | Are You There | none |
|
||
| S1F2 | | On-Line Data | `<L,2> [MDLN, SOFTREV]` |
|
||
| S1F3 | W | Selected Equipment Status Req | `<L,n> [SVID, SVID, ...]` |
|
||
| S1F4 | | Selected Equipment Status Data| `<L,n> [SV, SV, ...]` |
|
||
| S1F11 | W | Status Variable Namelist Req | `<L,n> [SVID, ...]` |
|
||
| S1F12 | | Status Variable Namelist | `<L,n> [<L,3> [SVID, SVNAME, UNITS]]` |
|
||
| S1F13 | W | Establish Communications Req | `<L,2> [MDLN, SOFTREV]` |
|
||
| S1F14 | | Establish Communications Ack | `<L,2> [COMMACK, <L,2> [MDLN, SOFTREV]]` |
|
||
| S1F15 | W | Request Offline | none |
|
||
| S1F16 | | OFLACK | `OFLACK` |
|
||
| S1F17 | W | Request Online | none |
|
||
| S1F18 | | ONLACK | `ONLACK` |
|
||
| S1F19 | W | Compliance Request | none |
|
||
| S1F20 | | Compliance Data | `<L,n> [CCODE, ...]` |
|
||
| S1F21 | W | DVID Namelist Request | `<L,n> [DVID, ...]` |
|
||
| S1F22 | | DVID Namelist | `<L,n> [<L,3> [DVID, DVNAME, UNITS]]` |
|
||
| S1F23 | W | CEID Namelist Request | `<L,n> [CEID, ...]` |
|
||
| S1F24 | | CEID Namelist | `<L,n> [<L,2> [CEID, [VID, VID, ...]]]` |
|
||
|
||
### S2 — Equipment constants, clock, events, commands, spool
|
||
|
||
S2F13/F14 (EC values), S2F15/F16 (set EC), S2F17/F18 (clock read),
|
||
S2F21/F22 (legacy RCMD), S2F23/F24 (trace init), S2F29/F30 (EC
|
||
namelist), S2F31/F32 (set clock), S2F33–F38 (report wiring),
|
||
S2F41/F42 (modern RCMD), S2F43/F44 (set spool streams),
|
||
S2F45–F48 (limits), S2F49/F50 (enhanced RCMD).
|
||
|
||
### S3 — Carrier management (E87)
|
||
|
||
S3F17/F18 (CarrierAction), S3F19/F20 (slot map verify),
|
||
S3F25/F26 (carrier transfer), S3F27/F28 (cancel carrier).
|
||
|
||
### S5 — Alarms + exception recovery
|
||
|
||
S5F1/F2 (alarm set/clear), S5F3/F4 (enable/disable alarm),
|
||
S5F5/F6 (list all alarms), S5F7/F8 (list enabled alarms),
|
||
S5F9–F18 (exception recovery, chapter 19).
|
||
|
||
### S6 — Data collection
|
||
|
||
S6F1/F2 (trace data), S6F11/F12 (event report), S6F15/F16 (event
|
||
report request), S6F19/F20 (individual report request),
|
||
S6F21/F22 (annotated individual report), S6F23/F24 (spool data
|
||
transmit/purge), S6F25/F26 (spool notification).
|
||
|
||
### S7 — Process program management
|
||
|
||
S7F1/F2 (PP load inquire), S7F3/F4 (PP send unformatted),
|
||
S7F5/F6 (PP request), S7F17/F18 (PP delete), S7F19/F20 (PP
|
||
namelist), S7F23/F24 (formatted PP send, E42), S7F25/F26
|
||
(formatted PP request).
|
||
|
||
### S9 — Protocol-error reports
|
||
|
||
S9F1 (unrecognized device ID), S9F3 (unrecognized stream),
|
||
S9F5 (unrecognized function), S9F7 (illegal data), S9F9 (T3
|
||
timeout), S9F11 (data too long), S9F13 (conversation timer
|
||
timeout). Auto-emitted; see chapter 11.
|
||
|
||
### S10 — Terminal services
|
||
|
||
S10F1/F2 (terminal display single, equipment→host),
|
||
S10F3/F4 (terminal display single, host→equipment),
|
||
S10F5/F6 (terminal display multi, host→equipment).
|
||
|
||
### S12 — Wafer maps
|
||
|
||
S12F* — Per E5 §13. Round-tripped through `raw_gem300_harness.py`.
|
||
|
||
### S14 — Object services (E39) + control jobs (E94)
|
||
|
||
S14F1/F2 (GetAttr), S14F3/F4 (SetAttr), S14F9/F10 (CreateCJ),
|
||
S14F11/F12 (DeleteCJ).
|
||
|
||
### S16 — Process jobs (E40)
|
||
|
||
S16F5/F6 (PRJobCommand), S16F7/F8 (PRJobMonitor), S16F9 (PRJobAlert
|
||
— unsolicited), S16F11/F12 (PRJobCreate), S16F13/F14 (PRJobDequeue),
|
||
S16F27/F28 (CJCommand).
|
||
|
||
For per-message body shapes, look up the YAML entry in
|
||
`data/messages.yaml`.
|
||
|
||
---
|
||
|
||
## YAML schema reference
|
||
|
||
### `data/messages.yaml`
|
||
|
||
```yaml
|
||
messages:
|
||
- id: S<X>F<Y> # required, must match (stream, function)
|
||
stream: <int 1-127>
|
||
function: <int 0-255>
|
||
w: <bool> # reply expected?
|
||
builder: <ident> # C++ builder function name
|
||
parser: <ident> # C++ parser function name
|
||
body: <body-shape> # see chapter 31 for the grammar
|
||
```
|
||
|
||
Body shapes:
|
||
|
||
```yaml
|
||
body: none
|
||
|
||
body:
|
||
kind: scalar
|
||
item_type: ASCII | BINARY_BYTE | BOOLEAN | U1..U8 | I1..I8 | F4 | F8 | ITEM
|
||
enum: <C++ enum type> # optional
|
||
param: <name> # optional, default 'value'
|
||
|
||
body:
|
||
kind: list
|
||
struct_name: <C++ type> # optional; if set, parser returns struct
|
||
fields:
|
||
- {name: <field>, shape: <body-shape>}
|
||
- ...
|
||
|
||
body:
|
||
kind: list_of
|
||
element: <body-shape>
|
||
name: <name> # parameter name, default 'values'
|
||
```
|
||
|
||
### `data/control_state.yaml`
|
||
|
||
```yaml
|
||
transitions:
|
||
- {from: <state>, on: <event>, to: <state>, then: <state>, ack: <code>}
|
||
```
|
||
|
||
`from`: one of `EquipmentOffline | AttemptOnline | HostOffline | OnlineLocal | OnlineRemote`.
|
||
`on`: one of `operator_switch_online | operator_switch_offline | operator_switch_local | operator_switch_remote | attempt_complete | attempt_failed | host_request_online | host_request_offline`.
|
||
`to`: optional new state.
|
||
`then`: optional chained state (for AttemptOnline pass-through).
|
||
`ack`: optional ACK code (`Accept`, `NotAccept`, `AlreadyOnline`).
|
||
|
||
### `data/process_job_state.yaml`
|
||
|
||
```yaml
|
||
transitions:
|
||
- {from: <state>, on: <event>, to: <state>}
|
||
```
|
||
|
||
`from` / `to`: `Queued | SettingUp | WaitingForStart | Processing | ProcessComplete | Paused | Stopping | Aborting`.
|
||
`on`: `select | setup_complete | start | pause | resume | stop | abort | process_complete | abort_complete`.
|
||
|
||
### `data/control_job_state.yaml`
|
||
|
||
Same shape, different state/event names — see
|
||
[`include/secsgem/gem/control_job_state.hpp`](../include/secsgem/gem/control_job_state.hpp).
|
||
|
||
### `data/equipment.yaml`
|
||
|
||
```yaml
|
||
device:
|
||
mdln: <ASCII>
|
||
softrev: <ASCII>
|
||
capabilities: [<list of GEM capability strings>]
|
||
|
||
svids:
|
||
- {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>}
|
||
|
||
dvids:
|
||
- {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>}
|
||
|
||
ecids:
|
||
- {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>, min: <num>, max: <num>}
|
||
|
||
ceids:
|
||
- {id: <U4>, name: <ASCII>}
|
||
|
||
alarms:
|
||
- {id: <U4>, alcd: <byte>, text: <ASCII>}
|
||
|
||
recipes:
|
||
- {id: <ASCII>, body: <bytes-or-string>}
|
||
|
||
host_commands:
|
||
- {name: <ASCII>, ack: <ACK enum>, emit_ceid: <U4>, set_alarm: <U4>}
|
||
|
||
events:
|
||
default_reports:
|
||
- {ceid: <U4>, vids: [<U4>, ...]}
|
||
|
||
spool:
|
||
whitelist: [<stream>, ...]
|
||
persistent_dir: <path> # optional
|
||
```
|
||
|
||
Type strings: `ASCII`, `BINARY`, `BOOLEAN`, `U1`–`U8`, `I1`–`I8`,
|
||
`F4`, `F8`. Same vocabulary as `data/messages.yaml` body shapes.
|
||
|
||
For required vs optional fields per record, see the validator
|
||
checks in
|
||
[`tests/test_config_validate.cpp`](../tests/test_config_validate.cpp).
|
||
|
||
---
|
||
|
||
## Where to go next
|
||
|
||
The last chapter is the practical companion to this one: the
|
||
**recipes** for extending the codebase — adding a new SVID, host
|
||
command, state, message, store, or persistence backend. Code
|
||
patches you can copy.
|
||
|
||
Next: [→ 51 Extending the codebase](51_extending_the_codebase.md)
|