raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: chapters 40, 41, 50, 51 — Operations + Reference (series complete)
tests / build-and-test (push) Successful in 2m7s
Details
tests / thread-sanitizer (push) Successful in 2m34s
Details
tests / tshark-dissector (push) Successful in 2m22s
Details
tests / secs4j-interop (push) Failing after 5s
Details
tests / libfuzzer (push) Successful in 3m7s
Details

Browse Source 
Last four chapters of the guided tour.

40 — Building, running, the demo.  Docker prerequisites, the build
flow, what each binary is for, running the 24-transaction demo
flow annotated step by step.  Running the 4 external-validator
sweeps + the libFuzzer pass.  Inspecting the demo with tcpdump and
tshark.  Reading source while running as the recommended learning
workflow.

41 — Integration: hardware, MES, production.  Four-phase tour:
wiring sensors / recipe engine / alarms / E84 GPIO; talking to a
real MES with the day-1 punch list + commercial-MES quirks (Wonderware
S2F21, Camstar Linktest cadence, etc.); production hardening
(nftables / stunnel / minisign / persistence layout / monitoring /
runbook); performance envelope + memory footprint + capacity
planning.  Pointers to the long-form INTEGRATION.md / MES_INTEROP.md /
SECURITY.md / BENCHMARKS.md.

50 — API + message catalog + YAML schemas reference.  Namespace-by-
namespace table of public symbols (secs2, hsms, secsi, gem, config,
metrics) with brief descriptions.  Stream-by-stream message catalog
reference (S1, S2, S3, S5, S6, S7, S9, S10, S12, S14, S16).  YAML
schema reference for messages.yaml + the three state-table files +
equipment.yaml.

51 — Extending the codebase.  Seven recipes ordered from no-code to
substantial: new SVID/DVID/ECID (YAML only), new CEID with reports
(YAML only), new host command (YAML + optional handler), new control-
state transition (YAML only), new SECS-II message (YAML + handler),
new store (header + tests), new persistence backend (drop-in vs
pluggable trade-off).  Each recipe has the actual mechanical steps,
the test pattern, and pointers to the chapter that explains why it
works.

Index updated to mark all 24 chapters published.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-09 20:28:21 +02:00
parent cae98d9a7d
commit 31f908e1bf
5 changed files with 1513 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/50_api_messages_yaml_reference.md
+345
View File
@@ -0,0 +1,345 @@
# 50 — API + message catalog + YAML schemas
← [41 Integration: hardware, MES, production](41_integration_hardware_mes_production.md) | [Back to index](00_index.md) | Next: [51 Extending the codebase](51_extending_the_codebase.md) →
This chapter is **reference**, not tutorial. Look up the namespace
or YAML key you need; cross-reference the code.
The whole codebase is small enough that "go read the header" is
often the right answer — this chapter helps you find which header.
---
## Namespace reference
### `secsgem::secs2` — codec (chapter 34)
```cpp
#include "secsgem/secs2/item.hpp"
#include "secsgem/secs2/codec.hpp"
#include "secsgem/secs2/message.hpp"
#include "secsgem/secs2/sml.hpp"
```
| Symbol | What it is |
|-------------------------------------------------|----------------------------------------------------|
| `enum class Format` | 16 SECS-II format codes. |
| `class Item` | Variant-based value type. |
| `class Message` | Stream + function + W-bit + system_bytes + body. |
| `class CodecError` | Thrown on malformed input. |
| `std::vector<uint8_t> encode(const Item&)` | Serialize an Item to bytes. |
| `void encode_into(const Item&, std::vector<uint8_t>&)` | Append-encode into existing buffer. |
| `Item decode(const std::vector<uint8_t>&)` | Decode one Item from a complete buffer. |
| `Item decode_at(const uint8_t*, size_t, size_t&)`| Decode one Item from a position; advances cursor. |
| `std::string to_sml(const Item&)` | Render as SML. |
| `std::optional<Item> try_parse_sml(const std::string&)` | Parse SML; returns nullopt on error. |
### `secsgem::hsms` — TCP transport (chapter 11, 33)
```cpp
#include "secsgem/hsms/header.hpp"
#include "secsgem/hsms/connection.hpp"
```
| Symbol | What it is |
|-------------------------------------------------|----------------------------------------------------|
| `enum class SType` | 9 session types. |
| `enum class SelectStatus / DeselectStatus / RejectReason` | Reply codes. |
| `struct Header` | 10-byte HSMS header. |
| `struct Frame` | Header + body, length-prefixed on wire. |
| `class FrameError` | Thrown on framing errors. |
| `struct Timers` | T3/T5/T6/T7/T8 + linktest cadence. |
| `class Connection` | One-socket session manager. |
| `Connection::Mode { Active, Passive }` | TCP role. |
| `Connection::State { NotSelected, Selected }` | Transport state. |
### `secsgem::secsi` — SECS-I transport (chapter 12, 33)
```cpp
#include "secsgem/secsi/header.hpp"
#include "secsgem/secsi/block.hpp"
#include "secsgem/secsi/protocol.hpp"
#include "secsgem/secsi/tcp_transport.hpp"
```
| Symbol | What it is |
|-------------------------------------------------|----------------------------------------------------|
| `struct Header` | 10-byte SECS-I block header (R/W/E + system bytes).|
| `class Block` | One block (header + body + checksum). |
| `split_message(msg)` / `assemble_message(blocks)`| Multi-block split / assemble. |
| `class Protocol` | IO-free FSM. |
| `enum class Timer { T1, T2, T3, T4 }` | Timer IDs (raised via `EventTimeout`). |
| `Action / Event` variants | FSM IO. |
| `class TcpTransport` | asio adapter for testing tunnels. |
### `secsgem::gem` — behavioural layer (chapters 13–19, 32, 35)
```cpp
#include "secsgem/gem/data_model.hpp" // composite
#include "secsgem/gem/router.hpp" // dispatch table
#include "secsgem/gem/control_state.hpp" // E30 control FSM
#include "secsgem/gem/communication_state.hpp" // E30 comm FSM
#include "secsgem/gem/process_job_state.hpp" // E40
#include "secsgem/gem/control_job_state.hpp" // E94
#include "secsgem/gem/carrier_state.hpp" // E87
#include "secsgem/gem/load_port_state.hpp" // E87
#include "secsgem/gem/substrate_state.hpp" // E90
#include "secsgem/gem/module_state.hpp" // E157
#include "secsgem/gem/ept_state.hpp" // E116
#include "secsgem/gem/exception_state.hpp" // E5 §13
#include "secsgem/gem/e84_state.hpp" // E84 FSM
#include "secsgem/gem/e84_timers.hpp" // E84 TA1/TA2/TA3
#include "secsgem/gem/e84_asio_timers.hpp" // asio wrapper
#include "secsgem/gem/host_handler.hpp" // host-side analogue
#include "secsgem/gem/messages_helpers.hpp" // identifier wildcards
// Plus build/generated/secsgem/gem/messages.hpp (codegen).
```
`include/secsgem/gem/store/` — 18 per-domain stores. See
chapter [32](32_stores_and_the_data_model.md) for the full table.
### `secsgem::config` — YAML loader + validator (chapter 31, 36)
```cpp
#include "secsgem/config/loader.hpp"
#include "secsgem/config/validate.hpp"
```
| Symbol | What it loads |
|-------------------------------------------------|----------------------------------------------------|
| `load_equipment(path)` | `data/equipment.yaml` → `EquipmentDescriptor`. |
| `load_control_state_table(path)` | `data/control_state.yaml` → `ControlStateConfig`. |
| `load_process_job_state(path)` | `data/process_job_state.yaml`. |
| `load_control_job_state(path)` | `data/control_job_state.yaml`. |
| `class ConfigValidator` | Multi-error YAML validator. |
### `secsgem::metrics` — Prometheus exporter (chapter 36)
```cpp
#include "secsgem/metrics/prometheus.hpp"
```
| Symbol | What it is |
|-------------------------------------------------|----------------------------------------------------|
| `class Registry` | Holds Counter + Gauge series with labels. |
| `enum class MetricType { Counter, Gauge }` | |
| `class PrometheusServer` | HTTP server on a configurable port. |
---
## Message catalog reference
164 entries in [`data/messages.yaml`](../data/messages.yaml).
Grouped by stream:
### S1 — Identification + status
| S/F | W | Name | Body |
|-------|---|-------------------------------|--------------------------------------------|
| S1F1 | W | Are You There | none |
| S1F2 | | On-Line Data | `<L,2> [MDLN, SOFTREV]` |
| S1F3 | W | Selected Equipment Status Req | `<L,n> [SVID, SVID, ...]` |
| S1F4 | | Selected Equipment Status Data| `<L,n> [SV, SV, ...]` |
| S1F11 | W | Status Variable Namelist Req | `<L,n> [SVID, ...]` |
| S1F12 | | Status Variable Namelist | `<L,n> [<L,3> [SVID, SVNAME, UNITS]]` |
| S1F13 | W | Establish Communications Req | `<L,2> [MDLN, SOFTREV]` |
| S1F14 | | Establish Communications Ack | `<L,2> [COMMACK, <L,2> [MDLN, SOFTREV]]` |
| S1F15 | W | Request Offline | none |
| S1F16 | | OFLACK | `OFLACK` |
| S1F17 | W | Request Online | none |
| S1F18 | | ONLACK | `ONLACK` |
| S1F19 | W | Compliance Request | none |
| S1F20 | | Compliance Data | `<L,n> [CCODE, ...]` |
| S1F21 | W | DVID Namelist Request | `<L,n> [DVID, ...]` |
| S1F22 | | DVID Namelist | `<L,n> [<L,3> [DVID, DVNAME, UNITS]]` |
| S1F23 | W | CEID Namelist Request | `<L,n> [CEID, ...]` |
| S1F24 | | CEID Namelist | `<L,n> [<L,2> [CEID, [VID, VID, ...]]]` |
### S2 — Equipment constants, clock, events, commands, spool
S2F13/F14 (EC values), S2F15/F16 (set EC), S2F17/F18 (clock read),
S2F21/F22 (legacy RCMD), S2F23/F24 (trace init), S2F29/F30 (EC
namelist), S2F31/F32 (set clock), S2F33–F38 (report wiring),
S2F41/F42 (modern RCMD), S2F43/F44 (set spool streams),
S2F45–F48 (limits), S2F49/F50 (enhanced RCMD).
### S3 — Carrier management (E87)
S3F17/F18 (CarrierAction), S3F19/F20 (slot map verify),
S3F25/F26 (carrier transfer), S3F27/F28 (cancel carrier).
### S5 — Alarms + exception recovery
S5F1/F2 (alarm set/clear), S5F3/F4 (enable/disable alarm),
S5F5/F6 (list all alarms), S5F7/F8 (list enabled alarms),
S5F9–F18 (exception recovery, chapter 19).
### S6 — Data collection
S6F1/F2 (trace data), S6F11/F12 (event report), S6F15/F16 (event
report request), S6F19/F20 (individual report request),
S6F21/F22 (annotated individual report), S6F23/F24 (spool data
transmit/purge), S6F25/F26 (spool notification).
### S7 — Process program management
S7F1/F2 (PP load inquire), S7F3/F4 (PP send unformatted),
S7F5/F6 (PP request), S7F17/F18 (PP delete), S7F19/F20 (PP
namelist), S7F23/F24 (formatted PP send, E42), S7F25/F26
(formatted PP request).
### S9 — Protocol-error reports
S9F1 (unrecognized device ID), S9F3 (unrecognized stream),
S9F5 (unrecognized function), S9F7 (illegal data), S9F9 (T3
timeout), S9F11 (data too long), S9F13 (conversation timer
timeout). Auto-emitted; see chapter 11.
### S10 — Terminal services
S10F1/F2 (terminal display single, equipment→host),
S10F3/F4 (terminal display single, host→equipment),
S10F5/F6 (terminal display multi, host→equipment).
### S12 — Wafer maps
S12F* — Per E5 §13. Round-tripped through `raw_gem300_harness.py`.
### S14 — Object services (E39) + control jobs (E94)
S14F1/F2 (GetAttr), S14F3/F4 (SetAttr), S14F9/F10 (CreateCJ),
S14F11/F12 (DeleteCJ).
### S16 — Process jobs (E40)
S16F5/F6 (PRJobCommand), S16F7/F8 (PRJobMonitor), S16F9 (PRJobAlert
— unsolicited), S16F11/F12 (PRJobCreate), S16F13/F14 (PRJobDequeue),
S16F27/F28 (CJCommand).
For per-message body shapes, look up the YAML entry in
`data/messages.yaml`.
---
## YAML schema reference
### `data/messages.yaml`
```yaml
messages:
- id: S<X>F<Y> # required, must match (stream, function)
stream: <int 1-127>
function: <int 0-255>
w: <bool> # reply expected?
builder: <ident> # C++ builder function name
parser: <ident> # C++ parser function name
body: <body-shape> # see chapter 31 for the grammar
```
Body shapes:
```yaml
body: none
body:
kind: scalar
item_type: ASCII | BINARY_BYTE | BOOLEAN | U1..U8 | I1..I8 | F4 | F8 | ITEM
enum: <C++ enum type> # optional
param: <name> # optional, default 'value'
body:
kind: list
struct_name: <C++ type> # optional; if set, parser returns struct
fields:
- {name: <field>, shape: <body-shape>}
- ...
body:
kind: list_of
element: <body-shape>
name: <name> # parameter name, default 'values'
```
### `data/control_state.yaml`
```yaml
transitions:
- {from: <state>, on: <event>, to: <state>, then: <state>, ack: <code>}
```
`from`: one of `EquipmentOffline | AttemptOnline | HostOffline | OnlineLocal | OnlineRemote`.
`on`: one of `operator_switch_online | operator_switch_offline | operator_switch_local | operator_switch_remote | attempt_complete | attempt_failed | host_request_online | host_request_offline`.
`to`: optional new state.
`then`: optional chained state (for AttemptOnline pass-through).
`ack`: optional ACK code (`Accept`, `NotAccept`, `AlreadyOnline`).
### `data/process_job_state.yaml`
```yaml
transitions:
- {from: <state>, on: <event>, to: <state>}
```
`from` / `to`: `Queued | SettingUp | WaitingForStart | Processing | ProcessComplete | Paused | Stopping | Aborting`.
`on`: `select | setup_complete | start | pause | resume | stop | abort | process_complete | abort_complete`.
### `data/control_job_state.yaml`
Same shape, different state/event names — see
[`include/secsgem/gem/control_job_state.hpp`](../include/secsgem/gem/control_job_state.hpp).
### `data/equipment.yaml`
```yaml
device:
mdln: <ASCII>
softrev: <ASCII>
capabilities: [<list of GEM capability strings>]
svids:
- {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>}
dvids:
- {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>}
ecids:
- {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>, min: <num>, max: <num>}
ceids:
- {id: <U4>, name: <ASCII>}
alarms:
- {id: <U4>, alcd: <byte>, text: <ASCII>}
recipes:
- {id: <ASCII>, body: <bytes-or-string>}
host_commands:
- {name: <ASCII>, ack: <ACK enum>, emit_ceid: <U4>, set_alarm: <U4>}
events:
default_reports:
- {ceid: <U4>, vids: [<U4>, ...]}
spool:
whitelist: [<stream>, ...]
persistent_dir: <path> # optional
```
Type strings: `ASCII`, `BINARY`, `BOOLEAN`, `U1`–`U8`, `I1`–`I8`,
`F4`, `F8`. Same vocabulary as `data/messages.yaml` body shapes.
For required vs optional fields per record, see the validator
checks in
[`tests/test_config_validate.cpp`](../tests/test_config_validate.cpp).
---
## Where to go next
The last chapter is the practical companion to this one: the
**recipes** for extending the codebase — adding a new SVID, host
command, state, message, store, or persistence backend. Code
patches you can copy.
Next: [→ 51 Extending the codebase](51_extending_the_codebase.md)