# 50 — API + message catalog + YAML schemas

← [41 Integration: hardware, MES, production](41_integration_hardware_mes_production.md) | [Back to index](00_index.md) | Next: [51 Extending the codebase](51_extending_the_codebase.md) →

This chapter is **reference**, not tutorial.  Look up the namespace
or YAML key you need; cross-reference the code.

The whole codebase is small enough that "go read the header" is
often the right answer — this chapter helps you find which header.

---

## Namespace reference

### `secsgem::secs2` — codec (chapter 34)

```cpp
#include "secsgem/secs2/item.hpp"
#include "secsgem/secs2/codec.hpp"
#include "secsgem/secs2/message.hpp"
#include "secsgem/secs2/sml.hpp"
```

| Symbol                                          | What it is                                         |
|-------------------------------------------------|----------------------------------------------------|
| `enum class Format`                             | 16 SECS-II format codes.                           |
| `class Item`                                    | Variant-based value type.                          |
| `class Message`                                 | Stream + function + W-bit + system_bytes + body.   |
| `class CodecError`                              | Thrown on malformed input.                         |
| `std::vector<uint8_t> encode(const Item&)`      | Serialize an Item to bytes.                        |
| `void encode_into(const Item&, std::vector<uint8_t>&)` | Append-encode into existing buffer.         |
| `Item decode(const std::vector<uint8_t>&)`      | Decode one Item from a complete buffer.            |
| `Item decode_at(const uint8_t*, size_t, size_t&)`| Decode one Item from a position; advances cursor. |
| `std::string to_sml(const Item&)`               | Render as SML.                                     |
| `std::optional<Item> try_parse_sml(const std::string&)` | Parse SML; returns nullopt on error.       |

### `secsgem::hsms` — TCP transport (chapter 11, 33)

```cpp
#include "secsgem/hsms/header.hpp"
#include "secsgem/hsms/connection.hpp"
```

| Symbol                                          | What it is                                         |
|-------------------------------------------------|----------------------------------------------------|
| `enum class SType`                              | 9 session types.                                   |
| `enum class SelectStatus / DeselectStatus / RejectReason` | Reply codes.                              |
| `struct Header`                                 | 10-byte HSMS header.                               |
| `struct Frame`                                  | Header + body, length-prefixed on wire.            |
| `class FrameError`                              | Thrown on framing errors.                          |
| `struct Timers`                                 | T3/T5/T6/T7/T8 + linktest cadence.                 |
| `class Connection`                              | One-socket session manager.                        |
| `Connection::Mode { Active, Passive }`          | TCP role.                                          |
| `Connection::State { NotSelected, Selected }`   | Transport state.                                   |

### `secsgem::secsi` — SECS-I transport (chapter 12, 33)

```cpp
#include "secsgem/secsi/header.hpp"
#include "secsgem/secsi/block.hpp"
#include "secsgem/secsi/protocol.hpp"
#include "secsgem/secsi/tcp_transport.hpp"
```

| Symbol                                          | What it is                                         |
|-------------------------------------------------|----------------------------------------------------|
| `struct Header`                                 | 10-byte SECS-I block header (R/W/E + system bytes).|
| `class Block`                                   | One block (header + body + checksum).              |
| `split_message(msg)` / `assemble_message(blocks)`| Multi-block split / assemble.                     |
| `class Protocol`                                | IO-free FSM.                                       |
| `enum class Timer { T1, T2, T3, T4 }`           | Timer IDs (raised via `EventTimeout`).             |
| `Action / Event` variants                       | FSM IO.                                            |
| `class TcpTransport`                            | asio adapter for testing tunnels.                  |

### `secsgem::gem` — behavioural layer (chapters 13–19, 32, 35)

```cpp
#include "secsgem/gem/data_model.hpp"          // composite
#include "secsgem/gem/router.hpp"              // dispatch table
#include "secsgem/gem/control_state.hpp"       // E30 control FSM
#include "secsgem/gem/communication_state.hpp" // E30 comm FSM
#include "secsgem/gem/process_job_state.hpp"   // E40
#include "secsgem/gem/control_job_state.hpp"   // E94
#include "secsgem/gem/carrier_state.hpp"       // E87
#include "secsgem/gem/load_port_state.hpp"     // E87
#include "secsgem/gem/substrate_state.hpp"     // E90
#include "secsgem/gem/module_state.hpp"        // E157
#include "secsgem/gem/ept_state.hpp"           // E116
#include "secsgem/gem/exception_state.hpp"     // E5 §13
#include "secsgem/gem/e84_state.hpp"           // E84 FSM
#include "secsgem/gem/e84_timers.hpp"          // E84 TA1/TA2/TA3
#include "secsgem/gem/e84_asio_timers.hpp"     // asio wrapper
#include "secsgem/gem/host_handler.hpp"        // host-side analogue
#include "secsgem/gem/messages_helpers.hpp"    // identifier wildcards
// Plus build/generated/secsgem/gem/messages.hpp (codegen).
```

`include/secsgem/gem/store/` — 18 per-domain stores.  See
chapter [32](32_stores_and_the_data_model.md) for the full table.

Engine-owner and default-behaviour entry points (added with the daemon
track; see [DAEMON_ROADMAP.md](DAEMON_ROADMAP.md)):

```cpp
#include "secsgem/gem/runtime.hpp"           // EquipmentRuntime: owns
        // io_context + Server + model + control FSM + Router; thread-safe
        // set_variable/emit_event/set_alarm/clear_alarm, on_command hook,
        // read_sync (the standard cross-thread read), control-state mirror,
        // add_control_state_observer / add_link_observer.
#include "secsgem/gem/default_handlers.hpp"  // the 56 GEM handlers as 15
        // per-capability register_* functions + register_default_handlers.
#include "secsgem/gem/handler_slot.hpp"      // primary + observer handler slots
#include "secsgem/gem/name_index.hpp"        // name -> VID/CEID resolution
```

### `secsgem::daemon` — the gRPC vendor surface

```cpp
#include "secsgem/daemon/equipment_service.hpp"  // EquipmentService: the
        // proto/secsgem/v1 Equipment service over an EquipmentRuntime
        // (SetVariables/GetVariables/FireEvent/SetAlarm/ClearAlarm/
        //  GetControlState/RequestControlState/WatchHealth/Subscribe/
        //  CompleteCommand). Built into apps/secs_gemd.cpp; the Python
        //  client in clients/python wraps the same proto.
```

### `secsgem::config` — YAML loader + validator (chapter 31, 36)

```cpp
#include "secsgem/config/loader.hpp"
#include "secsgem/config/validate.hpp"
```

| Symbol                                          | What it loads                                      |
|-------------------------------------------------|----------------------------------------------------|
| `load_equipment(path)`                          | `data/equipment.yaml` → `EquipmentDescriptor`.     |
| `load_control_state_table(path)`                | `data/control_state.yaml` → `ControlStateConfig`.  |
| `load_process_job_state(path)`                  | `data/process_job_state.yaml`.                     |
| `load_control_job_state(path)`                  | `data/control_job_state.yaml`.                     |
| `class ConfigValidator`                         | Multi-error YAML validator.                        |

### `secsgem::metrics` — Prometheus exporter (chapter 36)

```cpp
#include "secsgem/metrics/prometheus.hpp"
```

| Symbol                                          | What it is                                         |
|-------------------------------------------------|----------------------------------------------------|
| `class Registry`                                | Holds Counter + Gauge series with labels.          |
| `enum class MetricType { Counter, Gauge }`      |                                                    |
| `class PrometheusServer`                        | HTTP server on a configurable port.                |

---

## Message catalog reference

164 entries in [`data/messages.yaml`](../data/messages.yaml).
Grouped by stream:

### S1 — Identification + status

| S/F   | W | Name                          | Body                                       |
|-------|---|-------------------------------|--------------------------------------------|
| S1F1  | W | Are You There                 | none                                       |
| S1F2  |   | On-Line Data                  | `<L,2> [MDLN, SOFTREV]`                    |
| S1F3  | W | Selected Equipment Status Req | `<L,n> [SVID, SVID, ...]`                  |
| S1F4  |   | Selected Equipment Status Data| `<L,n> [SV, SV, ...]`                      |
| S1F11 | W | Status Variable Namelist Req  | `<L,n> [SVID, ...]`                        |
| S1F12 |   | Status Variable Namelist      | `<L,n> [<L,3> [SVID, SVNAME, UNITS]]`      |
| S1F13 | W | Establish Communications Req  | `<L,2> [MDLN, SOFTREV]`                    |
| S1F14 |   | Establish Communications Ack  | `<L,2> [COMMACK, <L,2> [MDLN, SOFTREV]]`   |
| S1F15 | W | Request Offline               | none                                       |
| S1F16 |   | OFLACK                        | `OFLACK`                                   |
| S1F17 | W | Request Online                | none                                       |
| S1F18 |   | ONLACK                        | `ONLACK`                                   |
| S1F19 | W | Compliance Request            | none                                       |
| S1F20 |   | Compliance Data               | `<L,n> [CCODE, ...]`                       |
| S1F21 | W | DVID Namelist Request         | `<L,n> [DVID, ...]`                        |
| S1F22 |   | DVID Namelist                 | `<L,n> [<L,3> [DVID, DVNAME, UNITS]]`      |
| S1F23 | W | CEID Namelist Request         | `<L,n> [CEID, ...]`                        |
| S1F24 |   | CEID Namelist                 | `<L,n> [<L,2> [CEID, [VID, VID, ...]]]`    |

### S2 — Equipment constants, clock, events, commands, spool

S2F13/F14 (EC values), S2F15/F16 (set EC), S2F17/F18 (clock read),
S2F21/F22 (legacy RCMD), S2F23/F24 (trace init), S2F29/F30 (EC
namelist), S2F31/F32 (set clock), S2F33–F38 (report wiring),
S2F41/F42 (modern RCMD), S2F43/F44 (set spool streams),
S2F45–F48 (limits), S2F49/F50 (enhanced RCMD).

### S3 — Carrier management (E87)

S3F17/F18 (CarrierAction), S3F19/F20 (slot map verify),
S3F25/F26 (carrier transfer), S3F27/F28 (cancel carrier).

### S5 — Alarms + exception recovery

S5F1/F2 (alarm set/clear), S5F3/F4 (enable/disable alarm),
S5F5/F6 (list all alarms), S5F7/F8 (list enabled alarms),
S5F9–F18 (exception recovery, chapter 19).

### S6 — Data collection

S6F1/F2 (trace data), S6F11/F12 (event report), S6F15/F16 (event
report request), S6F19/F20 (individual report request),
S6F21/F22 (annotated individual report), S6F23/F24 (spool data
transmit/purge), S6F25/F26 (spool notification).

### S7 — Process program management

S7F1/F2 (PP load inquire), S7F3/F4 (PP send unformatted),
S7F5/F6 (PP request), S7F17/F18 (PP delete), S7F19/F20 (PP
namelist), S7F23/F24 (formatted PP send, E42), S7F25/F26
(formatted PP request).

### S9 — Protocol-error reports

S9F1 (unrecognized device ID), S9F3 (unrecognized stream),
S9F5 (unrecognized function), S9F7 (illegal data), S9F9 (T3
timeout), S9F11 (data too long), S9F13 (conversation timer
timeout).  Auto-emitted; see chapter 11.

### S10 — Terminal services

S10F1/F2 (terminal display single, equipment→host),
S10F3/F4 (terminal display single, host→equipment),
S10F5/F6 (terminal display multi, host→equipment).

### S12 — Wafer maps

S12F* — Per E5 §13.  Round-tripped through `raw_gem300_harness.py`.

### S14 — Object services (E39) + control jobs (E94)

S14F1/F2 (GetAttr), S14F3/F4 (SetAttr), S14F9/F10 (CreateCJ),
S14F11/F12 (DeleteCJ).

### S16 — Process jobs (E40)

S16F5/F6 (PRJobCommand), S16F7/F8 (PRJobMonitor), S16F9 (PRJobAlert
— unsolicited), S16F11/F12 (PRJobCreate), S16F13/F14 (PRJobDequeue),
S16F27/F28 (CJCommand).

For per-message body shapes, look up the YAML entry in
`data/messages.yaml`.

---

## YAML schema reference

### `data/messages.yaml`

```yaml
messages:
  - id: S<X>F<Y>           # required, must match (stream, function)
    stream: <int 1-127>
    function: <int 0-255>
    w: <bool>              # reply expected?
    builder: <ident>       # C++ builder function name
    parser: <ident>        # C++ parser function name
    body: <body-shape>     # see chapter 31 for the grammar
```

Body shapes:

```yaml
body: none

body:
  kind: scalar
  item_type: ASCII | BINARY_BYTE | BOOLEAN | U1..U8 | I1..I8 | F4 | F8 | ITEM
  enum: <C++ enum type>   # optional
  param: <name>           # optional, default 'value'

body:
  kind: list
  struct_name: <C++ type> # optional; if set, parser returns struct
  fields:
    - {name: <field>, shape: <body-shape>}
    - ...

body:
  kind: list_of
  element: <body-shape>
  name: <name>            # parameter name, default 'values'
```

### `data/control_state.yaml`

```yaml
transitions:
  - {from: <state>, on: <event>, to: <state>, then: <state>, ack: <code>}
```

`from`: one of `EquipmentOffline | AttemptOnline | HostOffline | OnlineLocal | OnlineRemote`.
`on`: one of `operator_switch_online | operator_switch_offline | operator_switch_local | operator_switch_remote | attempt_complete | attempt_failed | host_request_online | host_request_offline`.
`to`: optional new state.
`then`: optional chained state (for AttemptOnline pass-through).
`ack`: optional ACK code (`Accept`, `NotAccept`, `AlreadyOnline`).

### `data/process_job_state.yaml`

```yaml
transitions:
  - {from: <state>, on: <event>, to: <state>}
```

`from` / `to`: `Queued | SettingUp | WaitingForStart | Processing | ProcessComplete | Paused | Stopping | Aborting`.
`on`: `select | setup_complete | start | pause | resume | stop | abort | process_complete | abort_complete`.

### `data/control_job_state.yaml`

Same shape, different state/event names — see
[`include/secsgem/gem/control_job_state.hpp`](../include/secsgem/gem/control_job_state.hpp).

### `data/equipment.yaml`

```yaml
device:
  mdln: <ASCII>
  softrev: <ASCII>
  capabilities: [<list of GEM capability strings>]

svids:
  - {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>}

dvids:
  - {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>}

ecids:
  - {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>, min: <num>, max: <num>}

ceids:
  - {id: <U4>, name: <ASCII>}

alarms:
  - {id: <U4>, alcd: <byte>, text: <ASCII>}

recipes:
  - {id: <ASCII>, body: <bytes-or-string>}

host_commands:
  - {name: <ASCII>, ack: <ACK enum>, emit_ceid: <U4>, set_alarm: <U4>}

events:
  default_reports:
    - {ceid: <U4>, vids: [<U4>, ...]}

spool:
  whitelist: [<stream>, ...]
  persistent_dir: <path>      # optional
```

Type strings: `ASCII`, `BINARY`, `BOOLEAN`, `U1`–`U8`, `I1`–`I8`,
`F4`, `F8`.  Same vocabulary as `data/messages.yaml` body shapes.

For required vs optional fields per record, see the validator
checks in
[`tests/test_config_validate.cpp`](../tests/test_config_validate.cpp).

---

## Where to go next

The last chapter is the practical companion to this one: the
**recipes** for extending the codebase — adding a new SVID, host
command, state, message, store, or persistence backend.  Code
patches you can copy.

Next: [→ 51 Extending the codebase](51_extending_the_codebase.md)