secs-gem/docs/50_api_messages_yaml_reference.md

50 — API + message catalog + YAML schemas

← 41 Integration: hardware, MES, production | Back to index | Next: 51 Extending the codebase →

This chapter is reference, not tutorial. Look up the namespace or YAML key you need; cross-reference the code.

The whole codebase is small enough that "go read the header" is often the right answer — this chapter helps you find which header.

---

Namespace reference

secsgem::secs2 — codec (chapter 34)

#include "secsgem/secs2/item.hpp"
#include "secsgem/secs2/codec.hpp"
#include "secsgem/secs2/message.hpp"
#include "secsgem/secs2/sml.hpp"

Symbol	What it is
enum class Format	16 SECS-II format codes.
class Item	Variant-based value type.
class Message	Stream + function + W-bit + system_bytes + body.
class CodecError	Thrown on malformed input.
std::vector<uint8_t> encode(const Item&)	Serialize an Item to bytes.
void encode_into(const Item&, std::vector<uint8_t>&)	Append-encode into existing buffer.
Item decode(const std::vector<uint8_t>&)	Decode one Item from a complete buffer.
Item decode_at(const uint8_t*, size_t, size_t&)	Decode one Item from a position; advances cursor.
std::string to_sml(const Item&)	Render as SML.
std::optional<Item> try_parse_sml(const std::string&)	Parse SML; returns nullopt on error.

secsgem::hsms — TCP transport (chapter 11, 33)

#include "secsgem/hsms/header.hpp"
#include "secsgem/hsms/connection.hpp"

Symbol	What it is
enum class SType	9 session types.
enum class SelectStatus / DeselectStatus / RejectReason	Reply codes.
struct Header	10-byte HSMS header.
struct Frame	Header + body, length-prefixed on wire.
class FrameError	Thrown on framing errors.
struct Timers	T3/T5/T6/T7/T8 + linktest cadence.
class Connection	One-socket session manager.
Connection::Mode { Active, Passive }	TCP role.
Connection::State { NotSelected, Selected }	Transport state.

secsgem::secsi — SECS-I transport (chapter 12, 33)

#include "secsgem/secsi/header.hpp"
#include "secsgem/secsi/block.hpp"
#include "secsgem/secsi/protocol.hpp"
#include "secsgem/secsi/tcp_transport.hpp"

Symbol	What it is
struct Header	10-byte SECS-I block header (R/W/E + system bytes).
class Block	One block (header + body + checksum).
split_message(msg) / assemble_message(blocks)	Multi-block split / assemble.
class Protocol	IO-free FSM.
enum class Timer { T1, T2, T3, T4 }	Timer IDs (raised via EventTimeout).
Action / Event variants	FSM IO.
class TcpTransport	asio adapter for testing tunnels.

secsgem::gem — behavioural layer (chapters 13–19, 32, 35)

#include "secsgem/gem/data_model.hpp"          // composite
#include "secsgem/gem/router.hpp"              // dispatch table
#include "secsgem/gem/control_state.hpp"       // E30 control FSM
#include "secsgem/gem/communication_state.hpp" // E30 comm FSM
#include "secsgem/gem/process_job_state.hpp"   // E40
#include "secsgem/gem/control_job_state.hpp"   // E94
#include "secsgem/gem/carrier_state.hpp"       // E87
#include "secsgem/gem/load_port_state.hpp"     // E87
#include "secsgem/gem/substrate_state.hpp"     // E90
#include "secsgem/gem/module_state.hpp"        // E157
#include "secsgem/gem/ept_state.hpp"           // E116
#include "secsgem/gem/exception_state.hpp"     // E5 §13
#include "secsgem/gem/e84_state.hpp"           // E84 FSM
#include "secsgem/gem/e84_timers.hpp"          // E84 TA1/TA2/TA3
#include "secsgem/gem/e84_asio_timers.hpp"     // asio wrapper
#include "secsgem/gem/host_handler.hpp"        // host-side analogue
#include "secsgem/gem/messages_helpers.hpp"    // identifier wildcards
// Plus build/generated/secsgem/gem/messages.hpp (codegen).

include/secsgem/gem/store/ — 18 per-domain stores. See chapter 32 for the full table.

Engine-owner and default-behaviour entry points (added with the daemon track; see DAEMON_ROADMAP.md):

#include "secsgem/gem/runtime.hpp"           // EquipmentRuntime: owns
        // io_context + Server + model + control FSM + Router; thread-safe
        // set_variable/emit_event/set_alarm/clear_alarm, on_command hook,
        // read_sync (the standard cross-thread read), control-state mirror,
        // add_control_state_observer / add_link_observer.
#include "secsgem/gem/default_handlers.hpp"  // the 56 GEM handlers as 15
        // per-capability register_* functions + register_default_handlers.
#include "secsgem/gem/handler_slot.hpp"      // primary + observer handler slots
#include "secsgem/gem/name_index.hpp"        // name -> VID/CEID resolution

secsgem::daemon — the gRPC vendor surface

#include "secsgem/daemon/equipment_service.hpp"  // EquipmentService: the
        // proto/secsgem/v1 Equipment service over an EquipmentRuntime
        // (SetVariables/GetVariables/FireEvent/SetAlarm/ClearAlarm/
        //  GetControlState/RequestControlState/WatchHealth/Subscribe/
        //  CompleteCommand). Built into apps/secs_gemd.cpp; the Python
        //  client in clients/python wraps the same proto.

secsgem::config — YAML loader + validator (chapter 31, 36)

#include "secsgem/config/loader.hpp"
#include "secsgem/config/validate.hpp"

Symbol	What it loads
load_equipment(path)	data/equipment.yaml → EquipmentDescriptor.
load_control_state_table(path)	data/control_state.yaml → ControlStateConfig.
load_process_job_state(path)	data/process_job_state.yaml.
load_control_job_state(path)	data/control_job_state.yaml.
class ConfigValidator	Multi-error YAML validator.

secsgem::metrics — Prometheus exporter (chapter 36)

#include "secsgem/metrics/prometheus.hpp"

Symbol	What it is
class Registry	Holds Counter + Gauge series with labels.
enum class MetricType { Counter, Gauge }
class PrometheusServer	HTTP server on a configurable port.

---

Message catalog reference

164 entries in data/messages.yaml. Grouped by stream:

S1 — Identification + status

S/F	W	Name	Body
S1F1	W	Are You There	none
S1F2		On-Line Data	<L,2> [MDLN, SOFTREV]
S1F3	W	Selected Equipment Status Req	<L,n> [SVID, SVID, ...]
S1F4		Selected Equipment Status Data	<L,n> [SV, SV, ...]
S1F11	W	Status Variable Namelist Req	<L,n> [SVID, ...]
S1F12		Status Variable Namelist	<L,n> [<L,3> [SVID, SVNAME, UNITS]]
S1F13	W	Establish Communications Req	<L,2> [MDLN, SOFTREV]
S1F14		Establish Communications Ack	<L,2> [COMMACK, <L,2> [MDLN, SOFTREV]]
S1F15	W	Request Offline	none
S1F16		OFLACK	OFLACK
S1F17	W	Request Online	none
S1F18		ONLACK	ONLACK
S1F19	W	Compliance Request	none
S1F20		Compliance Data	<L,n> [CCODE, ...]
S1F21	W	DVID Namelist Request	<L,n> [DVID, ...]
S1F22		DVID Namelist	<L,n> [<L,3> [DVID, DVNAME, UNITS]]
S1F23	W	CEID Namelist Request	<L,n> [CEID, ...]
S1F24		CEID Namelist	<L,n> [<L,2> [CEID, [VID, VID, ...]]]

S2 — Equipment constants, clock, events, commands, spool

S2F13/F14 (EC values), S2F15/F16 (set EC), S2F17/F18 (clock read), S2F21/F22 (legacy RCMD), S2F23/F24 (trace init), S2F29/F30 (EC namelist), S2F31/F32 (set clock), S2F33–F38 (report wiring), S2F41/F42 (modern RCMD), S2F43/F44 (set spool streams), S2F45–F48 (limits), S2F49/F50 (enhanced RCMD).

S3 — Carrier management (E87)

S3F17/F18 (CarrierAction), S3F19/F20 (slot map verify), S3F25/F26 (carrier transfer), S3F27/F28 (cancel carrier).

S5 — Alarms + exception recovery

S5F1/F2 (alarm set/clear), S5F3/F4 (enable/disable alarm), S5F5/F6 (list all alarms), S5F7/F8 (list enabled alarms), S5F9–F18 (exception recovery, chapter 19).

S6 — Data collection

S6F1/F2 (trace data), S6F11/F12 (event report), S6F15/F16 (event report request), S6F19/F20 (individual report request), S6F21/F22 (annotated individual report), S6F23/F24 (spool data transmit/purge), S6F25/F26 (spool notification).

S7 — Process program management

S7F1/F2 (PP load inquire), S7F3/F4 (PP send unformatted), S7F5/F6 (PP request), S7F17/F18 (PP delete), S7F19/F20 (PP namelist), S7F23/F24 (formatted PP send, E42), S7F25/F26 (formatted PP request).

S9 — Protocol-error reports

S9F1 (unrecognized device ID), S9F3 (unrecognized stream), S9F5 (unrecognized function), S9F7 (illegal data), S9F9 (T3 timeout), S9F11 (data too long), S9F13 (conversation timer timeout). Auto-emitted; see chapter 11.

S10 — Terminal services

S10F1/F2 (terminal display single, equipment→host), S10F3/F4 (terminal display single, host→equipment), S10F5/F6 (terminal display multi, host→equipment).

S12 — Wafer maps

S12F* — Per E5 §13. Round-tripped through raw_gem300_harness.py.

S14 — Object services (E39) + control jobs (E94)

S14F1/F2 (GetAttr), S14F3/F4 (SetAttr), S14F9/F10 (CreateCJ), S14F11/F12 (DeleteCJ).

S16 — Process jobs (E40)

S16F5/F6 (PRJobCommand), S16F7/F8 (PRJobMonitor), S16F9 (PRJobAlert — unsolicited), S16F11/F12 (PRJobCreate), S16F13/F14 (PRJobDequeue), S16F27/F28 (CJCommand).

For per-message body shapes, look up the YAML entry in data/messages.yaml.

---

YAML schema reference

data/messages.yaml

messages:
  - id: S<X>F<Y>           # required, must match (stream, function)
    stream: <int 1-127>
    function: <int 0-255>
    w: <bool>              # reply expected?
    builder: <ident>       # C++ builder function name
    parser: <ident>        # C++ parser function name
    body: <body-shape>     # see chapter 31 for the grammar

Body shapes:

body: none

body:
  kind: scalar
  item_type: ASCII | BINARY_BYTE | BOOLEAN | U1..U8 | I1..I8 | F4 | F8 | ITEM
  enum: <C++ enum type>   # optional
  param: <name>           # optional, default 'value'

body:
  kind: list
  struct_name: <C++ type> # optional; if set, parser returns struct
  fields:
    - {name: <field>, shape: <body-shape>}
    - ...

body:
  kind: list_of
  element: <body-shape>
  name: <name>            # parameter name, default 'values'

data/control_state.yaml

transitions:
  - {from: <state>, on: <event>, to: <state>, then: <state>, ack: <code>}

from: one of EquipmentOffline | AttemptOnline | HostOffline | OnlineLocal | OnlineRemote. on: one of operator_switch_online | operator_switch_offline | operator_switch_local | operator_switch_remote | attempt_complete | attempt_failed | host_request_online | host_request_offline. to: optional new state. then: optional chained state (for AttemptOnline pass-through). ack: optional ACK code (Accept, NotAccept, AlreadyOnline).

data/process_job_state.yaml

transitions:
  - {from: <state>, on: <event>, to: <state>}

from / to: Queued | SettingUp | WaitingForStart | Processing | ProcessComplete | Paused | Stopping | Aborting. on: select | setup_complete | start | pause | resume | stop | abort | process_complete | abort_complete.

data/control_job_state.yaml

Same shape, different state/event names — see include/secsgem/gem/control_job_state.hpp.

data/equipment.yaml

device:
  mdln: <ASCII>
  softrev: <ASCII>
  capabilities: [<list of GEM capability strings>]

svids:
  - {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>}

dvids:
  - {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>}

ecids:
  - {id: <U4>, name: <ASCII>, units: <ASCII>, type: <type>, value: <default>, min: <num>, max: <num>}

ceids:
  - {id: <U4>, name: <ASCII>}

alarms:
  - {id: <U4>, alcd: <byte>, text: <ASCII>}

recipes:
  - {id: <ASCII>, body: <bytes-or-string>}

host_commands:
  - {name: <ASCII>, ack: <ACK enum>, emit_ceid: <U4>, set_alarm: <U4>}

events:
  default_reports:
    - {ceid: <U4>, vids: [<U4>, ...]}

spool:
  whitelist: [<stream>, ...]
  persistent_dir: <path>      # optional

Type strings: ASCII, BINARY, BOOLEAN, U1–U8, I1–I8, F4, F8. Same vocabulary as data/messages.yaml body shapes.

For required vs optional fields per record, see the validator checks in tests/test_config_validate.cpp.

---

Where to go next

The last chapter is the practical companion to this one: the recipes for extending the codebase — adding a new SVID, host command, state, message, store, or persistence backend. Code patches you can copy.

Next: → 51 Extending the codebase