diff --git a/docs/00_index.md b/docs/00_index.md index cbe53a6..777fc83 100644 --- a/docs/00_index.md +++ b/docs/00_index.md @@ -138,6 +138,7 @@ how to run it. |---|-------|--------| | [40](40_building_running_demo.md) | Building, running, the demo | Docker setup, the two-container demo, every transaction it walks through. | | [41](41_integration_hardware_mes_production.md) | Integration | Wiring sensors and recipes, talking to a real MES, HSMS-GS for multi-MES, persistence layout, monitoring, security hardening, performance envelope. | +| [42](42_vendor_daemon_and_clients.md) | The vendor daemon + language clients | secs_gemd, the gRPC API and the HCACK-4 command contract, the Python client, EquipmentRuntime + per-capability registration, the threading contract, which tier to pick. | ### Part 5 — Reference diff --git a/docs/35_state_machines_and_dispatch.md b/docs/35_state_machines_and_dispatch.md index ddfd479..3c50c38 100644 --- a/docs/35_state_machines_and_dispatch.md +++ b/docs/35_state_machines_and_dispatch.md @@ -64,9 +64,11 @@ router->on(2, 41, [model](const auto& m) { // register_* functions; register_default_handlers(runtime) wires them all. ``` -The `examples/pvd_tool/main.cpp` §6 register 51 handlers in ~460 -lines. Each handler is a few lines: parse the body, mutate or read -a store, build the reply. +The `examples/pvd_tool/main.cpp` §6 gets all 56 with ONE call — +`register_default_handlers(runtime)` — then adds tool behaviour via +`commands.set_handler`. Inside the capability functions each handler is +still a few lines: parse the body, mutate or read a store, build the +reply. ### What happens for unhandled primaries diff --git a/docs/41_integration_hardware_mes_production.md b/docs/41_integration_hardware_mes_production.md index c11d6c1..39e83b1 100644 --- a/docs/41_integration_hardware_mes_production.md +++ b/docs/41_integration_hardware_mes_production.md @@ -41,7 +41,7 @@ the worked reference. Section by section: | §3 Recipe runner | PJ → SettingUp → Processing → ProcessComplete walk; per-step CEID emit | | §4 Alarm threshold monitor | Continuous threshold evaluation against ECID setpoints | | §5 EPT cycling | E116 transitions driven by PJ state + safety alarms | -| §6 Router handlers | 51 handlers in ~460 lines — every S/F a host might send to a PVD tool | +| §6 main() | `EquipmentRuntime` + `register_default_handlers` (all 56) + `set_handler` for START — was 51 hand-written handlers before the runtime | | §7 main() | YAML load → validate → compose → run | A real tool fork: diff --git a/docs/42_vendor_daemon_and_clients.md b/docs/42_vendor_daemon_and_clients.md new file mode 100644 index 0000000..28ef2c6 --- /dev/null +++ b/docs/42_vendor_daemon_and_clients.md @@ -0,0 +1,201 @@ +# 42 — The vendor daemon and language clients + +Chapters 30–41 teach the embedded C++ path: your `main()` owns the engine. +This chapter teaches the **daemon path** — the engine as its own process, +your tool software in any language on the other side of a socket — and the +runtime/capability API both paths share. If you are integrating a tool and +your controller is not C++, start here. + +Everything in this chapter is exercised by `tools/run_interop.sh` (the +`daemon`, `pyclient`, and `daemon-unit` steps) against the secsgem-py +reference host. Status and remaining work: [DAEMON_ROADMAP.md](DAEMON_ROADMAP.md). + +--- + +## 1. Why a daemon at all + +A fab host enforces timers: T3 reply timeouts, T6 control transactions, +linktest heartbeats. If the GEM stack lives inside your tool application, +every crash, upgrade, GC pause, or hung hardware call of that application +is a **communication failure the fab can see** — and in production that +can mean the tool gets taken offline and lots get held. + +`secs_gemd` inverts the deployment: the daemon owns the durable HSMS +relationship and answers the host from its own process, around the clock. +Your tool software connects over gRPC, pushes values and events in, and +receives host commands out. It can restart any time; the host never +notices. Spooling (chapter 13 §spool) covers the gap if the *host* link +drops; the daemon model covers the gap if *your software* drops. + +``` + tool software (any language) secs_gemd fab host / MES + ┌──────────────────────────┐ gRPC ┌──────────────────────────┐ HSMS ┌────────┐ + │ set / fire / alarm │◄─────►│ EquipmentRuntime │◄────►│ MES │ + │ @on("START") handlers │ :50051│ + register_default_* │SECS-II└────────┘ + │ (restartable, crashable) │ │ + spool, timers, FSMs │ + └──────────────────────────┘ └──────────────────────────┘ +``` + +Run it: + +```sh +build/secs_gemd --port 5000 --grpc 0.0.0.0:50051 --config-dir data +``` + +One process, two faces: passive HSMS equipment on `--port`, the gRPC tool +API on `--grpc`. + +--- + +## 2. The API contract (`proto/secsgem/v1/equipment.proto`) + +The proto is the source of truth; read it — it is heavily commented. The +shape in one breath: everything is **name-based** (the names from your +`equipment.yaml`; never numeric SVIDs/CEIDs/ALIDs) and **plain-typed** +(a `Value` oneof of text/integer/real/boolean/binary/list; the daemon +converts to each variable's declared SECS-II format, so an F4 variable +stays F4 on the wire no matter what you send). + +| RPC | What it does | +|---|---| +| `SetVariables` / `GetVariables` | write/read variables by name | +| `FireEvent` | trigger a collection event; daemon assembles the configured report → S6F11 | +| `SetAlarm` / `ClearAlarm` | S5F1 set/clear, by alarm `name:` (or stringified ALID) | +| `GetControlState` / `RequestControlState` | read the E30 control state / operator transitions | +| `WatchHealth` | server stream: link state, control state, spool depth | +| `Subscribe` | server stream: everything the host asks of the tool | +| `CompleteCommand` | close a streamed command's audit entry | + +### The HCACK-4 command contract + +The one piece of SEMI behaviour you must understand: when the host sends a +remote command (S2F41 START), the daemon does **not** wait for your tool. +It answers the host immediately: + +- **No tool subscribed** → the command's declarative ack from + `equipment.yaml` (exactly the pre-daemon behaviour; nothing buffered, + nothing replayed later). +- **Tool subscribed** → `HCACK=4` ("accepted, will finish later"), and the + command appears on your `Subscribe` stream with its parameters. + +The S2F42 transaction is already closed by the time you see the command. +The host learns the *real* outcome the way E30 intends: from the +**collection event you fire on success** (or the alarm you raise on +failure). `CompleteCommand` only feeds the daemon's audit log. This is the +same pattern secsgem-py applications and commercial GEM gateways use — the +protocol was designed for it. + +--- + +## 3. The Python client (`clients/python`) + +`pip install` the package (pure Python — pre-generated stubs, no compiled +extension) and the entire integration is: + +```python +from secsgem_client import Equipment + +eq = Equipment("localhost:50051") + +eq.set(ChamberPressure=2.5) # host sees it on its next S1F3 +eq["WaferCounter"] = 7 # item syntax, same thing +print(eq.get("ChamberPressure")) # read back through the daemon + +eq.fire("ProcessStarted", ChamberPressure=2.75) # values, then S6F11 +eq.alarm("chiller_temp_high") # S5F1 set +eq.clear("chiller_temp_high") # S5F1 clear + +@eq.on("START") # host S2F41 -> your function +def start(cmd): + run_recipe(cmd.params.get("PPID")) + eq.fire("ProcessStarted") # the host's completion signal + +eq.listen(background=True) # consume the Subscribe stream + +eq.control_state # "ONLINE_REMOTE" +eq.request_control_state("HOST_OFFLINE") # operator panel -> maintenance +eq.health() # link / control state / spool depth +``` + +Anything the daemon declines raises `SecsGemError` with its explanation +(`no variable named 'ChamberPresure'`). A complete runnable tool is +[clients/python/examples/mini_tool.py](../clients/python/examples/mini_tool.py) +(~25 lines). The package is validated end-to-end by +`interop/pyclient_interop.py`: 13 checks driving the published API while +secsgem-py judges the wire. + +Other languages: generate stubs from the proto (`protoc` supports 11+ +languages) and wrap them the same way — the Python client is ~200 lines +and is the reference for what a thin wrapper should feel like. + +--- + +## 4. The shared core: `EquipmentRuntime` + capability registration + +Both `secs_server` and `secs_gemd` (and any future surface) are thin +fronts over the same two calls: + +```cpp +#include "secsgem/gem/runtime.hpp" +#include "secsgem/gem/default_handlers.hpp" + +gem::EquipmentRuntime::Config cfg; +cfg.equipment_yaml = "data/equipment.yaml"; +cfg.control_state_yaml = "data/control_state.yaml"; +cfg.process_job_yaml = "data/process_job_state.yaml"; +cfg.control_job_yaml = "data/control_job_state.yaml"; +cfg.port = 5000; +cfg.log = [](const std::string& m) { std::cout << m << std::endl; }; + +gem::EquipmentRuntime R(cfg); +gem::register_default_handlers(R); // all 56 GEM handlers + emitters +R.run(); // accept + io_context (blocks) +``` + +`register_default_handlers` is the composition of **15 per-capability +functions** (`register_identification`, `register_alarms`, +`register_carriers`, `register_jobs`, …) mirroring how E30 itself lists +capabilities (S1F19). A sensor-class tool with no carriers or jobs +registers only what it is — the unregistered messages get the Router's +SxF0/S9 default treatment, which is exactly what "I don't implement that +capability" should look like on the wire. + +The ids the built-ins touch (the control-state/clock SVIDs the engine +refreshes, the CEIDs fired on CJ state changes) come from the `roles:` +block in `equipment.yaml` — visible coupling, no magic constants. + +### The threading contract (the one rule) + +One io_context thread owns the model. From any other thread: + +- **writes** go through the runtime's posting API + (`set_variable`, `emit_event`, `set_alarm`, `clear_alarm`); +- **reads of mutable state** go through `read_sync` (post + future with a + deadline) — `control_state()` alone is lock-free (atomic mirror); +- **behaviour hooks** (`commands.set_handler`, state-change observers) run + *on* the io thread: return promptly, post long work elsewhere. + +This is TSan-enforced in CI, daemon included. The first violation ever +caught was in our own test — the lane works. + +### Observers vs. the primary slot + +State machines expose `set_state_change_handler` (the primary slot — +yours, replaceable) **and** `add_state_change_handler` (append-only +observers that survive the primary being set). The runtime and daemon use +observers for the control-state mirror, `WatchHealth`, and the command +stream, so they never fight your application over the slot. + +--- + +## 5. Which tier do I pick? + +| Your situation | Tier | +|---|---| +| New tool, Python anywhere in the controller, fastest start | Python client | +| Existing controller in C#/Java/Go/…; multi-process architecture; tool app must be restartable without the host noticing | gRPC daemon + thin client | +| In-process integration, custom transports, hard real-time adjacency | Embedded C++ (`EquipmentRuntime`, chapter 41) | + +They compose: a C++ tool can still run the daemon for the HSMS face and +talk gRPC locally — that is precisely the "tool software + separate +SECS server" deployment many fabs already run. diff --git a/examples/pvd_tool/README.md b/examples/pvd_tool/README.md index db39fbb..a4c5959 100644 --- a/examples/pvd_tool/README.md +++ b/examples/pvd_tool/README.md @@ -21,7 +21,7 @@ customize. They're written to be a template, not an abstract demo. | §3 Recipe runner | Driving a PJ through SettingUp → Processing → ProcessComplete by walking the recipe body, with per-step CEID emission | | §4 Alarm threshold monitor | Continuous threshold-based alarm logic (chamber pressure, cleaning interval) with set/clear emission | | §5 EPT cycling | E116 state transitions driven by PJ state + safety alarms | -| §6 Router handlers | Every SECS/GEM message a host might send to a PVD tool, 51 handlers in ~460 lines | +| §6 main() | `EquipmentRuntime` + `register_default_handlers` (all 56 GEM handlers in one call) + `commands.set_handler` for the START behaviour | | §7 main() | Loading YAML → validating → composing → running, including the Prometheus exporter on `:9090` (§7.3) | ## Running it diff --git a/examples/pvd_tool/main.cpp b/examples/pvd_tool/main.cpp index 92f5fb7..06c407b 100644 --- a/examples/pvd_tool/main.cpp +++ b/examples/pvd_tool/main.cpp @@ -37,16 +37,12 @@ #include #include -#include "secsgem/config/loader.hpp" #include "secsgem/config/validate.hpp" -#include "secsgem/endpoint.hpp" -#include "secsgem/gem/control_state.hpp" #include "secsgem/gem/data_model.hpp" -#include "secsgem/gem/e116_constants.hpp" -#include "secsgem/gem/messages.hpp" -#include "secsgem/gem/router.hpp" +#include "secsgem/gem/default_handlers.hpp" +#include "secsgem/gem/runtime.hpp" #include "secsgem/metrics/prometheus.hpp" -#include "secsgem/secs2/message.hpp" +#include "secsgem/secs2/item.hpp" using namespace secsgem; using namespace std::chrono_literals; @@ -437,473 +433,15 @@ struct EptCycler { } // namespace pvd // ============================================================================= -// §6. Router handler registration +// §6. main() — the integration, on the modern stack // ============================================================================= // -// This is the smallest set of handlers a host needs to talk to the -// tool and run a recipe. apps/secs_server.cpp has the full -// catalogue (~30 more handlers) for terminal services, slot maps, -// E40/E94 jobs, etc.; in production you'd copy that here too. - -void register_handlers(gem::Router& router, - std::shared_ptr model, - std::shared_ptr sm, - const config::EquipmentDescriptor& desc, - std::function emit_event, - std::function emit_alarm_set, - std::shared_ptr recipe) { - - // S1F1 → S1F2 Are You There - router.on(1, 1, [desc](const s2::Message&) { - return gem::s1f2_on_line_data(desc.model_name, desc.software_rev); - }); - - // S1F3 → S1F4 Selected Status Request - router.on(1, 3, [model](const s2::Message& m) { - auto svids = gem::parse_s1f3(m); - if (!svids) return s2::Message(1, 0, false); - std::vector> values; - if (svids->empty()) { - for (const auto& sv : model->svids.all()) values.push_back(sv.value); - } else { - for (auto id : *svids) { - auto sv = model->svids.get(id); - values.push_back(sv ? std::optional(sv->value) : std::nullopt); - } - } - return gem::s1f4_selected_status_data(values); - }); - - // S1F11 → S1F12 Status Variable Namelist Request - router.on(1, 11, [model](const s2::Message&) { - std::vector rows; - for (const auto& sv : model->svids.all()) - rows.push_back({sv.id, sv.name, sv.units}); - return gem::s1f12_status_namelist_data(rows); - }); - - // S1F13 → S1F14 Establish Communications - router.on(1, 13, [desc](const s2::Message&) { - return gem::s1f14_establish_comms_ack(gem::CommAck::Accept, - {desc.model_name, desc.software_rev}); - }); - - // S1F17 → S1F18 Request Online - router.on(1, 17, [sm](const s2::Message&) { - auto ack = sm->on_host_request_online(); - return gem::s1f18_online_ack(ack); - }); - - // S2F13 → S2F14 EC Values - router.on(2, 13, [model](const s2::Message& m) { - auto ids = gem::parse_u4_list_body(m); - if (!ids) return s2::Message(2, 0, false); - std::vector values; - for (auto id : *ids) { - auto ec = model->ecids.get(id); - values.push_back(ec ? ec->value : s2::Item::list({})); - } - return gem::s2f14_ec_data(values); - }); - - // S2F17 → S2F18 Clock - router.on(2, 17, [model](const s2::Message&) { - return gem::s2f18_date_time_data(model->clock.current_time_string()); - }); - - // S2F41 → S2F42 Host Command - router.on(2, 41, [model, emit_event, emit_alarm_set, recipe] - (const s2::Message& m) { - auto cmd = gem::parse_s2f41(m); - if (!cmd) return gem::s2f42_host_command_ack(gem::HostCmdAck::ParameterInvalid, {}); - auto result = model->commands.dispatch(cmd->rcmd, cmd->params); - if (result.ack == gem::HostCmdAck::Accept) { - if (result.emit_ceid) emit_event(*result.emit_ceid); - if (result.set_alarm) emit_alarm_set(*result.set_alarm); - // Demo: RCMD=START with PJ in WaitingForStart triggers the - // recipe runner. Real tools would gate on richer state. - if (cmd->rcmd == "START") { - for (const auto& pjid : model->process_jobs.ids()) { - auto* pj = model->process_jobs.get(pjid); - if (pj && pj->fsm->state() == gem::ProcessJobState::WaitingForStart) { - model->process_jobs.fire_internal(pjid, gem::ProcessJobEvent::Start); - recipe->start(pjid); - break; - } - } - } - } - return gem::s2f42_host_command_ack(result.ack, {}); - }); - - // S5F5 → S5F6 List Alarms - router.on(5, 5, [model](const s2::Message& m) { - auto ids = gem::parse_u4_list_body(m); - std::vector alarms; - if (ids && ids->empty()) alarms = model->alarms.all(); - else if (ids) - for (auto id : *ids) { - auto a = model->alarms.get(id); - if (a) alarms.push_back(*a); - } - return gem::s5f6_list_alarms_data( - alarms, [model](uint32_t id) { return model->alarms.active(id); }); - }); - - // S7F5 → S7F6 Process Program Request - router.on(7, 5, [model](const s2::Message& m) { - auto ppid = gem::parse_s7f5(m); - if (!ppid) return gem::s7f6_process_program_data("", ""); - auto body = model->recipes.get(*ppid); - return gem::s7f6_process_program_data(*ppid, body ? *body : ""); - }); - - // S7F19 → S7F20 Current PP List - router.on(7, 19, [model](const s2::Message&) { - return gem::s7f20_current_eppd_data(model->recipes.list()); - }); - - // -------- Extended handlers (mirrors apps/secs_server.cpp) ---------- - // These follow the demo server's patterns one-for-one. A real - // vendor's main.cpp would either include them inline (as we do here) - // or extract them to a shared helper. - - // S1F15 → S1F16 Request Offline - router.on(1, 15, [sm](const s2::Message&) { - return gem::s1f16_offline_ack(sm->on_host_request_offline()); - }); - // S1F19 → S1F20 GEM Compliance - router.on(1, 19, [desc](const s2::Message&) { - std::vector caps; - for (const auto& c : desc.capabilities) caps.push_back({c.first, c.second}); - return gem::s1f20_get_gem_compliance_data( - desc.software_rev, desc.equipment_type, caps); - }); - // S1F21 → S1F22 DVID Namelist - router.on(1, 21, [model](const s2::Message&) { - std::vector rows; - for (const auto& dv : model->dvids.all()) - rows.push_back({dv.id, dv.name, dv.units}); - return gem::s1f22_data_variable_namelist_data(rows); - }); - // S1F23 → S1F24 CEID Namelist - router.on(1, 23, [model](const s2::Message& m) { - auto req = gem::parse_s1f23(m); - std::vector rows; - if (req && req->empty()) { - for (const auto& e : model->events.all_events()) - rows.push_back({e.id, e.name, model->events.vids_for(e.id)}); - } else if (req) { - for (auto id : *req) { - auto info = model->events.event_info(id); - rows.push_back({id, info ? info->name : "", model->events.vids_for(id)}); - } - } - return gem::s1f24_collection_event_namelist_data(rows); - }); - // S2F15 → S2F16 EC Set - router.on(2, 15, [model](const s2::Message& m) { - auto sets = gem::parse_s2f15(m); - auto eac = gem::EquipmentAck::Accept; - if (!sets) eac = gem::EquipmentAck::Denied_OutOfRange; - else for (const auto& s : *sets) { - auto r = model->ecids.set_value(s.ecid, s.value); - if (r != gem::EquipmentAck::Accept) eac = r; - } - return gem::s2f16_ec_ack(eac); - }); - // S2F29 → S2F30 EC Namelist - router.on(2, 29, [model](const s2::Message& m) { - auto ids = gem::parse_u4_list_body(m); - std::vector ecs; - if (ids && ids->empty()) ecs = model->ecids.all(); - else if (ids) for (auto id : *ids) { - auto ec = model->ecids.get(id); - if (ec) ecs.push_back(*ec); - } - std::vector rows; - for (const auto& ec : ecs) - rows.push_back({ec.id, ec.name, ec.min_str, ec.max_str, "", ec.units}); - return gem::s2f30_ec_namelist_data(rows); - }); - // S2F31 → S2F32 Set Clock - router.on(2, 31, [model](const s2::Message& m) { - auto t = gem::parse_s2f31(m); - return gem::s2f32_date_time_ack( - t ? model->clock.set_time_string(*t) : gem::TimeAck::Error); - }); - // S2F33/F35/F37 Dynamic event report config - router.on(2, 33, [model](const s2::Message& m) { - auto req = gem::parse_s2f33(m); - auto ack = gem::DefineReportAck::InvalidFormat; - if (req) { - std::vector>> rows; - for (const auto& r : req->reports) rows.emplace_back(r.rptid, r.vids); - ack = model->define_reports(rows); - } - return gem::s2f34_define_report_ack(ack); - }); - router.on(2, 35, [model](const s2::Message& m) { - auto req = gem::parse_s2f35(m); - auto ack = gem::LinkEventAck::InvalidFormat; - if (req) { - std::vector>> rows; - for (const auto& l : req->links) rows.emplace_back(l.ceid, l.rptids); - ack = model->link_event_reports(rows); - } - return gem::s2f36_link_event_report_ack(ack); - }); - router.on(2, 37, [model](const s2::Message& m) { - auto req = gem::parse_s2f37(m); - auto ack = req ? model->enable_events(req->enable, req->ceids) - : gem::EnableEventAck::UnknownCeid; - return gem::s2f38_enable_event_ack(ack); - }); - // S2F21 Legacy remote command - router.on(2, 21, [model, emit_event, emit_alarm_set](const s2::Message& m) { - auto rcmd = gem::parse_s2f21(m); - if (!rcmd) return gem::s2f22_remote_command_ack(gem::HostCmdAck::ParameterInvalid); - auto result = model->commands.dispatch(*rcmd, {}); - if (result.ack == gem::HostCmdAck::Accept) { - if (result.emit_ceid) emit_event(*result.emit_ceid); - if (result.set_alarm) emit_alarm_set(*result.set_alarm); - } - return gem::s2f22_remote_command_ack(result.ack); - }); - // S2F23 trace, S2F43 spool reset, S2F45 limits, S2F47 limit attrs - router.on(2, 23, [model](const s2::Message& m) { - auto req = gem::parse_s2f23(m); - auto ack = gem::TraceAck::Accept; - if (!req) ack = gem::TraceAck::InvalidPeriod; - else for (auto v : req->svids) - if (!model->vid_exists(v)) { ack = gem::TraceAck::UnknownVid; break; } - return gem::s2f24_trace_initialize_ack(ack); - }); - router.on(2, 43, [model](const s2::Message& m) { - auto streams = gem::parse_s2f43(m); - if (streams) model->spool.set_spoolable_streams(*streams); - return gem::s2f44_reset_spooling_ack( - streams ? gem::ResetSpoolAck::Accept : gem::ResetSpoolAck::Denied_NotAllowed, {}); - }); - router.on(2, 47, [model](const s2::Message& m) { - auto vids = gem::parse_s2f47(m); - std::vector rows; - if (vids) { - const auto target = vids->empty() ? model->limits.all_vids() : *vids; - for (auto v : target) rows.push_back({v, model->limits.get_for_vid(v)}); - } - return gem::s2f48_variable_limit_attribute_data(rows); - }); - // S2F49 Enhanced remote command - router.on(2, 49, [model, emit_event](const s2::Message& m) { - auto cmd = gem::parse_s2f49(m); - if (!cmd) return gem::s2f50_enhanced_host_command_ack( - gem::HostCmdAck::ParameterInvalid, {}); - auto result = model->commands.dispatch(cmd->rcmd, cmd->params); - if (result.ack == gem::HostCmdAck::Accept && result.emit_ceid) - emit_event(*result.emit_ceid); - return gem::s2f50_enhanced_host_command_ack(result.ack, {}); - }); - // S5F3 enable alarm, S5F7 list enabled - router.on(5, 3, [model](const s2::Message& m) { - auto req = gem::parse_s5f3(m); - return gem::s5f4_enable_alarm_ack( - req ? model->alarms.set_enabled(req->alid, (req->aled & 0x80) != 0) - : gem::AlarmAck::Error); - }); - router.on(5, 7, [model](const s2::Message&) { - std::vector rows; - for (const auto& a : model->alarms.all()) { - if (!model->alarms.enabled(a.id)) continue; - const uint8_t alcd = (a.severity_category & 0x7F) | - (model->alarms.active(a.id) ? 0x80 : 0x00); - rows.push_back({alcd, a.id, a.text}); - } - return gem::s5f8_list_enabled_alarms_data(rows); - }); - // S5F13/F17 exception recover - router.on(5, 13, [model](const s2::Message& m) { - auto req = gem::parse_s5f13(m); - return gem::s5f14_exception_recover_ack( - req ? model->exceptions.on_recover(req->exid, req->exrecvra) - : gem::AlarmAck::Error); - }); - router.on(5, 17, [model](const s2::Message& m) { - auto exid = gem::parse_s5f17(m); - return gem::s5f18_exception_recover_abort_ack( - exid ? model->exceptions.on_recover_abort(*exid) : gem::AlarmAck::Error); - }); - // S6F15/F19/F21 host-initiated event/report queries - router.on(6, 15, [model](const s2::Message& m) { - auto ceid = gem::parse_s6f15(m); - if (!ceid) return gem::s6f16_event_report_data({0, 0, {}}); - return gem::s6f16_event_report_data({0, *ceid, model->compose_reports_for(*ceid)}); - }); - router.on(6, 19, [model](const s2::Message& m) { - auto rptid = gem::parse_s6f19(m); - std::vector values; - if (rptid) for (const auto& r : model->events.all_reports()) { - if (r.id != *rptid) continue; - for (auto vid : r.vids) { - auto v = model->vid_value(vid); - values.push_back(v ? *v : s2::Item::list({})); - } - break; - } - return gem::s6f20_individual_report_data(values); - }); - router.on(6, 21, [model](const s2::Message& m) { - auto rptid = gem::parse_s6f21(m); - std::vector rows; - if (rptid) for (const auto& r : model->events.all_reports()) { - if (r.id != *rptid) continue; - for (auto vid : r.vids) { - auto v = model->vid_value(vid); - rows.push_back({vid, v ? *v : s2::Item::list({})}); - } - break; - } - return gem::s6f22_annotated_report_data(rows); - }); - // S6F23 spool data request - router.on(6, 23, [model](const s2::Message& m) { - auto rsdc = gem::parse_s6f23(m); - if (!rsdc) return gem::s6f24_request_spool_data_ack(gem::SpoolRequestAck::Denied); - if (*rsdc == gem::SpoolRequestCode::Purge) model->spool.clear(); - else model->spool.drain(); // demo: drop the drained messages - return gem::s6f24_request_spool_data_ack(gem::SpoolRequestAck::Accept); - }); - // S7F1 PP load inquire, S7F3 PP send, S7F17 PP delete - router.on(7, 1, [](const s2::Message& m) { - auto req = gem::parse_s7f1(m); - auto ack = gem::ProcessProgramAck::Accept; - if (!req || req->ppid.empty()) ack = gem::ProcessProgramAck::PpidNotFound; - return gem::s7f2_pp_load_grant(ack); - }); - router.on(7, 3, [model](const s2::Message& m) { - auto pp = gem::parse_s7f3(m); - if (!pp) return gem::s7f4_process_program_ack(gem::ProcessProgramAck::LengthError); - model->recipes.add(pp->ppid, pp->ppbody); - return gem::s7f4_process_program_ack(gem::ProcessProgramAck::Accept); - }); - router.on(7, 17, [model](const s2::Message& m) { - auto req = gem::parse_s7f17(m); - if (!req) return gem::s7f18_delete_pp_ack(gem::ProcessProgramAck::LengthError); - if (req->empty()) for (const auto& id : model->recipes.list()) model->recipes.remove(id); - else for (const auto& id : *req) model->recipes.remove(id); - return gem::s7f18_delete_pp_ack(gem::ProcessProgramAck::Accept); - }); - // S10F3 host→equipment terminal display, S10F5 multi-line - router.on(10, 3, [](const s2::Message& m) { - auto td = gem::parse_s10f3(m); - if (td) std::cout << "[TERMINAL " << static_cast(td->tid) - << "] " << td->text << "\n"; - return gem::s10f4_terminal_display_ack(gem::TerminalAck::Accepted); - }); - router.on(10, 5, [](const s2::Message& m) { - auto td = gem::parse_s10f5(m); - if (td) for (const auto& line : td->lines) - std::cout << "[TERMINAL " << static_cast(td->tid) << "] " << line << "\n"; - return gem::s10f6_terminal_display_multi_ack(gem::TerminalAck::Accepted); - }); - // S3 — E87 carriers (basic acceptance) - router.on(3, 17, [model](const s2::Message& m) { - auto req = gem::parse_s3f17(m); - if (!req) return gem::s3f18_carrier_action_ack(gem::CarrierActionAck::ParameterInvalid); - if (!model->carriers.has(req->carrierid)) - return gem::s3f18_carrier_action_ack(gem::CarrierActionAck::CarrierIDUnknown); - return gem::s3f18_carrier_action_ack(gem::CarrierActionAck::Accept); - }); - router.on(3, 19, [model](const s2::Message& m) { - auto req = gem::parse_s3f19(m); - if (!req) return gem::s3f20_slot_map_verify_ack(gem::SlotMapVerifyAck::Error); - return gem::s3f20_slot_map_verify_ack( - model->carriers.has(req->carrierid) ? gem::SlotMapVerifyAck::Accept - : gem::SlotMapVerifyAck::CarrierUnknown); - }); - router.on(3, 25, [](const s2::Message&) { - return gem::s3f26_carrier_transfer_ack(gem::CarrierActionAck::Accept); - }); - router.on(3, 27, [model](const s2::Message& m) { - auto cid = gem::parse_s3f27(m); - return gem::s3f28_cancel_carrier_ack( - cid && model->carriers.has(*cid) ? gem::CarrierActionAck::Accept - : gem::CarrierActionAck::CarrierIDUnknown); - }); - // S14 — E39 GetAttr + E94 CJ create/delete - router.on(14, 1, [model](const s2::Message& m) { - auto req = gem::parse_s14f1(m); - if (!req) return gem::s14f2_get_attr_data({}, gem::ObjectAck::Error); - auto* obj = model->cem.get(req->objspec); - if (!obj) return gem::s14f2_get_attr_data({}, gem::ObjectAck::Denied_UnknownObject); - std::vector attrs; - for (const auto& id : req->attrids) { - auto v = model->cem.get_attr(req->objspec, id); - attrs.push_back({id, v.value_or(s2::Item::ascii(""))}); - } - return gem::s14f2_get_attr_data(attrs, gem::ObjectAck::Success); - }); - router.on(14, 9, [model](const s2::Message& m) { - auto req = gem::parse_s14f9(m); - if (!req) return gem::s14f10_create_control_job_ack("", gem::ObjectAck::Error); - auto r = model->control_jobs.create(req->ctljobid, req->prjobids, - [model](const std::string& id) { return model->process_jobs.has(id); }); - auto ack = (r == gem::ControlJobStore::CreateResult::Created) - ? gem::ObjectAck::Success - : gem::ObjectAck::Denied_UnknownObject; - return gem::s14f10_create_control_job_ack(req->ctljobid, ack); - }); - router.on(14, 11, [model](const s2::Message& m) { - auto id = gem::parse_s14f11(m); - return gem::s14f12_delete_control_job_ack( - id && model->control_jobs.remove(*id) ? gem::ObjectAck::Success - : gem::ObjectAck::Denied_UnknownObject); - }); - // S16 — E40 PJ create/command/dequeue/monitor, E94 CJ command - router.on(16, 11, [model](const s2::Message& m) { - auto req = gem::parse_s16f11(m); - if (!req) return gem::s16f12_pr_job_create_ack(gem::HostCmdAck::ParameterInvalid); - auto r = model->process_jobs.create(req->prjobid, req->rcpspec.ppid, req->mtrloutspec, - [model](const std::string& ppid) { return model->recipes.get(ppid).has_value(); }); - auto ack = gem::HostCmdAck::Accept; - if (r == gem::ProcessJobStore::CreateResult::Denied_AlreadyExists) - ack = gem::HostCmdAck::Rejected; - else if (r == gem::ProcessJobStore::CreateResult::Denied_InvalidPpid) - ack = gem::HostCmdAck::ParameterInvalid; - return gem::s16f12_pr_job_create_ack(ack); - }); - router.on(16, 5, [model](const s2::Message& m) { - auto req = gem::parse_s16f5(m); - if (!req) return gem::s16f6_pr_job_command_ack(gem::HostCmdAck::ParameterInvalid); - auto ev = gem::pr_cmd_to_event(req->prcmd); - if (!ev) return gem::s16f6_pr_job_command_ack(gem::HostCmdAck::InvalidCommand); - return gem::s16f6_pr_job_command_ack(model->process_jobs.on_host_command(req->prjobid, *ev)); - }); - router.on(16, 7, [model](const s2::Message& m) { - auto req = gem::parse_s16f7(m); - if (!req) return gem::s16f8_pr_job_monitor_ack(gem::HostCmdAck::ParameterInvalid); - for (const auto& e : req->entries) - model->process_jobs.set_alert(e.prjobid, (e.pralert & 0x80) != 0); - return gem::s16f8_pr_job_monitor_ack(gem::HostCmdAck::Accept); - }); - router.on(16, 13, [model](const s2::Message& m) { - auto id = gem::parse_s16f13(m); - return gem::s16f14_pr_job_dequeue_ack( - id ? model->process_jobs.dequeue(*id) : gem::HostCmdAck::ParameterInvalid); - }); - router.on(16, 27, [model](const s2::Message& m) { - auto req = gem::parse_s16f27(m); - if (!req) return gem::s16f28_cj_command_ack(gem::HostCmdAck::ParameterInvalid); - auto ev = gem::ctl_cmd_to_event(req->ctljobcmd); - if (!ev) return gem::s16f28_cj_command_ack(gem::HostCmdAck::InvalidCommand); - return gem::s16f28_cj_command_ack( - model->control_jobs.on_host_command(req->ctljobid, *ev)); - }); -} - -// ============================================================================= -// §7. main() -// ============================================================================= +// Everything protocol-shaped is three calls now: construct an +// EquipmentRuntime from the YAML, register_default_handlers (all 56 GEM +// handlers + state-change emitters; ids bound via the config's roles: +// defaults), and hook tool behaviour onto the host commands with +// commands.set_handler. Compare with git history: this main() replaced +// ~650 lines of hand-wired Server/Router/handler plumbing. int main(int argc, char** argv) { const std::string config_path = (argc > 1) ? argv[1] : @@ -915,7 +453,7 @@ int main(int argc, char** argv) { const uint16_t metrics_port = (argc > 4) ? static_cast(std::stoi(argv[4])) : 9090; - // ---- §7.1 Validate the YAML configs before binding the port ---------- + // ---- Validate the YAML configs before binding the port ----------------- { config::ConfigValidator v; v.validate_equipment(config_path); @@ -927,167 +465,104 @@ int main(int argc, char** argv) { } } - auto logfn = [](const std::string& m) { - std::cout << "[pvd] " << m << "\n"; - }; + // std::endl (not "\n"): flush per line so logs are visible immediately + // when stdout is piped (docker logs, CI captures). + auto logfn = [](const std::string& m) { std::cout << "[pvd] " << m << std::endl; }; - // ---- §7.2 Build the data model --------------------------------------- - auto model = std::make_shared(); - config::EquipmentDescriptor desc; - config::ControlStateConfig sm_cfg; + // ---- The engine: one runtime + the default GEM behaviour --------------- + gem::EquipmentRuntime::Config cfg; + cfg.equipment_yaml = config_path; + cfg.control_state_yaml = state_path; + cfg.process_job_yaml = "/app/data/process_job_state.yaml"; + cfg.control_job_yaml = "/app/data/control_job_state.yaml"; + cfg.port = port; + cfg.log = logfn; + + std::unique_ptr runtime; try { - desc = config::load_equipment(config_path, *model); - sm_cfg = config::load_control_state(state_path); + runtime = std::make_unique(cfg); } catch (const std::exception& e) { std::cerr << "[pvd] config load failed: " << e.what() << "\n"; return 1; } - auto sm = std::make_shared( - sm_cfg.table, sm_cfg.initial); + auto& R = *runtime; + gem::register_default_handlers(R); + auto model = R.model_ptr(); logfn("loaded " + std::to_string(model->svids.size()) + " SVIDs, " + std::to_string(model->ecids.all().size()) + " ECIDs, " + std::to_string(model->events.all_events().size()) + " CEIDs, " + std::to_string(model->alarms.all().size()) + " alarms, " + std::to_string(model->recipes.list().size()) + " recipes"); - asio::io_context io; - - // ---- §7.3 Metrics exporter on a second port -------------------------- + // ---- Metrics exporter on a second port --------------------------------- auto registry = std::make_shared(); - registry->describe("pvd_messages_total", "SECS messages dispatched", + registry->describe("pvd_events_total", "Collection events fired", metrics::MetricType::Counter); registry->describe("pvd_chamber_pressure_torr", "Process chamber pressure", metrics::MetricType::Gauge); registry->describe("pvd_spool_depth", "Queued spool messages", metrics::MetricType::Gauge); - auto exporter = std::make_shared(io, metrics_port, registry); + auto exporter = std::make_shared( + R.io(), metrics_port, registry); exporter->start(); logfn("metrics exporter on :" + std::to_string(metrics_port) + "/metrics"); - // ---- §7.4 Sensor simulator, EPT cycler, alarm monitor --------------- - auto sim = std::make_shared(io, model); - sim->start(); - auto ept = std::make_shared(io, model); - ept->start(); - - // ---- §7.5 Server + handler wiring ------------------------------------ - Server::Config server_cfg{port, desc.device_id, {}}; - Server server(io, server_cfg); - server.on_log(logfn); - - auto active_conn = std::make_shared>(); - - // Shared event-emission helper. - auto deliver_or_spool = [active_conn, model, logfn](s2::Message msg) { - auto conn = active_conn->lock(); - if (!conn) { - model->spool.enqueue(msg); - return; - } - if (msg.reply_expected) { - conn->send_request(std::move(msg), [](std::error_code, const s2::Message&) {}); - } else { - conn->send_data(std::move(msg)); - } - }; - auto emit_event = [&io, model, deliver_or_spool, registry](uint32_t ceid) { - asio::post(io, [model, deliver_or_spool, ceid]() { - if (!model->is_event_enabled(ceid)) return; - auto reports = model->compose_reports_for(ceid); - deliver_or_spool(gem::s6f11_event_report(0, ceid, reports)); - }); + // ---- Tool behaviour: sensors, EPT, alarms, recipes ---------------------- + // The emit helpers are the runtime's thread-safe API plus a metrics bump. + auto emit_event = [&R, registry](uint32_t ceid) { + R.emit_event(ceid); registry->inc("pvd_events_total", {{"ceid", std::to_string(ceid)}}); }; - auto emit_alarm_set = [&io, model, deliver_or_spool, registry](uint32_t alid) { - asio::post(io, [model, deliver_or_spool, alid]() { - auto alarm = model->alarms.get(alid); - auto alcd = model->alarms.set_active(alid); - if (!alarm || !alcd || !model->alarms.enabled(alid)) return; - deliver_or_spool(gem::s5f1_alarm_report(*alcd, alid, alarm->text)); - }); - registry->inc("pvd_alarm_set_total", {{"alid", std::to_string(alid)}}); - }; - auto emit_alarm_clear = [&io, model, deliver_or_spool](uint32_t alid) { - asio::post(io, [model, deliver_or_spool, alid]() { - auto alarm = model->alarms.get(alid); - auto alcd = model->alarms.clear_active(alid); - if (!alarm || !alcd || !model->alarms.enabled(alid)) return; - deliver_or_spool(gem::s5f1_alarm_report(*alcd, alid, alarm->text)); - }); - }; + auto emit_alarm_set = [&R](uint32_t alid) { R.set_alarm(alid); }; + auto emit_alarm_clear = [&R](uint32_t alid) { R.clear_alarm(alid); }; + auto sim = std::make_shared(R.io(), model); + sim->start(); + auto ept = std::make_shared(R.io(), model); + ept->start(); auto alarm_mon = std::make_shared( - io, model, emit_alarm_set, emit_alarm_clear); + R.io(), model, emit_alarm_set, emit_alarm_clear); alarm_mon->start(); - auto recipe_runner = std::make_shared( - io, model, *sim, emit_event); + R.io(), model, *sim, emit_event); - // Wire control-state-change → CEID emission. - sm->set_state_change_handler( - [logfn, emit_event, desc](gem::ControlState from, gem::ControlState to, - gem::ControlEvent ev) { - logfn(std::string("control: ") + gem::control_state_name(from) + - " -> " + gem::control_state_name(to) + - " (" + gem::control_event_name(ev) + ")"); - if (desc.emit_on_control_change) - emit_event(*desc.emit_on_control_change); + // Host command behaviour via the set_handler hook (runs on the io thread + // during S2F41/F21/F49 dispatch) — this used to be a hand-rolled S2F41 + // router override. The YAML still declares the static ack + emit_ceid; + // this adds the part config can't express: actually starting the recipe. + model->commands.set_handler( + "START", [model, recipe_runner](const std::string&, + const std::vector&) { + for (const auto& pjid : model->process_jobs.ids()) { + auto* pj = model->process_jobs.get(pjid); + if (pj && pj->fsm->state() == gem::ProcessJobState::WaitingForStart) { + model->process_jobs.fire_internal(pjid, gem::ProcessJobEvent::Start); + recipe_runner->start(pjid); + break; + } + } + return gem::HostCmdAck::Accept; }); - // ---- §7.6 Periodic gauge updates for Prometheus ---------------------- - auto gauge_timer = std::make_shared(io); - std::function gauge_tick = - [&gauge_tick, gauge_timer, model, registry](std::error_code ec) { - if (ec) return; - auto p = model->svids.get(pvd::kSvidChamberPressure); - if (p) { - const float v = std::get>(p->value.storage())[0]; - registry->set_gauge("pvd_chamber_pressure_torr", v); - } - registry->set_gauge("pvd_spool_depth", model->spool.size()); - gauge_timer->expires_after(5s); - gauge_timer->async_wait(gauge_tick); - }; + // ---- Periodic gauge updates for Prometheus ------------------------------ + auto gauge_timer = std::make_shared(R.io()); + auto gauge_tick = std::make_shared>(); + *gauge_tick = [gauge_tick, gauge_timer, model, registry](std::error_code ec) { + if (ec) return; + auto p = model->svids.get(pvd::kSvidChamberPressure); + if (p && std::holds_alternative>(p->value.storage())) { + registry->set_gauge("pvd_chamber_pressure_torr", + std::get>(p->value.storage())[0]); + } + registry->set_gauge("pvd_spool_depth", model->spool.size()); + gauge_timer->expires_after(5s); + gauge_timer->async_wait(*gauge_tick); + }; gauge_timer->expires_after(5s); - gauge_timer->async_wait(gauge_tick); + gauge_timer->async_wait(*gauge_tick); - // ---- §7.7 Router + per-connection handler wiring -------------------- - gem::Router router; - register_handlers(router, model, sm, desc, - emit_event, emit_alarm_set, recipe_runner); - logfn("registered " + std::to_string(router.size()) + " SECS-II handlers"); - - server.on_connection([&io, sm, model, logfn, active_conn, &router, registry] - (std::shared_ptr conn) { - *active_conn = conn; - conn->set_closed_handler([active_conn](const std::string&) { - active_conn->reset(); - }); - conn->set_selected_handler([logfn, sm]() { - logfn(std::string("host SELECTED; control=") + - gem::control_state_name(sm->state())); - }); - conn->set_message_handler( - [&router, model, conn, registry](const s2::Message& msg) - -> std::optional { - registry->inc("pvd_messages_total", - {{"dir", "rx"}, - {"stream", std::to_string(msg.stream)}, - {"function", std::to_string(msg.function)}}); - return router.dispatch_with_s9( - [&](uint8_t f, const std::array& mh) { - conn->emit_s9(f, mh); - }, - [&]() -> std::optional> { - auto* h = conn->current_header(); - return h ? std::optional{h->encode()} : std::nullopt; - }, msg); - }); - }); - - server.start(); logfn("ACME-PVD-3000 ready, listening on :" + std::to_string(port)); - io.run(); + R.run(); // accept connections + run the io_context (blocks) return 0; }