metrics: Prometheus exporter sample + worked INTEGRATION example
README §3 promised a monitoring story ("aggregate into Prometheus via
a sidecar that polls the data model"). Nothing shipped. Customers
running a real fab without a metrics pipeline find out about T7
storms, spool blowups, and stalled CJs after their MES does — not
the position you want SRE in.
This commit ships:
- include/secsgem/metrics/prometheus.hpp: header-only. A Registry
(counters + gauges + HELP/TYPE descriptions, label-keyed,
mutex-guarded so updates from the io thread and scrape renders from
the same io serialize cleanly) plus a PrometheusServer (asio
acceptor, replies to any GET with the text-exposition rendering,
no auth — drop nginx in front for that).
- tests/test_metrics_prometheus.cpp: 3 cases / 19 assertions.
Render counter+gauge with labels, scrape via raw TCP and parse the
HTTP body, verify live updates land on subsequent scrapes.
- INTEGRATION.md §6.4: worked example that pairs the exporter with the
Connection + EquipmentDataModel hooks documented in §6.1/§6.2.
Shows the wrap-around-handler trick for message counters, a 5s
polling timer for gauges (spool depth, active alarms), and the
expected /metrics output.
Deliberately *not* shipped:
- A StandardMetrics helper that auto-wires everything — would force
a single hook owner per store, breaking customers who want
composable observers. Customers wire what they need; the registry
gives them counters + gauges + an HTTP endpoint, no policy.
- TLS / auth on the HTTP endpoint. Reverse-proxy territory.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -450,6 +450,87 @@ body arrived; S9F9 means a reply didn't arrive in T3; S9F11 means
|
||||
a frame exceeded the 16 MiB cap. None of these are normal — they're
|
||||
real diagnostic events.
|
||||
|
||||
### 6.4. Prometheus exporter (worked example)
|
||||
|
||||
`include/secsgem/metrics/prometheus.hpp` ships a minimal Registry +
|
||||
asio-backed HTTP server. Drop it next to your equipment and scrape
|
||||
from your fab's Prometheus.
|
||||
|
||||
```cpp
|
||||
#include "secsgem/metrics/prometheus.hpp"
|
||||
namespace metrics = secsgem::metrics;
|
||||
|
||||
auto reg = std::make_shared<metrics::Registry>();
|
||||
reg->describe("secsgem_messages_total", "messages dispatched",
|
||||
metrics::MetricType::Counter);
|
||||
reg->describe("secsgem_alarms_active", "currently-active alarms",
|
||||
metrics::MetricType::Gauge);
|
||||
reg->describe("secsgem_spool_depth", "queued spool messages",
|
||||
metrics::MetricType::Gauge);
|
||||
reg->describe("secsgem_t_timer_total", "T-timer expiry by id",
|
||||
metrics::MetricType::Counter);
|
||||
|
||||
// HTTP /metrics on :9090. Same io_context as the HSMS connection —
|
||||
// scraping runs on the strand, so updates and reads serialize for free.
|
||||
auto exporter = std::make_shared<metrics::PrometheusServer>(io, 9090, reg);
|
||||
exporter->start();
|
||||
|
||||
// Wire counters into the connection + model hooks you already set up
|
||||
// in §6.1 / §6.2. These all fire on the io_context thread.
|
||||
conn->set_selected_handler([reg, conn] {
|
||||
reg->set_gauge("secsgem_hsms_selected", 1);
|
||||
});
|
||||
conn->set_closed_handler([reg](const std::string& reason) {
|
||||
reg->set_gauge("secsgem_hsms_selected", 0);
|
||||
// T-timer expirations surface here as `reason` starting with "T".
|
||||
if (!reason.empty() && reason[0] == 'T')
|
||||
reg->inc("secsgem_t_timer_total", {{"timer", reason.substr(0, 2)}});
|
||||
});
|
||||
|
||||
// Per-message dispatch — wrap your existing router.dispatch() call.
|
||||
auto orig_handler = conn->message_handler(); // (or whatever you set)
|
||||
conn->set_message_handler([reg, orig_handler](const secs2::Message& m) {
|
||||
reg->inc("secsgem_messages_total",
|
||||
{{"dir", "rx"},
|
||||
{"stream", std::to_string(m.stream)},
|
||||
{"function", std::to_string(m.function)}});
|
||||
return orig_handler(m);
|
||||
});
|
||||
|
||||
// Push gauge snapshots from a periodic timer on the same io.
|
||||
auto poll = std::make_shared<asio::steady_timer>(io);
|
||||
std::function<void(std::error_code)> tick = [reg, model, poll, &tick](std::error_code ec) {
|
||||
if (ec) return;
|
||||
reg->set_gauge("secsgem_spool_depth",
|
||||
static_cast<double>(model->spool.size()));
|
||||
std::size_t active = 0;
|
||||
for (auto& a : model->alarms.all())
|
||||
if (model->alarms.active(a.id)) ++active;
|
||||
reg->set_gauge("secsgem_alarms_active", static_cast<double>(active));
|
||||
poll->expires_after(std::chrono::seconds(5));
|
||||
poll->async_wait(tick);
|
||||
};
|
||||
poll->expires_after(std::chrono::seconds(5));
|
||||
poll->async_wait(tick);
|
||||
```
|
||||
|
||||
What lands at `/metrics`:
|
||||
|
||||
```
|
||||
# HELP secsgem_messages_total messages dispatched
|
||||
# TYPE secsgem_messages_total counter
|
||||
secsgem_messages_total{dir="rx",function="13",stream="1"} 42
|
||||
# TYPE secsgem_spool_depth gauge
|
||||
secsgem_spool_depth 7
|
||||
# TYPE secsgem_hsms_selected gauge
|
||||
secsgem_hsms_selected 1
|
||||
```
|
||||
|
||||
Wire this into your fab's Prometheus + Grafana and you've got the
|
||||
starter dashboard the README §3 table describes. The exporter has
|
||||
**no auth and no TLS** — drop nginx or Caddy in front with mTLS for
|
||||
production.
|
||||
|
||||
---
|
||||
|
||||
## 7. Recommended layout for a vendor application
|
||||
|
||||
Reference in New Issue
Block a user