bench: secs_bench harness + BENCHMARKS.md baseline

Customer SREs and capacity planners had nothing to point at.
INTEGRATION.md asked the right questions ("how many tx/sec?"
"how much memory per active CJ?") but had no numbers.

secs_bench spins up an in-process passive equipment + active host
on an OS-allocated port, runs three canned workloads, and emits a
markdown table customers can capture and diff across commits:

- S1F1/F2 header-only round-trip   — dispatch + framing baseline
- S1F3/F4 with N SVIDs             — encode + decode throughput
- S6F11 push (W=0)                  — one-way emission ceiling
- PJ + CJ pair memory footprint    — bytes per active job

Latency reports p50/p95/p99/max via std::nth_element over the
sample vector.  RSS is read from /proc/self/statm on Linux,
mach_task_basic_info on macOS.

CLI: --requests / --concurrency / --svid-count / --store-pairs.
Default 20k req @ 16 concurrent.

BENCHMARKS.md checks in a reference run (Docker on M-series
macOS): ~140k req/s S1F1, ~79k req/s S1F3 with 32-SVID list,
~572k S6F11/s push, ~450 bytes per PJ+CJ pair.  Three orders of
magnitude headroom over typical fab tool load.

The doc is explicit about what the bench does NOT measure (real
network, persistence I/O, TLS tunnel overhead, multi-session GS
dispatch) — customers should re-run on their target hardware.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
2026-06-09 14:36:50 +02:00
parent a4599b3b9d
commit 9c5d67fdad
4 changed files with 439 additions and 2 deletions
+69
View File
@@ -0,0 +1,69 @@
# Performance baseline
Numbers from `build/secs_bench --requests 20000 --concurrency 16` on
Docker / Ubuntu 24.04 inside Docker Desktop on macOS (M-series), single
io_context thread. Treat as **rough envelope for capacity planning**,
not lab-grade benchmarks; re-run on your target hardware before
sizing pods or VMs.
## Round-trip throughput / latency
| Scenario | Ops | Elapsed | Ops/sec | p50 µs | p95 µs | p99 µs |
|----------------------------------|--------:|--------:|-----------:|--------:|--------:|--------:|
| S1F1/F2 (header-only) | 20000 | 0.14 | ~140000 | 74 | 103 | 161 |
| S1F3/F4 (32 SVIDs) | 20000 | 0.25 | ~79000 | 165 | 186 | 260 |
| S6F11 push (W=0) | 20000 | 0.03 | ~572000 | n/a | n/a | n/a |
**Read the table this way.** A real fab tool needs to handle tens to a
few hundred S6F11 events/second sustained. We're three orders of
magnitude above that on the push path, two orders above on synchronous
round-trips. Throughput is not the bottleneck; latency tail under
contention is.
## Memory footprint
A `ProcessJob` + `ControlJob` pair (no persistence enabled) is around
**~450 bytes** of heap (1000 pairs ≈ 0.45 MiB, measured on a fresh
process). With persistence enabled add ~200 bytes of in-memory journal
path tracking per record.
| Active entity | Approx bytes / instance |
|----------------------|------------------------:|
| PJ + CJ pair | ~450 |
| Carrier (no slots) | ~80 |
| Carrier slot | ~24 |
| Substrate | ~120 |
| Spool entry | ~40 + encoded body size |
A busy fab tool tracking 50 carriers × 25 slots, 200 substrates, 20
active PJ+CJ pairs comes in well under 1 MiB of model state. RSS will
be dominated by the binary itself + asio's buffers (~10-20 MiB),
not the model.
## How to re-run
```sh
docker compose run --rm builder /app/build/secs_bench \
--requests 50000 \
--concurrency 32 \
--svid-count 32 \
--store-pairs 10000
```
Output is markdown — pipe to a file and commit it to your CI so
regressions show up as diffs.
## What this does NOT measure
- **Real network**. Loopback TCP has no MTU fragmentation, no
retransmits, no jitter. Production HSMS over a fab control LAN will
see higher tail latency.
- **Persistence write amplification**. The bench runs with persistence
disabled. Each store mutation with persistence enabled is one
atomic-rename to disk; on rotational media that limits you to a few
hundred mutations/sec. SSD-backed deployments are fine.
- **Concurrent S6F11 enable filtering**. Real CEID emission gates on
the host's enable/disable list — this bench fires raw S6F11s.
- **Multi-session HSMS-GS** dispatch overhead — single-session only.
- **TLS-tunneled sockets** (via stunnel/sidecar) — these add ~50 µs
per round-trip on modern hardware.