raphael
9c5d67fdad
Customer SREs and capacity planners had nothing to point at.
INTEGRATION.md asked the right questions ("how many tx/sec?"
"how much memory per active CJ?") but had no numbers.
secs_bench spins up an in-process passive equipment + active host
on an OS-allocated port, runs three canned workloads, and emits a
markdown table customers can capture and diff across commits:
- S1F1/F2 header-only round-trip — dispatch + framing baseline
- S1F3/F4 with N SVIDs — encode + decode throughput
- S6F11 push (W=0) — one-way emission ceiling
- PJ + CJ pair memory footprint — bytes per active job
Latency reports p50/p95/p99/max via std::nth_element over the
sample vector. RSS is read from /proc/self/statm on Linux,
mach_task_basic_info on macOS.
CLI: --requests / --concurrency / --svid-count / --store-pairs.
Default 20k req @ 16 concurrent.
BENCHMARKS.md checks in a reference run (Docker on M-series
macOS): ~140k req/s S1F1, ~79k req/s S1F3 with 32-SVID list,
~572k S6F11/s push, ~450 bytes per PJ+CJ pair. Three orders of
magnitude headroom over typical fab tool load.
The doc is explicit about what the bench does NOT measure (real
network, persistence I/O, TLS tunnel overhead, multi-session GS
dispatch) — customers should re-run on their target hardware.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3.0 KiB
Performance baseline
Numbers from build/secs_bench --requests 20000 --concurrency 16 on
Docker / Ubuntu 24.04 inside Docker Desktop on macOS (M-series), single
io_context thread. Treat as rough envelope for capacity planning,
not lab-grade benchmarks; re-run on your target hardware before
sizing pods or VMs.
Round-trip throughput / latency
| Scenario | Ops | Elapsed | Ops/sec | p50 µs | p95 µs | p99 µs |
|---|---|---|---|---|---|---|
| S1F1/F2 (header-only) | 20000 | 0.14 | ~140000 | 74 | 103 | 161 |
| S1F3/F4 (32 SVIDs) | 20000 | 0.25 | ~79000 | 165 | 186 | 260 |
| S6F11 push (W=0) | 20000 | 0.03 | ~572000 | n/a | n/a | n/a |
Read the table this way. A real fab tool needs to handle tens to a few hundred S6F11 events/second sustained. We're three orders of magnitude above that on the push path, two orders above on synchronous round-trips. Throughput is not the bottleneck; latency tail under contention is.
Memory footprint
A ProcessJob + ControlJob pair (no persistence enabled) is around
~450 bytes of heap (1000 pairs ≈ 0.45 MiB, measured on a fresh
process). With persistence enabled add ~200 bytes of in-memory journal
path tracking per record.
| Active entity | Approx bytes / instance |
|---|---|
| PJ + CJ pair | ~450 |
| Carrier (no slots) | ~80 |
| Carrier slot | ~24 |
| Substrate | ~120 |
| Spool entry | ~40 + encoded body size |
A busy fab tool tracking 50 carriers × 25 slots, 200 substrates, 20 active PJ+CJ pairs comes in well under 1 MiB of model state. RSS will be dominated by the binary itself + asio's buffers (~10-20 MiB), not the model.
How to re-run
docker compose run --rm builder /app/build/secs_bench \
--requests 50000 \
--concurrency 32 \
--svid-count 32 \
--store-pairs 10000
Output is markdown — pipe to a file and commit it to your CI so regressions show up as diffs.
What this does NOT measure
- Real network. Loopback TCP has no MTU fragmentation, no retransmits, no jitter. Production HSMS over a fab control LAN will see higher tail latency.
- Persistence write amplification. The bench runs with persistence disabled. Each store mutation with persistence enabled is one atomic-rename to disk; on rotational media that limits you to a few hundred mutations/sec. SSD-backed deployments are fine.
- Concurrent S6F11 enable filtering. Real CEID emission gates on the host's enable/disable list — this bench fires raw S6F11s.
- Multi-session HSMS-GS dispatch overhead — single-session only.
- TLS-tunneled sockets (via stunnel/sidecar) — these add ~50 µs per round-trip on modern hardware.