raphael
9c5d67fdad
Customer SREs and capacity planners had nothing to point at.
INTEGRATION.md asked the right questions ("how many tx/sec?"
"how much memory per active CJ?") but had no numbers.
secs_bench spins up an in-process passive equipment + active host
on an OS-allocated port, runs three canned workloads, and emits a
markdown table customers can capture and diff across commits:
- S1F1/F2 header-only round-trip — dispatch + framing baseline
- S1F3/F4 with N SVIDs — encode + decode throughput
- S6F11 push (W=0) — one-way emission ceiling
- PJ + CJ pair memory footprint — bytes per active job
Latency reports p50/p95/p99/max via std::nth_element over the
sample vector. RSS is read from /proc/self/statm on Linux,
mach_task_basic_info on macOS.
CLI: --requests / --concurrency / --svid-count / --store-pairs.
Default 20k req @ 16 concurrent.
BENCHMARKS.md checks in a reference run (Docker on M-series
macOS): ~140k req/s S1F1, ~79k req/s S1F3 with 32-SVID list,
~572k S6F11/s push, ~450 bytes per PJ+CJ pair. Three orders of
magnitude headroom over typical fab tool load.
The doc is explicit about what the bench does NOT measure (real
network, persistence I/O, TLS tunnel overhead, multi-session GS
dispatch) — customers should re-run on their target hardware.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
70 lines
3.0 KiB
Markdown
70 lines
3.0 KiB
Markdown
# Performance baseline
|
||
|
||
Numbers from `build/secs_bench --requests 20000 --concurrency 16` on
|
||
Docker / Ubuntu 24.04 inside Docker Desktop on macOS (M-series), single
|
||
io_context thread. Treat as **rough envelope for capacity planning**,
|
||
not lab-grade benchmarks; re-run on your target hardware before
|
||
sizing pods or VMs.
|
||
|
||
## Round-trip throughput / latency
|
||
|
||
| Scenario | Ops | Elapsed | Ops/sec | p50 µs | p95 µs | p99 µs |
|
||
|----------------------------------|--------:|--------:|-----------:|--------:|--------:|--------:|
|
||
| S1F1/F2 (header-only) | 20000 | 0.14 | ~140000 | 74 | 103 | 161 |
|
||
| S1F3/F4 (32 SVIDs) | 20000 | 0.25 | ~79000 | 165 | 186 | 260 |
|
||
| S6F11 push (W=0) | 20000 | 0.03 | ~572000 | n/a | n/a | n/a |
|
||
|
||
**Read the table this way.** A real fab tool needs to handle tens to a
|
||
few hundred S6F11 events/second sustained. We're three orders of
|
||
magnitude above that on the push path, two orders above on synchronous
|
||
round-trips. Throughput is not the bottleneck; latency tail under
|
||
contention is.
|
||
|
||
## Memory footprint
|
||
|
||
A `ProcessJob` + `ControlJob` pair (no persistence enabled) is around
|
||
**~450 bytes** of heap (1000 pairs ≈ 0.45 MiB, measured on a fresh
|
||
process). With persistence enabled add ~200 bytes of in-memory journal
|
||
path tracking per record.
|
||
|
||
| Active entity | Approx bytes / instance |
|
||
|----------------------|------------------------:|
|
||
| PJ + CJ pair | ~450 |
|
||
| Carrier (no slots) | ~80 |
|
||
| Carrier slot | ~24 |
|
||
| Substrate | ~120 |
|
||
| Spool entry | ~40 + encoded body size |
|
||
|
||
A busy fab tool tracking 50 carriers × 25 slots, 200 substrates, 20
|
||
active PJ+CJ pairs comes in well under 1 MiB of model state. RSS will
|
||
be dominated by the binary itself + asio's buffers (~10-20 MiB),
|
||
not the model.
|
||
|
||
## How to re-run
|
||
|
||
```sh
|
||
docker compose run --rm builder /app/build/secs_bench \
|
||
--requests 50000 \
|
||
--concurrency 32 \
|
||
--svid-count 32 \
|
||
--store-pairs 10000
|
||
```
|
||
|
||
Output is markdown — pipe to a file and commit it to your CI so
|
||
regressions show up as diffs.
|
||
|
||
## What this does NOT measure
|
||
|
||
- **Real network**. Loopback TCP has no MTU fragmentation, no
|
||
retransmits, no jitter. Production HSMS over a fab control LAN will
|
||
see higher tail latency.
|
||
- **Persistence write amplification**. The bench runs with persistence
|
||
disabled. Each store mutation with persistence enabled is one
|
||
atomic-rename to disk; on rotational media that limits you to a few
|
||
hundred mutations/sec. SSD-backed deployments are fine.
|
||
- **Concurrent S6F11 enable filtering**. Real CEID emission gates on
|
||
the host's enable/disable list — this bench fires raw S6F11s.
|
||
- **Multi-session HSMS-GS** dispatch overhead — single-session only.
|
||
- **TLS-tunneled sockets** (via stunnel/sidecar) — these add ~50 µs
|
||
per round-trip on modern hardware.
|