raphael
dae6bfd747
Tone pass across the non-tutorial markdown — README, PROOFS,
ARCHITECTURE, BENCHMARKS, COMPLIANCE, FAQ, MES_INTEROP, SECURITY,
and interop/README. Three patterns came out:
- Bug-history war stories ("Past interop sweeps surfaced…",
"What these harnesses caught: 1. Strict U-width parsing…").
- Chat-with-reader framing ("Don't skip TLS unless…", "Treat as a
punch list", "If you're running in a pod…", "Misconfiguration
incidents drop dramatically").
- Self-referential narration ("we ship", "our codec", "the
codebase's most-tested layer", "three orders of magnitude above
fab load", "the gift that keeps giving").
README also drops the standalone ThreadSanitizer subsection under
Build details (now a single line under the new Testing section).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
6.5 KiB
6.5 KiB
Proofs
Eight commands. If they all exit zero on a fresh clone, the codebase implements what COMPLIANCE.md claims.
| # | Command | What it proves |
|---|---|---|
| 1 | docker compose run --rm tests |
445 test cases / 2 753 assertions pass: every store, FSM, codec, parser, persistence path |
| 2 | docker compose run --rm builder /app/build/secs_conformance --host server --port 5000 |
47 wire-level conformance checks PASS against a live passive equipment |
| 3 | docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py --host server |
31 interop checks PASS against secsgem-py 0.3.0 (the Python reference impl) |
| 4 | SECSGEM_ROBUSTNESS_SOAK=1 docker compose run --rm builder /app/build/secsgem_tests -tc='*soak*' |
100 000 random tool operations execute with all invariants and persistence round-trips holding |
| 5 | docker compose run --rm builder /app/build/secs_server --validate-config --config /app/data/equipment.yaml --state-table /app/data/control_state.yaml --pj-state-table /app/data/process_job_state.yaml --cj-state-table /app/data/control_job_state.yaml |
Every shipped YAML config passes structural + referential validation |
| 6 | docker compose run --rm builder bash /app/interop/tshark_validate.sh |
69 HSMS frames dissected by Wireshark's HSMS dissector (independent third codec) with no malformed packets |
| 7 | bash interop/secs4j_validate.sh |
55 cross-validation checks PASS against secs4java8 (independent Java implementation), covering S1/S2/S3/S5/S6/S7/S10/S14/S16, the full E40 PJ body, dynamic event reports + unsolicited S6F11 / S5F1 observation, alarm management, spool, PP management, terminal services, limits, trace, E39, and the GEM 300 streams secsgem-py couldn't easily drive |
| 8 | cmake -B build-fuzz -DSECSGEM_FUZZ=ON && build-fuzz/fuzz_secs2_decode -max_total_time=60 |
200 000+ random inputs through secs2::decode, 1.4 M+ through try_parse_sml per 60 s lane, ASan + UBSan + libFuzzer coverage, 0 crashes |
CI (Gitea Actions) runs a Release build +
full suite and a separate -fsanitize=thread lane on every push to
main. All 445 cases / 2 753 assertions pass under TSan clean.
Per-standard test coverage
Every claimed standard has dedicated tests. Counts are
grep -c TEST_CASE; cross-cutting tests (e.g. test_robustness_fuzz,
test_gem300_scenario) exercise multiple standards in concert.
| Standard | Test files | Cases |
|---|---|---|
| E5 — SECS-II encoding | test_secs2, test_sml, test_messages, test_e5_kat, test_identifier_wildcards, test_fuzz |
139 |
| E5 §13 — exceptions | test_exceptions, test_exception_persistence |
16 |
| E4 — SECS-I transport | test_secsi, test_secsi_timers, test_secsi_tcp |
27 |
| E37 — HSMS (SS + GS) | test_hsms, test_hsms_connection, test_hsms_timers, test_hsms_s9, test_hsms_gs, test_hsms_gs_integration, test_s9_fallback, test_concurrency |
34 |
| E30 — GEM core | test_control_state, test_communication_state, test_host_handler, test_data_model, test_loader, test_config_validate |
71 |
| E40 — process jobs | test_process_jobs |
21 |
| E94 — control jobs | test_control_jobs |
9 |
| E42 — formatted PP | test_e42_formatted_pp |
6 |
| E87 — carriers + load ports | test_carriers, test_carrier_state, test_carrier_persistence, test_e87_wire_scenarios |
27 |
| E90 — substrate tracking | test_substrates, test_substrate_persistence |
21 |
| E116 — EPT | test_ept |
7 |
| E120 / E39 — common equip / object service | test_cem_objects |
3 |
| E157 — module process tracking | test_modules |
5 |
| E84 — parallel I/O + timers | test_e84, test_e84_ports, test_e84_timers, test_e84_asio_timers |
27 |
| Persistence + cross-cutting | test_job_persistence, test_persistence_upgrade, test_wire_ceid_emission, test_gem300_scenario, test_live_gem300, test_thread_safety, test_metrics_prometheus, test_robustness_fuzz |
32 |
| Total | 445 |
docker compose run --rm builder /app/build/secsgem_tests --list-test-cases | wc -l
currently reports 445.
Categories of evidence
The eight commands split into four kinds:
- Internal (#1, #2, #4, #5) — unit suite, conformance harness, soak property test, config validator. Necessary but not independent.
- External, second implementation (#3, #7) — secsgem-py (Python) and secs4java8 (Java). Two independent SECS implementations agree on every frame.
- External, third codec (#6) — Wireshark's HSMS dissector, written by network-protocol authors with no shared code. Catches framing bugs the implementations might share.
- External, structural search (#8) — libFuzzer + ASan + UBSan over the decoder and SML parser surface.
VERIFICATION.md covers each external validator in detail.