raphael
af1a159c59
A large gap had opened between the docs and the code: the README and INTEGRATION guide did not mention the gRPC daemon or the Python client at all (the entire vendor surface), ARCHITECTURE still described secs_server as the ~1200-line canonical wiring example (it is a ~110-line thin main over EquipmentRuntime), and test counts across six files were stale (445/2753 -> 473/3087 core + the separate 125-assertion daemon suite). - README: new "Integrating your tool (pick a tier)" section — Python client / any-language gRPC / embedded C++ — plus daemon tests and tools/run_interop.sh in the Testing section. - ARCHITECTURE: layer diagram gains the vendor-surface and EquipmentRuntime/default_handlers tiers; stale wiring row fixed. - INTEGRATION: three-tier chooser up front (this guide = the C++ tier). - ch30 tour: secs_gemd + secs_gemd_tests in the binaries table. - ch31: example alarm used a nonexistent `alcd:` field with bit 7 set (which the validator forbids) -> real `category:`/`name:` fields, and the roles: block documented. - ch35: handler-location note now points at default_handlers.cpp's 15 per-capability register_* functions. - ch40: built-artifacts list + sample output counts. - ch50: secsgem::gem runtime/default_handlers/handler_slot/name_index includes + new secsgem::daemon namespace section. - PROOFS: test-count table gains the runtime/handlers/daemon row so the tally adds up; daemon suite noted. VERIFICATION/COMPLIANCE counts. - interop/README: the one-command runner + the two daemon-track harnesses (daemon_interop, pyclient_interop). Audited via a docs-vs-code sweep (the audit itself under-reported: it validated counts textually; reality was 473/3087). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
70 lines
6.9 KiB
Markdown
70 lines
6.9 KiB
Markdown
# Proofs
|
|
|
|
Eight commands. If they all exit zero on a fresh clone, the codebase
|
|
implements what [COMPLIANCE.md](COMPLIANCE.md) claims.
|
|
|
|
| # | Command | What it proves |
|
|
|---|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|
|
|
| 1 | `docker compose run --rm tests` | **473 test cases / 3 087 assertions** pass: every store, FSM, codec, parser, persistence path |
|
|
| 2 | `docker compose run --rm builder /app/build/secs_conformance --host server --port 5000` | **47 wire-level conformance checks** PASS against a live passive equipment |
|
|
| 3 | `docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py --host server` | **31 interop checks** PASS against secsgem-py 0.3.0 (the Python reference impl) |
|
|
| 4 | `SECSGEM_ROBUSTNESS_SOAK=1 docker compose run --rm builder /app/build/secsgem_tests -tc='*soak*'` | **100 000 random tool operations** execute with all invariants and persistence round-trips holding |
|
|
| 5 | `docker compose run --rm builder /app/build/secs_server --validate-config --config /app/data/equipment.yaml --state-table /app/data/control_state.yaml --pj-state-table /app/data/process_job_state.yaml --cj-state-table /app/data/control_job_state.yaml` | Every shipped YAML config passes structural + referential validation |
|
|
| 6 | `docker compose run --rm builder bash /app/interop/tshark_validate.sh` | **69 HSMS frames** dissected by Wireshark's HSMS dissector (independent third codec) with no malformed packets |
|
|
| 7 | `bash interop/secs4j_validate.sh` | **55 cross-validation checks** PASS against [secs4java8](https://github.com/kenta-shimizu/secs4java8) (independent Java implementation), covering S1/S2/S3/S5/S6/S7/S10/S14/S16, the full E40 PJ body, dynamic event reports + unsolicited S6F11 / S5F1 observation, alarm management, spool, PP management, terminal services, limits, trace, E39, and the GEM 300 streams secsgem-py couldn't easily drive |
|
|
| 8 | `cmake -B build-fuzz -DSECSGEM_FUZZ=ON && build-fuzz/fuzz_secs2_decode -max_total_time=60` | **200 000+ random inputs** through `secs2::decode`, **1.4 M+** through `try_parse_sml` per 60 s lane, ASan + UBSan + libFuzzer coverage, **0 crashes** |
|
|
|
|
CI ([Gitea Actions](.gitea/workflows/ci.yml)) runs a Release build +
|
|
full suite and a separate `-fsanitize=thread` lane on every push to
|
|
`main`. All 473 cases / 3 087 assertions pass under TSan clean.
|
|
|
|
## Per-standard test coverage
|
|
|
|
Every claimed standard has dedicated tests. Counts are
|
|
`grep -c TEST_CASE`; cross-cutting tests (e.g. `test_robustness_fuzz`,
|
|
`test_gem300_scenario`) exercise multiple standards in concert.
|
|
|
|
| Standard | Test files | Cases |
|
|
|-----------------------------------|-------------------------------------------------------------------------------------------|------:|
|
|
| **E5** — SECS-II encoding | `test_secs2`, `test_sml`, `test_messages`, `test_e5_kat`, `test_identifier_wildcards`, `test_fuzz` | 139 |
|
|
| **E5 §13** — exceptions | `test_exceptions`, `test_exception_persistence` | 16 |
|
|
| **E4** — SECS-I transport | `test_secsi`, `test_secsi_timers`, `test_secsi_tcp` | 27 |
|
|
| **E37** — HSMS (SS + GS) | `test_hsms`, `test_hsms_connection`, `test_hsms_timers`, `test_hsms_s9`, `test_hsms_gs`, `test_hsms_gs_integration`, `test_s9_fallback`, `test_concurrency` | 34 |
|
|
| **E30** — GEM core | `test_control_state`, `test_communication_state`, `test_host_handler`, `test_data_model`, `test_loader`, `test_config_validate` | 71 |
|
|
| **E40** — process jobs | `test_process_jobs` | 21 |
|
|
| **E94** — control jobs | `test_control_jobs` | 9 |
|
|
| **E42** — formatted PP | `test_e42_formatted_pp` | 6 |
|
|
| **E87** — carriers + load ports | `test_carriers`, `test_carrier_state`, `test_carrier_persistence`, `test_e87_wire_scenarios` | 27 |
|
|
| **E90** — substrate tracking | `test_substrates`, `test_substrate_persistence` | 21 |
|
|
| **E116** — EPT | `test_ept` | 7 |
|
|
| **E120 / E39** — common equip / object service | `test_cem_objects` | 3 |
|
|
| **E157** — module process tracking | `test_modules` | 5 |
|
|
| **E84** — parallel I/O + timers | `test_e84`, `test_e84_ports`, `test_e84_timers`, `test_e84_asio_timers` | 27 |
|
|
| Persistence + cross-cutting | `test_job_persistence`, `test_persistence_upgrade`, `test_wire_ceid_emission`, `test_gem300_scenario`, `test_live_gem300`, `test_thread_safety`, `test_metrics_prometheus`, `test_robustness_fuzz` | 32 |
|
|
| Runtime / handlers / daemon glue | `test_runtime`, `test_default_handlers`, `test_handler_conformance` (53-handler sweep + golden frames), `test_name_index`, loader/validator additions | 28 |
|
|
| **Total** | | **473** |
|
|
|
|
`docker compose run --rm builder /app/build/secsgem_tests --list-test-cases | wc -l`
|
|
currently reports 473. The gRPC daemon has its own binary on top:
|
|
`secs_gemd_tests` (in-process gRPC service tests, 125 assertions, also run
|
|
under ThreadSanitizer in CI).
|
|
|
|
## Categories of evidence
|
|
|
|
The eight commands split into four kinds:
|
|
|
|
- **Internal** (#1, #2, #4, #5) — unit suite, conformance harness,
|
|
soak property test, config validator. Necessary but not
|
|
independent.
|
|
- **External, second implementation** (#3, #7) — secsgem-py (Python)
|
|
and secs4java8 (Java). Two independent SECS implementations agree
|
|
on every frame.
|
|
- **External, third codec** (#6) — Wireshark's HSMS dissector,
|
|
written by network-protocol authors with no shared code. Catches
|
|
framing bugs the implementations might share.
|
|
- **External, structural search** (#8) — libFuzzer + ASan + UBSan
|
|
over the decoder and SML parser surface.
|
|
|
|
[VERIFICATION.md](VERIFICATION.md) covers each external validator in
|
|
detail.
|