docs: customer-ready sweep + README restructure + tshark CI fix
Audit pass over the public-facing surface so a customer can read it end-to-end without tripping on stale numbers or self-contradictions. README + docs accuracy: - Test counts 426 → 445, assertions 2 557 → 2 753 (verified via doctest run); E5 row was missing test_e5_kat (19 cases) - Interop checks 24 → 31, COMPLIANCE.md message count 149 → 164, COMPLIANCE.md "291 cases / 1515 assertions" → 445 / 2 753 - README "60+ test IDs" for MES_INTEROP → actual 59 - PVD example counts: 32 SVIDs/17 CEIDs → 29/21, "~40 handlers in ~200 lines" → 51 in ~460, "~700 lines" → ~1,100; main.cpp header table-of-contents resynced with the actual 7 sections Out-of-scope honesty (COMPLIANCE.md §8 + FAQ.md): - Removed HSMS-GS (was both ✅ implemented in §1 and "out of scope" in §8; INTEGRATION.md §7 documents using it) - Removed multi-block SECS-I (split_message/assemble_message exist with 4 dedicated tests) - Added serial-port wiring as the genuine open ⬜ item — FSM is tested end-to-end over TCP; only the asio serial_port glue is deferred - COMPLIANCE.md intro now lists E42 and notes "E37 (SS + GS)" README restructure: - Moved the 8-command proof table and per-standard test-coverage table to a new PROOFS.md (72 lines) - README now leads with what / Quick start / Documentation map, then a one-paragraph "How it's proved" linking to PROOFS.md - Updated cross-refs in FAQ.md, GLOSSARY.md, VERIFICATION.md, and interop/README.md to point at PROOFS.md CI fix — tshark-dissector job: - interop/tshark_validate.sh hardcoded /app/build/secs_server etc. which only works inside the docker image. Now derives ROOT from the script's own location and accepts BUILD/SERVER/CLIENT/DATA env overrides, so CI can run it from the workspace dir - Verified still passes in docker (69 frames, 0 malformed) .gitignore: - Added build-fuzz/ and build-tsan/ (were showing as untracked) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -13,67 +13,13 @@ E157**.
|
||||
|
||||
---
|
||||
|
||||
## Proof of feature-completeness
|
||||
|
||||
"Feature-complete" is a claim that the code must prove, not the
|
||||
README. These five commands are the proof. If they all exit zero on
|
||||
a fresh clone, the codebase implements what
|
||||
[COMPLIANCE.md](COMPLIANCE.md) claims.
|
||||
|
||||
| # | Command | What it proves |
|
||||
|---|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|
|
||||
| 1 | `docker compose run --rm tests` | **426 test cases / 2 557 assertions** pass: every store, FSM, codec, parser, persistence path |
|
||||
| 2 | `docker compose run --rm builder /app/build/secs_conformance --host server --port 5000` | **47 wire-level conformance checks** PASS against a live passive equipment |
|
||||
| 3 | `docker compose run --rm interop python3 /app/interop/host_vs_cpp_server.py --host server` | **24 interop checks** PASS against secsgem-py 0.3.0 (the Python reference impl) |
|
||||
| 4 | `SECSGEM_ROBUSTNESS_SOAK=1 docker compose run --rm builder /app/build/secsgem_tests -tc='*soak*'` | **100 000 random tool operations** execute with all invariants and persistence round-trips holding |
|
||||
| 5 | `docker compose run --rm builder /app/build/secs_server --validate-config --config /app/data/equipment.yaml --state-table /app/data/control_state.yaml --pj-state-table /app/data/process_job_state.yaml --cj-state-table /app/data/control_job_state.yaml` | Every shipped YAML config passes structural + referential validation |
|
||||
| 6 | `docker compose run --rm builder bash /app/interop/tshark_validate.sh` | **69 HSMS frames** dissected by Wireshark's HSMS dissector (independent third codec) with no malformed packets |
|
||||
| 7 | `bash interop/secs4j_validate.sh` | **55 cross-validation checks** PASS against [secs4java8](https://github.com/kenta-shimizu/secs4java8) (independent Java implementation), covering S1/S2/S3/S5/S6/S7/S10/S14/S16, the full E40 PJ body, dynamic event reports + unsolicited S6F11 / S5F1 observation, alarm management, spool, PP management, terminal services, limits, trace, E39, and the GEM 300 streams secsgem-py couldn't easily drive |
|
||||
| 8 | `cmake -B build-fuzz -DSECSGEM_FUZZ=ON && build-fuzz/fuzz_secs2_decode -max_total_time=60` | **~70 000 random inputs** through `secs2::decode`, **~285 000** through `try_parse_sml`, ASan + UBSan + libFuzzer coverage, **0 crashes** |
|
||||
|
||||
Plus, on every push to `main`, [Gitea Actions](.gitea/workflows/ci.yml)
|
||||
runs both a **Release build + full test suite** and a separate
|
||||
**ThreadSanitizer lane** that builds with `-fsanitize=thread` and
|
||||
fails on any race. All 426 cases / 2 557 assertions pass under TSan
|
||||
clean.
|
||||
|
||||
### Per-standard test coverage
|
||||
|
||||
Every claimed standard has dedicated tests. Counts are
|
||||
`grep -c TEST_CASE`; cross-cutting tests (e.g. `test_robustness_fuzz`,
|
||||
`test_gem300_scenario`) exercise multiple standards in concert.
|
||||
|
||||
| Standard | Test files | Cases |
|
||||
|-----------------------------------|-------------------------------------------------------------------------------------------|------:|
|
||||
| **E5** — SECS-II encoding | `test_secs2`, `test_sml`, `test_messages`, `test_identifier_wildcards`, `test_fuzz` | 120 |
|
||||
| **E5 §13** — exceptions | `test_exceptions`, `test_exception_persistence` | 16 |
|
||||
| **E4** — SECS-I transport | `test_secsi`, `test_secsi_timers`, `test_secsi_tcp` | 27 |
|
||||
| **E37** — HSMS (SS + GS) | `test_hsms`, `test_hsms_connection`, `test_hsms_timers`, `test_hsms_s9`, `test_hsms_gs`, `test_hsms_gs_integration`, `test_s9_fallback`, `test_concurrency` | 34 |
|
||||
| **E30** — GEM core | `test_control_state`, `test_communication_state`, `test_host_handler`, `test_data_model`, `test_loader`, `test_config_validate` | 71 |
|
||||
| **E40** — process jobs | `test_process_jobs` | 21 |
|
||||
| **E94** — control jobs | `test_control_jobs` | 9 |
|
||||
| **E42** — formatted PP | `test_e42_formatted_pp` | 6 |
|
||||
| **E87** — carriers + load ports | `test_carriers`, `test_carrier_state`, `test_carrier_persistence`, `test_e87_wire_scenarios` | 27 |
|
||||
| **E90** — substrate tracking | `test_substrates`, `test_substrate_persistence` | 21 |
|
||||
| **E116** — EPT | `test_ept` | 7 |
|
||||
| **E120 / E39** — common equip / object service | `test_cem_objects` | 3 |
|
||||
| **E157** — module process tracking | `test_modules` | 5 |
|
||||
| **E84** — parallel I/O + timers | `test_e84`, `test_e84_ports`, `test_e84_timers`, `test_e84_asio_timers` | 27 |
|
||||
| Persistence + cross-cutting | `test_job_persistence`, `test_persistence_upgrade`, `test_wire_ceid_emission`, `test_gem300_scenario`, `test_live_gem300`, `test_thread_safety`, `test_metrics_prometheus`, `test_robustness_fuzz` | 32 |
|
||||
| **Total** | | **426** |
|
||||
|
||||
A single command to see this live: `docker compose run --rm builder
|
||||
/app/build/secsgem_tests --list-test-cases | wc -l` (currently 426).
|
||||
|
||||
---
|
||||
|
||||
## Quick start
|
||||
|
||||
Everything runs in Docker — no compiler or build tools on the host.
|
||||
|
||||
```bash
|
||||
docker compose run --rm builder # configure + compile
|
||||
docker compose run --rm tests # 426 cases / 2 557 assertions
|
||||
docker compose run --rm tests # 445 cases / 2 753 assertions
|
||||
docker compose up --no-deps server client # live two-container demo
|
||||
```
|
||||
|
||||
@@ -89,9 +35,10 @@ through the data model. Watch the logs interleave.
|
||||
| [COMPLIANCE.md](COMPLIANCE.md) | Per-capability audit against every SEMI standard implemented |
|
||||
| [INTEGRATION.md](INTEGRATION.md) | Vendor-side tutorial: YAML → callbacks → production deploy |
|
||||
| [ARCHITECTURE.md](ARCHITECTURE.md) | How the pieces fit + how to extend (new store / FSM / message) |
|
||||
| [PROOFS.md](PROOFS.md) | The eight commands that prove the feature-completeness claim |
|
||||
| [VERIFICATION.md](VERIFICATION.md) | Test plan for the external validators behind the proof table |
|
||||
| [BENCHMARKS.md](BENCHMARKS.md) | Performance envelope (throughput, latency, memory) + how to re-run |
|
||||
| [MES_INTEROP.md](MES_INTEROP.md) | Day-1 punch list to run against your commercial MES (60+ test IDs) |
|
||||
| [MES_INTEROP.md](MES_INTEROP.md) | Day-1 punch list to run against your commercial MES (59 test IDs) |
|
||||
| [SECURITY.md](SECURITY.md) | Concrete configs: nftables, stunnel, minisign, SIEM audit-log schema |
|
||||
| [GLOSSARY.md](GLOSSARY.md) | SEMI vocabulary: SVID, CEID, PPID, ALCD, HCACK, T-timers, … |
|
||||
| [FAQ.md](FAQ.md) | Common questions and their canonical answers |
|
||||
@@ -100,6 +47,24 @@ through the data model. Watch the logs interleave.
|
||||
|
||||
---
|
||||
|
||||
## How it's proved
|
||||
|
||||
"Feature-complete" is a claim that the code must prove, not the
|
||||
README. On a fresh clone, **eight commands** demonstrate it:
|
||||
unit + integration suite (445 cases / 2 753 assertions), a live
|
||||
conformance harness (47 wire-level checks), interop against
|
||||
secsgem-py (31 checks) and secs4java8 (55 checks), a 100 000-op
|
||||
soak property test, YAML config validation, Wireshark's HSMS
|
||||
dissector on a recorded pcap (69 frames, 0 malformed), and libFuzzer
|
||||
(ASan + UBSan, 0 crashes). Each command, exit code, and the
|
||||
external-vs-internal breakdown live in [PROOFS.md](PROOFS.md).
|
||||
|
||||
CI runs the full suite plus a separate ThreadSanitizer lane on
|
||||
every push to `main` ([Gitea Actions](.gitea/workflows/ci.yml));
|
||||
all 445 cases pass clean under `-fsanitize=thread`.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
The project is **spec-as-data**: the SEMI behavioural rules live in
|
||||
@@ -142,6 +107,9 @@ YAML; the C++ is the engine that reads them.
|
||||
secsgem::metrics Prometheus exporter (Registry + HTTP server).
|
||||
```
|
||||
|
||||
See [ARCHITECTURE.md](ARCHITECTURE.md) for how to extend it (new
|
||||
store / FSM / message).
|
||||
|
||||
---
|
||||
|
||||
## Adding a capability
|
||||
|
||||
Reference in New Issue
Block a user