secs-gem/docs/30_repository_tour.md

30 — Repository tour

← 19 E42 + E148 + S9 — Misc | Back to index | Next: 31 Spec-as-data + codegen →

You've seen what every SEMI standard does. Now we shift to how this codebase is laid out. This chapter answers: when you git clone this repo, what are you looking at?

The repo is small — about 15 k lines of C++ + tests + tooling. It fits in your head with a little patience. By the end of this chapter you'll know:

• What each top-level directory contains.
• Which binaries get built.
• The dependency graph between modules.
• How the build system finds and links them.

---

Top-level layout

secs-gem/
├── README.md                One-page project summary.
├── LICENSE                  Proprietary terms.
├── CMakeLists.txt           Build config (CMake 3.16+, single file).
├── Dockerfile               Ubuntu 24.04 + g++-13 + libasio + yaml-cpp.
├── docker-compose.yml       Multi-container demo wiring.
├── .gitea/workflows/ci.yml  CI pipeline.
│
├── include/secsgem/         Public headers.  All API here.
│   ├── secs2/               E5 codec + SML.
│   ├── hsms/                E37 transport (TCP + framing).
│   ├── secsi/               E4 transport (FSM + TCP tunnel).
│   ├── config/              YAML loader + multi-error validator.
│   ├── metrics/             Prometheus exporter.
│   ├── endpoint.hpp         asio::ip::tcp::endpoint factory.
│   └── gem/                 E30 + every GEM 300 standard.
│       ├── store/           Per-domain bundles (SVIDs, alarms, …).
│       └── *.hpp            State machines + composers.
│
├── src/                     Implementations.  Mirrors include/.
│   ├── secs2/{codec,sml}.cpp
│   ├── hsms/{header,connection}.cpp
│   ├── secsi/{header,block,protocol,tcp_transport}.cpp
│   ├── config/...
│   ├── gem/...
│   └── endpoint.cpp
│
├── apps/                    Standalone binaries.
│   ├── secs_server.cpp      Passive equipment (demo + integration target).
│   ├── secs_client.cpp      Active host driving the demo flow.
│   ├── secs_conformance.cpp 47-check wire-level conformance harness.
│   ├── secs_interop_probe.cpp Probe against secsgem-py passive equip.
│   ├── secs_bench.cpp       Throughput / latency / memory bench.
│   ├── fuzz_secs2_decode.cpp libFuzzer harness for secs2::decode.
│   └── fuzz_sml_parse.cpp    libFuzzer harness for try_parse_sml.
│
├── tests/                   doctest unit + integration tests.
│   └── test_*.cpp           55 files, 473 cases, 3087 assertions.
│
├── data/                    YAML configs (the spec-as-data).
│   ├── messages.yaml        SECS-II message catalog (164 msgs).
│   ├── control_state.yaml   E30 §6.2 transition table.
│   ├── process_job_state.yaml  E40 transition table.
│   ├── control_job_state.yaml  E94 transition table.
│   └── equipment.yaml       Demo SVIDs/ECIDs/CEIDs/alarms/recipes.
│
├── tools/                   Build-time scripts.
│   └── gen_messages.py      Codegen: messages.yaml → messages.hpp.
│
├── interop/                 External-validator harnesses.
│   ├── README.md            Harness-by-harness detail.
│   ├── host_vs_cpp_server.py    secsgem-py active host driving us.
│   ├── passive_equipment.py     secsgem-py passive equipment for us to drive.
│   ├── raw_gem300_harness.py    Raw S3/S14/S16/S12 round-trip.
│   ├── tshark_validate.sh       pcap + tshark HSMS dissector check.
│   ├── secs4j_validate.sh       secs4java8 (Java) cross-validation.
│   └── secs4j/                  Dockerfile + harness for secs4java8.
│
├── examples/
│   └── pvd_tool/            Worked vendor example: fictional PVD tool.
│       ├── README.md        What the example shows.
│       ├── equipment.yaml   Realistic SVIDs/ECIDs/CEIDs/alarms/recipes.
│       └── main.cpp         Sensor sim, recipe runner, alarm monitor.
│
└── docs/                    This guide + reference docs.
    ├── 00_index.md          The series TOC.
    ├── 01–51_*.md           Tutorial chapters.
    ├── ARCHITECTURE.md      One-page architecture overview.
    ├── COMPLIANCE.md        Per-capability audit.
    ├── INTEGRATION.md       Vendor-side production deploy.
    ├── PROOFS.md            8 commands proving feature-completeness.
    ├── VERIFICATION.md      External-validator test plan.
    ├── BENCHMARKS.md        Performance envelope.
    ├── MES_INTEROP.md       Commercial-MES day-1 punch list.
    ├── SECURITY.md          nftables / stunnel / minisign configs.
    ├── GLOSSARY.md          SEMI vocabulary cheat sheet.
    └── FAQ.md               Canonical answers.

---

The dependency graph

                            data/*.yaml
                                │
                  ┌─────────────┼──────────────────┐
                  │  (codegen)  │   (runtime load) │
                  ▼             ▼                  ▼
       generated/messages.hpp                  config::loader
                  │                                │
                  └──────────► gem::EquipmentDataModel
                                       │
                                       │ used by
                                       ▼
                                  gem::Router
                                       │
                                       │ wraps
                                       ▼
                               secs2::Message  ◄─── codec / SML
                                       │
                                       │ over
                                       ▼
                           hsms::Connection  /  secsi::TcpTransport
                                       │
                                       ▼
                                   TCP socket

Read it bottom-up: a TCP socket carries bytes; hsms::Connection frames them into secs2::Messages; gem::Router dispatches by (stream, function) to handlers; handlers read/write EquipmentDataModel; the model composes per-domain stores; the stores were built from the YAML at startup.

No layer ever calls up the graph. secs2::Item has no idea HSMS exists. hsms::Connection doesn't know about CEIDs. gem::Router doesn't know whether the bytes came over HSMS or SECS-I. Strict layering is what keeps the codebase small.

---

The binaries

Built by CMakeLists.txt (one file, ~250 lines). Each binary lives in build/ after cmake --build.

Binary	Source	What it does
secs_server	apps/secs_server.cpp	Passive equipment. Listens on TCP, dispatches via Router.
secs_client	apps/secs_client.cpp	Active host. Drives ~24 transactions in the demo.
secs_conformance	apps/secs_conformance.cpp	47 wire-level conformance checks against a live server.
secs_interop_probe	apps/secs_interop_probe.cpp	Active host probing a secsgem-py passive equipment.
secs_bench	apps/secs_bench.cpp	Throughput / latency / memory harness.
secsgem_tests	All tests/*.cpp	The 473-case doctest binary.
secs_gemd	apps/secs_gemd.cpp + proto/secsgem/v1	The gRPC daemon: HSMS equipment + name-based tool API.
secs_gemd_tests	tests/test_daemon_service.cpp	In-process gRPC service tests (built when grpc++ is).
fuzz_secs2_decode	apps/fuzz_secs2_decode.cpp	libFuzzer (clang only, opt-in -DSECSGEM_FUZZ=ON).
fuzz_sml_parse	apps/fuzz_sml_parse.cpp	libFuzzer for the SML parser.

A worked example binary pvd_tool (from examples/pvd_tool/) is also built by the same CMakeLists.txt when the example is included.

---

How the build system finds everything

CMakeLists.txt does five things in order:

1. Pull in dependencies — find_package(Threads), find_package(yaml-cpp), FetchContent for doctest. Standalone Asio is header-only (no link step).
2. Run codegen — invokes tools/gen_messages.py to turn data/messages.yaml into build/generated/secsgem/gem/messages.hpp. Listed as a custom command so it re-runs when messages.yaml changes.
3. Build the library — add_library(secsgem ...) with every source under src/ plus the generated header.
4. Build the apps — one add_executable per apps/*.cpp, each linking against secsgem.
5. Build the tests — add_executable(secsgem_tests ...) with every tests/*.cpp, linked against doctest + secsgem.

Build flags:

• -DSECSGEM_TSAN=ON — adds -fsanitize=thread to a separate build dir. CI runs this lane.
• -DSECSGEM_FUZZ=ON — requires clang; adds libFuzzer + ASan + UBSan; builds the two fuzz harnesses.

Everything else (Release / Debug, parallelism, output dirs) is standard CMake.

---

Test layout

50 test files; 473 test cases; 3 087 assertions. One file per concern. Naming is test_<thing>.cpp consistently:

• test_secs2.cpp, test_e5_kat.cpp, test_sml.cpp, test_messages.cpp — codec.
• test_hsms*.cpp (5 files), test_secsi*.cpp (3 files) — transport.
• test_control_state.cpp, test_communication_state.cpp, test_data_model.cpp, test_host_handler.cpp, test_loader.cpp, test_config_validate.cpp — E30.
• test_process_jobs.cpp, test_control_jobs.cpp, test_carriers.cpp, test_substrates.cpp, test_ept.cpp, test_modules.cpp, test_cem_objects.cpp, test_e84*.cpp, test_e42_formatted_pp.cpp — GEM 300.
• test_*_persistence.cpp (4) — file-backed journal.
• test_robustness_fuzz.cpp — randomized property test.
• test_thread_safety.cpp — TSan-validated single-threaded contract.
• test_metrics_prometheus.cpp — Prometheus exporter.
• test_wire_ceid_emission.cpp — CEID firings observed on a real socket.
• test_live_gem300.cpp, test_gem300_scenario.cpp — multi-FSM cascades.

Full per-standard breakdown: docs/PROOFS.md "Per-standard test coverage" table.

---

Where to go next

Now that you know what's where, the next chapter explains the philosophy that makes the codebase this small: the spec-as-data principle, and how the YAML files + codegen + runtime loader work together so adding a new SVID / state / message rarely requires C++.

Next: → 31 Spec-as-data + codegen