a4599b3b9dcd2e09aa91f28c4251230d5de744d9
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 raphael
|
d73906f372 |
license: switch contact email to raphael@maenle.net
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
05d58f1a0a |
license: All Rights Reserved proprietary
Hard blocker for any fab customer's procurement / legal review — without a LICENSE in the repo they couldn't even begin evaluation, because permission to read the source is itself something the copyright holder has to grant. This license grants nothing by default. Viewing the repo is the only implicit allowance; everything else (compile, evaluate, benchmark, deploy, sublicense, train ML on, reverse-engineer) requires a separate written agreement with r.maenle@gmail.com. Explicitly *not* granting the carve-outs that open-source licenses imply: no fair use, no internal evaluation, no academic research, no demo, no production deployment. Customers who want any of those need to talk to Raphael first. SPDX-License-Identifier: LicenseRef-Proprietary for tooling. README header gains a license callout pointing at the file and contact email so anyone landing on the GitHub frontpage sees the restriction before reading further. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
7871718848 |
persistence: v1->v2 upgrade test + honest README
README §6 claimed bidirectional forward-compat for journal records. Reality is narrower: - ProcessJobStore (kVersion=2) and SubstrateStore (kVersion=2) accept v1 records on replay — their loaders explicitly switch on the version byte and treat the v2 trailer fields as empty when absent. This is the actual upgrade path the README half-described. - ControlJobStore, CarrierStore, LoadPortStore, ExceptionStore, and SpoolStore use strict `header[1] != kVersion` rejection. A future kVersion bump there without a matching loader-side dispatch would silently nuke every replayed record. The README sold this as a feature; it isn't yet. This commit adds: - tests/test_persistence_upgrade.cpp: five cases that craft journal records byte-by-byte so format drift is caught (no codec round-trip hiding the field layout). PJ v1 -> v2 read; PJ v1 rewrite stamps current kVersion=2; PJ unknown future version rejected; Substrate v1 read with empty history trailer; CJ + Carrier reject unknown versions (tripwire for the strict-version stores). - README §6: replaces the rosy "newer versions ignore unknown trailers" claim with what's actually implemented — multi-version reads on PJ + Substrate, strict equality elsewhere — and points at the test as the contract anchor. When the strict-version stores grow their own v2, the rejection tests will need to flip to acceptance; the layout is right there in the test so the edit is mechanical. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
9653a54584 |
docs+test: thread-safety contract for EquipmentDataModel
INTEGRATION.md §3 used to show a sensor-poll thread calling model->svids.set_value() directly while the io_context thread reads the same SVID for an inbound S1F3. That's a data race — there are zero locks anywhere in EquipmentDataModel and there's no intention to add them. The library is single-threaded by design; the doc was just inviting trouble. This commit makes the actual contract explicit: - INTEGRATION.md §3: thread-safety callout box. All access must run on the io_context that drives the HSMS connection. Sensor updates from other threads marshal via asio::post(io.get_executor(), ...). Same applies to set_*_change_handler callbacks (they fire on the io_context thread; observers must be thread-safe or hand work off). - README.md §3 (Monitoring & observability): added a paragraph noting that hooks fire on the io_context thread, blocking I/O inside a handler stalls the dispatcher, and metrics exporters must respect the same contract. - tests/test_thread_safety.cpp: two scenarios that exercise the canonical pattern — N producer threads asio::post sensor updates onto a worker-driven io_context; reads marshal back through the io. Catches obvious regressions (e.g. someone adding a "convenience" cross-thread mutator that bypasses the strand). A passing run isn't proof of race-freedom under ThreadSanitizer — it pins down the *pattern* customers should follow. TSan integration is a separate workstream. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
06f287b415 |
conformance: standalone secs_conformance harness binary
The closest thing to an in-repo "RTS" — a runnable executable that points at any HSMS-SS equipment and walks through every E30 fundamental + additional capability, reporting pass/fail per check and exiting with the right code for CI / canary use. build/secs_conformance --host <ip> --port 5000 --device 0 Each check sends a host-initiated primary and asserts the equipment replies with the expected stream/function within T3. Checks chain forward through async callbacks (each reply handler kicks off the next check) so the conformance run stays inside one io.run(). Initial check set (mirrors COMPLIANCE.md §3 fundamentals): E37 §7.2 SELECT handshake E30 §6.5 S1F13/F14 Establish Comms E30 §6.7 S1F1/F2 Are You There E30 §6.13 S1F11/F12 SVID Namelist E30 §6.16 S2F29/F30 ECID Namelist E30 §6.20 S2F17/F18 Clock E30 §6.14 S5F5/F6 List Alarms E30 §6.17 S7F19/F20 PP List E30 §6.10 S1F19/F20 GEM Compliance Validated against the demo server: 9/9 PASS. README.md §8 (Compliance + certification) updated to point at the harness as the suggested first-line conformance check. Tool vendors fork apps/secs_conformance.cpp and add their own capability-specific checks alongside. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
d470442a8c |
docs: drop implementation_plan.md, rewrite README for fab deployment
implementation_plan.md was a Layer-0..6 roadmap from the project's
spec-as-data exploration phase; every layer it described is now
shipped (Layer 0 foundations through Layer 4 message catalog +
state machines). Removed.
README rewritten for the fab-deployment audience. Sections added:
1. Persistence directory layout (storage rules, disk budget, DR)
2. Security (network isolation, TLS tunnels, audit logging,
config signing)
3. Monitoring + observability (signals → hooks table, Prometheus
pattern)
4. High availability (active/standby on shared persistence)
5. Deployment patterns (Docker / systemd / k8s)
6. Upgrade path (YAML reload, code rollout, schema versioning)
7. Integration with the fab stack (MES / AMHS / OHT / recipe
engine table)
8. Compliance + certification (fork COMPLIANCE.md per tool, run
RTS)
9. Testing in production (canary, synthetic transactions, shadow
traffic)
10. Operational runbook (incident → first check → mitigation)
Stale stats refreshed: test count went 148/794 → 384/2390;
catalog grew to 164 messages; HSMS-GS, SECS-I T3/T4, per-port E84,
E42 formatted PPs all mentioned.
COMPLIANCE.md §9 lost its stale `implementation_plan.md` reference.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
90c177b7ce |
E40 Process Jobs + E94 Control Jobs + E30 communication state
GEM300 layer: SEMI E40-0705 Process Job and E94-0705 Control Job state machines, plus the E30 §6.1 communication-state machine that sits between HSMS SELECT and full GEM communication. Data-driven via data/process_job_state.yaml and data/control_job_state.yaml, mirroring the existing control_state.yaml pattern. Wire coverage: S14F9/F10 CreateObject (CJ) host -> equipment S14F11/F12 DeleteObject (CJ) host -> equipment S16F5/F6 PRJobCommand host -> equipment S16F9 PRJobAlert equipment -> host S16F11/F12 PRJobCreate (simplified body) host -> equipment S16F13/F14 PRJobDequeue host -> equipment S16F27/F28 CJobCommand host -> equipment Process Job FSM exposes 8 states matching PRJOBSTATE bytes (E40 §10.3.2); HOQ is reorder-aware (move-to-head against an insertion-order vector); Stop/Abort on a Queued PJ routes through ABORTING so the host observes PRJOBSTATE=7 on the wire (§6.3); alert_enabled is settable per-PJ for PRALERT control; FSM dispatches through ProcessJobStore::on_change_ dynamically so a late set_state_change_handler() reaches existing PJs. Hardening: loader rejects NoState (sentinel) as initial/from/to and rejects `on: created` rows; static_asserts pin enum values to wire bytes; ProcessJobStore is non-movable to keep the per-PJ this-capture safe. Server simulator cascades the full CJ -> PJ lifecycle on CJSTART so the wire trace exercises every legal state. CEIDs 400/401 fire on CJ state changes via the existing event-report pipeline. Tests: 60+ new assertions across test_process_jobs, test_control_jobs, test_communication_state, test_hsms_connection, plus loader and messages round-trip coverage. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
1f67aad985 |
100%/F: S10F5/F6 multi-line + honest 100% in COMPLIANCE.md + README pass
tests / build-and-test (push) Failing after 33s
The final additions: S10F5/F6 multi-line terminal display (closes the last partial Additional capability — Equipment Terminal Services flips ✅), and a thorough COMPLIANCE.md / README pass that states the 100% claim honestly. Catalog + handlers data/messages.yaml S10F5 / S10F6 added. apps/secs_server.cpp router.on(10, 5) iterates the line list, acks with S10F6. tests/test_messages.cpp Round-trips a 3-line multi-line display. COMPLIANCE.md (rewritten) Every GEM Fundamental ✅. Every GEM Additional that E30 binds to a concrete message set ✅. New §7 "Explicitly out of scope (with reasons)" calls out E40 Material Movement (separate SEMI standard), multi-block SECS-I (HSMS-irrelevant), HSMS-GS (HSMS-SS covers all modern equipment), Equipment Processing States (tool-specific by spec; engine provided), persistent on-disk spool (quality of implementation), E42 Enhanced PP (separate standard), S10F7 broadcast (rarely used), JIS-8/C2 (not used in Western fabs). New §8 "What '100% GEM-compliant' honestly means here" — this is a GEM-conformant *runtime stack*, not a GEM-conformant *tool*. Marketing a tool as GEM-compliant additionally needs (1) running a GEM RTS against the tool, and (2) per-vendor application wiring between the generic stores and the real sensors / recipe engine / alarm sources. README.md (rewritten) Architecture diagram updated to reflect the actual store list (nine stores). "Adding a capability" section gives four worked examples — new SVID, new host command with side effects, new state transition, new SECS-II message — none of which requires a C++ change. Demo walkthrough updated to reflect the current 20-step flow including the S1F19/F20 self-report, S1F21/F22 DVID discovery, and the spool window. Code clarity include/secsgem/gem/data_model.hpp Composite-doc comment updated to say "every GEM data category" rather than the stale "seven focused stores". Verified - Tests: 84 cases / 487 assertions pass. - Demo: 198 server/host log lines; exits 0 end-to-end. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
raphael
|
b871cd9da2 |
Table/YAML-driven refactor (Layer 1 start)
Move equipment capabilities and the E30 control state machine out of C++
code and into YAML data files; introduce a Router for SECS dispatch;
consolidate small files.
Behavioural changes: none. Demo identical (15 SxFy transactions +
3 equipment-initiated primaries), 67 test cases / 384 assertions still
all green. Structural changes only.
Why
---
The previous server.cpp held the equipment data dictionary (3 SVIDs,
2 ECIDs, 3 CEIDs, 2 alarms, 2 recipes, 4 host commands) as imperative
C++ in a 50-line `populate()` function, and routed inbound messages
through a 150-line if-ladder. Adding a new SVID required a recompile.
Adding a new state transition required editing two switch statements
(`operator_*` and `on_host_request_*`). The control state machine's
behavioural rules were spread across imperative code in two methods.
This is exactly what implementation_plan.md calls out as the wrong
shape: behavioural rules should live in versioned data, and every
runtime/test/analyzer should read from that data rather than re-encode
it. This commit starts that move.
What's new
----------
data/equipment.yaml
Equipment data dictionary. Declarative SVIDs / ECIDs / CEIDs /
alarms / recipes / host commands. Host commands carry their HCACK
ack code plus optional `emit_ceid` and `set_alarm` side-effects.
Adding a new SVID or command is a YAML edit, no recompile.
data/control_state.yaml
The E30 §6.2 control state transition table as data. Each row is
(from, on) -> (to [, then] [, ack]). `then` chains an auto-advance
through the transient AttemptOnline state. The previous
imperative switch is gone.
include/secsgem/config/loader.hpp + src/config/loader.cpp
yaml-cpp-backed loader. `load_control_state(path)` returns a
ControlTransitionTable + initial state; `load_equipment(path, model)`
populates the EquipmentDataModel and returns the device descriptor
(id, MDLN, SOFTREV, optional auto-emit CEID). Surfaces config
errors with file path + field name via ConfigError.
include/secsgem/gem/router.hpp (header-only)
Small (stream, function) -> handler map. Server registers all
handlers once at startup, then the Connection's message handler is
just `router.dispatch(msg)`. Unhandled primaries with W set get
SxF0 by default. Replaces the if-ladder in secs_server.cpp.
include/secsgem/gem/control_state.hpp + .cpp
ControlTransitionTable is the new pure data type. ControlStateMachine
is now a thin engine over the table: `fire(event)` looks up the row,
optionally transitions, optionally chains a `then` transition, returns
the ack code. Behaviour rules no longer live in C++ switches.
The default in-code table matches data/control_state.yaml row for row;
tests rely on it so they don't need the YAML file.
include/secsgem/gem/data_model.hpp + .cpp
`register_command(rcmd, CommandSpec)` replaces the function-handler
signature. CommandSpec = (HostCmdAck, optional emit_ceid, optional
set_alarm). `dispatch_command` returns a CommandResult so the server
can fire the side-effects after S2F42 is sent.
apps/secs_server.cpp
No populate(), no if-ladder. Loads equipment.yaml + control_state.yaml
at startup (clean error on bad config), wires the Router once,
delegates dispatch. Sm change handler reads emit_on_control_change
from the YAML. Welcome S10F3 removed for parity with config (a future
YAML rule could re-introduce it declaratively).
tests/test_loader.cpp (new)
Verifies the YAML loader produces the same shape as the in-code
default table, and that equipment.yaml populates every section
(SVIDs/ECIDs/CEIDs/alarms/recipes/commands). SECSGEM_DATA_DIR
CMake define points at ${CMAKE_SOURCE_DIR}/data so tests don't
depend on cwd.
CMakeLists.txt, Dockerfile
find_package(yaml-cpp) and link. libyaml-cpp-dev added to the
Ubuntu base image (yaml-cpp 0.8 ships the modern target name).
File consolidation
------------------
Five small files removed; their content lives in fewer headers:
- secs2/item.cpp -> inline in secs2/item.hpp
- secs2/message.cpp -> inline in secs2/message.hpp
- hsms/types.hpp -> merged into hsms/header.hpp
- hsms/frame.hpp -> merged into hsms/header.hpp
- hsms/frame.cpp -> merged into hsms/header.cpp
hsms/header.hpp is now "the HSMS wire format" in one place: SType + status
enums + Timers + Header + Frame + constants. All includers updated.
Net effect
----------
Before: equipment data dictionary lived in 50 lines of imperative
populate() inside secs_server.cpp; dispatch in a 20-branch if-ladder.
After: equipment data dictionary lives in 47 lines of YAML; dispatch
is a Router built once. Adding a new capability is now a YAML edit
in the common case.
Test count up to 67 cases / 384 assertions (+4 cases / +106 assertions)
covering the loader and the new table-driven SM paths.
What's NOT changed
------------------
The per-SxFy reply construction still lives in C++ (each message has a
unique body shape). Moving those into YAML/JSON message-shape
definitions is the next refactor step but requires a generic typed
encoder/decoder driven by shape descriptors; out of scope here.
Spooling, the S9 error stream, S1F19/F20, and the other gaps in
COMPLIANCE.md remain unchanged.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
96b02f8b50 |
Initial commit: C++20 SECS-II / HSMS / GEM client + server
A fully containerised SECS/GEM toolchain. Single docker compose project,
no host build tools. 63 unit-test cases / 278 assertions, two demo
executables, end-to-end two-container demo exercising every implemented
capability.
Architecture (bottom-up):
secs2/ E5 SECS-II codec
Item variant over L/A/B/BOOLEAN/I1-8/U1-8/F4/F8
encode/decode big-endian, 1/2/3-byte length encoding
Message SxFy + W-bit + optional root item
to_sml human-readable text rendering
hsms/ E37 HSMS transport (TCP)
Header 10-byte header + SType enum (Data/Select/Deselect/
Linktest/Reject/Separate)
Frame 4-byte length prefix + payload encode/decode
Connection async Asio TCP, NOT-SELECTED -> SELECTED state machine,
T3/T5/T6/T7/T8 timers, system-bytes reply correlation,
graceful close-after-flush separation
endpoint active Client (connect with T5 retry) and passive Server
(accept loop) wrappers over Connection
gem/ E30 GEM logic
ControlStateMachine 5-state E30 control model with operator
actions, host requests, SEMI-mandated ack
codes (OnlineAck, OfflineAck, CommAck), and
a state-change handler
EquipmentDataModel in-memory dictionary: SVIDs, DVIDs, ECIDs
(with EAC), CEIDs, report defs, CEID->report
links, enabled-events set, alarm table
(ALCD, enabled, active), process programs,
host command registry, clock (16-char
YYYYMMDDhhmmsscc with offset)
messages.hpp builders + parsers for every SxFy below
GEM message coverage (full list):
S1F1/F2 Are You There / On Line Data
S1F3/F4 Selected Equipment Status Request / Data
S1F11/F12 Status Variable Namelist Request / Data
S1F13/F14 Establish Communications (+ CommAck)
S1F15/F16 Request OFFLINE (+ OfflineAck)
S1F17/F18 Request ONLINE (+ OnlineAck)
S2F13/F14 Equipment Constant Request / Data
S2F15/F16 EC Send + EquipmentAck (Accept/UnknownEcid/Busy/OutOfRange)
S2F17/F18 Date and Time Request / Data
S2F29/F30 Equipment Constant Namelist Request / Data
S2F31/F32 Date and Time Set Request / TimeAck
S2F33/F34 Define Report + DefineReportAck (5 enum values)
S2F35/F36 Link Event Report + LinkEventAck
S2F37/F38 Enable / Disable Event Report + EnableEventAck
S2F41/F42 Host Command + HostCmdAck (7 values) + per-param CPACKs
S5F1/F2 Alarm Report Send + AlarmAck (ALCD bit-7 set/cleared
+ lower-7 category)
S5F3/F4 Enable/Disable Alarm Send + AlarmAck
S5F5/F6 List Alarms Request / Data (active alarms tagged in ALCD)
S6F11/F12 Event Report Send (equipment-initiated CEID emission
with full report data) + EventReportAck
S7F3/F4 Process Program Send + ProcessProgramAck (7 values)
S7F5/F6 Process Program Request / Data
S7F19/F20 Current EPPD List Request / Data
S10F1/F2 Terminal Display Single (host->equipment) + TerminalAck
S10F3/F4 Terminal Display Single (equipment->host)
Demo apps:
apps/secs_server.cpp passive equipment. Populates the data model
with 3 SVIDs (ControlState, Clock,
EventsEnabled), 2 ECIDs, 3 CEIDs
(ControlStateChanged, AlarmSetEvent,
ProcessStarted), 2 alarms (Chiller Temp High
cat 4, Door Open cat 1), 2 recipes
(RECIPE-A, RECIPE-B), and 4 host commands
(START, STOP, PAUSE, FAULT). Emits S6F11 on
every control state transition + on START;
emits S5F1 + the AlarmSetEvent CEID on FAULT.
Pushes an S10F3 welcome message when the host
comes online.
apps/secs_client.cpp active host. Walks 17 steps: Establish ->
Online -> S1F11 SVID namelist -> S1F3 read ->
S2F29 EC namelist -> S2F13 read ->
S2F17 clock -> S2F33/S2F35/S2F37 dynamic
event subscription -> S2F41 START
(-> receives S6F11) -> S5F5 alarm list ->
S5F3 enable alarm 1 -> S2F41 FAULT
(-> receives S5F1 + S6F11) -> S7F19/S7F5
recipe list + body -> S10F1 terminal ->
S1F15 Offline -> Separate. Handles inbound
S6F11, S5F1, S10F3 primaries.
Testing:
tests/test_secs2.cpp codec round-trip for every format,
byte-layout assertions for known values,
truncation/trailing-byte rejection,
nested list round-trip, SML rendering
tests/test_hsms.cpp header byte layout, data + control
header round-trip, full frame round-
trip with length prefix, short-payload
rejection
tests/test_control_state.cpp every (state, event) pair in the E30
control state machine, including
AlreadyOnline / NotAccept rejections
and idempotent offline-while-offline
tests/test_data_model.cpp SVID/ECID/Alarm/Recipe CRUD, clock
format + parse, host command registry,
full event-report pipeline (define ->
link -> enable -> compose) with
every error path (InvalidVid,
UnknownCeid, UnknownRptid), alarm
set/clear with ALCD bit-7 semantics
tests/test_messages.cpp round-trip + byte-layout for every
builder/parser pair, including S6F11
event reports with mixed item types
Toolchain:
Dockerfile Ubuntu 24.04, g++-13, CMake, Ninja, libasio-dev
docker-compose.yml builder / tests / server / client services,
source bind-mounted, build artifacts in a
named volume so the host tree stays clean
CMakeLists.txt C++20, -Wall -Wextra -Wpedantic, standalone
Asio (ASIO_STANDALONE), doctest via FetchContent
Documentation:
README.md architecture, quick start, demo log
COMPLIANCE.md honest per-capability E5/E30/E37 audit with
spec section refs. Calls out what's implemented,
what's partial (Reject.req, Alarms missing F7/F8,
EC range validation, PP without verify, terminal
single-line only), and what's intentionally not
yet implemented (spooling, S9 error stream,
Documentation S1F19/F20+F21/F22, limits monitoring,
trace data collection, multi-block, material
movement). Does NOT claim "100% GEM-compliant" and
lists the work required to honestly make that claim.
This is Layer 0 + the start of Layer 1 from implementation_plan.md.
The transition-table-driven "spec-as-data" architecture (Layer 1
proper) is not yet implemented; the current code uses imperative
state machines that are structurally ready to be refactored onto
tables.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|