8686654b15e85ddd41d74c0db660ae10ab672431
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 raphael
|
8686654b15 |
feat(client): the Python client — a GEM tool in plain Python (Phase C)
clients/python: pip-installable "secsgem-client", pure Python (stubs
pre-generated from equipment.proto, import made package-relative; no
compiled extension, no SEMI knowledge, no C++ toolchain). The API the whole
effort aimed at:
eq = Equipment("localhost:50051")
eq.set(ChamberPressure=2.5); eq["WaferCounter"] = 7
eq.fire("ProcessStarted", ChamberPressure=2.75)
eq.alarm("chiller_temp_high"); eq.clear("chiller_temp_high")
@eq.on("START")
def start(cmd): ... # auto-CompleteCommand after return
eq.listen(background=True)
eq.control_state; eq.request_control_state("HOST_OFFLINE"); eq.health()
Errors raise SecsGemError carrying the daemon's message ("no variable named
..."). bool checked before int in conversion (isinstance(True, int)).
examples/mini_tool.py is a complete GEM tool in ~25 lines.
PROOF — interop/pyclient_interop.py drives the PUBLISHED package (not raw
stubs) against a live secs_gemd with secsgem-py as the fab host: 13 checks
all green on first run — set/get round-trips, item syntax, SecsGemError on
unknown names, control state, health, fire->S6F11 on the host's wire,
alarm/clear->S5F1 with correct set bit, the full command loop (host S2F41 ->
HCACK=4 -> @eq.on handler -> completion event back at the host), operator
offline. Conversion layer unit-tested standalone; both wired into
tools/run_interop.sh as the pyclient step.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
raphael
|
cf230d4119 |
chore(phase0): name validation, golden frames, daemon into library tree, TSan daemon lane
Item 8a — ConfigValidator warns on non-identifier variable/event/alarm/ command names ([A-Za-z_][A-Za-z0-9_]*): language bindings expose names as kwargs/attributes, so 'Chamber Pressure' would be unusable in the planned Python client. Warning not error — the wire doesn't care. Tested (4 warning sites + good-name negative). Item 4 tail — golden frames for S5F1 (Binary ALCD / U4 ALID / ASCII ALTX) and a composed S6F11 (the production-critical report shape), bytes hand- computed from E5 encoding rules: external pins on message composition. Item 7 — equipment_service.hpp moved to include/secsgem/daemon/ (apps/ include-path hack removed) and a TSan daemon lane added locally + in CI. tools/tsan.supp suppresses races whose accesses sit entirely inside the UNinstrumented system libgrpc/libgpr/libabsl (epoll wakeups, absl Mutex GraphCycles bookkeeping); our frames stay fully checked. The lane earned its keep on first run: it caught a REAL threading-contract violation — a daemon test reading model stores from the test thread while the io thread serviced posted writes — fixed to use read_sync, exactly per the documented contract. Now TSan-clean under halt_on_error=1 in the full production threading shape. Suites: core 470/3068, daemon Release+TSan 125/125 each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
raphael
|
42044e92e2 |
ci(interop): one-command external-validation suite + CI lanes for the daemon
tests / build-and-test (push) Successful in 2m42s
tests / thread-sanitizer (push) Successful in 2m50s
tests / tshark-dissector (push) Successful in 2m24s
tests / secs4j-interop (push) Successful in 37s
tests / python-interop (push) Successful in 2m56s
tests / libfuzzer (push) Successful in 3m44s
tools/run_interop.sh runs ALL nine validation steps with a PASS/FAIL summary: build, unit (464), daemon-unit (41), secsgem-py host vs server (31 checks), secs_conformance (47), gRPC+secsgem-py daemon bridge, spool persistence across restart, tshark HSMS dissector, secs4java8 (55 checks). Verified green end-to-end. The unit suite is partly self-referential (our parsers validate our builders); these external validators are the real oracle — now they run with one command instead of by hand. Two bugs found by running it: unbounded ninja at -O3 OOM-kills cc1plus in memory-constrained Docker VMs (build with -j 2) and bash-3.2 lacks negative array subscripts. CI: grpc deps added to the build job so secs_gemd + secs_gemd_tests build and RUN in CI (previously the daemon silently dropped out — now fails loudly if missing), plus a python-interop lane running py-host/conformance/daemon harnesses against localhost in one container (no docker-in-docker). Service hardening while in there: reject proto Values with no kind set at the RPC edge (previously silently became ASCII ""), TODO markers for list element formats and daemon graceful shutdown. New tests: unset-Value guard + a property test iterating ALL configured variables via gRPC asserting each keeps its declared SECS-II format (daemon tests 16 -> 41 assertions). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
raphael
|
2d60571a9c |
interop: secsgem-py cross-validation harness + lenient identifier parsing
Adds a Docker-based interop harness that drives the C++ server with
secsgem-py 0.3.0 as the active host and probes a secsgem-py-passive
equipment from a minimal C++ active client. Surfaces and fixes four
interoperability bugs uncovered by cross-testing:
* SEMI E5 identifier formatcodes are a U1|U2|U4|U8 wildcard;
secsgem-py picks the narrowest fitting width while our parsers
only accepted U4. `as_uN_scalar` / `as_iN_scalar` now accept
any unsigned/signed width and range-check the downcast.
* PPBODY (S7F3/F6) is "ASCII | Binary | List" per the spec;
secsgem-py defaults to ASCII. Added BINARY_OR_ASCII codegen
item type with `as_text_or_binary` accessor.
* S1F23/F24 Collection Event Namelist was unimplemented; added
schema + `vids_for(ceid)` accessor on EventReportSubscriptions
plus the dispatch handler.
* S10F1 was registered as a host->equipment handler, but per
SEMI E5 §12 S10F1 is equipment->host; S10F3 is the actual
host->equipment Terminal Display Single. Added an S10F3
handler alongside (we keep S10F1 too for backward compat).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
raphael
|
29db1caedb |
#6 SxFy codegen from YAML message catalog
The bulk of the per-SxFy boilerplate — ~90 hand-written builders and parsers
across 30+ message pairs — is now generated at build time from a single YAML
catalog. Adding a new SECS-II message becomes a YAML edit; the C++ code is
generated, not maintained.
What changed
------------
data/messages.yaml
The catalog. Describes every SxFy currently supported: stream, function,
W-bit, builder name, optional parser name, and a recursive body shape
grammar (scalar / list / list_of). Shapes carry SECS-II item types
(ASCII, BINARY_BYTE, U4, F8, ITEM, ...) and optional C++ enum types for
typed ack codes. Inner-most fields can be marked external_struct: true
so structs already defined elsewhere (ReportData, CommandParameter) are
referenced rather than redefined.
tools/gen_messages.py
Python codegen. Reads the catalog and emits one inline header. Handles
nested shapes via depth-unique variable names in the generated IIFEs, so
S6F11's three-level nesting compiles without lambda capture conflicts.
Post-order traversal ensures inner structs are emitted before outer ones
that reference them. Generates positional and (where applicable) struct
builder overloads, plus struct-returning parsers for messages with a
`parser:` entry.
CMakeLists.txt
Custom command runs gen_messages.py at configure/build time and emits
${CMAKE_BINARY_DIR}/generated/secsgem/gem/messages.hpp. Added to the
secsgem target's include path so `#include "secsgem/gem/messages.hpp"`
resolves to the generated file. Depends on the YAML + the script, so
edits trigger regen automatically.
Dockerfile
Added python3 + python3-yaml to the toolchain image.
include/secsgem/gem/messages_helpers.hpp (new)
The small set of hand-written helpers the generated header relies on:
scalar accessors (as_ascii / as_u4_scalar / ...), parse_u4_list_body,
u4_list_item, ack_byte, ALED byte constants, and the two special-case
messages whose shape doesn't fit the codegen schema (S1F4 needs
per-row std::optional<Item> semantics; S5F6 needs a per-row ALCD
callback).
include/secsgem/gem/messages.hpp (deleted)
The hand-written builder/parser file is gone. Its content now flows
through the catalog + codegen.
include/secsgem/gem/data_model.hpp
Moved CommandParameter to namespace scope so it can be shared between
the data model and the messages.yaml's external_struct entry. Added
`using CommandParam = CommandParameter` for back-compat.
apps/secs_server.cpp + apps/secs_client.cpp
Updated the call sites that the codegen renamed or restructured:
- parse_terminal_display() split into parse_s10f1 / parse_s10f3.
- s1f14_establish_comms_ack now takes a McAck struct for the nested
identity (mdln, softrev) — call site uses brace init.
- S2F33/S2F35 parsers return strongly-typed entries (DefineReportEntry,
LinkEventEntry); the server adapts these to the model's pair-based
API at the call site.
- S2F15 parser returns vector<EcSet>; iterate by .ecid/.value.
- S5F3 parser returns EnableAlarmRequest{aled, alid}; bool comes from
(aled & 0x80) != 0.
- AlarmReport's is_set()/category() methods removed; callers use the
raw alcd byte with bit math (alcd & 0x80, alcd & 0x7F).
- s2f42_host_command_ack and s2f41_host_command always take their
second list argument explicitly (no defaulted arg from codegen).
tests/test_messages.cpp
Updated to construct the generated typed structs (EcSet, StatusName,
EnableAlarmRequest, CommandParameter, CommandParameterAck) and to read
the new field names (.ecid/.value, .rptid/.vids, .ceid/.rptids,
.name/.code).
Coverage
--------
Generated by codegen (44 SxFy in catalog):
S1F1, S1F2, S1F3, S1F11, S1F12, S1F13, S1F14, S1F15, S1F16, S1F17, S1F18
S2F13, S2F14, S2F15, S2F16, S2F17, S2F18, S2F29, S2F30, S2F31, S2F32
S2F33, S2F34, S2F35, S2F36, S2F37, S2F38, S2F41, S2F42
S5F1, S5F2, S5F3, S5F4, S5F5
S6F11, S6F12
S7F3, S7F4, S7F5, S7F6, S7F19, S7F20
S10F1, S10F2, S10F3, S10F4
Hand-written (in messages_helpers.hpp):
S1F4 list-of-optional-items shape (nullopt -> <L,0>)
S5F6 per-row ALCD via callback
Adding a new SxFy
-----------------
Append a single entry to data/messages.yaml describing the body shape.
The builder + parser appear in messages.hpp after the next build. The
host command above for S2F41 (or any other added SxFy) requires no C++
changes if the body fits the recursive scalar/list/list_of grammar.
Tests: 67 cases / 384 assertions still passing.
Demo: byte-for-byte identical behaviour (Select, Establish, Online,
S1F11/F3 namelist+values, S2F29 EC namelist, S2F33/F35/F37 dynamic event
subscription, S2F41 START -> S6F11 emission, S5F5/F3 alarm directory +
enable, S2F41 FAULT -> S5F1 alarm + S6F11, S7F19/F5 recipe ops, S10F1
terminal, S1F15 offline, Separate).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|