ProcessJobStore and SubstrateStore already implemented the
loader-accepts-any-version-in-[1, kVersion] pattern. The other five
stores (ControlJobStore, CarrierStore, LoadPortStore, ExceptionStore,
SpoolStore) used strict `header[1] != kVersion` rejection, meaning
a future kVersion bump there would silently nuke every persisted
record on first replay. That's a footgun the test_persistence_upgrade
test already flagged as a tripwire.
This commit flips the strict checks to `< 1 || > kVersion`, mirroring
PJ + Substrate. No format change (kVersion stays at 1 across the
five stores), but:
- Future v2 of any store now Just Works: add fields at the end of
write_record_, bump kVersion to 2, gate the new reads behind
`if (version >= 2)`. Old v1 records on disk continue to replay
with the new fields defaulted.
- Future versions beyond kVersion still get rejected (downgrade
protection — older code can't try to decode trailers it doesn't
understand).
Comment blocks on each kVersion declaration now describe the upgrade
discipline so the next contributor doesn't reinvent it.
Test additions:
- Positive test that v1 ControlJob records load on current code
(will continue to pass when kVersion bumps to 2, proving v1 is
still readable)
- ExceptionStore rejects a v9 (future) record, matching CJ + Carrier
- The existing tripwire tests get retitled from "rejects unknown
version" to "rejects a future version" to reflect the new contract
README §6 gets honest: every store is now multi-version-aware, not
just PJ + Substrate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds opt-in disk persistence to SpoolStore. `enable_persistence(dir)`
turns every enqueue into a single `<seq>.spool` file alongside the
in-memory queue; drain and clear delete the matching files; restart
replays the directory sorted by seq.
Writes are atomic: serialize the message via the SECS-II codec, write
to `.tmp`, then `std::filesystem::rename` to the final name. Malformed
records are dropped silently so a single bad file can't poison the
whole spool.
`secs_server --spool-dir <path>` enables persistence at startup.
Without the flag the behaviour is identical to before (in-memory only).
Two new tests: enqueue → restart → replay → drain restores the wire
order, and clear deletes the journal files.
Test suite: 291 cases / 1515 assertions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements the largest functional gap from the compliance audit. The
equipment now queues events the host can't immediately receive (either
because there's no SELECTED session or because the demo's force-spool
flag is on) and transmits the queue on host request.
What's new
include/secsgem/gem/store/spool.hpp
SpoolStore: a deque queue with a configurable per-stream whitelist
(so only streams 5+6 spool by default), a max_size cap with FIFO
eviction on overflow, and a `force_spool` test flag. Enqueue
returns one of Queued / Dropped_NotSpoolable / Dropped_Full so the
caller can fall back to live delivery when appropriate. Drain
pops the entire queue in FIFO order. Two new ack enums:
ResetSpoolAck (S2F44 RSPACK) and SpoolRequestAck (S6F24 RSDA), plus
SpoolRequestCode (S6F23 RSDC, Transmit/Purge).
data/messages.yaml + auto-regenerated messages.hpp
S2F43 W <L,n <B stream>> Reset Spooling
S2F44 <L,2 <B RSPACK> <L,a ...>> Reset Spooling Ack
S6F23 W <B RSDC> Request Spooled Data
S6F24 <B RSDA> Request Spooled Data Ack
data/equipment.yaml
`spool:` section: max_size + spoolable_streams list. Two new host
commands SPOOL_ON / SPOOL_OFF that flip the force-spool flag (these
stand in for "host link down" in the demo without dropping TCP).
include/secsgem/gem/store/host_commands.hpp
Spec/Result gain an optional<bool> force_spool field. S2F41
dispatch returns the result, the server applies it after S2F42 is
queued.
src/config/loader.cpp
Reads `spool:` from equipment.yaml; reads `force_spool` from each
host_commands entry; populates SpoolStore + CommandSpec.
apps/secs_server.cpp
New `deliver_or_spool(msg, what)` helper. emit_event and
emit_alarm_set funnel through it: if force_spool is on (or there's
no active session), msg.stream is checked against the spoolable
list and the message is enqueued; otherwise it's sent live.
Two new handlers:
S2F43 parses the stream list, updates SpoolStore, replies S2F44
S6F23 RSDC=Transmit drains and re-sends each as a fresh primary
(posted on the executor so the S6F24 ack flushes first);
RSDC=Purge clears the queue and acks.
The S2F41 handler now also propagates result.force_spool into the
SpoolStore.
apps/secs_client.cpp
Demo extended with 4 new steps after the FAULT branch:
SPOOL_ON -> S2F42 Accept
START -> S2F42 Accept; CEID 300 emission spooled (no live S6F11)
SPOOL_OFF -> S2F42 Accept; queue still has the message
S6F23(Transmit) -> S6F24 Accept; spooled S6F11 arrives next
Then the existing S7F19/S7F5/S10F1/S1F15/Separate flow continues.
tests/test_data_model.cpp
Four new TEST_CASEs for SpoolStore (whitelist, FIFO eviction at
max_size, drain ordering, force flag).
tests/test_loader.cpp
Confirms equipment.yaml's `spool:` section populates the store and
`force_spool: true/false` flows through to dispatch results.
COMPLIANCE.md
Spooling moves from ⬜ to 🟡. Adds S2F43/F44 + S6F23/F24 as ✅ in
the message coverage matrix; calls out what's still missing
(S6F25/F26 notification, automatic activation on HSMS NOT-SELECTED,
persistent on-disk spool).
Verified
- Tests: 73 cases / 383 assertions pass (+4 spool cases).
- Demo (docker compose up server client) walks the full happy path
and the spool path, observed in the server log as:
spool: force_spool=true (depth=0)
spool: S6F11 CEID=300 queued (depth=1)
spool: force_spool=false (depth=1)
S6F23 transmit: draining 1 messages
and on the host side as the queued S6F11 arriving in the correct
order after S6F24.
Known limitations (logged in COMPLIANCE.md)
- Spool activation is manual via SPOOL_ON/OFF rather than
automatically triggered by HSMS NOT-SELECTED.
- No S6F25/F26 spooled-data-ready notification on re-SELECT.
- In-memory only; an equipment restart loses queued events.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raphael