secs-gem/docs/11_e37_hsms.md

11 — E37: HSMS — TCP transport for SECS-II

← 10 E5 — SECS-II data items | Back to index | Next: 12 E4 — SECS-I →

You have an Item. You have its byte encoding from chapter 10. Now those bytes need to travel from equipment to host (or vice versa).

SEMI E37 — HSMS, High-Speed SECS Message Services — published 1995, defines how SECS-II bytes travel over TCP/IP. It's not a codec (chapter 10 was the codec); it's a framing + connection-state

• timer protocol that sits between SECS-II and TCP.

This chapter covers:

• The 14-byte HSMS framing.
• The connection state machine (NOT-CONNECTED → NOT-SELECTED → SELECTED).
• The five T-timers (T3, T5, T6, T7, T8) and what each one bounds.
• The control-message handshakes: Select.req, Linktest.req, Separate.req, Reject.req.
• HSMS-SS (single-session) vs HSMS-GS (multi-session).
• The auto-S9 protocol-error replies the implementation emits.
• Where every part of this is in include/secsgem/hsms/ and src/hsms/.

---

The frame

Every HSMS message on the wire looks the same:

┌─────────────────┬──────────────────────────┬──────────────────────────┐
│ 4-byte length   │ 10-byte header           │ body (0+ bytes)          │
│ big-endian      │ session_id / byte2 / …   │ SECS-II Item bytes       │
└─────────────────┴──────────────────────────┴──────────────────────────┘

The length prefix counts the header + body bytes (it does not include itself). So a frame on the wire is at least 14 bytes long (4-byte length + 10-byte header, with an empty body — typical for control messages).

The length is big-endian: the high byte first, then the next, etc. Defined in include/secsgem/hsms/header.hpp:113:

inline constexpr std::size_t kLengthPrefixSize = 4;
inline constexpr std::size_t kHeaderSize       = 10;

The body is a fully-encoded SECS-II Item (chapter 10). For control messages (Select, Linktest, etc.) the body is empty. For data messages it carries whatever Item the higher layer wants to send.

---

The 10-byte header

The header is the only HSMS-specific structure besides the length prefix. Six fields, packed in a fixed order:

 byte  0  1  2     3        4       5       6  7  8  9
      ┌──────┬──────────┬──────────┬─────┬─────┬───────────┐
      │ ses  │ byte2    │ byte3    │ Ptype│ Stype│ system_bytes │
      │ _id  │ (W+strm) │ function │ (=0) │     │ (correlation)│
      └──────┴──────────┴──────────┴─────┴─────┴───────────┘
        ↑       ↑           ↑         ↑      ↑       ↑
       u16    bit7 W      func id    0     SType   u32
              bits6-0
              stream

Per field:

• session_id (u16, big-endian). For HSMS-SS this is the sentinel 0xFFFF (kControlSessionId in header.hpp:48). For HSMS-GS, it carries the device ID identifying which session this frame belongs to.
• byte2 (u8). Bit 7 is the W-bit (1 = reply expected, 0 = no reply). Bits 0–6 are the stream number (S1–S127).
• byte3 (u8). The function number (F0–F255).
• PType (u8). Presentation type. Always 0 for SECS-II bodies. Any other value triggers S9F1 "Unrecognized Device ID" (or RejectReq PtypeNotSupported, depending on state).
• SType (u8). Session type. 0 for data messages; 1–9 for the various control messages (see next section).
• system_bytes (u32, big-endian). An opaque correlation token. Picked by the sender; the receiver MUST echo the same value in its reply. This is how we match a S1F2 to the S1F1 that asked for it.

The header is encoded by Header::encode() and decoded by Header::decode(); both implementations in src/hsms/header.cpp are ~30 lines each.

Helpful constructors

The header has two named constructors that wire the right fields together:

// Data message: W-bit + stream + function + system_bytes.
Header::data_message(session_id, stream, function, w_bit, system_bytes);

// Control message: SType + system_bytes (other fields default).
Header::control(stype, system_bytes);

Both in header.hpp:74-97.

A worked frame

Encode the data message S1F1 W (Are You There, reply expected), in HSMS-SS, with system_bytes = 1:

length prefix:  00 00 00 0A     (header only = 10 bytes; body is empty)
session_id:     FF FF           (0xFFFF — kControlSessionId)
byte2:          81              (bit 7 W=1 | stream 1 = 0x80 | 0x01)
byte3:          01              (function 1)
PType:          00              (SECS-II)
SType:          00              (data)
system_bytes:   00 00 00 01     (correlation = 1)

That's a complete frame. Send those 14 bytes over TCP and you've just asked an equipment "are you there?"

The reply, S1F2, would echo system_bytes = 1 and carry a body (the MDLN + SOFTREV <L[2] A "MDLN" A "SOFTREV">).

---

The session types (SType)

Defined as enum class SType:

SType	Name	Purpose
0	Data	Carries a SECS-II message body.
1	SelectReq	Request to enter SELECTED state.
2	SelectRsp	Reply to Select.req. Body carries a 1-byte SelectStatus.
3	DeselectReq	Request to leave SELECTED state (rarely used).
4	DeselectRsp	Reply to Deselect.req.
5	LinktestReq	"Are you still there?" probe.
6	LinktestRsp	Reply to Linktest.req.
7	RejectReq	"I don't understand that." Sent in response to malformed control.
9	SeparateReq	"I'm closing the connection." No reply expected.

SType 8 is reserved.

For every primary control message (Req types 1, 3, 5), the matching Rsp type comes back — except for RejectReq and SeparateReq which are fire-and-forget. All control messages use the kControlSessionId = 0xFFFF in HSMS-SS.

---

The connection state machine

A new TCP connection is NOT-SELECTED. Until both sides SELECT, no data messages can be exchanged — even if data frames arrive, they're rejected with RejectReq(EntityNotSelected).

 (TCP connected; T7 armed)
  ┌────────────────┐    Select.req →           ┌────────────────┐
  │  NOT-SELECTED  │ ────────────────────────► │   SELECTED     │
  │                │ ← Select.rsp(Ok)          │                │
  └────────────────┘                           └────────────────┘
       │                                              │
       │ T7 expires                                   │ Separate.req → close
       │ ─────────────────► close                     │ or TCP FIN
       │                                              │
       │ data frame received while NOT-SELECTED       │
       │ ─────────────────► RejectReq(EntityNotSelected)
       │                                              │
       └─────── close (any reason) ───────────────────┘

Per side:

• Active initiates the TCP connect, then sends Select.req.
• Passive binds and listens; T7 arms when the TCP connection comes in; it waits for the peer's Select.req.

The state transitions live in src/hsms/connection.cpp — start() either calls connect() (active) or accept() (passive); reaching SELECTED fires on_selected_ and arms the linktest timer.

The convention for SECS/GEM is that the equipment is passive (binds the port) and the host is active (initiates connect + Select). This is configurable — Connection::Mode is Active or Passive — but every example in this codebase follows the GEM default.

---

The five T-timers

The HSMS spec defines T1, T2, T3, T4, T5, T6, T7, T8 — but HSMS only uses T3, T5, T6, T7, T8. T1, T2, T4 are SECS-I (chapter 12).

Defaults in Timers struct, line 51:

struct Timers {
  std::chrono::milliseconds t3{45000};    // reply
  std::chrono::milliseconds t5{10000};    // connect separation
  std::chrono::milliseconds t6{5000};     // control transaction
  std::chrono::milliseconds t7{10000};    // not-selected
  std::chrono::milliseconds t8{5000};     // intercharacter
  std::chrono::milliseconds linktest{0};  // 0 disables
};

T3 — reply timeout (45 s default)

When a side sends a W=1 data message, it arms T3 for the matching reply. T3 cancels when the reply arrives (same system_bytes). If it expires, the implementation auto-emits S9F9 Transaction Timer Timeout carrying the original 10-byte MHEAD so the peer knows which transaction timed out.

Tested in tests/test_hsms_timers.cpp:164.

T5 — connect separation timeout (10 s default)

After a TCP connection fails or drops, the active side waits T5 before retrying. Stops a misbehaving peer from getting hammered with reconnects.

T6 — control transaction timeout (5 s default)

For Select.req → Select.rsp, Linktest.req → Linktest.rsp, and Deselect.req → Deselect.rsp. If the response doesn't come back within T6, the connection closes.

Tested in tests/test_hsms_timers.cpp:206.

T7 — not-selected timeout (10 s default)

On a passive side: when TCP comes up but no Select.req has arrived yet, T7 is armed. If it expires before Select.req, the passive side closes the connection. This prevents an unauthenticated connection from camping forever.

Tested in tests/test_hsms_timers.cpp:230.

T8 — inter-character timeout (5 s default)

After reading the 4-byte length prefix, the implementation waits up to T8 for the next byte. If the peer goes silent mid-payload, the connection closes. Protects against half-open connections that think they're alive but never finish sending.

Tested in tests/test_hsms_timers.cpp:248.

Linktest cadence

Timers::linktest (not a timeout — a cadence). If non-zero, the side periodically emits Linktest.req to verify the peer is still responsive. Default is 0 (disabled); turn it on when you suspect silent drops. Linktest is the only HSMS health-check between data messages.

---

The auto-S9 paths

When the connection sees a protocol error — malformed bytes, unknown SType, unhandled stream/function — it doesn't silently drop. It emits an S9F<n> message back to the peer so the peer knows what went wrong:

Trigger	Reply	Where
Unknown PType	S9F1	(auto)
Unrecognized device ID (SS mode with non-control sid)	S9F1	Connection::handle_data
Frame body fails SECS-II decode	S9F7	Connection::handle_data
W=1 message arrived for unknown stream	S9F3	Router::dispatch_with_s9 → Connection::emit_s9
W=1 message arrived for unknown function in known stream	S9F5	Router::dispatch_with_s9 → Connection::emit_s9
W=1 reply timed out (T3 expired)	S9F9	Connection::on_t3_expire
Body larger than configured cap	S9F11	Connection::on_payload (16 MiB-ish cap)
Frame valid but received in NOT-SELECTED	RejectReq(EntityNotSelected)	handle_data

emit_s9(function, mhead) is exposed publicly (connection.hpp:97) so a higher-level dispatcher (like gem::Router) can call it after its own unknown-stream / unknown-function check.

Tested across tests/test_hsms_s9.cpp (3 cases for the malformed-body paths) and tests/test_s9_fallback.cpp (2 cases for the Router dispatch path).

---

HSMS-SS vs HSMS-GS

Single-Session (HSMS-SS)

The default. One TCP socket carries one SECS conversation between one equipment and one host. The session_id field in every frame header carries the sentinel 0xFFFF (control session ID), even for data messages — it's not used for routing.

In code: pass any value to Connection's constructor as device_id — it's stored, but for SS-mode handshakes the header's session_id is forced to 0xFFFF.

General-Session (HSMS-GS)

E37 §11. One TCP socket multiplexes several sessions, each with its own device_id. The Select.req frame's session_id field carries the device_id of the session being selected; data frames likewise carry the session's device_id in the header.

Use case: one equipment with one TCP socket to a host that wants several logical conversations — say, production traffic on one session, maintenance traffic on another. Real-world example: a fab where the production MES and the maintenance MES connect to the same equipment over one link via separate gateways.

In code:

auto conn = std::make_shared<Connection>(std::move(sock), Mode::Passive,
                                          /*primary device_id=*/100, timers);
conn->add_session(/*device_id=*/200);   // second session
conn->add_session(/*device_id=*/300);   // third session

conn->set_session_message_handler(100, ...);
conn->set_session_message_handler(200, ...);
conn->set_session_message_handler(300, ...);
conn->start();

Each session has its own SELECTED state, independent message handler, independent send queue. When a Select.req arrives, the implementation routes it by session_id field to find the right registered session.

Tested in tests/test_hsms_gs.cpp (5 wire-level cases) and tests/test_hsms_gs_integration.cpp (one end-to-end three-session scenario).

Implementation walk-through with code snippets: docs/INTEGRATION.md §7.

---

The single-threaded contract

Connection is single-threaded. All state mutations — the read loop, the send queue, the timers, the SELECTED transition — run on the socket's asio::executor. Callers that want to send from another thread must marshal onto the executor via asio::post().

The contract is documented in docs/INTEGRATION.md §3 and exercised under ThreadSanitizer by tests/test_thread_safety.cpp. N producer threads asio::post updates; TSan reports zero races.

This avoids every mutex you'd otherwise need for the queue, the session map, the in-flight request table, the timer state machine. The cost is that the caller has to know about it. See chapter 33 for a deeper look at the asio strand model.

---

How send/receive feels from the caller

Send a request (W=1):

conn->send_request(secs2::Message{stream, function, /*w=*/true, body},
    [](std::error_code ec, const secs2::Message& reply) {
      if (ec) {
        // T3 expired, or connection closed before reply.
      } else {
        // reply.body() is the decoded Item.
      }
    });

Send a one-way data message (W=0):

conn->send_data(secs2::Message{6, 11, /*w=*/false, event_report_body});

Register a primary-message handler (called for inbound W=1 messages in SELECTED state):

conn->set_message_handler([](const secs2::Message& m) -> std::optional<secs2::Message> {
  // Build a reply Message and return it; system_bytes are auto-filled.
  return secs2::Message{m.stream(), m.function() + 1, /*w=*/false, reply_body};
});

Higher layers (gem::Router) wrap this with stream/function dispatch.

---

What's still ahead at this layer

The connection is just transport. It doesn't know about GEM states, doesn't know about CEIDs or alarms, doesn't validate the body's shape (just decodes it as an Item). When gem::Router hands a request to the alarm handler and the alarm handler returns a reply, all the Router does is bundle that reply into a secs2::Message and tell the connection to send it.

Three things to revisit in later chapters:

• The behavioural layer above HSMS is E30 (chapter 13). GEM has its own communication state machine on top of HSMS's transport state machine — they look similar but are different state machines.
• The Router that dispatches stream/function to handlers lives in secsgem::gem::Router and is covered in chapter 35.
• The asio strand model that makes the single-threaded contract work is in chapter 33.

But first, the other transport — the one this one replaced, but that hasn't gone away. SECS-I over serial.

Next: → 12 E4 — SECS-I: the serial origin