secs-gem/docs/11_e37_hsms.md

# 11 — E37: HSMS — TCP transport for SECS-II

← [10 E5 — SECS-II data items](10_e5_secs_ii_data_items.md) | [Back to index](00_index.md) | Next: [12 E4 — SECS-I](12_e4_secs_i.md) →

You have an Item.  You have its byte encoding from chapter 10.  Now
those bytes need to travel from equipment to host (or vice versa).

**SEMI E37 — HSMS, High-Speed SECS Message Services** — published
1995, defines how SECS-II bytes travel over TCP/IP.  It's not a
codec (chapter 10 was the codec); it's a framing + connection-state
+ timer protocol that sits between SECS-II and TCP.

This chapter covers:

- The 14-byte HSMS framing.
- The connection state machine (NOT-CONNECTED → NOT-SELECTED →
  SELECTED).
- The five T-timers (T3, T5, T6, T7, T8) and what each one bounds.
- The control-message handshakes: Select.req, Linktest.req,
  Separate.req, Reject.req.
- HSMS-SS (single-session) vs HSMS-GS (multi-session).
- The auto-S9 protocol-error replies the implementation emits.
- Where every part of this is in
  [`include/secsgem/hsms/`](../include/secsgem/hsms/) and
  [`src/hsms/`](../src/hsms/).

---

## The frame

Every HSMS message on the wire looks the same:

```
┌─────────────────┬──────────────────────────┬──────────────────────────┐
│ 4-byte length   │ 10-byte header           │ body (0+ bytes)          │
│ big-endian      │ session_id / byte2 / …   │ SECS-II Item bytes       │
└─────────────────┴──────────────────────────┴──────────────────────────┘
```

The length prefix counts the **header + body** bytes (it does *not*
include itself).  So a frame on the wire is at least 14 bytes long
(4-byte length + 10-byte header, with an empty body — typical for
control messages).

The length is big-endian: the high byte first, then the next, etc.
Defined in [`include/secsgem/hsms/header.hpp:113`](../include/secsgem/hsms/header.hpp):

```cpp
inline constexpr std::size_t kLengthPrefixSize = 4;
inline constexpr std::size_t kHeaderSize       = 10;
```

The body is a fully-encoded SECS-II Item (chapter 10).  For control
messages (Select, Linktest, etc.) the body is empty.  For data
messages it carries whatever Item the higher layer wants to send.

---

## The 10-byte header

The header is the only HSMS-specific structure besides the length
prefix.  Six fields, packed in a fixed order:

```
 byte  0  1  2     3        4       5       6  7  8  9
      ┌──────┬──────────┬──────────┬─────┬─────┬───────────┐
      │ ses  │ byte2    │ byte3    │ Ptype│ Stype│ system_bytes │
      │ _id  │ (W+strm) │ function │ (=0) │     │ (correlation)│
      └──────┴──────────┴──────────┴─────┴─────┴───────────┘
        ↑       ↑           ↑         ↑      ↑       ↑
       u16    bit7 W      func id    0     SType   u32
              bits6-0
              stream
```

Per field:

- **`session_id`** (u16, big-endian).  For HSMS-SS this is the
  sentinel `0xFFFF` (`kControlSessionId` in
  [`header.hpp:48`](../include/secsgem/hsms/header.hpp)).  For
  HSMS-GS, it carries the device ID identifying which session this
  frame belongs to.
- **`byte2`** (u8).  Bit 7 is the **W-bit** (1 = reply expected,
  0 = no reply).  Bits 0–6 are the **stream** number (S1–S127).
- **`byte3`** (u8).  The **function** number (F0–F255).
- **`PType`** (u8).  Presentation type.  Always 0 for SECS-II
  bodies.  Any other value triggers `S9F1` "Unrecognized Device ID"
  (or `RejectReq` PtypeNotSupported, depending on state).
- **`SType`** (u8).  Session type.  0 for data messages; 1–9 for the
  various control messages (see next section).
- **`system_bytes`** (u32, big-endian).  An opaque correlation
  token.  Picked by the sender; the receiver MUST echo the same
  value in its reply.  This is how we match a `S1F2` to the
  `S1F1` that asked for it.

The header is encoded by [`Header::encode()`](../include/secsgem/hsms/header.hpp)
and decoded by [`Header::decode()`](../include/secsgem/hsms/header.hpp);
both implementations in
[`src/hsms/header.cpp`](../src/hsms/header.cpp) are ~30 lines each.

### Helpful constructors

The header has two named constructors that wire the right fields
together:

```cpp
// Data message: W-bit + stream + function + system_bytes.
Header::data_message(session_id, stream, function, w_bit, system_bytes);

// Control message: SType + system_bytes (other fields default).
Header::control(stype, system_bytes);
```

Both in [`header.hpp:74-97`](../include/secsgem/hsms/header.hpp).

### A worked frame

Encode the data message `S1F1 W` (Are You There, reply expected), in
HSMS-SS, with `system_bytes = 1`:

```
length prefix:  00 00 00 0A     (header only = 10 bytes; body is empty)
session_id:     FF FF           (0xFFFF — kControlSessionId)
byte2:          81              (bit 7 W=1 | stream 1 = 0x80 | 0x01)
byte3:          01              (function 1)
PType:          00              (SECS-II)
SType:          00              (data)
system_bytes:   00 00 00 01     (correlation = 1)
```

That's a complete frame.  Send those 14 bytes over TCP and you've
just asked an equipment "are you there?"

The reply, `S1F2`, would echo `system_bytes = 1` and carry a body
(the MDLN + SOFTREV `<L[2] A "MDLN" A "SOFTREV">`).

---

## The session types (`SType`)

Defined as
[`enum class SType`](../include/secsgem/hsms/header.hpp):

| SType | Name           | Purpose                                                       |
|-------|----------------|---------------------------------------------------------------|
| 0     | `Data`         | Carries a SECS-II message body.                               |
| 1     | `SelectReq`    | Request to enter SELECTED state.                              |
| 2     | `SelectRsp`    | Reply to Select.req.  Body carries a 1-byte `SelectStatus`.   |
| 3     | `DeselectReq`  | Request to leave SELECTED state (rarely used).                |
| 4     | `DeselectRsp`  | Reply to Deselect.req.                                        |
| 5     | `LinktestReq`  | "Are you still there?" probe.                                 |
| 6     | `LinktestRsp`  | Reply to Linktest.req.                                        |
| 7     | `RejectReq`    | "I don't understand that."  Sent in response to malformed control. |
| 9     | `SeparateReq`  | "I'm closing the connection."  No reply expected.             |

SType 8 is reserved.

For every primary control message (Req types 1, 3, 5), the matching
Rsp type comes back — except for `RejectReq` and `SeparateReq`
which are fire-and-forget.  All control messages use the
`kControlSessionId = 0xFFFF` in HSMS-SS.

---

## The connection state machine

A new TCP connection is `NOT-SELECTED`.  Until both sides
SELECT, no data messages can be exchanged — even if data frames
arrive, they're rejected with `RejectReq(EntityNotSelected)`.

```
 (TCP connected; T7 armed)
  ┌────────────────┐    Select.req →           ┌────────────────┐
  │  NOT-SELECTED  │ ────────────────────────► │   SELECTED     │
  │                │ ← Select.rsp(Ok)          │                │
  └────────────────┘                           └────────────────┘
       │                                              │
       │ T7 expires                                   │ Separate.req → close
       │ ─────────────────► close                     │ or TCP FIN
       │                                              │
       │ data frame received while NOT-SELECTED       │
       │ ─────────────────► RejectReq(EntityNotSelected)
       │                                              │
       └─────── close (any reason) ───────────────────┘
```

Per side:

- **Active** initiates the TCP connect, then sends `Select.req`.
- **Passive** binds and listens; T7 arms when the TCP connection
  comes in; it waits for the peer's `Select.req`.

The state transitions live in
[`src/hsms/connection.cpp`](../src/hsms/connection.cpp) — `start()`
either calls `connect()` (active) or `accept()` (passive); reaching
SELECTED fires `on_selected_` and arms the linktest timer.

The convention for SECS/GEM is that **the equipment is passive**
(binds the port) and **the host is active** (initiates connect +
Select).  This is configurable — `Connection::Mode` is `Active` or
`Passive` — but every example in this codebase follows the GEM
default.

---

## The five T-timers

The HSMS spec defines T1, T2, T3, T4, T5, T6, T7, T8 — but **HSMS
only uses T3, T5, T6, T7, T8**.  T1, T2, T4 are SECS-I (chapter 12).

Defaults in
[`Timers` struct](../include/secsgem/hsms/header.hpp), line 51:

```cpp
struct Timers {
  std::chrono::milliseconds t3{45000};    // reply
  std::chrono::milliseconds t5{10000};    // connect separation
  std::chrono::milliseconds t6{5000};     // control transaction
  std::chrono::milliseconds t7{10000};    // not-selected
  std::chrono::milliseconds t8{5000};     // intercharacter
  std::chrono::milliseconds linktest{0};  // 0 disables
};
```

### T3 — reply timeout (45 s default)

When a side sends a W=1 data message, it arms T3 for the matching
reply.  T3 cancels when the reply arrives (same system_bytes).  If
it expires, the implementation **auto-emits `S9F9` Transaction Timer
Timeout** carrying the original 10-byte MHEAD so the peer knows
which transaction timed out.

Tested in [`tests/test_hsms_timers.cpp:164`](../tests/test_hsms_timers.cpp).

### T5 — connect separation timeout (10 s default)

After a TCP connection fails or drops, the active side waits T5
before retrying.  Stops a misbehaving peer from getting hammered
with reconnects.

### T6 — control transaction timeout (5 s default)

For Select.req → Select.rsp, Linktest.req → Linktest.rsp, and
Deselect.req → Deselect.rsp.  If the response doesn't come back
within T6, the connection closes.

Tested in [`tests/test_hsms_timers.cpp:206`](../tests/test_hsms_timers.cpp).

### T7 — not-selected timeout (10 s default)

On a passive side: when TCP comes up but no Select.req has arrived
yet, T7 is armed.  If it expires before Select.req, the passive
side closes the connection.  This prevents an unauthenticated
connection from camping forever.

Tested in [`tests/test_hsms_timers.cpp:230`](../tests/test_hsms_timers.cpp).

### T8 — inter-character timeout (5 s default)

After reading the 4-byte length prefix, the implementation waits up
to T8 for the next byte.  If the peer goes silent mid-payload, the
connection closes.  Protects against half-open connections that
think they're alive but never finish sending.

Tested in [`tests/test_hsms_timers.cpp:248`](../tests/test_hsms_timers.cpp).

### Linktest cadence

`Timers::linktest` (not a timeout — a cadence).  If non-zero, the
side periodically emits `Linktest.req` to verify the peer is still
responsive.  Default is 0 (disabled); turn it on when you suspect
silent drops.  Linktest is the only HSMS health-check between
data messages.

---

## The auto-S9 paths

When the connection sees a *protocol* error — malformed bytes,
unknown SType, unhandled stream/function — it doesn't silently
drop.  It emits an `S9F<n>` message back to the peer so the peer
knows what went wrong:

| Trigger                                              | Reply           | Where                                         |
|------------------------------------------------------|-----------------|-----------------------------------------------|
| Unknown PType                                         | `S9F1`          | (auto)                                        |
| Unrecognized device ID (SS mode with non-control sid) | `S9F1`          | `Connection::handle_data`                     |
| Frame body fails SECS-II decode                       | `S9F7`          | `Connection::handle_data`                     |
| W=1 message arrived for unknown stream                | `S9F3`          | Router::dispatch_with_s9 → `Connection::emit_s9` |
| W=1 message arrived for unknown function in known stream | `S9F5`       | Router::dispatch_with_s9 → `Connection::emit_s9` |
| W=1 reply timed out (T3 expired)                      | `S9F9`          | `Connection::on_t3_expire`                    |
| Body larger than configured cap                        | `S9F11`         | `Connection::on_payload` (16 MiB-ish cap)     |
| Frame valid but received in NOT-SELECTED               | `RejectReq(EntityNotSelected)` | `handle_data`                  |

`emit_s9(function, mhead)` is exposed publicly
([`connection.hpp:97`](../include/secsgem/hsms/connection.hpp)) so a
higher-level dispatcher (like `gem::Router`) can call it after its
own unknown-stream / unknown-function check.

Tested across
[`tests/test_hsms_s9.cpp`](../tests/test_hsms_s9.cpp) (3 cases for
the malformed-body paths) and
[`tests/test_s9_fallback.cpp`](../tests/test_s9_fallback.cpp) (2
cases for the Router dispatch path).

---

## HSMS-SS vs HSMS-GS

### Single-Session (HSMS-SS)

The default.  One TCP socket carries one SECS conversation between
one equipment and one host.  The `session_id` field in every frame
header carries the sentinel `0xFFFF` (control session ID), even for
data messages — it's not used for routing.

In code: pass any value to `Connection`'s constructor as
`device_id` — it's stored, but for SS-mode handshakes the header's
session_id is forced to `0xFFFF`.

### General-Session (HSMS-GS)

E37 §11.  One TCP socket multiplexes *several* sessions, each with
its own device_id.  The Select.req frame's `session_id` field
carries the device_id of the session being selected; data frames
likewise carry the session's device_id in the header.

Use case: one equipment with one TCP socket to a host that wants
several logical conversations — say, production traffic on one
session, maintenance traffic on another.  Real-world example: a
fab where the production MES and the maintenance MES connect to
the same equipment over one link via separate gateways.

In code:

```cpp
auto conn = std::make_shared<Connection>(std::move(sock), Mode::Passive,
                                          /*primary device_id=*/100, timers);
conn->add_session(/*device_id=*/200);   // second session
conn->add_session(/*device_id=*/300);   // third session

conn->set_session_message_handler(100, ...);
conn->set_session_message_handler(200, ...);
conn->set_session_message_handler(300, ...);
conn->start();
```

Each session has its own SELECTED state, independent message
handler, independent send queue.  When a Select.req arrives, the
implementation routes it by `session_id` field to find the right
registered session.

Tested in
[`tests/test_hsms_gs.cpp`](../tests/test_hsms_gs.cpp) (5 wire-level
cases) and
[`tests/test_hsms_gs_integration.cpp`](../tests/test_hsms_gs_integration.cpp)
(one end-to-end three-session scenario).

Implementation walk-through with code snippets:
[`docs/INTEGRATION.md`](INTEGRATION.md) §7.

---

## The single-threaded contract

`Connection` is **single-threaded**.  All state mutations — the
read loop, the send queue, the timers, the SELECTED transition —
run on the socket's `asio::executor`.  Callers that want to send
from another thread must marshal onto the executor via
`asio::post()`.

The contract is documented in
[`docs/INTEGRATION.md`](INTEGRATION.md) §3 and exercised under
ThreadSanitizer by
[`tests/test_thread_safety.cpp`](../tests/test_thread_safety.cpp).
N producer threads `asio::post` updates; TSan reports zero races.

This avoids every mutex you'd otherwise need for the queue, the
session map, the in-flight request table, the timer state machine.
The cost is that the caller has to know about it.  See chapter
[33](33_transport.md) for a deeper look at the asio strand model.

---

## How send/receive feels from the caller

Send a request (W=1):

```cpp
conn->send_request(secs2::Message{stream, function, /*w=*/true, body},
    [](std::error_code ec, const secs2::Message& reply) {
      if (ec) {
        // T3 expired, or connection closed before reply.
      } else {
        // reply.body() is the decoded Item.
      }
    });
```

Send a one-way data message (W=0):

```cpp
conn->send_data(secs2::Message{6, 11, /*w=*/false, event_report_body});
```

Register a primary-message handler (called for inbound W=1 messages
in SELECTED state):

```cpp
conn->set_message_handler([](const secs2::Message& m) -> std::optional<secs2::Message> {
  // Build a reply Message and return it; system_bytes are auto-filled.
  return secs2::Message{m.stream(), m.function() + 1, /*w=*/false, reply_body};
});
```

Higher layers (`gem::Router`) wrap this with stream/function dispatch.

---

## What's still ahead at this layer

The connection is **just transport**.  It doesn't know about GEM
states, doesn't know about CEIDs or alarms, doesn't validate the
body's shape (just decodes it as an Item).  When `gem::Router`
hands a request to the alarm handler and the alarm handler returns
a reply, all the Router does is bundle that reply into a
`secs2::Message` and tell the connection to send it.

Three things to revisit in later chapters:

- **The behavioural layer above HSMS** is E30 (chapter
  [13](13_e30_gem.md)).  GEM has its own communication state machine
  on top of HSMS's transport state machine — they look similar but
  are *different* state machines.
- **The Router** that dispatches stream/function to handlers lives
  in `secsgem::gem::Router` and is covered in chapter
  [35](35_state_machines_and_dispatch.md).
- **The asio strand model** that makes the single-threaded contract
  work is in chapter [33](33_transport.md).

But first, the *other* transport — the one this one replaced, but
that hasn't gone away.  **SECS-I over serial**.

Next: [→ 12 E4 — SECS-I: the serial origin](12_e4_secs_i.md)