raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: start guided-tour tutorial series under docs/

Browse Source 
A linear teach-from-zero tutorial that walks both SECS/GEM as a
protocol family and this codebase as an implementation.  Each
chapter explains a SEMI concept and shows where it lives in code,
so a reader builds a mental model of the standards and the
repository simultaneously.

Structure (24 chapters across 5 parts):
- Part 1 (3 ch) — Foundations: what SECS/GEM is, the cast of
  characters, vocabulary + a wafer's end-to-end journey
- Part 2 (10 ch) — Standards in detail: E5, E37, E4, E30,
  E40+E94, E87, E90+E157, E116+E120+E39, E84, E42+E148+S9
- Part 3 (7 ch) — Codebase: repository tour, spec-as-data + codegen,
  stores, transport, codec, state machines, persistence
- Part 4 (2 ch) — Operations: build/run/demo, integration
- Part 5 (2 ch) — Reference: API + messages + YAML, extension guide

Published in this commit:
- 00_index.md — guide layout, audience map, reading paths,
  conventions, status table
- 01_what_is_secs_gem.md — the N×M integration problem, what SECS
  vs. HSMS vs. GEM each actually refer to, the GEM 300 suite, the
  transport→message→behaviour layering, where each layer lives in
  this codebase, an end-to-end S2F17/F18 example

Chapters publish iteratively from here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-09 19:16:35 +02:00
parent b01dedfaa5
commit 01acac97d4
2 changed files with 587 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/00_index.md
+259
View File
@@ -0,0 +1,259 @@
# The secs-gem guided tour
A tutorial series that teaches **SECS/GEM the protocol** and
**secs-gem the codebase** at the same time. Starts from zero — no
prior knowledge of semiconductor manufacturing or factory automation
required — and ends with a working mental model of every namespace,
state machine, and YAML file in this repository.
Each chapter does two things in parallel:
1. **Explains a SEMI concept** in plain language, with diagrams and a
concrete example.
2. **Shows where it lives in this codebase**, with file paths and
line references you can click straight into.
By the end you'll be able to read any commit, audit any YAML, or
extend any subsystem without having to ask "what does that even
mean?"
---
## Who this is for
- **Software engineers new to fab automation** who need to make a
semiconductor tool talk to a Manufacturing Execution System (MES).
- **Vendor integrators** who own the C++ side of an equipment
deployment and need to wire `secs-gem` into a real recipe engine,
PLC, or sensor stack.
- **Fab IT / MES owners** who need to understand what their
equipment is sending them and why.
- **Auditors and reviewers** who want a structured walk through the
codebase before signing off on compliance claims.
If you already know SECS/GEM and just want the codebase, skip to
[Part 3](#part-3--the-codebase). If you know neither, start at
[Part 1, Chapter 01](01_what_is_secs_gem.md) and read straight
through.
---
## How the standards stack fits together
Before we dive in, here's the one-screen mental model. Every chapter
in Part 2 fills in one of these boxes:
```
┌──────────────────────────────────────────────────────────────┐
│ GEM 300 — fab-floor behaviour (one rule book per concern) │
│ │
│ E40 process jobs E94 control jobs E87 carriers │
│ E90 substrates E157 modules E116 perf time │
│ E84 AMHS handoff E120 common equip E148 time sync │
│ E42 formatted PP E39 object services E5 §13 wafer maps │
└──────────────────────────────────────────────────────────────┘
▲ uses messages defined by
│
┌──────────────────────────────────────────────────────────────┐
│ E30 — GEM: communication state + control state + scenarios │
└──────────────────────────────────────────────────────────────┘
▲ uses messages encoded by
│
┌──────────────────────────────────────────────────────────────┐
│ E5 — SECS-II: items, lists, format codes, message bodies │
└──────────────────────────────────────────────────────────────┘
▲ carried over
│
┌──────────────────────────────────────────────────────────────┐
│ E37 — HSMS (TCP) │ E4 — SECS-I (RS-232 / RS-422) │
└──────────────────────────────────────────────────────────────┘
```
Read top-to-bottom: a GEM 300 chapter (say, E40 process jobs)
*reasons* about lifecycle states and emits SECS-II messages defined
by E5, which travel over an HSMS connection defined by E37. The
codebase mirrors that layering: `secsgem::gem` (top) sits on
`secsgem::secs2` (codec) which is moved by `secsgem::hsms` or
`secsgem::secsi` (bottom).
---
## The series
Twenty-four chapters in five parts. Read linearly the first time;
on later visits, jump to whatever section answers your question.
### Part 1 — Foundations
You can read these without ever opening a code file.
| # | Title | What you'll know after |
|---|-------|------------------------|
| [01](01_what_is_secs_gem.md) | What is SECS/GEM? | Why a fab needs a protocol at all; the SEMI standards body; the one-paragraph history from SECS-I to GEM 300. |
| [02](02_the_cast.md) | The cast of characters | Equipment vs. host vs. MES vs. scheduler vs. AMHS — who talks to whom and why. |
| [03](03_vocabulary_and_a_wafers_journey.md) | Vocabulary + a wafer's journey | Every acronym you'll meet (SVID, CEID, ALID, PPID, ALCD, HCACK, …) traced through one wafer moving end-to-end through a fab. |
### Part 2 — The standards in detail
Each chapter takes one SEMI standard (or a tight family), explains
what problem it solves, walks through the on-the-wire messages, and
shows where the spec is implemented in this codebase. Hexdumps and
state diagrams included.
| # | Title | Covers |
|---|-------|--------|
| [10](10_e5_secs_ii_data_items.md) | **E5** — SECS-II data items | The 14 format codes, length-byte arithmetic, lists, the variant `Item` type, encode/decode. |
| [11](11_e37_hsms.md) | **E37** — HSMS transport | TCP framing, the SELECT handshake, T1–T8 timers, HSMS-SS vs. HSMS-GS, S9 error replies. |
| [12](12_e4_secs_i.md) | **E4** — SECS-I serial | The original RS-232 transport; ENQ/EOT/ACK/NAK, blocking, T1/T2/T3/T4, why it still matters. |
| [13](13_e30_gem.md) | **E30** — GEM behaviour | Communication state, control state, GEM Fundamentals + Additionals, scenarios, host commands. |
| [14](14_e40_e94_jobs.md) | **E40 + E94** — process and control jobs | The PJ and CJ lifecycles, S16/S14 messages, the start-stop dance, cascading state. |
| [15](15_e87_carriers.md) | **E87** — carriers and load ports | FOUPs, load-port states, slot maps, carrier transfer, ProceedWithCarrier, CancelCarrier. |
| [16](16_e90_e157_substrates_modules.md) | **E90 + E157** — substrate and module tracking | Per-wafer state, process-module state, the relationship between PJ ↔ substrate ↔ module. |
| [17](17_e116_e120_e39_objects.md) | **E116 + E120 + E39** — performance, CEM, objects | Equipment Performance Tracking time-buckets, Common Equipment Model, object-services GetAttr/SetAttr. |
| [18](18_e84_parallel_io.md) | **E84** — parallel I/O handoff | The 8-line AMHS handshake, TA1/TA2/TA3 timers, why a robot doesn't drop a $20k FOUP. |
| [19](19_e42_e148_s9_misc.md) | **E42 + E148 + S9** — enhanced PPs, time sync, exceptions | Formatted process programs, distributed clock, S5F9–F18 exception recovery, the auto-S9 paths. |
### Part 3 — The codebase
Now we open the source. Every chapter is a guided walk through a
specific namespace, with the call graphs, ownership rules, and
extension points spelled out.
| # | Title | Covers |
|---|-------|--------|
| [30](30_repository_tour.md) | Repository tour | Directory layout, build system, the eight apps, the test suite. |
| [31](31_spec_as_data_and_codegen.md) | Spec-as-data + codegen | The five YAML files, how `tools/generate_messages.py` turns `messages.yaml` into typed C++. |
| [32](32_stores_and_the_data_model.md) | Stores + the data model | `EquipmentDataModel`, every per-domain store (SVIDs, alarms, carriers, substrates, …) and how they compose. |
| [33](33_transport.md) | Transport | `hsms::Connection` (asio TCP) and `secsi::Protocol` (FSM-only); the strand-threading contract; T-timer wiring. |
| [34](34_codec_and_sml.md) | Codec + SML | `secs2::Item` variant, `encode`/`decode`, the SML parser and printer, the identifier-wildcard rule. |
| [35](35_state_machines_and_dispatch.md) | State machines + dispatch | Control / PJ / CJ / EPT / E84 FSMs, the YAML-driven `ControlTransitionTable`, `gem::Router`. |
| [36](36_persistence_validation_metrics.md) | Persistence + validation + metrics | Per-store journals, multi-version reads, `--validate-config`, the Prometheus exporter. |
### Part 4 — Operations
Reading the code teaches you what it does; this section teaches you
how to run it.
| # | Title | Covers |
|---|-------|--------|
| [40](40_building_running_demo.md) | Building, running, the demo | Docker setup, the two-container demo, every transaction it walks through. |
| [41](41_integration_hardware_mes_production.md) | Integration | Wiring sensors and recipes, talking to a real MES, HSMS-GS for multi-MES, persistence layout, monitoring, security hardening, performance envelope. |
### Part 5 — Reference
Look-up material rather than narrative.
| # | Title | Covers |
|---|-------|--------|
| [50](50_api_messages_yaml_reference.md) | API + message catalog + YAML schemas | Every namespace, every message in `data/messages.yaml`, every YAML key the config loader recognises. |
| [51](51_extending_the_codebase.md) | Extending the codebase | How to add a new SVID, host command, state, message, store, FSM, or persistence backend — the actual mechanical steps. |
---
## How to read this guide
Pick a path based on what you're trying to do.
**"I'm new to SECS/GEM and to this codebase."**
Read Parts 1, 2, 3 in order. Skim Part 4 to know what's there. Use
Part 5 as reference. Budget: a long afternoon.
**"I know SECS/GEM, I just need to learn this codebase."**
Skim Part 1.03 for vocabulary, skip Part 2, read Part 3 in full,
then Part 4. Budget: 2 hours.
**"I'm new to SECS/GEM but only need to consume what this tool
emits."**
Read Parts 1, 2. Skip Parts 3, 4, 5. Budget: 3 hours.
**"I'm integrating a real tool right now and need answers fast."**
Read Part 4 chapter 41; cross-reference Part 5 chapter 51 for each
new behaviour you have to add. Budget: as long as the integration
takes.
**"I'm auditing for compliance / signing off on a deployment."**
Read [`../COMPLIANCE.md`](../COMPLIANCE.md) first. Then read each
Part 2 chapter for the standards in scope. Cross-check the code
references against [`../PROOFS.md`](../PROOFS.md).
---
## Conventions used throughout
**File references** look like `src/secs2/codec.cpp:123` — a path
relative to the repo root, optional line number after a colon. When
a function or symbol is the target, the form is `namespace::Symbol`
followed by the file where it lives.
**Wire dumps** are shown in two forms — annotated SML (the
human-readable SECS-II) on the left, raw hex on the right:
```
S1F1 W │ 00 00 00 0A length prefix
. │ 00 00 session_id
│ 81 01 S=1, W=1, F=1
│ 00 00 PType/SType (data)
│ 00 00 00 01 system_bytes
```
**Diagrams** use the box-drawing characters above. No Mermaid — the
repo's render targets (Gitea, GitHub, plain text) all handle the
box-drawing characters uniformly.
**Cross-references**: chapter X.YZ refers to Part X, chapter YZ.
E.g. "see 3.32 §3" means Part 3, chapter 32, section 3.
**Spec citations** look like `E30 §6.5` — SEMI standard E30,
section 6.5. The standards themselves are paywalled and *not*
included in this repo. This guide is written to be readable without
them; the section numbers are there so a reader who *does* have
access can cross-check.
---
## What this guide is not
- **Not a substitute for the SEMI standards** if you're certifying
for production. We aim for accuracy, but if you're shipping into
a fab, buy the official PDFs.
- **Not a GEM RTS run.** [`../COMPLIANCE.md`](../COMPLIANCE.md) §8
explains the difference between "spec-implementing codebase" and
"third-party-certified compliant equipment."
- **Not a replacement for `../PROOFS.md`.** The proof table is the
empirical claim; this guide is the explanatory text.
---
## Where the rest of the docs live
Existing root-level docs are reference / audit artifacts. This guide
is the *tutorial path* that ties them together.
| Root doc | What it is | When to read |
|-----------------------------------------|-------------------------------------------------------------|--------------|
| [`../README.md`](../README.md) | One-page project summary + quick start | First contact |
| [`../PROOFS.md`](../PROOFS.md) | The eight commands that prove feature-completeness | Verifying claims |
| [`../COMPLIANCE.md`](../COMPLIANCE.md) | Per-capability audit against every SEMI standard | Compliance review |
| [`../ARCHITECTURE.md`](../ARCHITECTURE.md) | One-page architecture overview | Quick mental model |
| [`../INTEGRATION.md`](../INTEGRATION.md) | Vendor-side integration tutorial | Going to production |
| [`../VERIFICATION.md`](../VERIFICATION.md) | External validator test plan | Verification deep dive |
| [`../BENCHMARKS.md`](../BENCHMARKS.md) | Performance envelope | Capacity planning |
| [`../MES_INTEROP.md`](../MES_INTEROP.md) | Day-1 punch list for commercial MES integration | Pre-flight before MES connect |
| [`../SECURITY.md`](../SECURITY.md) | Concrete nftables / stunnel / minisign / SIEM configs | Production hardening |
| [`../GLOSSARY.md`](../GLOSSARY.md) | SEMI vocabulary cheat sheet | Quick term lookup |
| [`../FAQ.md`](../FAQ.md) | Common questions, canonical answers | Stuck? Check here first |
| [`../examples/pvd_tool/`](../examples/pvd_tool/) | A complete fictional PVD tool — YAML + main.cpp | Concrete reference deployment |
---
## Status of this guide
Chapters publish as they're written. The list above is the table of
contents; individual files exist once the chapter has been written.
A chapter without a working link is on the to-write list.
**Currently published:** Chapter 00 (this index).
**In progress:** Chapter 01 — *What is SECS/GEM?*
Next chapter: [→ 01 What is SECS/GEM?](01_what_is_secs_gem.md)
docs/01_what_is_secs_gem.md
+328
View File
@@ -0,0 +1,328 @@
# 01 — What is SECS/GEM?
← [Back to index](00_index.md) | Next: [02 The cast of characters](02_the_cast.md) →
A semiconductor fabrication plant — a **fab** — is one of the most
automation-dense environments on Earth. A 300 mm wafer fab runs
**50 to 200 tools** simultaneously, each manufactured by a different
vendor (Applied Materials, Tokyo Electron, Lam Research, ASML, KLA,
Hitachi, dozens more), each performing one step in a recipe that
takes 8–12 weeks and 500–1500 steps to turn a bare silicon wafer
into a finished chip.
Every one of those tools needs to talk to a central computer — the
**Manufacturing Execution System** (MES) — to receive recipe
instructions, report progress, surface alarms, and prove every wafer
ended up where it was supposed to.
SECS/GEM is the protocol they use to do that.
This chapter explains why the protocol exists, what its parts are
called, and the one-screen history from the original 1980s standard
to the modern GEM 300 suite that this codebase implements.
---
## The N × M problem
Imagine a fab with 100 tools and 1 MES. Without a standard:
- Every **tool vendor** has to write a custom integration for every
**MES vendor** they want to ship into.
- Every **MES vendor** has to write a custom driver for every **tool
vendor**'s API.
- Every fab has to negotiate, test, and maintain N × M integration
pairs — and a 100-tool fab with even 2 MES generations in flight
is suddenly looking at 200 distinct integrations.
- When a tool gets a firmware update, every MES integration breaks.
This is the **N × M integration problem**. It's the same problem
that USB solved for peripherals, that TCP/IP solved for networking,
that POSIX solved for system calls.
The semiconductor industry's answer is a family of standards
published by **SEMI** (Semiconductor Equipment and Materials
International), the trade body for the industry. The relevant ones
have names like **E4**, **E5**, **E30**, **E37**, **E40**, **E84**,
**E87**, **E90**, **E94** and a dozen more. Together they describe:
1. **What bytes go on the wire** (the protocol stack).
2. **What those bytes mean** (the message catalog).
3. **What the equipment must DO when it receives them** (the
behavioural contract).
Once a tool implements those standards, *any* MES can drive it.
Once an MES implements them, *any* tool will respond. N × M
collapses to N + M.
---
## The three names you'll keep seeing
You will see the abbreviations **SECS**, **HSMS**, and **GEM** in
every diagram and every doc. They mean three different things, and
people use them sloppily. Pin them down once:
### SECS — Semiconductor Equipment Communications Standard
SECS is the **message layer**. It defines:
- The set of named messages (`S1F1`, `S1F3`, `S6F11`, `S16F11`, …).
- The bytes that encode each message (format codes, length bytes,
body structure).
- The conversational pattern (every primary message either has a
reply or is explicitly fire-and-forget).
There are two SECS specs you'll encounter, and they do different
things:
- **SECS-I (E4)** — the **transport** for SECS messages over
RS-232 / RS-422 serial cables. Defined in 1980. Still used on
older 200 mm fabs and on smaller specialty tools. Block-oriented,
half-duplex, ENQ/EOT/ACK/NAK style.
- **SECS-II (E5)** — the **message structure** itself, independent of
transport. Defined in 1982. A message is a list of *items*,
where each item has a SEMI-defined format code (U1, U2, F4, ASCII,
List, …). Every modern SECS-based protocol still uses E5 for
message encoding.
You'll often see "**SECS**" used loosely as a synonym for **SECS-II**
(the message structure). When someone says "a SECS message," they
mean an E5-encoded message; the transport (E4 or HSMS) is a separate
concern.
### HSMS — High-Speed SECS Message Services
HSMS is the **modern replacement for SECS-I** as a transport.
Defined as **E37** in 1995, it carries SECS-II messages over a TCP/IP
connection instead of a serial cable.
If you set up a SECS-II message and want to send it to a 21st-century
tool, you send it over HSMS, not SECS-I.
HSMS has its own framing (4-byte length prefix + 10-byte header +
SECS-II body) and its own connection state machine (NOT-CONNECTED →
NOT-SELECTED → SELECTED) and its own timers (T3 reply, T5 separation,
T6 control transaction, T7 not-selected, T8 inter-character).
Chapter [11](11_e37_hsms.md) covers it in detail.
HSMS comes in two flavours:
- **HSMS-SS** (Single-Session) — one TCP socket carries one SECS
conversation between one equipment and one host. This is the
default.
- **HSMS-GS** (General-Session) — one TCP socket multiplexes
*multiple* sessions, identified by a session ID in each frame's
header. Used in fabs where one piece of equipment must talk to
several MES servers (production, maintenance, engineering) over
the same physical link.
### GEM — Generic Equipment Model
GEM, defined as **E30** in 1992, is the **behavioural layer** on top
of SECS-II. It answers questions like:
- When a host sends `S1F13 Establish Communications`, what state must
the equipment enter?
- When the operator presses the **Online** button, which messages
fire to the host?
- When an alarm becomes active, must the equipment send `S5F1`?
Under what conditions can the host suppress it?
- What does it mean for equipment to be in the `EquipmentOffline`
state vs. `OnlineRemote`?
E30 spells out:
- The **communication state machine** (DISABLED, WAIT-CRA,
WAIT-DELAY, COMMUNICATING) that runs above HSMS's transport-level
states.
- The **control state machine** (Equipment Offline, Attempting
Online, Host Offline, Online, Online Local, Online Remote) that
governs who's allowed to issue commands.
- The required **scenarios** — like "Establish Communications,"
"On-Line Identification," "Event Notification," "Alarm Management"
— that every GEM-compliant tool must support.
- Two **capability tiers**: **Fundamentals** (mandatory) and
**Additionals** (optional but very commonly required by MES
procurement).
A tool that obeys E30 is called **GEM-compliant** and can be
integrated by *any* MES that speaks GEM without custom code.
### GEM 300 — the 300 mm wafer suite
When fabs migrated from 200 mm to 300 mm wafers around 2000, the
extra automation (robot-driven wafer handling, no human touching a
substrate) needed new behavioural contracts. GEM 300 is the
collective name for:
| Standard | Year | What it adds |
|----------|------|---------------------------------------------------------------|
| **E39** | 1999 | Object Services — generic CRUD over typed equipment objects |
| **E40** | 1999 | Process Job management — submit, track, cancel a wafer process |
| **E84** | 2000 | Parallel I/O — the 8-line AMHS robot-to-tool handshake |
| **E87** | 2000 | Carrier Management — FOUPs and load ports |
| **E90** | 2000 | Substrate Tracking — per-wafer location and state |
| **E94** | 2001 | Control Job management — scheduling of multiple process jobs |
| **E116** | 2003 | Equipment Performance Tracking — time-buckets per state |
| **E120** | 2003 | Common Equipment Model — generic object hierarchy |
| **E148** | 2005 | Time Synchronization — distributed clock |
| **E157** | 2006 | Module Process Tracking — per-process-module state |
| **E42** | 2004 | Formatted Process Programs — typed recipe payloads |
Every modern 300 mm tool ships with all of these. This codebase
implements all of them. Chapters [14–19](14_e40_e94_jobs.md) cover
them one family at a time.
---
## Why it's structured this way
The layering — transport → message → behaviour — is **deliberate
and load-bearing**, and the codebase mirrors it exactly.
```
Behavioural contract E30 (GEM) + GEM 300 suite
(what equipment must DO) secsgem::gem
──────────────
│
│ emits / receives
▼
Message structure E5 (SECS-II)
(what the bytes mean) secsgem::secs2
──────────────
│
│ encoded over
▼
Transport E37 (HSMS, TCP/IP) E4 (SECS-I, serial)
(how bytes get there) secsgem::hsms secsgem::secsi
────────────── ──────────────
```
The separation matters because:
1. **You can swap transports.** The same SECS-II message and the
same E30 behaviour work whether the bytes travel over HSMS-SS,
HSMS-GS, or SECS-I. In this codebase, `gem::Router` doesn't know
which transport delivered the bytes — it just sees a decoded
`secs2::Message`.
2. **You can evolve layers independently.** E5's format codes
haven't changed in 40 years; HSMS replaced SECS-I as the
transport in 1995; GEM 300 added new behaviour without disturbing
E5 or E37. The layers ship at different cadences and the spec
only had to evolve the layers that needed to change.
3. **You can test each layer in isolation.** This codebase has
139 tests for the E5 codec alone, 34 for HSMS, 27 for SECS-I,
71 for E30 behaviour, and dozens per GEM 300 standard. None of
the codec tests need a transport; none of the transport tests
need a behaviour. See [PROOFS.md](../PROOFS.md) for the
per-standard test counts.
---
## Where the layers live in this codebase
| Layer | Standard | Namespace | Headers | Tests |
|------------------|----------------|--------------------|--------------------------------------------------------|----------------------------------------|
| Behavioural | E30 + GEM 300 | `secsgem::gem` | `include/secsgem/gem/*.hpp` | `tests/test_control_state.cpp`, `tests/test_communication_state.cpp`, `tests/test_data_model.cpp`, and one file per GEM 300 standard |
| Messages | E5 | `secsgem::secs2` | `include/secsgem/secs2/{item,codec,message,sml}.hpp` | `tests/test_secs2.cpp`, `tests/test_e5_kat.cpp`, `tests/test_sml.cpp`, `tests/test_messages.cpp` |
| Transport (TCP) | E37 | `secsgem::hsms` | `include/secsgem/hsms/{frame,header,connection}.hpp` | `tests/test_hsms.cpp`, `tests/test_hsms_connection.cpp`, `tests/test_hsms_timers.cpp`, `tests/test_hsms_gs.cpp`, `tests/test_hsms_s9.cpp` |
| Transport (ser.) | E4 | `secsgem::secsi` | `include/secsgem/secsi/{header,block,protocol,tcp_transport}.hpp` | `tests/test_secsi.cpp`, `tests/test_secsi_timers.cpp`, `tests/test_secsi_tcp.cpp` |
| Catalog (codegen)| E5 + GEM | `secsgem::gem` | `build/generated/secsgem/gem/messages.hpp` | `tests/test_messages.cpp` |
Read it top-to-bottom: the behavioural layer (`gem`) uses the message
layer (`secs2`) which is moved by the transport layer (`hsms` or
`secsi`). Each row has its own chapter in Parts 2 and 3.
The **codegen** row is worth a footnote: SECS-II has ~160 named
messages and each one has a typed struct body. Writing all 160
builders + parsers by hand would be 5000+ lines of boilerplate, so
`tools/generate_messages.py` reads `data/messages.yaml` at build time
and emits `messages.hpp` with one typed struct + builder + parser per
message. Chapter [31](31_spec_as_data_and_codegen.md) walks through
how it works.
---
## One example, end-to-end
Just so the abstractions feel less abstract: here's what happens when
an MES asks an equipment "what time is it?"
1. **MES (the host)** wants to read the equipment's clock. In SECS
terms that's stream 2, function 17 — `S2F17` — defined in E30
§6.20 as part of the Clock capability.
2. The MES encodes a `S2F17` request. The E5 body is empty (an
empty `List`), so the SECS-II encoding is just the format byte +
length: `01 00` (format=0=List, length-byte-count=1, length=0).
3. The MES wraps the SECS-II body in an HSMS frame: 4-byte length
prefix + 10-byte header (`session_id`, `byte2`, `byte3`, `PType`,
`SType`, `system_bytes`) + body. The W-bit in the header is set
to 1 because S2F17 expects a reply.
4. The HSMS frame travels over TCP to the equipment.
5. The equipment's `hsms::Connection` reads the 4-byte length,
reads the 10-byte header, reads the body, and dispatches the
decoded `secs2::Message` to `gem::Router`.
6. `gem::Router` looks up the registered handler for `(stream=2,
function=17)` and calls it.
7. The handler reads the equipment's clock — say, `2026-06-09 19:30:00.42`
— formats it as a 16-char ASCII string `"2026060919300042"` per E30
§6.20, builds a `S2F18` reply with the string as its only item,
and hands it back to `gem::Router`.
8. The router asks `hsms::Connection` to send the reply. The same
layers run in reverse: SECS-II encoding (`A[16] '2026...'` →
`41 10 32 30 32 36 …`), HSMS framing, TCP send.
9. The MES decodes the reply, reads the timestamp, displays it on a
dashboard.
That's one transaction. A 300 mm fab tool exchanges **hundreds to
thousands of these per minute** during normal operation —
status polls, event reports, recipe management, job tracking, alarm
notifications, terminal messages. Every one of them flows through
exactly that stack. All the rest of this guide is filling in the
detail of each layer.
---
## A brief history (one paragraph)
- **1980** — SECS-I (E4) published. RS-232 framing, intended for
early 200 mm and pre-200 mm tools.
- **1982** — SECS-II (E5) published. Standardised the message
*structure* so it could outlive any one transport.
- **1992** — GEM (E30) published. Standardised the *behaviour* on
top of SECS-II. Made the message layer useful by giving every
message a defined role.
- **1995** — HSMS (E37) published. Replaced RS-232 with TCP/IP
while keeping E5 + E30 unchanged.
- **1999–2006** — GEM 300 suite published one standard at a time
(E39, E40, E84, E87, E90, E94, E116, E120, E148, E157, E42), adding
the behaviour needed for 300 mm wafer automation.
- **Today** — every modern 300 mm tool ships with E5 + E37 + E30 +
the GEM 300 suite. This codebase implements all of them.
---
## What's next
You now know:
- Why a fab needs a protocol at all.
- The three names — SECS, HSMS, GEM — and what each one actually
refers to.
- How the standards stack into transport → message → behaviour.
- Where each layer lives in this codebase.
The next chapter introduces the **cast of characters** — equipment,
host, MES, scheduler, AMHS — and shows who talks to whom in a typical
fab.
Next: [→ 02 The cast of characters](02_the_cast.md)