secs-gem/docs/01_what_is_secs_gem.md

# 01 — What is SECS/GEM?

← [Back to index](00_index.md) | Next: [02 The cast of characters](02_the_cast.md) →

A semiconductor fabrication plant — a **fab** — is one of the most
automation-dense environments on Earth.  A 300 mm wafer fab runs
**50 to 200 tools** simultaneously, each manufactured by a different
vendor (Applied Materials, Tokyo Electron, Lam Research, ASML, KLA,
Hitachi, dozens more), each performing one step in a recipe that
takes 8–12 weeks and 500–1500 steps to turn a bare silicon wafer
into a finished chip.

Every one of those tools needs to talk to a central computer — the
**Manufacturing Execution System** (MES) — to receive recipe
instructions, report progress, surface alarms, and prove every wafer
ended up where it was supposed to.

SECS/GEM is the protocol they use to do that.

This chapter explains why the protocol exists, what its parts are
called, and the one-screen history from the original 1980s standard
to the modern GEM 300 suite that this codebase implements.

---

## The N × M problem

Imagine a fab with 100 tools and 1 MES.  Without a standard:

- Every **tool vendor** has to write a custom integration for every
  **MES vendor** they want to ship into.
- Every **MES vendor** has to write a custom driver for every **tool
  vendor**'s API.
- Every fab has to negotiate, test, and maintain N × M integration
  pairs — and a 100-tool fab with even 2 MES generations in flight
  is suddenly looking at 200 distinct integrations.
- When a tool gets a firmware update, every MES integration breaks.

This is the **N × M integration problem**.  It's the same problem
that USB solved for peripherals, that TCP/IP solved for networking,
that POSIX solved for system calls.

The semiconductor industry's answer is a family of standards
published by **SEMI** (Semiconductor Equipment and Materials
International), the trade body for the industry.  The relevant ones
have names like **E4**, **E5**, **E30**, **E37**, **E40**, **E84**,
**E87**, **E90**, **E94** and a dozen more.  Together they describe:

1. **What bytes go on the wire** (the protocol stack).
2. **What those bytes mean** (the message catalog).
3. **What the equipment must DO when it receives them** (the
   behavioural contract).

Once a tool implements those standards, *any* MES can drive it.
Once an MES implements them, *any* tool will respond.  N × M
collapses to N + M.

---

## The three names you'll keep seeing

You will see the abbreviations **SECS**, **HSMS**, and **GEM** in
every diagram and every doc.  They mean three different things, and
people use them sloppily.  Pin them down once:

### SECS — Semiconductor Equipment Communications Standard

SECS is the **message layer**.  It defines:

- The set of named messages (`S1F1`, `S1F3`, `S6F11`, `S16F11`, …).
- The bytes that encode each message (format codes, length bytes,
  body structure).
- The conversational pattern (every primary message either has a
  reply or is explicitly fire-and-forget).

There are two SECS specs you'll encounter, and they do different
things:

- **SECS-I (E4)** — the **transport** for SECS messages over
  RS-232 / RS-422 serial cables.  Defined in 1980.  Still used on
  older 200 mm fabs and on smaller specialty tools.  Block-oriented,
  half-duplex, ENQ/EOT/ACK/NAK style.
- **SECS-II (E5)** — the **message structure** itself, independent of
  transport.  Defined in 1982.  A message is a list of *items*,
  where each item has a SEMI-defined format code (U1, U2, F4, ASCII,
  List, …).  Every modern SECS-based protocol still uses E5 for
  message encoding.

You'll often see "**SECS**" used loosely as a synonym for **SECS-II**
(the message structure).  When someone says "a SECS message," they
mean an E5-encoded message; the transport (E4 or HSMS) is a separate
concern.

### HSMS — High-Speed SECS Message Services

HSMS is the **modern replacement for SECS-I** as a transport.
Defined as **E37** in 1995, it carries SECS-II messages over a TCP/IP
connection instead of a serial cable.

If you set up a SECS-II message and want to send it to a 21st-century
tool, you send it over HSMS, not SECS-I.

HSMS has its own framing (4-byte length prefix + 10-byte header +
SECS-II body) and its own connection state machine (NOT-CONNECTED →
NOT-SELECTED → SELECTED) and its own timers (T3 reply, T5 separation,
T6 control transaction, T7 not-selected, T8 inter-character).
Chapter [11](11_e37_hsms.md) covers it in detail.

HSMS comes in two flavours:

- **HSMS-SS** (Single-Session) — one TCP socket carries one SECS
  conversation between one equipment and one host.  This is the
  default.
- **HSMS-GS** (General-Session) — one TCP socket multiplexes
  *multiple* sessions, identified by a session ID in each frame's
  header.  Used in fabs where one piece of equipment must talk to
  several MES servers (production, maintenance, engineering) over
  the same physical link.

### GEM — Generic Equipment Model

GEM, defined as **E30** in 1992, is the **behavioural layer** on top
of SECS-II.  It answers questions like:

- When a host sends `S1F13 Establish Communications`, what state must
  the equipment enter?
- When the operator presses the **Online** button, which messages
  fire to the host?
- When an alarm becomes active, must the equipment send `S5F1`?
  Under what conditions can the host suppress it?
- What does it mean for equipment to be in the `EquipmentOffline`
  state vs. `OnlineRemote`?

E30 spells out:

- The **communication state machine** (DISABLED, WAIT-CRA,
  WAIT-DELAY, COMMUNICATING) that runs above HSMS's transport-level
  states.
- The **control state machine** (Equipment Offline, Attempting
  Online, Host Offline, Online, Online Local, Online Remote) that
  governs who's allowed to issue commands.
- The required **scenarios** — like "Establish Communications,"
  "On-Line Identification," "Event Notification," "Alarm Management"
  — that every GEM-compliant tool must support.
- Two **capability tiers**: **Fundamentals** (mandatory) and
  **Additionals** (optional but very commonly required by MES
  procurement).

A tool that obeys E30 is called **GEM-compliant** and can be
integrated by *any* MES that speaks GEM without custom code.

### GEM 300 — the 300 mm wafer suite

When fabs migrated from 200 mm to 300 mm wafers around 2000, the
extra automation (robot-driven wafer handling, no human touching a
substrate) needed new behavioural contracts.  GEM 300 is the
collective name for:

| Standard | Year | What it adds                                                  |
|----------|------|---------------------------------------------------------------|
| **E39**  | 1999 | Object Services — generic CRUD over typed equipment objects   |
| **E40**  | 1999 | Process Job management — submit, track, cancel a wafer process |
| **E84**  | 2000 | Parallel I/O — the 8-line AMHS robot-to-tool handshake        |
| **E87**  | 2000 | Carrier Management — FOUPs and load ports                     |
| **E90**  | 2000 | Substrate Tracking — per-wafer location and state             |
| **E94**  | 2001 | Control Job management — scheduling of multiple process jobs  |
| **E116** | 2003 | Equipment Performance Tracking — time-buckets per state       |
| **E120** | 2003 | Common Equipment Model — generic object hierarchy             |
| **E148** | 2005 | Time Synchronization — distributed clock                      |
| **E157** | 2006 | Module Process Tracking — per-process-module state            |
| **E42**  | 2004 | Formatted Process Programs — typed recipe payloads            |

Every modern 300 mm tool ships with all of these.  This codebase
implements all of them.  Chapters [14–19](14_e40_e94_jobs.md) cover
them one family at a time.

---

## Why it's structured this way

The layering — transport → message → behaviour — is **deliberate
and load-bearing**, and the codebase mirrors it exactly.

```
   Behavioural contract        E30 (GEM) + GEM 300 suite
   (what equipment must DO)    secsgem::gem
                               ──────────────
                                        │
                                        │  emits / receives
                                        ▼
   Message structure           E5 (SECS-II)
   (what the bytes mean)       secsgem::secs2
                               ──────────────
                                        │
                                        │  encoded over
                                        ▼
   Transport                   E37 (HSMS, TCP/IP)        E4 (SECS-I, serial)
   (how bytes get there)       secsgem::hsms             secsgem::secsi
                               ──────────────            ──────────────
```

The separation matters because:

1. **You can swap transports.**  The same SECS-II message and the
   same E30 behaviour work whether the bytes travel over HSMS-SS,
   HSMS-GS, or SECS-I.  In this codebase, `gem::Router` doesn't know
   which transport delivered the bytes — it just sees a decoded
   `secs2::Message`.
2. **You can evolve layers independently.**  E5's format codes
   haven't changed in 40 years; HSMS replaced SECS-I as the
   transport in 1995; GEM 300 added new behaviour without disturbing
   E5 or E37.  The layers ship at different cadences and the spec
   only had to evolve the layers that needed to change.
3. **You can test each layer in isolation.**  This codebase has
   139 tests for the E5 codec alone, 34 for HSMS, 27 for SECS-I,
   71 for E30 behaviour, and dozens per GEM 300 standard.  None of
   the codec tests need a transport; none of the transport tests
   need a behaviour.  See [PROOFS.md](PROOFS.md) for the
   per-standard test counts.

---

## Where the layers live in this codebase

| Layer            | Standard       | Namespace          | Headers                                                | Tests                                  |
|------------------|----------------|--------------------|--------------------------------------------------------|----------------------------------------|
| Behavioural      | E30 + GEM 300  | `secsgem::gem`     | `include/secsgem/gem/*.hpp`                            | `tests/test_control_state.cpp`, `tests/test_communication_state.cpp`, `tests/test_data_model.cpp`, and one file per GEM 300 standard |
| Messages         | E5             | `secsgem::secs2`   | `include/secsgem/secs2/{item,codec,message,sml}.hpp`   | `tests/test_secs2.cpp`, `tests/test_e5_kat.cpp`, `tests/test_sml.cpp`, `tests/test_messages.cpp` |
| Transport (TCP)  | E37            | `secsgem::hsms`    | `include/secsgem/hsms/{frame,header,connection}.hpp`   | `tests/test_hsms.cpp`, `tests/test_hsms_connection.cpp`, `tests/test_hsms_timers.cpp`, `tests/test_hsms_gs.cpp`, `tests/test_hsms_s9.cpp` |
| Transport (ser.) | E4             | `secsgem::secsi`   | `include/secsgem/secsi/{header,block,protocol,tcp_transport}.hpp` | `tests/test_secsi.cpp`, `tests/test_secsi_timers.cpp`, `tests/test_secsi_tcp.cpp` |
| Catalog (codegen)| E5 + GEM       | `secsgem::gem`     | `build/generated/secsgem/gem/messages.hpp`              | `tests/test_messages.cpp`               |

Read it top-to-bottom: the behavioural layer (`gem`) uses the message
layer (`secs2`) which is moved by the transport layer (`hsms` or
`secsi`).  Each row has its own chapter in Parts 2 and 3.

The **codegen** row is worth a footnote: SECS-II has ~160 named
messages and each one has a typed struct body.  Writing all 160
builders + parsers by hand would be 5000+ lines of boilerplate, so
`tools/generate_messages.py` reads `data/messages.yaml` at build time
and emits `messages.hpp` with one typed struct + builder + parser per
message.  Chapter [31](31_spec_as_data_and_codegen.md) walks through
how it works.

---

## One example, end-to-end

Just so the abstractions feel less abstract: here's what happens when
an MES asks an equipment "what time is it?"

1. **MES (the host)** wants to read the equipment's clock.  In SECS
   terms that's stream 2, function 17 — `S2F17` — defined in E30
   §6.20 as part of the Clock capability.

2. The MES encodes a `S2F17` request.  The E5 body is empty (an
   empty `List`), so the SECS-II encoding is just the format byte +
   length: `01 00` (format=0=List, length-byte-count=1, length=0).

3. The MES wraps the SECS-II body in an HSMS frame: 4-byte length
   prefix + 10-byte header (`session_id`, `byte2`, `byte3`, `PType`,
   `SType`, `system_bytes`) + body.  The W-bit in the header is set
   to 1 because S2F17 expects a reply.

4. The HSMS frame travels over TCP to the equipment.

5. The equipment's `hsms::Connection` reads the 4-byte length,
   reads the 10-byte header, reads the body, and dispatches the
   decoded `secs2::Message` to `gem::Router`.

6. `gem::Router` looks up the registered handler for `(stream=2,
   function=17)` and calls it.

7. The handler reads the equipment's clock — say, `2026-06-09 19:30:00.42`
   — formats it as a 16-char ASCII string `"2026060919300042"` per E30
   §6.20, builds a `S2F18` reply with the string as its only item,
   and hands it back to `gem::Router`.

8. The router asks `hsms::Connection` to send the reply.  The same
   layers run in reverse: SECS-II encoding (`A[16] '2026...'` →
   `41 10 32 30 32 36 …`), HSMS framing, TCP send.

9. The MES decodes the reply, reads the timestamp, displays it on a
   dashboard.

That's one transaction.  A 300 mm fab tool exchanges **hundreds to
thousands of these per minute** during normal operation —
status polls, event reports, recipe management, job tracking, alarm
notifications, terminal messages.  Every one of them flows through
exactly that stack.  All the rest of this guide is filling in the
detail of each layer.

---

## A brief history (one paragraph)

- **1980** — SECS-I (E4) published.  RS-232 framing, intended for
  early 200 mm and pre-200 mm tools.
- **1982** — SECS-II (E5) published.  Standardised the message
  *structure* so it could outlive any one transport.
- **1992** — GEM (E30) published.  Standardised the *behaviour* on
  top of SECS-II.  Made the message layer useful by giving every
  message a defined role.
- **1995** — HSMS (E37) published.  Replaced RS-232 with TCP/IP
  while keeping E5 + E30 unchanged.
- **1999–2006** — GEM 300 suite published one standard at a time
  (E39, E40, E84, E87, E90, E94, E116, E120, E148, E157, E42), adding
  the behaviour needed for 300 mm wafer automation.
- **Today** — every modern 300 mm tool ships with E5 + E37 + E30 +
  the GEM 300 suite.  This codebase implements all of them.

---

## What's next

You now know:

- Why a fab needs a protocol at all.
- The three names — SECS, HSMS, GEM — and what each one actually
  refers to.
- How the standards stack into transport → message → behaviour.
- Where each layer lives in this codebase.

The next chapter introduces the **cast of characters** — equipment,
host, MES, scheduler, AMHS — and shows who talks to whom in a typical
fab.

Next: [→ 02 The cast of characters](02_the_cast.md)