raphael
0355c73211
generate_messages.py → gen_messages.py and several gem/ headers moved under gem/store/ (carrier_store.hpp → store/carriers.hpp, etc.); e84.hpp split into e84_state.hpp. The guided-tour chapters still pointed at the old paths — relink them so the deep-link footnotes resolve. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
329 lines
15 KiB
Markdown
329 lines
15 KiB
Markdown
# 01 — What is SECS/GEM?
|
||
|
||
← [Back to index](00_index.md) | Next: [02 The cast of characters](02_the_cast.md) →
|
||
|
||
A semiconductor fabrication plant — a **fab** — is one of the most
|
||
automation-dense environments on Earth. A 300 mm wafer fab runs
|
||
**50 to 200 tools** simultaneously, each manufactured by a different
|
||
vendor (Applied Materials, Tokyo Electron, Lam Research, ASML, KLA,
|
||
Hitachi, dozens more), each performing one step in a recipe that
|
||
takes 8–12 weeks and 500–1500 steps to turn a bare silicon wafer
|
||
into a finished chip.
|
||
|
||
Every one of those tools needs to talk to a central computer — the
|
||
**Manufacturing Execution System** (MES) — to receive recipe
|
||
instructions, report progress, surface alarms, and prove every wafer
|
||
ended up where it was supposed to.
|
||
|
||
SECS/GEM is the protocol they use to do that.
|
||
|
||
This chapter explains why the protocol exists, what its parts are
|
||
called, and the one-screen history from the original 1980s standard
|
||
to the modern GEM 300 suite that this codebase implements.
|
||
|
||
---
|
||
|
||
## The N × M problem
|
||
|
||
Imagine a fab with 100 tools and 1 MES. Without a standard:
|
||
|
||
- Every **tool vendor** has to write a custom integration for every
|
||
**MES vendor** they want to ship into.
|
||
- Every **MES vendor** has to write a custom driver for every **tool
|
||
vendor**'s API.
|
||
- Every fab has to negotiate, test, and maintain N × M integration
|
||
pairs — and a 100-tool fab with even 2 MES generations in flight
|
||
is suddenly looking at 200 distinct integrations.
|
||
- When a tool gets a firmware update, every MES integration breaks.
|
||
|
||
This is the **N × M integration problem**. It's the same problem
|
||
that USB solved for peripherals, that TCP/IP solved for networking,
|
||
that POSIX solved for system calls.
|
||
|
||
The semiconductor industry's answer is a family of standards
|
||
published by **SEMI** (Semiconductor Equipment and Materials
|
||
International), the trade body for the industry. The relevant ones
|
||
have names like **E4**, **E5**, **E30**, **E37**, **E40**, **E84**,
|
||
**E87**, **E90**, **E94** and a dozen more. Together they describe:
|
||
|
||
1. **What bytes go on the wire** (the protocol stack).
|
||
2. **What those bytes mean** (the message catalog).
|
||
3. **What the equipment must DO when it receives them** (the
|
||
behavioural contract).
|
||
|
||
Once a tool implements those standards, *any* MES can drive it.
|
||
Once an MES implements them, *any* tool will respond. N × M
|
||
collapses to N + M.
|
||
|
||
---
|
||
|
||
## The three names you'll keep seeing
|
||
|
||
You will see the abbreviations **SECS**, **HSMS**, and **GEM** in
|
||
every diagram and every doc. They mean three different things, and
|
||
people use them sloppily. Pin them down once:
|
||
|
||
### SECS — Semiconductor Equipment Communications Standard
|
||
|
||
SECS is the **message layer**. It defines:
|
||
|
||
- The set of named messages (`S1F1`, `S1F3`, `S6F11`, `S16F11`, …).
|
||
- The bytes that encode each message (format codes, length bytes,
|
||
body structure).
|
||
- The conversational pattern (every primary message either has a
|
||
reply or is explicitly fire-and-forget).
|
||
|
||
There are two SECS specs you'll encounter, and they do different
|
||
things:
|
||
|
||
- **SECS-I (E4)** — the **transport** for SECS messages over
|
||
RS-232 / RS-422 serial cables. Defined in 1980. Still used on
|
||
older 200 mm fabs and on smaller specialty tools. Block-oriented,
|
||
half-duplex, ENQ/EOT/ACK/NAK style.
|
||
- **SECS-II (E5)** — the **message structure** itself, independent of
|
||
transport. Defined in 1982. A message is a list of *items*,
|
||
where each item has a SEMI-defined format code (U1, U2, F4, ASCII,
|
||
List, …). Every modern SECS-based protocol still uses E5 for
|
||
message encoding.
|
||
|
||
You'll often see "**SECS**" used loosely as a synonym for **SECS-II**
|
||
(the message structure). When someone says "a SECS message," they
|
||
mean an E5-encoded message; the transport (E4 or HSMS) is a separate
|
||
concern.
|
||
|
||
### HSMS — High-Speed SECS Message Services
|
||
|
||
HSMS is the **modern replacement for SECS-I** as a transport.
|
||
Defined as **E37** in 1995, it carries SECS-II messages over a TCP/IP
|
||
connection instead of a serial cable.
|
||
|
||
If you set up a SECS-II message and want to send it to a 21st-century
|
||
tool, you send it over HSMS, not SECS-I.
|
||
|
||
HSMS has its own framing (4-byte length prefix + 10-byte header +
|
||
SECS-II body) and its own connection state machine (NOT-CONNECTED →
|
||
NOT-SELECTED → SELECTED) and its own timers (T3 reply, T5 separation,
|
||
T6 control transaction, T7 not-selected, T8 inter-character).
|
||
Chapter [11](11_e37_hsms.md) covers it in detail.
|
||
|
||
HSMS comes in two flavours:
|
||
|
||
- **HSMS-SS** (Single-Session) — one TCP socket carries one SECS
|
||
conversation between one equipment and one host. This is the
|
||
default.
|
||
- **HSMS-GS** (General-Session) — one TCP socket multiplexes
|
||
*multiple* sessions, identified by a session ID in each frame's
|
||
header. Used in fabs where one piece of equipment must talk to
|
||
several MES servers (production, maintenance, engineering) over
|
||
the same physical link.
|
||
|
||
### GEM — Generic Equipment Model
|
||
|
||
GEM, defined as **E30** in 1992, is the **behavioural layer** on top
|
||
of SECS-II. It answers questions like:
|
||
|
||
- When a host sends `S1F13 Establish Communications`, what state must
|
||
the equipment enter?
|
||
- When the operator presses the **Online** button, which messages
|
||
fire to the host?
|
||
- When an alarm becomes active, must the equipment send `S5F1`?
|
||
Under what conditions can the host suppress it?
|
||
- What does it mean for equipment to be in the `EquipmentOffline`
|
||
state vs. `OnlineRemote`?
|
||
|
||
E30 spells out:
|
||
|
||
- The **communication state machine** (DISABLED, WAIT-CRA,
|
||
WAIT-DELAY, COMMUNICATING) that runs above HSMS's transport-level
|
||
states.
|
||
- The **control state machine** (Equipment Offline, Attempting
|
||
Online, Host Offline, Online, Online Local, Online Remote) that
|
||
governs who's allowed to issue commands.
|
||
- The required **scenarios** — like "Establish Communications,"
|
||
"On-Line Identification," "Event Notification," "Alarm Management"
|
||
— that every GEM-compliant tool must support.
|
||
- Two **capability tiers**: **Fundamentals** (mandatory) and
|
||
**Additionals** (optional but very commonly required by MES
|
||
procurement).
|
||
|
||
A tool that obeys E30 is called **GEM-compliant** and can be
|
||
integrated by *any* MES that speaks GEM without custom code.
|
||
|
||
### GEM 300 — the 300 mm wafer suite
|
||
|
||
When fabs migrated from 200 mm to 300 mm wafers around 2000, the
|
||
extra automation (robot-driven wafer handling, no human touching a
|
||
substrate) needed new behavioural contracts. GEM 300 is the
|
||
collective name for:
|
||
|
||
| Standard | Year | What it adds |
|
||
|----------|------|---------------------------------------------------------------|
|
||
| **E39** | 1999 | Object Services — generic CRUD over typed equipment objects |
|
||
| **E40** | 1999 | Process Job management — submit, track, cancel a wafer process |
|
||
| **E84** | 2000 | Parallel I/O — the 8-line AMHS robot-to-tool handshake |
|
||
| **E87** | 2000 | Carrier Management — FOUPs and load ports |
|
||
| **E90** | 2000 | Substrate Tracking — per-wafer location and state |
|
||
| **E94** | 2001 | Control Job management — scheduling of multiple process jobs |
|
||
| **E116** | 2003 | Equipment Performance Tracking — time-buckets per state |
|
||
| **E120** | 2003 | Common Equipment Model — generic object hierarchy |
|
||
| **E148** | 2005 | Time Synchronization — distributed clock |
|
||
| **E157** | 2006 | Module Process Tracking — per-process-module state |
|
||
| **E42** | 2004 | Formatted Process Programs — typed recipe payloads |
|
||
|
||
Every modern 300 mm tool ships with all of these. This codebase
|
||
implements all of them. Chapters [14–19](14_e40_e94_jobs.md) cover
|
||
them one family at a time.
|
||
|
||
---
|
||
|
||
## Why it's structured this way
|
||
|
||
The layering — transport → message → behaviour — is **deliberate
|
||
and load-bearing**, and the codebase mirrors it exactly.
|
||
|
||
```
|
||
Behavioural contract E30 (GEM) + GEM 300 suite
|
||
(what equipment must DO) secsgem::gem
|
||
──────────────
|
||
│
|
||
│ emits / receives
|
||
▼
|
||
Message structure E5 (SECS-II)
|
||
(what the bytes mean) secsgem::secs2
|
||
──────────────
|
||
│
|
||
│ encoded over
|
||
▼
|
||
Transport E37 (HSMS, TCP/IP) E4 (SECS-I, serial)
|
||
(how bytes get there) secsgem::hsms secsgem::secsi
|
||
────────────── ──────────────
|
||
```
|
||
|
||
The separation matters because:
|
||
|
||
1. **You can swap transports.** The same SECS-II message and the
|
||
same E30 behaviour work whether the bytes travel over HSMS-SS,
|
||
HSMS-GS, or SECS-I. In this codebase, `gem::Router` doesn't know
|
||
which transport delivered the bytes — it just sees a decoded
|
||
`secs2::Message`.
|
||
2. **You can evolve layers independently.** E5's format codes
|
||
haven't changed in 40 years; HSMS replaced SECS-I as the
|
||
transport in 1995; GEM 300 added new behaviour without disturbing
|
||
E5 or E37. The layers ship at different cadences and the spec
|
||
only had to evolve the layers that needed to change.
|
||
3. **You can test each layer in isolation.** This codebase has
|
||
139 tests for the E5 codec alone, 34 for HSMS, 27 for SECS-I,
|
||
71 for E30 behaviour, and dozens per GEM 300 standard. None of
|
||
the codec tests need a transport; none of the transport tests
|
||
need a behaviour. See [PROOFS.md](PROOFS.md) for the
|
||
per-standard test counts.
|
||
|
||
---
|
||
|
||
## Where the layers live in this codebase
|
||
|
||
| Layer | Standard | Namespace | Headers | Tests |
|
||
|------------------|----------------|--------------------|--------------------------------------------------------|----------------------------------------|
|
||
| Behavioural | E30 + GEM 300 | `secsgem::gem` | `include/secsgem/gem/*.hpp` | `tests/test_control_state.cpp`, `tests/test_communication_state.cpp`, `tests/test_data_model.cpp`, and one file per GEM 300 standard |
|
||
| Messages | E5 | `secsgem::secs2` | `include/secsgem/secs2/{item,codec,message,sml}.hpp` | `tests/test_secs2.cpp`, `tests/test_e5_kat.cpp`, `tests/test_sml.cpp`, `tests/test_messages.cpp` |
|
||
| Transport (TCP) | E37 | `secsgem::hsms` | `include/secsgem/hsms/{frame,header,connection}.hpp` | `tests/test_hsms.cpp`, `tests/test_hsms_connection.cpp`, `tests/test_hsms_timers.cpp`, `tests/test_hsms_gs.cpp`, `tests/test_hsms_s9.cpp` |
|
||
| Transport (ser.) | E4 | `secsgem::secsi` | `include/secsgem/secsi/{header,block,protocol,tcp_transport}.hpp` | `tests/test_secsi.cpp`, `tests/test_secsi_timers.cpp`, `tests/test_secsi_tcp.cpp` |
|
||
| Catalog (codegen)| E5 + GEM | `secsgem::gem` | `build/generated/secsgem/gem/messages.hpp` | `tests/test_messages.cpp` |
|
||
|
||
Read it top-to-bottom: the behavioural layer (`gem`) uses the message
|
||
layer (`secs2`) which is moved by the transport layer (`hsms` or
|
||
`secsi`). Each row has its own chapter in Parts 2 and 3.
|
||
|
||
The **codegen** row is worth a footnote: SECS-II has ~160 named
|
||
messages and each one has a typed struct body. Writing all 160
|
||
builders + parsers by hand would be 5000+ lines of boilerplate, so
|
||
`tools/gen_messages.py` reads `data/messages.yaml` at build time
|
||
and emits `messages.hpp` with one typed struct + builder + parser per
|
||
message. Chapter [31](31_spec_as_data_and_codegen.md) walks through
|
||
how it works.
|
||
|
||
---
|
||
|
||
## One example, end-to-end
|
||
|
||
Just so the abstractions feel less abstract: here's what happens when
|
||
an MES asks an equipment "what time is it?"
|
||
|
||
1. **MES (the host)** wants to read the equipment's clock. In SECS
|
||
terms that's stream 2, function 17 — `S2F17` — defined in E30
|
||
§6.20 as part of the Clock capability.
|
||
|
||
2. The MES encodes a `S2F17` request. The E5 body is empty (an
|
||
empty `List`), so the SECS-II encoding is just the format byte +
|
||
length: `01 00` (format=0=List, length-byte-count=1, length=0).
|
||
|
||
3. The MES wraps the SECS-II body in an HSMS frame: 4-byte length
|
||
prefix + 10-byte header (`session_id`, `byte2`, `byte3`, `PType`,
|
||
`SType`, `system_bytes`) + body. The W-bit in the header is set
|
||
to 1 because S2F17 expects a reply.
|
||
|
||
4. The HSMS frame travels over TCP to the equipment.
|
||
|
||
5. The equipment's `hsms::Connection` reads the 4-byte length,
|
||
reads the 10-byte header, reads the body, and dispatches the
|
||
decoded `secs2::Message` to `gem::Router`.
|
||
|
||
6. `gem::Router` looks up the registered handler for `(stream=2,
|
||
function=17)` and calls it.
|
||
|
||
7. The handler reads the equipment's clock — say, `2026-06-09 19:30:00.42`
|
||
— formats it as a 16-char ASCII string `"2026060919300042"` per E30
|
||
§6.20, builds a `S2F18` reply with the string as its only item,
|
||
and hands it back to `gem::Router`.
|
||
|
||
8. The router asks `hsms::Connection` to send the reply. The same
|
||
layers run in reverse: SECS-II encoding (`A[16] '2026...'` →
|
||
`41 10 32 30 32 36 …`), HSMS framing, TCP send.
|
||
|
||
9. The MES decodes the reply, reads the timestamp, displays it on a
|
||
dashboard.
|
||
|
||
That's one transaction. A 300 mm fab tool exchanges **hundreds to
|
||
thousands of these per minute** during normal operation —
|
||
status polls, event reports, recipe management, job tracking, alarm
|
||
notifications, terminal messages. Every one of them flows through
|
||
exactly that stack. All the rest of this guide is filling in the
|
||
detail of each layer.
|
||
|
||
---
|
||
|
||
## A brief history (one paragraph)
|
||
|
||
- **1980** — SECS-I (E4) published. RS-232 framing, intended for
|
||
early 200 mm and pre-200 mm tools.
|
||
- **1982** — SECS-II (E5) published. Standardised the message
|
||
*structure* so it could outlive any one transport.
|
||
- **1992** — GEM (E30) published. Standardised the *behaviour* on
|
||
top of SECS-II. Made the message layer useful by giving every
|
||
message a defined role.
|
||
- **1995** — HSMS (E37) published. Replaced RS-232 with TCP/IP
|
||
while keeping E5 + E30 unchanged.
|
||
- **1999–2006** — GEM 300 suite published one standard at a time
|
||
(E39, E40, E84, E87, E90, E94, E116, E120, E148, E157, E42), adding
|
||
the behaviour needed for 300 mm wafer automation.
|
||
- **Today** — every modern 300 mm tool ships with E5 + E37 + E30 +
|
||
the GEM 300 suite. This codebase implements all of them.
|
||
|
||
---
|
||
|
||
## What's next
|
||
|
||
You now know:
|
||
|
||
- Why a fab needs a protocol at all.
|
||
- The three names — SECS, HSMS, GEM — and what each one actually
|
||
refers to.
|
||
- How the standards stack into transport → message → behaviour.
|
||
- Where each layer lives in this codebase.
|
||
|
||
The next chapter introduces the **cast of characters** — equipment,
|
||
host, MES, scheduler, AMHS — and shows who talks to whom in a typical
|
||
fab.
|
||
|
||
Next: [→ 02 The cast of characters](02_the_cast.md)
|