docs: start guided-tour tutorial series under docs/
A linear teach-from-zero tutorial that walks both SECS/GEM as a protocol family and this codebase as an implementation. Each chapter explains a SEMI concept and shows where it lives in code, so a reader builds a mental model of the standards and the repository simultaneously. Structure (24 chapters across 5 parts): - Part 1 (3 ch) — Foundations: what SECS/GEM is, the cast of characters, vocabulary + a wafer's end-to-end journey - Part 2 (10 ch) — Standards in detail: E5, E37, E4, E30, E40+E94, E87, E90+E157, E116+E120+E39, E84, E42+E148+S9 - Part 3 (7 ch) — Codebase: repository tour, spec-as-data + codegen, stores, transport, codec, state machines, persistence - Part 4 (2 ch) — Operations: build/run/demo, integration - Part 5 (2 ch) — Reference: API + messages + YAML, extension guide Published in this commit: - 00_index.md — guide layout, audience map, reading paths, conventions, status table - 01_what_is_secs_gem.md — the N×M integration problem, what SECS vs. HSMS vs. GEM each actually refer to, the GEM 300 suite, the transport→message→behaviour layering, where each layer lives in this codebase, an end-to-end S2F17/F18 example Chapters publish iteratively from here. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,328 @@
|
||||
# 01 — What is SECS/GEM?
|
||||
|
||||
← [Back to index](00_index.md) | Next: [02 The cast of characters](02_the_cast.md) →
|
||||
|
||||
A semiconductor fabrication plant — a **fab** — is one of the most
|
||||
automation-dense environments on Earth. A 300 mm wafer fab runs
|
||||
**50 to 200 tools** simultaneously, each manufactured by a different
|
||||
vendor (Applied Materials, Tokyo Electron, Lam Research, ASML, KLA,
|
||||
Hitachi, dozens more), each performing one step in a recipe that
|
||||
takes 8–12 weeks and 500–1500 steps to turn a bare silicon wafer
|
||||
into a finished chip.
|
||||
|
||||
Every one of those tools needs to talk to a central computer — the
|
||||
**Manufacturing Execution System** (MES) — to receive recipe
|
||||
instructions, report progress, surface alarms, and prove every wafer
|
||||
ended up where it was supposed to.
|
||||
|
||||
SECS/GEM is the protocol they use to do that.
|
||||
|
||||
This chapter explains why the protocol exists, what its parts are
|
||||
called, and the one-screen history from the original 1980s standard
|
||||
to the modern GEM 300 suite that this codebase implements.
|
||||
|
||||
---
|
||||
|
||||
## The N × M problem
|
||||
|
||||
Imagine a fab with 100 tools and 1 MES. Without a standard:
|
||||
|
||||
- Every **tool vendor** has to write a custom integration for every
|
||||
**MES vendor** they want to ship into.
|
||||
- Every **MES vendor** has to write a custom driver for every **tool
|
||||
vendor**'s API.
|
||||
- Every fab has to negotiate, test, and maintain N × M integration
|
||||
pairs — and a 100-tool fab with even 2 MES generations in flight
|
||||
is suddenly looking at 200 distinct integrations.
|
||||
- When a tool gets a firmware update, every MES integration breaks.
|
||||
|
||||
This is the **N × M integration problem**. It's the same problem
|
||||
that USB solved for peripherals, that TCP/IP solved for networking,
|
||||
that POSIX solved for system calls.
|
||||
|
||||
The semiconductor industry's answer is a family of standards
|
||||
published by **SEMI** (Semiconductor Equipment and Materials
|
||||
International), the trade body for the industry. The relevant ones
|
||||
have names like **E4**, **E5**, **E30**, **E37**, **E40**, **E84**,
|
||||
**E87**, **E90**, **E94** and a dozen more. Together they describe:
|
||||
|
||||
1. **What bytes go on the wire** (the protocol stack).
|
||||
2. **What those bytes mean** (the message catalog).
|
||||
3. **What the equipment must DO when it receives them** (the
|
||||
behavioural contract).
|
||||
|
||||
Once a tool implements those standards, *any* MES can drive it.
|
||||
Once an MES implements them, *any* tool will respond. N × M
|
||||
collapses to N + M.
|
||||
|
||||
---
|
||||
|
||||
## The three names you'll keep seeing
|
||||
|
||||
You will see the abbreviations **SECS**, **HSMS**, and **GEM** in
|
||||
every diagram and every doc. They mean three different things, and
|
||||
people use them sloppily. Pin them down once:
|
||||
|
||||
### SECS — Semiconductor Equipment Communications Standard
|
||||
|
||||
SECS is the **message layer**. It defines:
|
||||
|
||||
- The set of named messages (`S1F1`, `S1F3`, `S6F11`, `S16F11`, …).
|
||||
- The bytes that encode each message (format codes, length bytes,
|
||||
body structure).
|
||||
- The conversational pattern (every primary message either has a
|
||||
reply or is explicitly fire-and-forget).
|
||||
|
||||
There are two SECS specs you'll encounter, and they do different
|
||||
things:
|
||||
|
||||
- **SECS-I (E4)** — the **transport** for SECS messages over
|
||||
RS-232 / RS-422 serial cables. Defined in 1980. Still used on
|
||||
older 200 mm fabs and on smaller specialty tools. Block-oriented,
|
||||
half-duplex, ENQ/EOT/ACK/NAK style.
|
||||
- **SECS-II (E5)** — the **message structure** itself, independent of
|
||||
transport. Defined in 1982. A message is a list of *items*,
|
||||
where each item has a SEMI-defined format code (U1, U2, F4, ASCII,
|
||||
List, …). Every modern SECS-based protocol still uses E5 for
|
||||
message encoding.
|
||||
|
||||
You'll often see "**SECS**" used loosely as a synonym for **SECS-II**
|
||||
(the message structure). When someone says "a SECS message," they
|
||||
mean an E5-encoded message; the transport (E4 or HSMS) is a separate
|
||||
concern.
|
||||
|
||||
### HSMS — High-Speed SECS Message Services
|
||||
|
||||
HSMS is the **modern replacement for SECS-I** as a transport.
|
||||
Defined as **E37** in 1995, it carries SECS-II messages over a TCP/IP
|
||||
connection instead of a serial cable.
|
||||
|
||||
If you set up a SECS-II message and want to send it to a 21st-century
|
||||
tool, you send it over HSMS, not SECS-I.
|
||||
|
||||
HSMS has its own framing (4-byte length prefix + 10-byte header +
|
||||
SECS-II body) and its own connection state machine (NOT-CONNECTED →
|
||||
NOT-SELECTED → SELECTED) and its own timers (T3 reply, T5 separation,
|
||||
T6 control transaction, T7 not-selected, T8 inter-character).
|
||||
Chapter [11](11_e37_hsms.md) covers it in detail.
|
||||
|
||||
HSMS comes in two flavours:
|
||||
|
||||
- **HSMS-SS** (Single-Session) — one TCP socket carries one SECS
|
||||
conversation between one equipment and one host. This is the
|
||||
default.
|
||||
- **HSMS-GS** (General-Session) — one TCP socket multiplexes
|
||||
*multiple* sessions, identified by a session ID in each frame's
|
||||
header. Used in fabs where one piece of equipment must talk to
|
||||
several MES servers (production, maintenance, engineering) over
|
||||
the same physical link.
|
||||
|
||||
### GEM — Generic Equipment Model
|
||||
|
||||
GEM, defined as **E30** in 1992, is the **behavioural layer** on top
|
||||
of SECS-II. It answers questions like:
|
||||
|
||||
- When a host sends `S1F13 Establish Communications`, what state must
|
||||
the equipment enter?
|
||||
- When the operator presses the **Online** button, which messages
|
||||
fire to the host?
|
||||
- When an alarm becomes active, must the equipment send `S5F1`?
|
||||
Under what conditions can the host suppress it?
|
||||
- What does it mean for equipment to be in the `EquipmentOffline`
|
||||
state vs. `OnlineRemote`?
|
||||
|
||||
E30 spells out:
|
||||
|
||||
- The **communication state machine** (DISABLED, WAIT-CRA,
|
||||
WAIT-DELAY, COMMUNICATING) that runs above HSMS's transport-level
|
||||
states.
|
||||
- The **control state machine** (Equipment Offline, Attempting
|
||||
Online, Host Offline, Online, Online Local, Online Remote) that
|
||||
governs who's allowed to issue commands.
|
||||
- The required **scenarios** — like "Establish Communications,"
|
||||
"On-Line Identification," "Event Notification," "Alarm Management"
|
||||
— that every GEM-compliant tool must support.
|
||||
- Two **capability tiers**: **Fundamentals** (mandatory) and
|
||||
**Additionals** (optional but very commonly required by MES
|
||||
procurement).
|
||||
|
||||
A tool that obeys E30 is called **GEM-compliant** and can be
|
||||
integrated by *any* MES that speaks GEM without custom code.
|
||||
|
||||
### GEM 300 — the 300 mm wafer suite
|
||||
|
||||
When fabs migrated from 200 mm to 300 mm wafers around 2000, the
|
||||
extra automation (robot-driven wafer handling, no human touching a
|
||||
substrate) needed new behavioural contracts. GEM 300 is the
|
||||
collective name for:
|
||||
|
||||
| Standard | Year | What it adds |
|
||||
|----------|------|---------------------------------------------------------------|
|
||||
| **E39** | 1999 | Object Services — generic CRUD over typed equipment objects |
|
||||
| **E40** | 1999 | Process Job management — submit, track, cancel a wafer process |
|
||||
| **E84** | 2000 | Parallel I/O — the 8-line AMHS robot-to-tool handshake |
|
||||
| **E87** | 2000 | Carrier Management — FOUPs and load ports |
|
||||
| **E90** | 2000 | Substrate Tracking — per-wafer location and state |
|
||||
| **E94** | 2001 | Control Job management — scheduling of multiple process jobs |
|
||||
| **E116** | 2003 | Equipment Performance Tracking — time-buckets per state |
|
||||
| **E120** | 2003 | Common Equipment Model — generic object hierarchy |
|
||||
| **E148** | 2005 | Time Synchronization — distributed clock |
|
||||
| **E157** | 2006 | Module Process Tracking — per-process-module state |
|
||||
| **E42** | 2004 | Formatted Process Programs — typed recipe payloads |
|
||||
|
||||
Every modern 300 mm tool ships with all of these. This codebase
|
||||
implements all of them. Chapters [14–19](14_e40_e94_jobs.md) cover
|
||||
them one family at a time.
|
||||
|
||||
---
|
||||
|
||||
## Why it's structured this way
|
||||
|
||||
The layering — transport → message → behaviour — is **deliberate
|
||||
and load-bearing**, and the codebase mirrors it exactly.
|
||||
|
||||
```
|
||||
Behavioural contract E30 (GEM) + GEM 300 suite
|
||||
(what equipment must DO) secsgem::gem
|
||||
──────────────
|
||||
│
|
||||
│ emits / receives
|
||||
▼
|
||||
Message structure E5 (SECS-II)
|
||||
(what the bytes mean) secsgem::secs2
|
||||
──────────────
|
||||
│
|
||||
│ encoded over
|
||||
▼
|
||||
Transport E37 (HSMS, TCP/IP) E4 (SECS-I, serial)
|
||||
(how bytes get there) secsgem::hsms secsgem::secsi
|
||||
────────────── ──────────────
|
||||
```
|
||||
|
||||
The separation matters because:
|
||||
|
||||
1. **You can swap transports.** The same SECS-II message and the
|
||||
same E30 behaviour work whether the bytes travel over HSMS-SS,
|
||||
HSMS-GS, or SECS-I. In this codebase, `gem::Router` doesn't know
|
||||
which transport delivered the bytes — it just sees a decoded
|
||||
`secs2::Message`.
|
||||
2. **You can evolve layers independently.** E5's format codes
|
||||
haven't changed in 40 years; HSMS replaced SECS-I as the
|
||||
transport in 1995; GEM 300 added new behaviour without disturbing
|
||||
E5 or E37. The layers ship at different cadences and the spec
|
||||
only had to evolve the layers that needed to change.
|
||||
3. **You can test each layer in isolation.** This codebase has
|
||||
139 tests for the E5 codec alone, 34 for HSMS, 27 for SECS-I,
|
||||
71 for E30 behaviour, and dozens per GEM 300 standard. None of
|
||||
the codec tests need a transport; none of the transport tests
|
||||
need a behaviour. See [PROOFS.md](../PROOFS.md) for the
|
||||
per-standard test counts.
|
||||
|
||||
---
|
||||
|
||||
## Where the layers live in this codebase
|
||||
|
||||
| Layer | Standard | Namespace | Headers | Tests |
|
||||
|------------------|----------------|--------------------|--------------------------------------------------------|----------------------------------------|
|
||||
| Behavioural | E30 + GEM 300 | `secsgem::gem` | `include/secsgem/gem/*.hpp` | `tests/test_control_state.cpp`, `tests/test_communication_state.cpp`, `tests/test_data_model.cpp`, and one file per GEM 300 standard |
|
||||
| Messages | E5 | `secsgem::secs2` | `include/secsgem/secs2/{item,codec,message,sml}.hpp` | `tests/test_secs2.cpp`, `tests/test_e5_kat.cpp`, `tests/test_sml.cpp`, `tests/test_messages.cpp` |
|
||||
| Transport (TCP) | E37 | `secsgem::hsms` | `include/secsgem/hsms/{frame,header,connection}.hpp` | `tests/test_hsms.cpp`, `tests/test_hsms_connection.cpp`, `tests/test_hsms_timers.cpp`, `tests/test_hsms_gs.cpp`, `tests/test_hsms_s9.cpp` |
|
||||
| Transport (ser.) | E4 | `secsgem::secsi` | `include/secsgem/secsi/{header,block,protocol,tcp_transport}.hpp` | `tests/test_secsi.cpp`, `tests/test_secsi_timers.cpp`, `tests/test_secsi_tcp.cpp` |
|
||||
| Catalog (codegen)| E5 + GEM | `secsgem::gem` | `build/generated/secsgem/gem/messages.hpp` | `tests/test_messages.cpp` |
|
||||
|
||||
Read it top-to-bottom: the behavioural layer (`gem`) uses the message
|
||||
layer (`secs2`) which is moved by the transport layer (`hsms` or
|
||||
`secsi`). Each row has its own chapter in Parts 2 and 3.
|
||||
|
||||
The **codegen** row is worth a footnote: SECS-II has ~160 named
|
||||
messages and each one has a typed struct body. Writing all 160
|
||||
builders + parsers by hand would be 5000+ lines of boilerplate, so
|
||||
`tools/generate_messages.py` reads `data/messages.yaml` at build time
|
||||
and emits `messages.hpp` with one typed struct + builder + parser per
|
||||
message. Chapter [31](31_spec_as_data_and_codegen.md) walks through
|
||||
how it works.
|
||||
|
||||
---
|
||||
|
||||
## One example, end-to-end
|
||||
|
||||
Just so the abstractions feel less abstract: here's what happens when
|
||||
an MES asks an equipment "what time is it?"
|
||||
|
||||
1. **MES (the host)** wants to read the equipment's clock. In SECS
|
||||
terms that's stream 2, function 17 — `S2F17` — defined in E30
|
||||
§6.20 as part of the Clock capability.
|
||||
|
||||
2. The MES encodes a `S2F17` request. The E5 body is empty (an
|
||||
empty `List`), so the SECS-II encoding is just the format byte +
|
||||
length: `01 00` (format=0=List, length-byte-count=1, length=0).
|
||||
|
||||
3. The MES wraps the SECS-II body in an HSMS frame: 4-byte length
|
||||
prefix + 10-byte header (`session_id`, `byte2`, `byte3`, `PType`,
|
||||
`SType`, `system_bytes`) + body. The W-bit in the header is set
|
||||
to 1 because S2F17 expects a reply.
|
||||
|
||||
4. The HSMS frame travels over TCP to the equipment.
|
||||
|
||||
5. The equipment's `hsms::Connection` reads the 4-byte length,
|
||||
reads the 10-byte header, reads the body, and dispatches the
|
||||
decoded `secs2::Message` to `gem::Router`.
|
||||
|
||||
6. `gem::Router` looks up the registered handler for `(stream=2,
|
||||
function=17)` and calls it.
|
||||
|
||||
7. The handler reads the equipment's clock — say, `2026-06-09 19:30:00.42`
|
||||
— formats it as a 16-char ASCII string `"2026060919300042"` per E30
|
||||
§6.20, builds a `S2F18` reply with the string as its only item,
|
||||
and hands it back to `gem::Router`.
|
||||
|
||||
8. The router asks `hsms::Connection` to send the reply. The same
|
||||
layers run in reverse: SECS-II encoding (`A[16] '2026...'` →
|
||||
`41 10 32 30 32 36 …`), HSMS framing, TCP send.
|
||||
|
||||
9. The MES decodes the reply, reads the timestamp, displays it on a
|
||||
dashboard.
|
||||
|
||||
That's one transaction. A 300 mm fab tool exchanges **hundreds to
|
||||
thousands of these per minute** during normal operation —
|
||||
status polls, event reports, recipe management, job tracking, alarm
|
||||
notifications, terminal messages. Every one of them flows through
|
||||
exactly that stack. All the rest of this guide is filling in the
|
||||
detail of each layer.
|
||||
|
||||
---
|
||||
|
||||
## A brief history (one paragraph)
|
||||
|
||||
- **1980** — SECS-I (E4) published. RS-232 framing, intended for
|
||||
early 200 mm and pre-200 mm tools.
|
||||
- **1982** — SECS-II (E5) published. Standardised the message
|
||||
*structure* so it could outlive any one transport.
|
||||
- **1992** — GEM (E30) published. Standardised the *behaviour* on
|
||||
top of SECS-II. Made the message layer useful by giving every
|
||||
message a defined role.
|
||||
- **1995** — HSMS (E37) published. Replaced RS-232 with TCP/IP
|
||||
while keeping E5 + E30 unchanged.
|
||||
- **1999–2006** — GEM 300 suite published one standard at a time
|
||||
(E39, E40, E84, E87, E90, E94, E116, E120, E148, E157, E42), adding
|
||||
the behaviour needed for 300 mm wafer automation.
|
||||
- **Today** — every modern 300 mm tool ships with E5 + E37 + E30 +
|
||||
the GEM 300 suite. This codebase implements all of them.
|
||||
|
||||
---
|
||||
|
||||
## What's next
|
||||
|
||||
You now know:
|
||||
|
||||
- Why a fab needs a protocol at all.
|
||||
- The three names — SECS, HSMS, GEM — and what each one actually
|
||||
refers to.
|
||||
- How the standards stack into transport → message → behaviour.
|
||||
- Where each layer lives in this codebase.
|
||||
|
||||
The next chapter introduces the **cast of characters** — equipment,
|
||||
host, MES, scheduler, AMHS — and shows who talks to whom in a typical
|
||||
fab.
|
||||
|
||||
Next: [→ 02 The cast of characters](02_the_cast.md)
|
||||
Reference in New Issue
Block a user