secs-gem/docs/01_what_is_secs_gem.md

01 — What is SECS/GEM?

← Back to index | Next: 02 The cast of characters →

A semiconductor fabrication plant — a fab — is one of the most automation-dense environments on Earth. A 300 mm wafer fab runs 50 to 200 tools simultaneously, each manufactured by a different vendor (Applied Materials, Tokyo Electron, Lam Research, ASML, KLA, Hitachi, dozens more), each performing one step in a recipe that takes 8–12 weeks and 500–1500 steps to turn a bare silicon wafer into a finished chip.

Every one of those tools needs to talk to a central computer — the Manufacturing Execution System (MES) — to receive recipe instructions, report progress, surface alarms, and prove every wafer ended up where it was supposed to.

SECS/GEM is the protocol they use to do that.

This chapter explains why the protocol exists, what its parts are called, and the one-screen history from the original 1980s standard to the modern GEM 300 suite that this codebase implements.

---

The N × M problem

Imagine a fab with 100 tools and 1 MES. Without a standard:

• Every tool vendor has to write a custom integration for every MES vendor they want to ship into.
• Every MES vendor has to write a custom driver for every tool vendor's API.
• Every fab has to negotiate, test, and maintain N × M integration pairs — and a 100-tool fab with even 2 MES generations in flight is suddenly looking at 200 distinct integrations.
• When a tool gets a firmware update, every MES integration breaks.

This is the N × M integration problem. It's the same problem that USB solved for peripherals, that TCP/IP solved for networking, that POSIX solved for system calls.

The semiconductor industry's answer is a family of standards published by SEMI (Semiconductor Equipment and Materials International), the trade body for the industry. The relevant ones have names like E4, E5, E30, E37, E40, E84, E87, E90, E94 and a dozen more. Together they describe:

1. What bytes go on the wire (the protocol stack).
2. What those bytes mean (the message catalog).
3. What the equipment must DO when it receives them (the behavioural contract).

Once a tool implements those standards, any MES can drive it. Once an MES implements them, any tool will respond. N × M collapses to N + M.

---

The three names you'll keep seeing

You will see the abbreviations SECS, HSMS, and GEM in every diagram and every doc. They mean three different things, and people use them sloppily. Pin them down once:

SECS — Semiconductor Equipment Communications Standard

SECS is the message layer. It defines:

• The set of named messages (S1F1, S1F3, S6F11, S16F11, …).
• The bytes that encode each message (format codes, length bytes, body structure).
• The conversational pattern (every primary message either has a reply or is explicitly fire-and-forget).

There are two SECS specs you'll encounter, and they do different things:

• SECS-I (E4) — the transport for SECS messages over RS-232 / RS-422 serial cables. Defined in 1980. Still used on older 200 mm fabs and on smaller specialty tools. Block-oriented, half-duplex, ENQ/EOT/ACK/NAK style.
• SECS-II (E5) — the message structure itself, independent of transport. Defined in 1982. A message is a list of items, where each item has a SEMI-defined format code (U1, U2, F4, ASCII, List, …). Every modern SECS-based protocol still uses E5 for message encoding.

You'll often see "SECS" used loosely as a synonym for SECS-II (the message structure). When someone says "a SECS message," they mean an E5-encoded message; the transport (E4 or HSMS) is a separate concern.

HSMS — High-Speed SECS Message Services

HSMS is the modern replacement for SECS-I as a transport. Defined as E37 in 1995, it carries SECS-II messages over a TCP/IP connection instead of a serial cable.

If you set up a SECS-II message and want to send it to a 21st-century tool, you send it over HSMS, not SECS-I.

HSMS has its own framing (4-byte length prefix + 10-byte header + SECS-II body) and its own connection state machine (NOT-CONNECTED → NOT-SELECTED → SELECTED) and its own timers (T3 reply, T5 separation, T6 control transaction, T7 not-selected, T8 inter-character). Chapter 11 covers it in detail.

HSMS comes in two flavours:

• HSMS-SS (Single-Session) — one TCP socket carries one SECS conversation between one equipment and one host. This is the default.
• HSMS-GS (General-Session) — one TCP socket multiplexes multiple sessions, identified by a session ID in each frame's header. Used in fabs where one piece of equipment must talk to several MES servers (production, maintenance, engineering) over the same physical link.

GEM — Generic Equipment Model

GEM, defined as E30 in 1992, is the behavioural layer on top of SECS-II. It answers questions like:

• When a host sends S1F13 Establish Communications, what state must the equipment enter?
• When the operator presses the Online button, which messages fire to the host?
• When an alarm becomes active, must the equipment send S5F1? Under what conditions can the host suppress it?
• What does it mean for equipment to be in the EquipmentOffline state vs. OnlineRemote?

E30 spells out:

• The communication state machine (DISABLED, WAIT-CRA, WAIT-DELAY, COMMUNICATING) that runs above HSMS's transport-level states.
• The control state machine (Equipment Offline, Attempting Online, Host Offline, Online, Online Local, Online Remote) that governs who's allowed to issue commands.
• The required scenarios — like "Establish Communications," "On-Line Identification," "Event Notification," "Alarm Management" — that every GEM-compliant tool must support.
• Two capability tiers: Fundamentals (mandatory) and Additionals (optional but very commonly required by MES procurement).

A tool that obeys E30 is called GEM-compliant and can be integrated by any MES that speaks GEM without custom code.

GEM 300 — the 300 mm wafer suite

When fabs migrated from 200 mm to 300 mm wafers around 2000, the extra automation (robot-driven wafer handling, no human touching a substrate) needed new behavioural contracts. GEM 300 is the collective name for:

Standard	Year	What it adds
E39	1999	Object Services — generic CRUD over typed equipment objects
E40	1999	Process Job management — submit, track, cancel a wafer process
E84	2000	Parallel I/O — the 8-line AMHS robot-to-tool handshake
E87	2000	Carrier Management — FOUPs and load ports
E90	2000	Substrate Tracking — per-wafer location and state
E94	2001	Control Job management — scheduling of multiple process jobs
E116	2003	Equipment Performance Tracking — time-buckets per state
E120	2003	Common Equipment Model — generic object hierarchy
E148	2005	Time Synchronization — distributed clock
E157	2006	Module Process Tracking — per-process-module state
E42	2004	Formatted Process Programs — typed recipe payloads

Every modern 300 mm tool ships with all of these. This codebase implements all of them. Chapters 14–19 cover them one family at a time.

---

Why it's structured this way

The layering — transport → message → behaviour — is deliberate and load-bearing, and the codebase mirrors it exactly.

   Behavioural contract        E30 (GEM) + GEM 300 suite
   (what equipment must DO)    secsgem::gem
                               ──────────────
                                        │
                                        │  emits / receives
                                        ▼
   Message structure           E5 (SECS-II)
   (what the bytes mean)       secsgem::secs2
                               ──────────────
                                        │
                                        │  encoded over
                                        ▼
   Transport                   E37 (HSMS, TCP/IP)        E4 (SECS-I, serial)
   (how bytes get there)       secsgem::hsms             secsgem::secsi
                               ──────────────            ──────────────

The separation matters because:

1. You can swap transports. The same SECS-II message and the same E30 behaviour work whether the bytes travel over HSMS-SS, HSMS-GS, or SECS-I. In this codebase, gem::Router doesn't know which transport delivered the bytes — it just sees a decoded secs2::Message.
2. You can evolve layers independently. E5's format codes haven't changed in 40 years; HSMS replaced SECS-I as the transport in 1995; GEM 300 added new behaviour without disturbing E5 or E37. The layers ship at different cadences and the spec only had to evolve the layers that needed to change.
3. You can test each layer in isolation. This codebase has 139 tests for the E5 codec alone, 34 for HSMS, 27 for SECS-I, 71 for E30 behaviour, and dozens per GEM 300 standard. None of the codec tests need a transport; none of the transport tests need a behaviour. See PROOFS.md for the per-standard test counts.

---

Where the layers live in this codebase

Layer	Standard	Namespace	Headers	Tests
Behavioural	E30 + GEM 300	secsgem::gem	include/secsgem/gem/*.hpp	tests/test_control_state.cpp, tests/test_communication_state.cpp, tests/test_data_model.cpp, and one file per GEM 300 standard
Messages	E5	secsgem::secs2	include/secsgem/secs2/{item,codec,message,sml}.hpp	tests/test_secs2.cpp, tests/test_e5_kat.cpp, tests/test_sml.cpp, tests/test_messages.cpp
Transport (TCP)	E37	secsgem::hsms	include/secsgem/hsms/{frame,header,connection}.hpp	tests/test_hsms.cpp, tests/test_hsms_connection.cpp, tests/test_hsms_timers.cpp, tests/test_hsms_gs.cpp, tests/test_hsms_s9.cpp
Transport (ser.)	E4	secsgem::secsi	include/secsgem/secsi/{header,block,protocol,tcp_transport}.hpp	tests/test_secsi.cpp, tests/test_secsi_timers.cpp, tests/test_secsi_tcp.cpp
Catalog (codegen)	E5 + GEM	secsgem::gem	build/generated/secsgem/gem/messages.hpp	tests/test_messages.cpp

Read it top-to-bottom: the behavioural layer (gem) uses the message layer (secs2) which is moved by the transport layer (hsms or secsi). Each row has its own chapter in Parts 2 and 3.

The codegen row is worth a footnote: SECS-II has ~160 named messages and each one has a typed struct body. Writing all 160 builders + parsers by hand would be 5000+ lines of boilerplate, so tools/gen_messages.py reads data/messages.yaml at build time and emits messages.hpp with one typed struct + builder + parser per message. Chapter 31 walks through how it works.

---

One example, end-to-end

Just so the abstractions feel less abstract: here's what happens when an MES asks an equipment "what time is it?"

1. MES (the host) wants to read the equipment's clock. In SECS terms that's stream 2, function 17 — S2F17 — defined in E30 §6.20 as part of the Clock capability.

2. The MES encodes a S2F17 request. The E5 body is empty (an empty List), so the SECS-II encoding is just the format byte + length: 01 00 (format=0=List, length-byte-count=1, length=0).

3. The MES wraps the SECS-II body in an HSMS frame: 4-byte length prefix + 10-byte header (session_id, byte2, byte3, PType, SType, system_bytes) + body. The W-bit in the header is set to 1 because S2F17 expects a reply.

4. The HSMS frame travels over TCP to the equipment.

5. The equipment's hsms::Connection reads the 4-byte length, reads the 10-byte header, reads the body, and dispatches the decoded secs2::Message to gem::Router.

6. gem::Router looks up the registered handler for (stream=2, function=17) and calls it.

7. The handler reads the equipment's clock — say, 2026-06-09 19:30:00.42 — formats it as a 16-char ASCII string "2026060919300042" per E30 §6.20, builds a S2F18 reply with the string as its only item, and hands it back to gem::Router.

8. The router asks hsms::Connection to send the reply. The same layers run in reverse: SECS-II encoding (A[16] '2026...' → 41 10 32 30 32 36 …), HSMS framing, TCP send.

9. The MES decodes the reply, reads the timestamp, displays it on a dashboard.

That's one transaction. A 300 mm fab tool exchanges hundreds to thousands of these per minute during normal operation — status polls, event reports, recipe management, job tracking, alarm notifications, terminal messages. Every one of them flows through exactly that stack. All the rest of this guide is filling in the detail of each layer.

---

A brief history (one paragraph)

• 1980 — SECS-I (E4) published. RS-232 framing, intended for early 200 mm and pre-200 mm tools.
• 1982 — SECS-II (E5) published. Standardised the message structure so it could outlive any one transport.
• 1992 — GEM (E30) published. Standardised the behaviour on top of SECS-II. Made the message layer useful by giving every message a defined role.
• 1995 — HSMS (E37) published. Replaced RS-232 with TCP/IP while keeping E5 + E30 unchanged.
• 1999–2006 — GEM 300 suite published one standard at a time (E39, E40, E84, E87, E90, E94, E116, E120, E148, E157, E42), adding the behaviour needed for 300 mm wafer automation.
• Today — every modern 300 mm tool ships with E5 + E37 + E30 + the GEM 300 suite. This codebase implements all of them.

---

What's next

You now know:

• Why a fab needs a protocol at all.
• The three names — SECS, HSMS, GEM — and what each one actually refers to.
• How the standards stack into transport → message → behaviour.
• Where each layer lives in this codebase.

The next chapter introduces the cast of characters — equipment, host, MES, scheduler, AMHS — and shows who talks to whom in a typical fab.

Next: → 02 The cast of characters