raphael/secs-gem
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test: SEMI E5 known-answer tests for SECS-II encoding

Browse Source 
Hex-string fixtures constructed directly from the SEMI E5 §9
format-byte encoding rules:

  format_byte = (format_code << 2) | length_byte_count
  length_byte_count ∈ {1, 2, 3}

Coverage:
- Every format code (L, B, BOOLEAN, A, J, C, U1-U8, I1-I8, F4, F8)
- Every length-byte-count variant (1, 2, 3 bytes — exercises the
  255 → 256 → 65 536 transitions)
- Numeric edges: 0, ±1, MIN, MAX, ±Inf, NaN, -0.0, multi-element vectors
- Empty and single-element variants
- Nested lists
- A "format byte layout per format code" regression tripwire that
  pins every code → byte mapping

19 test cases, 196 assertions.  Every fixture round-trips
byte-identical against the codec.

Why this is the strongest single codec test: every other validator
(secsgem-py interop, conformance harness, in-house unit tests) is
one implementer's interpretation.  KAT is the standard's own
arithmetic.  If our encoder matches these canonical bytes and our
decoder reverses them to the same Item, our SECS-II layer is wire-
compatible with anything else that obeys E5 §9.

NaN / signed-zero / Inf use a bit-pattern compare (IEEE NaN != NaN
breaks the default Item == path) — decode the canonical, re-encode
the decoded, assert byte-identical.

The 3-byte-length fixture (ASCII 65 536 × 'X') generates a ~200 KB
expected-bytes string in the test — slow to write but trivial to
check and forces the 3-byte length-prefix path that 99 % of real
traffic doesn't exercise.

Plan: VERIFICATION.md §1.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Raphael Maenle
2026-06-09 15:50:57 +02:00
parent 257a148d34
commit a79973ed4c
2 changed files with 387 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CMakeLists.txt
+1
View File
@@ -164,6 +164,7 @@ add_executable(secsgem_tests
tests/test_metrics_prometheus.cpp
tests/test_hsms_gs_integration.cpp
tests/test_robustness_fuzz.cpp
tests/test_e5_kat.cpp
)
target_link_libraries(secsgem_tests PRIVATE secsgem doctest::doctest)
target_compile_definitions(secsgem_tests PRIVATE
tests/test_e5_kat.cpp
+386
View File
@@ -0,0 +1,386 @@
// SEMI E5 known-answer tests for SECS-II encoding.
//
// Hex-string fixtures constructed directly from the SEMI E5 §9
// format-byte encoding rules:
//
// format_byte = (format_code << 2) | length_byte_count
// length_byte_count ∈ {1, 2, 3}
// followed by length_byte_count big-endian length bytes
// followed by `length` body bytes (element-typed)
//
// Format codes (octal, §9.5 Table 5):
// L=000 B=010 BOOLEAN=011 A=020 J=021 C=022
// I8=030 I1=031 I2=032 I4=034
// F8=040 F4=044
// U8=050 U1=051 U2=052 U4=054
//
// This file is the strongest single proof of codec correctness:
// every other validator is one implementer's interpretation; KAT is
// the standard's own arithmetic. If our encoder matches these
// canonical bytes and our decoder reverses them to the same Item,
// our SECS-II layer is wire-compatible with anything else that obeys
// E5 §9.
//
// Coverage:
// - Every format code with at least one non-trivial value
// - Every length-byte-count variant (1, 2, 3 bytes)
// - Numeric edges: 0, ±1, MIN, MAX, ±Inf, NaN, multi-element vectors
// - Empty and single-element variants
// - Nested lists
//
// On a failure, do NOT change the fixture — fix the codec to match
// the spec.
#include <doctest/doctest.h>
#include <cmath>
#include <cstdint>
#include <cstring>
#include <string>
#include <vector>
#include "secsgem/secs2/codec.hpp"
#include "secsgem/secs2/item.hpp"
using namespace secsgem::secs2;
namespace {
// Convert "41 05 48 65 6C 6C 6F" → {0x41, 0x05, 0x48, 0x65, 0x6C, 0x6C, 0x6F}.
std::vector<uint8_t> hex(const std::string& s) {
std::vector<uint8_t> out;
uint8_t cur = 0;
bool nibble = false;
for (char c : s) {
if (c == ' ' || c == '\n' || c == '\t') continue;
int v;
if (c >= '0' && c <= '9') v = c - '0';
else if (c >= 'a' && c <= 'f') v = 10 + (c - 'a');
else if (c >= 'A' && c <= 'F') v = 10 + (c - 'A');
else throw std::runtime_error(std::string("bad hex char: ") + c);
cur = static_cast<uint8_t>((cur << 4) | v);
if (nibble) {
out.push_back(cur);
cur = 0;
}
nibble = !nibble;
}
if (nibble) throw std::runtime_error("odd nibble count");
return out;
}
std::string to_hex(const std::vector<uint8_t>& b) {
static const char* digits = "0123456789ABCDEF";
std::string s;
s.reserve(b.size() * 3);
for (std::size_t i = 0; i < b.size(); ++i) {
if (i) s += ' ';
s += digits[b[i] >> 4];
s += digits[b[i] & 0xF];
}
return s;
}
// Round-trip a fixture: encode the Item, assert exact byte equality
// with the canonical; decode the canonical, assert equality with the
// Item. The label appears in failure messages.
void check_kat(const char* label, const Item& item,
const std::string& canonical_hex) {
const auto canonical = hex(canonical_hex);
const auto encoded = encode(item);
INFO(label);
CHECK_MESSAGE(encoded == canonical,
"encode mismatch:\n got " << to_hex(encoded)
<< "\n expected " << canonical_hex);
const auto decoded = decode(canonical);
CHECK_MESSAGE(decoded == item,
"decode mismatch for " << label);
}
// Float specials (NaN, ±Inf, -0.0) need bit-level checks because NaN
// doesn't compare equal to itself by IEEE rules. We decode the
// canonical bytes (no construction needed — the bit pattern IS the
// fixture), then re-encode and assert byte-identical output.
void check_kat_float_bits(const char* label, Format fmt,
const std::vector<uint8_t>& canonical) {
const auto decoded = decode(canonical);
INFO(label);
REQUIRE(decoded.format() == fmt);
const auto re_encoded = encode(decoded);
CHECK_MESSAGE(re_encoded == canonical,
"round-trip mismatch:\n got " << to_hex(re_encoded)
<< "\n canonical " << to_hex(canonical));
}
} // namespace
// -------------------- List --------------------
TEST_CASE("E5 KAT: List (format code 000)") {
// Empty list: fmt=(0<<2)|1=0x01, len=0
check_kat("L empty", Item::list({}), "01 00");
// List of two U1 elements [1, 2]: fmt=0x01, len=0x02 (2 elements),
// then U1(1) = A5 01 01 and U1(2) = A5 01 02.
check_kat("L [U1(1), U1(2)]",
Item::list({Item::u1(1), Item::u1(2)}),
"01 02 A5 01 01 A5 01 02");
// Nested list: L [L[U1(1)], A "OK"]
// outer: 01 02
// inner L[U1(1)]: 01 01 A5 01 01
// A "OK": 41 02 4F 4B
check_kat("L nested",
Item::list({Item::list({Item::u1(1)}), Item::ascii("OK")}),
"01 02 01 01 A5 01 01 41 02 4F 4B");
}
// -------------------- Binary --------------------
TEST_CASE("E5 KAT: Binary (format code 010)") {
// Empty: fmt=(8<<2)|1=0x21, len=0
check_kat("B empty", Item::binary({}), "21 00");
// Single byte 0xDE: fmt=0x21, len=0x01, body=DE
check_kat("B {0xDE}", Item::binary({0xDE}), "21 01 DE");
// Multi-byte: fmt=0x21, len=0x04, body=DE AD BE EF
check_kat("B {0xDE, 0xAD, 0xBE, 0xEF}",
Item::binary({0xDE, 0xAD, 0xBE, 0xEF}),
"21 04 DE AD BE EF");
}
// -------------------- Boolean --------------------
TEST_CASE("E5 KAT: Boolean (format code 011)") {
// TRUE: fmt=(9<<2)|1=0x25, len=0x01, body=01
check_kat("BOOLEAN TRUE", Item::boolean(true), "25 01 01");
// FALSE: 25 01 00
check_kat("BOOLEAN FALSE", Item::boolean(false), "25 01 00");
// Vector {true, false, true}: 25 03 01 00 01
check_kat("BOOLEAN vec",
Item::boolean(std::vector<uint8_t>{1, 0, 1}),
"25 03 01 00 01");
}
// -------------------- ASCII --------------------
TEST_CASE("E5 KAT: ASCII (format code 020)") {
// Empty: fmt=(16<<2)|1=0x41, len=0
check_kat("A empty", Item::ascii(""), "41 00");
// "H": 41 01 48
check_kat("A \"H\"", Item::ascii("H"), "41 01 48");
// "Hello": 41 05 48 65 6C 6C 6F
check_kat("A \"Hello\"", Item::ascii("Hello"), "41 05 48 65 6C 6C 6F");
}
TEST_CASE("E5 KAT: ASCII length-byte transitions") {
// 255-byte body (max for 1-byte length): fmt=0x41, len=0xFF, body=255× 'A'
std::string s255(255, 'A');
std::string expected255 = "41 FF";
for (int i = 0; i < 255; ++i) expected255 += " 41";
check_kat("A 255× 'A' (1-byte length)", Item::ascii(s255), expected255);
// 256-byte body — transitions to 2-byte length:
// fmt=(16<<2)|2=0x42, len=0x01 0x00 (256), body=256× 'A'
std::string s256(256, 'A');
std::string expected256 = "42 01 00";
for (int i = 0; i < 256; ++i) expected256 += " 41";
check_kat("A 256× 'A' (2-byte length transition)",
Item::ascii(s256), expected256);
}
TEST_CASE("E5 KAT: ASCII 3-byte length") {
// 65 536-byte body: transitions to 3-byte length.
// fmt=(16<<2)|3=0x43, len=0x01 0x00 0x00 (65 536), body=65 536× 'X'
std::string s(65536, 'X');
std::string expected = "43 01 00 00";
// 65 536 bytes of 'X' (0x58) — build incrementally to avoid a giant literal.
expected.reserve(4 + 3 + 65536 * 3);
for (int i = 0; i < 65536; ++i) expected += " 58";
check_kat("A 65536× 'X' (3-byte length transition)",
Item::ascii(s), expected);
}
// -------------------- JIS-8 --------------------
TEST_CASE("E5 KAT: JIS-8 (format code 021)") {
// Empty: fmt=(17<<2)|1=0x45, len=0
check_kat("J empty", Item::jis8(""), "45 00");
// 4 bytes (typical JIS-X-0201 katakana range): A1 A2 A3 A4
check_kat("J 4 bytes",
Item::jis8(std::string{char(0xA1), char(0xA2),
char(0xA3), char(0xA4)}),
"45 04 A1 A2 A3 A4");
}
// -------------------- C2 (Unicode 2-byte) --------------------
TEST_CASE("E5 KAT: C2 (format code 022)") {
// Empty: fmt=(18<<2)|1=0x49, len=0
check_kat("C empty", Item::c2(std::vector<uint16_t>{}), "49 00");
// [0x0041, 0x0042] ('A', 'B' as BMP code points):
// fmt=0x49, len=0x04 (4 body bytes for 2 code points × 2 bytes BE)
check_kat("C [0x0041, 0x0042]",
Item::c2(std::vector<uint16_t>{0x0041, 0x0042}),
"49 04 00 41 00 42");
}
// -------------------- I1 / I2 / I4 / I8 --------------------
TEST_CASE("E5 KAT: I1 (format code 031)") {
// fmt=(25<<2)|1=0x65
check_kat("I1 0", Item::i1(int8_t{0}), "65 01 00");
check_kat("I1 1", Item::i1(int8_t{1}), "65 01 01");
check_kat("I1 -1", Item::i1(int8_t{-1}), "65 01 FF");
check_kat("I1 127", Item::i1(int8_t{127}), "65 01 7F");
check_kat("I1 -128", Item::i1(int8_t{-128}), "65 01 80");
check_kat("I1 vec [1,-1,2]",
Item::i1(std::vector<int8_t>{1, -1, 2}),
"65 03 01 FF 02");
}
TEST_CASE("E5 KAT: I2 (format code 032)") {
// fmt=(26<<2)|1=0x69
check_kat("I2 0", Item::i2(int16_t{0}), "69 02 00 00");
check_kat("I2 -1", Item::i2(int16_t{-1}), "69 02 FF FF");
check_kat("I2 INT16_MIN", Item::i2(int16_t{-32768}), "69 02 80 00");
check_kat("I2 INT16_MAX", Item::i2(int16_t{32767}), "69 02 7F FF");
}
TEST_CASE("E5 KAT: I4 (format code 034)") {
// fmt=(28<<2)|1=0x71
check_kat("I4 0", Item::i4(int32_t{0}), "71 04 00 00 00 00");
check_kat("I4 1", Item::i4(int32_t{1}), "71 04 00 00 00 01");
check_kat("I4 -1", Item::i4(int32_t{-1}), "71 04 FF FF FF FF");
check_kat("I4 0x01020304",
Item::i4(int32_t{0x01020304}),
"71 04 01 02 03 04");
check_kat("I4 INT32_MIN",
Item::i4(int32_t{INT32_MIN}),
"71 04 80 00 00 00");
}
TEST_CASE("E5 KAT: I8 (format code 030)") {
// fmt=(24<<2)|1=0x61
check_kat("I8 0", Item::i8(int64_t{0}), "61 08 00 00 00 00 00 00 00 00");
check_kat("I8 -1", Item::i8(int64_t{-1}), "61 08 FF FF FF FF FF FF FF FF");
check_kat("I8 INT64_MIN",
Item::i8(int64_t{INT64_MIN}),
"61 08 80 00 00 00 00 00 00 00");
}
// -------------------- U1 / U2 / U4 / U8 --------------------
TEST_CASE("E5 KAT: U1 (format code 051)") {
// fmt=(41<<2)|1=0xA5
check_kat("U1 0", Item::u1(uint8_t{0}), "A5 01 00");
check_kat("U1 1", Item::u1(uint8_t{1}), "A5 01 01");
check_kat("U1 0x7F", Item::u1(uint8_t{0x7F}), "A5 01 7F");
check_kat("U1 0xFF", Item::u1(uint8_t{0xFF}), "A5 01 FF");
check_kat("U1 vec [1,2,3]",
Item::u1(std::vector<uint8_t>{1, 2, 3}),
"A5 03 01 02 03");
}
TEST_CASE("E5 KAT: U2 (format code 052)") {
// fmt=(42<<2)|1=0xA9
check_kat("U2 0", Item::u2(uint16_t{0}), "A9 02 00 00");
check_kat("U2 0x0102", Item::u2(uint16_t{0x0102}), "A9 02 01 02");
check_kat("U2 0xFFFF", Item::u2(uint16_t{0xFFFF}), "A9 02 FF FF");
}
TEST_CASE("E5 KAT: U4 (format code 054)") {
// fmt=(44<<2)|1=0xB1
check_kat("U4 0",
Item::u4(uint32_t{0}),
"B1 04 00 00 00 00");
check_kat("U4 0x01020304",
Item::u4(uint32_t{0x01020304}),
"B1 04 01 02 03 04");
check_kat("U4 0xFFFFFFFF",
Item::u4(uint32_t{0xFFFFFFFF}),
"B1 04 FF FF FF FF");
}
TEST_CASE("E5 KAT: U8 (format code 050)") {
// fmt=(40<<2)|1=0xA1
check_kat("U8 0",
Item::u8(uint64_t{0}),
"A1 08 00 00 00 00 00 00 00 00");
check_kat("U8 0x0102030405060708",
Item::u8(uint64_t{0x0102030405060708ULL}),
"A1 08 01 02 03 04 05 06 07 08");
check_kat("U8 UINT64_MAX",
Item::u8(UINT64_MAX),
"A1 08 FF FF FF FF FF FF FF FF");
}
// -------------------- F4 (IEEE 754 single) --------------------
TEST_CASE("E5 KAT: F4 (format code 044)") {
// fmt=(36<<2)|1=0x91
check_kat("F4 0.0", Item::f4(0.0f), "91 04 00 00 00 00");
check_kat("F4 1.0", Item::f4(1.0f), "91 04 3F 80 00 00");
check_kat("F4 -1.0", Item::f4(-1.0f), "91 04 BF 80 00 00");
check_kat("F4 2.0", Item::f4(2.0f), "91 04 40 00 00 00");
// Specials need bit-pattern compare (NaN != NaN, signed zero, etc.)
check_kat_float_bits("F4 +Inf", Format::F4, hex("91 04 7F 80 00 00"));
check_kat_float_bits("F4 -Inf", Format::F4, hex("91 04 FF 80 00 00"));
check_kat_float_bits("F4 quiet NaN", Format::F4, hex("91 04 7F C0 00 00"));
check_kat_float_bits("F4 -0.0", Format::F4, hex("91 04 80 00 00 00"));
}
// -------------------- F8 (IEEE 754 double) --------------------
TEST_CASE("E5 KAT: F8 (format code 040)") {
// fmt=(32<<2)|1=0x81
check_kat("F8 0.0", Item::f8(0.0), "81 08 00 00 00 00 00 00 00 00");
check_kat("F8 1.0", Item::f8(1.0), "81 08 3F F0 00 00 00 00 00 00");
check_kat("F8 -1.0", Item::f8(-1.0), "81 08 BF F0 00 00 00 00 00 00");
check_kat("F8 2.0", Item::f8(2.0), "81 08 40 00 00 00 00 00 00 00");
check_kat_float_bits("F8 +Inf", Format::F8, hex("81 08 7F F0 00 00 00 00 00 00"));
check_kat_float_bits("F8 -Inf", Format::F8, hex("81 08 FF F0 00 00 00 00 00 00"));
check_kat_float_bits("F8 quiet NaN", Format::F8, hex("81 08 7F F8 00 00 00 00 00 00"));
}
// -------------------- Format byte table (regression tripwire) --------------
// One-shot test that pins every format byte's bit-layout. If anyone
// edits the Format enum without updating the encoder this fires
// loudly.
TEST_CASE("E5 KAT: format byte layout per format code") {
struct Case { Format fmt; uint8_t code; };
Case cases[] = {
{Format::List, 000}, {Format::Binary, 010}, {Format::Boolean, 011},
{Format::ASCII, 020}, {Format::JIS8, 021}, {Format::C2, 022},
{Format::I8, 030}, {Format::I1, 031}, {Format::I2, 032},
{Format::I4, 034},
{Format::F8, 040}, {Format::F4, 044},
{Format::U8, 050}, {Format::U1, 051}, {Format::U2, 052},
{Format::U4, 054},
};
auto empty_for = [](Format f) -> Item {
switch (f) {
case Format::List: return Item::list({});
case Format::Binary: return Item::binary({});
case Format::Boolean: return Item::boolean(std::vector<uint8_t>{});
case Format::ASCII: return Item::ascii("");
case Format::JIS8: return Item::jis8("");
case Format::C2: return Item::c2(std::vector<uint16_t>{});
case Format::U1: return Item::u1(std::vector<uint8_t>{});
case Format::U2: return Item::u2(std::vector<uint16_t>{});
case Format::U4: return Item::u4(std::vector<uint32_t>{});
case Format::U8: return Item::u8(std::vector<uint64_t>{});
case Format::I1: return Item::i1(std::vector<int8_t>{});
case Format::I2: return Item::i2(std::vector<int16_t>{});
case Format::I4: return Item::i4(std::vector<int32_t>{});
case Format::I8: return Item::i8(std::vector<int64_t>{});
case Format::F4: return Item::f4(std::vector<float>{});
case Format::F8: return Item::f8(std::vector<double>{});
}
return Item{};
};
for (auto c : cases) {
INFO("format=" << static_cast<int>(c.fmt));
CHECK(static_cast<uint8_t>(c.fmt) == c.code);
// Per the spec the format byte for a 1-byte-length, empty body is
// (code << 2) | 1. Encoder must produce that.
const auto enc = encode(empty_for(c.fmt));
REQUIRE(enc.size() >= 2);
const uint8_t expected_fmt_byte = static_cast<uint8_t>((c.code << 2) | 1);
CHECK(enc[0] == expected_fmt_byte);
CHECK(enc[1] == 0); // length = 0
}
}