Benchmarks

Use this page to select a reference case, see exactly what a passing run checks, and reproduce the result. For validation scope and recommended engineering checks, read Cross-Validation and Accuracy.

validation/crossval/manifest.json defines the case list, dependencies, exit codes, checks, and saved artifacts. Automated tests compare this page with that manifest so a case cannot be added or reclassified without updating the documentation.

Pass criteria are case-specific

The metrics and tolerances below come from each script’s docstring and check block. A pass applies to the stated geometry, mesh, frequency range, and reference only. It does not by itself validate other geometries, arbitrary discretizations, or an entire port or component family.

Result codes

Each script returns one of these exit codes:

Exit	Meaning
0	Every self-check and numeric criterion passed, including each reference marked required_for_script_pass in the manifest.
1	A required run, self-check, or numeric criterion failed.
2	The rfx self-check completed, but a required external reference or optional dependency such as OpenEMS was unavailable. This result is inconclusive, not a pass.

Some manifest entries also define a failure-line sentinel. The CPU runner checks that sentinel after exit 0 and treats a contradictory failure message as a failure.

Reference cases

Validated comparison means that the listed command or commands check the references explicitly identified as gated in that row. Diagnostic only means that the script reports a comparison for the stated configuration or checks a specific regression condition; it must not be used as evidence for broader support.

Use	Case	Purpose	Reference	Pass criteria	Reproduce
Validated comparison	01_waveguide_bend	90° dielectric waveguide-bend transmittance	Meep “Basics” bend tutorial and a straight-guide self-check	Smoothed mean T ∈ [0.3, 1.0]; straight-guide T ∈ [0.95, 1.05]; |rfx − Meep| < 0.10	python validation/crossval/01_waveguide_bend.py
Validated comparison	02_ring_resonator	Ring-resonator resonant-mode frequencies	Meep “Basics” ring-mode tutorial and Harminv	Mean mode-frequency error < 5%; at least two rfx and Meep modes matched	python validation/crossval/02_ring_resonator.py
Validated comparison	03_straight_waveguide_flux	Straight-waveguide transmission and energy conservation	Meep “Basics” straight-waveguide tutorial and unit-transmission self-check	Over the central source band (fcen ± 0.15*df), rfx and Meep band-mean T are each in [0.95, 1.05], and their band-mean difference is < 0.05. Peak-bin values are informational only.	python validation/crossval/03_straight_waveguide_flux.py
Validated comparison	04_multilayer_fresnel	Normal-incidence slab R(f) and T(f)	Analytic transfer-matrix/Fresnel solution (Taflove Ch. 5) is gated. Meep values are informational; Meep availability is required for exit 0, but numeric Meep agreement is not gated.	Against the analytic result, mean T error, mean R error, and R+T energy deviation are each < 0.05; every masked bin additionally satisfies |R+T−1| ≤ 0.06, and a post-pulse settling-tail witness is gated.	python validation/crossval/04_multilayer_fresnel.py
Diagnostic only	05_patch_antenna	2.4 GHz rectangular patch resonance on FR4 (coarse-mesh integration check)	OpenEMS “Simple Patch Antenna” tutorial and Balanis transmission-line estimate, both reported not gated	Reports a coarse-mesh (dx=1 mm) resonance and exercises the probe-fed patch / finite-GP / CPML / non-uniform-z / openEMS-lumped-port path end to end. It does not gate accuracy; patch-accuracy evidence lives in the committed tests tests/test_issue80_patch_resonance_harminv.py, tests/test_issue80_patch_s11_regression.py, tests/test_patch_cavity_eps_oracle.py, and tests/test_patch_canonical_farfield_e4.py (committed thirds-rule OpenEMS far-field reference, envelope-locked).	python validation/crossval/05_patch_antenna.py pytest -q -m slow tests/test_patch_canonical_farfield_e4.py
Validated comparison	06b_msl_notch_filter_uniform	MSL open-stub notch filter using distributed add_msl_port on a uniform mesh	Analytic quarter-wave notch plus a separately gated, committed OpenEMS dx=50 µm result	Script gate: notch-frequency error < 15%, depth < −10 dB, median Z₀ ∈ (40, 65) Ω. Fixture-test gate: both results passive, notch-frequency difference ≤ 7%, off-notch mean/max |S21| difference ≤ 0.13/0.25.	python validation/crossval/06b_msl_notch_filter_uniform.py pytest -q tests/test_msl_notch_e4_comparison_gates.py
Diagnostic only	07_sheen_lpf	Sheen 1990 stepped-impedance microstrip low-pass filter stopband	OpenEMS matched-geometry run, whose own passivity and Re(Z₀) sanity are gated first, plus a committed Palace FEM conformal-mesh referee (tests/fixtures/sheen_lpf_e4/)	Characterization locks only, no accuracy gate. The referee shows the stopband is a double transmission zero (~7.0 and ~8.0 GHz) that OpenEMS matches to ~0.7%. The rfx leg was regenerated 2026-07-26 with the measured recipe (num_periods = 60, settling witness −72 dB vs the −24.7 dB truncation failure of the original num_periods = 20 leg; n_probe_offset = 30, which removes a measured probe near-field contamination — moving the rfx argmin from 7.218 to 7.874 GHz, the ~8 GHz doublet member, 1.4% from OpenEMS — while keeping the λ_g/4 reflector clearance that offset = 40 violated). Quoted rfx |S| is passivity-enforced (strict ≤ 1, max column power 0.9938); the raw excess is recorded per bin in passivity_correction (84/120 bins > 0.05, worst 0.365 at 18.0 GHz — the dx = 200 µm coarse-mesh envelope, bounded and recorded rather than quoted). The fitted passband median Re(Z₀) (~67 Ω vs OpenEMS’s ~51 Ω) is printed, not gated — a low-frequency fit-conditioning artifact; the in-band median is 49.8 Ω. Gated: OpenEMS reference passive (column power ≤ 1.10), median Re(Z₀) ∈ (40, 65) Ω, passband mean |S21| ≥ 0.85, stopband zero ≤ −20 dB; committed bin counts; the committed argmin nulls to 1%, the rfx passband mean to 1%, and the referee verdict; and the evidence chain (strict bound, settling witness < −40 dB, correction footprint) failing closed when a leg lacks the witness/enforcement fields.	python validation/crossval/07_sheen_lpf.py pytest -q tests/test_sheen_lpf_palace_referee_gates.py
Validated comparison	09_half_symmetric_waveguide	PMC symmetry plane versus a full PEC cavity for TE₁₀₁	Analytic rectangular-cavity result (Pozar Ch. 6) and full-versus-half self-check	f_full and f_half are each within 10% of analytic; |f_full − f_half|/f_full < 5%	python validation/crossval/09_half_symmetric_waveguide.py
Diagnostic only	10_pmc_cpml_half_symmetric	PMC+CPML per-face composition on uniform and nonuniform grids	Peak-stability self-check; no external solver	For each path, peak spread (max−min)/max < 0.02; no NaN or Inf	python validation/crossval/10_pmc_cpml_half_symmetric.py
Diagnostic only	11_waveguide_port_wr90	WR-90 port results for empty guide, PEC short, and a dielectric slab	Analytic matched-guide, short-circuit, and Airy references are gated; optional Meep, OpenEMS, and Palace result files populate comparison tables	Empty guide: band-mean linear-magnitude difference < 0.02 for S11 and < 0.03 for S21. PEC short: band-mean magnitude difference < 0.05, band-mean round-trip phase difference < 15°, per-frequency |S11| ∈ [0.93, 1.07], and a max|S11| ≤ 1.05 passivity ceiling (a start-up self-test proves the per-frequency gate rejects a synthetic spike). Slab: band-mean magnitude difference < 0.10 for S11 and < 0.07 for S21; masked band-mean phase difference < 60° where |S_ref| ≥ 0.30; maximum complex-S difference < 0.30.	python validation/crossval/11_waveguide_port_wr90.py
Validated comparison	14_rect_cavity_pozar	Rectangular air-filled PEC cavity eigenfrequencies	Exact closed-form Pozar cavity resonance f_mnl = (c/2)·√((m/a)²+(n/b)²+(l/d)²), re-derived in the script; no external solver	TE₁₀₁ frequency error < 1% and at least one higher mode of TM₁₁₀/TE₀₁₁/TM₁₁₁/TE₂₀₁/TM₂₁₀/TE₁₀₂ within 2%, for the (a,b,d) = (50,30,40) mm cavity on a dx = 1 mm mesh that divides all three extents exactly. The cavity is closed and lossless, so the ring-down settling witness does not apply and harminv Q is a window-length artifact that is printed but not gated; only the frequency is gated.	python validation/crossval/14_rect_cavity_pozar.py
Diagnostic only	15_patch_antenna_rt5880	Probe-fed 40×50 mm patch on RT/Duroid 5880 (eps_r 2.2, h = 3.175 mm), uniform-mesh integration study	OpenEMS same-geometry run is gated; its canonical “Simple Patch Antenna” tutorial reproduce-check and a Balanis transmission-line estimate are reported, not gated	Same-geometry integration envelope, not a patch-accuracy claim: rfx ring-down f0 versus the OpenEMS S11 dip within 8%, the −40 dB open-domain settling witness, passivity on both legs (max|S11| ≤ 1.05), and broadside directivity within 3 dB. The substrate is asserted to rasterize to exactly four z-cells on the uniform mesh before any f0 is quoted. Patch accuracy remains delegated to 05_patch_antenna and the committed tests it names.	python validation/crossval/15_patch_antenna_rt5880.py
Diagnostic only	16_pec_sphere_mie_ka_sweep	PEC-sphere monostatic RCS versus the exact Mie series, ka 0.5–4.0	Exact conducting-sphere Mie backscatter series (Ruck 1970), re-implemented twice (the script’s committed self-witnessing oracle, and again in the frozen-fixture gates from scipy.special; a four-way convention-independent check is recorded in the PR #475 review); no external solver	Two-tier posture at a derived CPU-scale operating point (cells-per-radius 6.4 coarse / 12.8 fine — the sphere is never cell-starved at low ka). Gated: coarse ka ≤ 1.25 and the fine rung at ka = 2.0 only, at gate = round-up(measured clearance-scan envelope × 1.5); the envelope population (a 7-point clearance scan plus three domain realizations) is committed and the gate test recomputes it, so gate and envelope cannot drift apart. Reported, not gated: coarse ka ≥ 1.5 and fine ka = 3.0/4.0 — near the deep Mie interference nulls, under a domain-size-only change, the domain-to-domain spread reaches 8.0 dB (coarse, at ka = 1.75) and 14.5 dB (fine, at ka = 3.0 across the committed clearance scan), the worst single-point deviation is 11.1 dB (coarse ka = 3.0) / 9.3 dB (fine ka = 3.0), the local-minimum positions move with domain size, and fine ka = 4.0 fails a 3.5 dB gate at 9 of 13 clearances (the review’s anti-aliasing scan; the original 3-clearance sample happened to hit passing values). Attribution: truncation, domain and clearance witnesses are committed fixture data; the effective-radius and 9.6-rung resolution probes are recorded as provenance of the 2026-07-27 offline session, not data; incident leakage at the monostatic bin is excluded by the documented backscatter leakage null (the formerly-quoted sub-vs-unsub probe was retracted as a same-array tautology — monostatic_rcs is unsubtracted by construction). Extends, does not replace, the committed coarse ladder and fine ka≈1 fixtures.	python validation/crossval/16_pec_sphere_mie_ka_sweep.py pytest -q tests/test_rcs_mie_ka_sweep_gates.py
Diagnostic only	17_dielectric_sphere_mie	Lossless dielectric sphere (eps_r 2.56) monostatic RCS versus the exact Mie series, ka 0.5–2.5	Exact Bohren–Huffman dielectric Mie backscatter series, re-implemented twice with self-witnesses (Rayleigh, m→1, convergence, unitarity); no external solver	The material-path twin of the PEC ka-sweep and the first cross-method record of the binary dielectric rasterize path (no sub-cell interface averaging exists — this measures that staircase, it does not certify smooth interfaces). Single gated tier: coarse ka ≤ 1.25 within 6.3 dB = round-up(committed 7-point clearance-scan envelope 4.18 dB × 1.5), a regression lock rather than a converged-accuracy claim. No fine rung is gated — a measured decision: the committed cpr-12.8 witness scan envelopes 3.75/3.04 dB, barely below the coarse 4.18 (the single-clearance apparent convergence was the aliasing class the PEC review exposed, caught here before commit). Reported, not gated: every bin with ka ≥ 1.5 — domain-size-only spreads of 11.7 dB (ka = 1.75) and 29.0 dB (ka = 2.5), worse than the PEC twin; truncation falsified by direct probes; incident leakage at the monostatic bin excluded by the documented backscatter leakage null (monostatic_rcs is unsubtracted by construction, so a sub-vs-unsub probe is a same-array tautology for this observable). Bempp PMCHWT corroboration is an offline follow-up.	python validation/crossval/17_dielectric_sphere_mie.py pytest -q tests/test_rcs_dielectric_sphere_mie_gates.py
Diagnostic only	18_wr90_iris_modematch	Single symmetric inductive iris in WR-90, flux-normalized |S11| versus a mode-matching oracle, 8.2–12.4 GHz (29 points)	TEn0 mode-matching cascade (two H-plane width-step junctions + a below-cutoff section), re-implemented twice with self-witnesses (unitarity 1.1e-16; Marcuvitz cot² thin-limit anchor 10.8%); the pre-merge review independently reproduced it with a formulation-independent 2-D H-plane FDFD to 6e-4; no external solver	The first calibrated evidence for a PEC obstacle in the rectangular-waveguide S-parameter lane (iris/post/septum remain experimental; multi-iris filters stay fenced — this is the stage-1 prerequisite). Gated: fine rung dx = a/60 within 0.04 = round-up(measured envelope 0.0232 × 1.5) across 8 configurations (three apertures × centred/off-centre iris, plus two guide lengths — every configuration lands within 0.023), and the Richardson extrapolation 2·S(a/60) − S(a/30) on the oracle within 0.01 at every one of those pairs, which cross-confirms the oracle together with the first-order discretization attribution (gap ratios 0.527–0.604). Reported, not gated: the coarse rung (0.018–0.043), the raw-extraction record (worse than flux; pointwise difference up to 0.033), and residual detrended ripple, measured against the oracle so its own curvature is not counted (fine ≤ 0.0077, coarse ≤ 0.0158). Three setup defects found during the campaign are each fenced by an assert or a derived setting: a parasitic wall-slot, half-ulp-fragile node-plane box corners, and a fin footprint that made the electrical aperture d + 2·dx — the last, together with a too-thin absorber, had inflated an earlier envelope 4–6×. The earlier modal-extraction fence is retracted: its column-power 1.112–1.164 evidence does not survive the corrected setup (now 1.0013–1.0207, zero extractor warnings), and the withdrawn fence is replaced by a committed witness that records modal accuracy too — modal gaps run 0.0002–0.0055 worse than flux, which is why flux still carries the gate.	python validation/crossval/18_wr90_iris_modematch.py pytest -q tests/test_wr90_iris_modematch_gates.py
Diagnostic only	19_wr90_iris_filter_aghanim	A published 4th-order WR-90 inductive-iris bandpass filter (five irises, four coupled cavities), flux-normalized |S11| versus a mode-matching cascade oracle, 10.40–11.70 GHz (131 points at 10 MHz)	TEn0 mode-matching cascade with arbitrary aperture position, carrying the evanescent modes across every cavity. Self-witnessed (unitarity 2.2e-15, reciprocity and mirror symmetry exact, mode convergence 8.8e-4, an L → 0 collapse witness at 2.5e-05 measured), and its N=1 limit reproduces the stage-1 single-iris result to 1.05e-04 in an independent odd-mode formulation — the object a pre-merge review confirmed against a formulation-independent 2-D H-plane FDFD to 6e-4. Those witnesses have known limits: unitarity constrains only the propagating modes, mirror symmetry holds by construction here, and a formulation-independent check of the five-iris cascade now backs the oracle: a 2-D H-plane FDFD sharing only numpy/scipy with it agrees to 1.1 MHz in centre frequency and 1.0 MHz in bandwidth after first-order Richardson extrapolation over three refinement levels whose two extrapolation estimates agree to 0.4 MHz — a same-model consistency check, not an error bound: the three levels fit convergence order ≈1.25 rather than exactly 1, worth ~0.7–1.0 MHz of order-assumption uncertainty in the extrapolate, and f0 and bandwidth derive from the same two band edges, one confirmation rather than two — while rfx differs from both routes by the same ~12–13 MHz, placing that residual on the FDTD side. The published HFSS/CST curves are a digitized anchor, not a solver run — this case does not compare rfx against either commercial code.	The first resonant multi-obstacle case in the rectangular-waveguide lane: unlike a single iris, a per-face geometry error here is a passband shift rather than a magnitude tolerance. Gated: centre frequency within 19 MHz = round-up(envelope 12.1230 × 1.5); the structural reflection-zero count, witnessed invariant across the 8.0–8.5-cell iris-thickness ambiguity band by a committed oracle-side sweep; and passband contiguity as a regression lock (span_holes ≤ 1 — the f0 gate alone cannot see a split passband, since band edges are the outermost crossings), against the oracle on the as-realized geometry — measured +12.08 MHz, 3 zeros versus 3, one interior hole bin. The envelope is a population of nine configurations over four setup axes (guide height, run length, port standoff, absorber depth), each axis carrying an interior sample as well as an endpoint — an endpoint-only design cannot detect non-monotonic sensitivity, which is how PR #475 failed — spanning 12.0605–12.1230 MHz, a 0.06 MHz spread: the residual is a reproducible systematic difference, about 0.12 cell of cavity length at the measured −105 MHz/cell, not a setup artifact. Band edges (+17.08/+7.09 MHz) and bandwidth (−9.99 MHz) are reported, not gated, and they are one fact rather than two since the bandwidth residual is identically the difference of the edge residuals. They are not gated because the oracle is fed the geometry that was built and one leg of that convention is unsettled: the cavity leg (L+1)·dx is confirmed to 0.04–0.17 cell and the transverse aperture leg to better than 0.05 cell, but the iris-thickness leg measures ≈(t−0.68)·dx across four drawn thicknesses, matching neither integer rule, which leaves a half-cell comparator-input uncertainty worth ~40 MHz of bandwidth against f0’s ~2.4 MHz. Handing the oracle the drawn counts instead biases f0 by +107.5 MHz, and the envelope×1.5 rule does not catch that class — it bounds scatter, and that is bias. Also reported: worst in-band return loss, ripple levels, every zero depth, contiguity detail beyond the span_holes lock (the built filter has one 10 MHz bin at 9.80 dB inside its 340 MHz span, longest contiguous run 270 MHz, where the oracle is contiguous), the coarse a/60 rung, and phase. Setup is gated separately from physics — a num_periods doubling with a column-power settling criterion (one witness family, not two: the energy ring-down witness is not populated on the waveguide S-matrix path, #538), feed clearance and absorber depth each hold the edges to one 10 MHz bin. The built structure is a snapped version of the published filter: centre frequency within the reference’s own solver scatter of CST though not of HFSS, one of four structural zeros lost, and worst return loss degraded 13.82 → 10.65 dB by rasterization alone. Multi-iris filters, posts and septa remain experimental; this measures one published design on one mesh and supports no broader claim.	pytest -q tests/test_wr90_iris_filter_gates.py python validation/crossval/19_wr90_iris_filter_aghanim.py (≈19 min, excluded from the CPU lane)
Diagnostic only	20_msl_phase_referee	MSL de-embedded phase (angle(S21)) versus an external OpenEMS referee, on a matched-mesh (dx = 50 µm) RO4350B thru line, W = 600 µm, h = 254 µm, L12 = 5 mm	OpenEMS MSL_NotchFilter.py tutorial reproduce-gate (Stage A, against a Hammerstad–Jensen closed-form notch frequency) plus a matched-mesh thru-line comparison against rfx’s own compute_msl_s_matrix (Stage B)	Answers whether rfx’s un-gated MSL phase was a time-convention mismatch or a reference-plane mismatch: both tools’ DFT kernels are identical at the source, so it was reference-plane, resolved by placing OpenEMS’s measurement plane at rfx’s own probe-0 coordinate. Gated: each solver’s own angle(S21) against its own measured beta (self-consistency, a 3° budget from a ±4-cell plane-position allowance, resolving-power tested against a planted referral defect), plus passivity and settling guards, over 3.0–4.5 GHz. Reported, not gated: the raw cross-solver angle(S21) difference — measured ≤ 1° in 22 of 30 bins, ≤ 0.304° inside the gated band, and only the 500 MHz bin exceeds 3° (4.13°, where the 5 mm line is ~0.03 guide wavelengths and three-point port extraction is ill-conditioned; excluded from any claim). OpenEMS’s CalcPort reference-plane-referral rotation is unexercised on this fixture (the effective shift measures 0.0 by construction) and remains untested off-grid.	python validation/crossval/20_msl_phase_referee.py pytest -q tests/test_msl_phase_referee_header.py
Diagnostic only	21_coax_two_port_referee	Coax two-port through-line |S21|, phase, and group delay versus an external OpenEMS referee, on a matched-mesh (dx = 0.37474 mm) SMA-class PTFE coax (a = 0.635 mm, b = 2.055 mm, eps_r = 2.1, L12 = 58.4595 mm)	OpenEMS Coax.m tutorial reproduce-gate (Stage A, against Coax.m’s own closed-form line-impedance check) plus a matched-mesh CoaxialPort thru-line comparison against rfx’s own compute_coaxial_two_port fixture (Stage B, forward + reverse drive, full S-matrix)	Comparator-leg referee — brackets, does not judge rfx’s own numbers; this run was one leg of the evidence chain behind a SEPARATE, later decision (issue #489, 2026-08-06) that lifted compute_coaxial_two_port’s own EXPERIMENTAL label to VALIDATED WITH SCOPE (through-line class, this coax geometry family; see docs/guides/sparameter_support_matrix.md for the full scope statement — coax-to-planar transitions stay EXPERIMENTAL, diagnostic-only). Gated: Stage A reproduces Coax.m’s own documented ZL check (measured 50.432 Ω vs analytic 49.940 Ω, max dev 0.511 Ω); Stage B gates each solver’s own self-consistency — passivity (|S11|² + |S21|² ≤ 1.05, measured ≤ 1.0001), a one-sided matched-through magnitude band ([0.5, 1.1]; the gated central-band leg measured 0.9999966–1.0000282 combined across S21/S12), and a phase/group-delay witness against the port’s own measured beta, not an idealized analytic one: the analytic-beta path fails by 111.38° (>30° tolerance) even though its own magnitude and group-delay legs pass (282.6 ps expected, 35.4 ps deviation, inside a 200 ps tolerance) — switching to the port’s own measured beta collapses the same run-3 data to < 0.44° phase deviation on both S21 and S12. This is staircase dispersion (the referral hypothesis is ruled out by the implied-length mismatch matching no candidate discrete referral-cell count) from this coax’s own coarse a-to-b PTFE annulus (measured/analytic beta ratio ~1.12×, 1.1205–1.1212 in the gated central band; ~3.8 cells across) — now measured by a mesh-refinement convergence witness (issue #489 leg 1, VESSL 369367251845, 2026-08-05, one-sided acceptance band pre-declared before the run): a 1.5× mesh refinement (annulus 3.79 → 5.68 cells) moves the measured/analytic beta ratio from 1.1208 to 1.0662 (implied convergence order p ≈ 1.5 (two-point, from this single 1.5× step — not a multi-level fit), between the naive first-order and second-order bounds — consistent with openEMS’s own material averaging at a dielectric interface, not a fault) — the pre-declared one-sided acceptance band CONFIRMED. The four agreeing beta readings (port1/port2, both drives) at the original mesh density remain one witness, not four (same run, same solver, same CalcPort extractor); the mesh-refinement run above is the independent second axis. Reported, not gated: run-3’s own |S21| ≈ 1.000 (0.99997–1.00003) across the full 4–12 GHz sweep for this independent, differently-discretized geometry — consistent with, not a refutation of, rfx’s own raw 0.960 → 0.737 decline (a different geometry/discretization); reciprocity (|S21| vs |S12|, max magnitude deviation 1.2 × 10⁻⁵, max phase deviation 0.0012° — essentially exact for this passive fixture). Does not characterize a different geometry, mesh density, coax-to-planar transition, AD, or the reverse_drive call-site wiring (the active #489 dev track).	python validation/crossval/21_coax_two_port_referee.py pytest -q tests/test_coax_two_port_referee_header.py

Case-specific cautions

05_patch_antenna

The OpenEMS comparison requires CSXCAD and openEMS. If either is unavailable, the script exits 2: the rfx Harminv and lumped-port \|S11\| self-checks run, but the OpenEMS comparison does not.

The single-cell lumped port has parasitic reactance. Use the Harminv frequency as the resonance result. The shallow local \|S11\| dip is only a passivity and local-dip check; it is not an absolute feed calibration.

The script’s own OpenEMS comparison is coarse (no thirds-rule edge mesh) and is reported, not gated. A separate committed reference from the canonical thirds-rule recipe (scripts/diagnostics/patch_tutorial_openems.py; f_res 2.4221 GHz, broadside D 6.79 dBi, from our source-built openEMS run — full provenance in the fixture) is locked by tests/test_patch_canonical_farfield_e4.py: broadside directivity within 0.5 dB of the reference (measured difference 0.08 dB), beam peaks within 15° of broadside, and the resonance inside the documented dx = 2 mm low-bias band of −12% to −5% (measured −8.6%). The frequency band is a discretization-bias regression lock, not an accuracy claim.

06b_msl_notch_filter_uniform

The script applies loose analytic tolerances to the uniform-mesh add_msl_port result. The committed dx=50 µm rfx/OpenEMS comparison is checked separately by tests/test_msl_notch_e4_comparison_gates.py. It covers that geometry and resolution only, not arbitrary MSL structures or meshes.

10_pmc_cpml_half_symmetric

This is a regression check with no external solver. It verifies only that far-face absorber thickness does not materially change the interior peak for the scripted uniform and nonuniform cases.

11_waveguide_port_wr90

This script prints a diagnostic report. The empty-guide and PEC-short magnitude criteria are direct checks. The dielectric slab phase and complex-S tolerances account for different analytic and solver-monitor reference planes and therefore must not be read as general phase-accuracy limits. The regression checks are in tests/test_waveguide_port_validation_battery.py.

Missing optional external result files do not skip the analytic checks. Read the per-frequency table before interpreting a passing exit code.

Cases not included

Research-only subgrid scripts are under validation/research/subgrid/. They are not in the cross-validation manifest and are not documented as supported reference cases.

Reproduce the CPU subset

Run the CPU-feasible cases with:

PYTHONPATH=. python scripts/run_crossval_cpu.py

The runner starts each script in a fresh subprocess, applies a per-script timeout, and prints one of these results:

Result	Meaning
PASS	The script returned 0 and no failure sentinel was found.
SELF-CHECK-ONLY	The script returned 2; required external evidence is unavailable, so the result is inconclusive.
FAIL	The script returned 1, another unexpected exit code, or a contradictory failure sentinel.
ENV-SKIP	An optional reference solver such as Meep is installed but cannot be imported. This is an environment or packaging problem, not an rfx result.
TIMEOUT	The script exceeded its configured time limit.

The runner exits 0 only when no script ends in FAIL or TIMEOUT.

The CPU runner does not attempt these cases:

• 06b_msl_notch_filter_uniform: more than about 10 minutes on CPU because of the fine uniform mesh and distributed MSL-port de-embedding.

Cases 01 through 04 require a Meep build compatible with the host NumPy ABI. If Meep cannot be imported, the runner reports ENV-SKIP instead of FAIL; cases 01 and 04 still complete their rfx-side self-checks.