Use this page to select a reference case, see exactly what a passing run checks, and reproduce the result. For validation scope and recommended engineering checks, read Cross-Validation and Accuracy.
validation/crossval/manifest.json defines the case list, dependencies, exit
codes, checks, and saved artifacts. Automated tests compare
this page with that manifest so a case cannot be added or reclassified without
updating the documentation.
Result codes
Each script returns one of these exit codes:
| Exit | Meaning |
|---|---|
0 | Every self-check and numeric criterion passed, including each reference marked required_for_script_pass in the manifest. |
1 | A required run, self-check, or numeric criterion failed. |
2 | The rfx self-check completed, but a required external reference or optional dependency such as OpenEMS was unavailable. This result is inconclusive, not a pass. |
Some manifest entries also define a failure-line sentinel. The CPU runner
checks that sentinel after exit 0 and treats a contradictory failure message
as a failure.
Reference cases
Validated comparison means that the listed command or commands check the references explicitly identified as gated in that row. Diagnostic only means that the script reports a comparison for the stated configuration or checks a specific regression condition; it must not be used as evidence for broader support.
| Use | Case | Purpose | Reference | Pass criteria | Reproduce |
|---|---|---|---|---|---|
| Validated comparison | 01_waveguide_bend | 90° dielectric waveguide-bend transmittance | Meep “Basics” bend tutorial and a straight-guide self-check | Smoothed mean T ∈ [0.3, 1.0]; straight-guide T ∈ [0.95, 1.05]; |rfx − Meep| < 0.10 | python validation/crossval/01_waveguide_bend.py |
| Validated comparison | 02_ring_resonator | Ring-resonator resonant-mode frequencies | Meep “Basics” ring-mode tutorial and Harminv | Every Meep mode assigned a distinct rfx mode by a tolerance-free one-to-one assignment (an unmatched Meep mode fails); at least two modes; mean and per-mode frequency error < 5%; and mode Q within the record-length window tau_ref/T for each mode whose decay the harminv record observed (T/tau >= 1/4) — modes whose Q the record cannot resolve are reported, not gated | python validation/crossval/02_ring_resonator.py |
| Validated comparison | 03_straight_waveguide_flux | Straight-waveguide guided-mode dispersion, transmission, and energy conservation | Closed-form symmetric-slab TE0 n_eff for the declared recipe, the Meep “Basics” straight-waveguide tutorial, and the unit-transmission identity | Over the central source band (fcen ± 0.15*df): the measured n_eff(f) is within 2.0% of the analytic slab mode at every bin (max, not mean), and rfx and Meep band-mean T are each in [0.95, 1.05] with a band-mean difference < 0.05. T = flux_out/flux_in = 1 is an energy-conservation identity that holds for any lossless guide, so it is reported as a self-check and is not on its own evidence about the guide (issue #812). Peak-bin values are informational only. | python validation/crossval/03_straight_waveguide_flux.py |
| Validated comparison | 22_dispersive_slab_fresnel | Dispersive single-pole slab (Debye, Lorentz, Drude) normal-incidence R(f) and T(f), 4–10 GHz, on the 04_multilayer_fresnel rig with the record length derived from the slab’s own ring-down (−40 dB settling witness) | Analytic transfer matrix with the complex, frequency-dependent ε(f) of the declared pole, gated per bin and band-mean; Meep at 40 px/cm (the converged rung of a committed resolution ladder) with the same material (mapped into LorentzianSusceptibility / DrudeSusceptibility by validation/crossval/comparators/dispersive_eps.py, unit-tested to 1e-9 with wrong-convention falsifiers) is gated against the transfer matrix and rfx is gated against Meep. | Band-mean |ΔR| / |ΔT| vs the transfer matrix: Debye validation/crossval/_22_dispersive_results/rfx.json::arms.debye.mean_dR_gated = 0.0023 / validation/crossval/_22_dispersive_results/rfx.json::arms.debye.mean_dT_gated = 0.0028, Lorentz validation/crossval/_22_dispersive_results/rfx.json::arms.lorentz.mean_dR_gated = 0.0028 / validation/crossval/_22_dispersive_results/rfx.json::arms.lorentz.mean_dT_gated = 0.0016, Drude validation/crossval/_22_dispersive_results/rfx.json::arms.drude.mean_dR_gated = 0.00049 / validation/crossval/_22_dispersive_results/rfx.json::arms.drude.mean_dT_gated = 0.0017 against derived means of validation/crossval/_22_dispersive_results/rfx.json::arms.debye.mean_window_R = 0.01 / validation/crossval/_22_dispersive_results/rfx.json::arms.debye.mean_window_T = 0.017 (04_multilayer_fresnel committed envelope × 1.5 plus the ADE’s own temporal-discretization term); rfx vs Meep: Lorentz validation/crossval/_22_dispersive_results/rfx.json::arms.lorentz.meep.mean_dR_rfx_meep_gated = 0.004 / validation/crossval/_22_dispersive_results/rfx.json::arms.lorentz.meep.mean_dT_rfx_meep_gated = 0.004, Drude validation/crossval/_22_dispersive_results/rfx.json::arms.drude.meep.mean_dR_rfx_meep_gated = 0.0017 / validation/crossval/_22_dispersive_results/rfx.json::arms.drude.meep.mean_dT_rfx_meep_gated = 0.0078; verdict validation/crossval/_22_dispersive_results/rfx.json::verdict.exit_code = 0. Pre-declared before any run in docs/design_notes/20260902_cv22_dispersive_slab_predeclaration.md with falsifiers (pole frequency or relaxation time wrong, dispersion removed, Meep material mapped with the wrong convention) that each exited 1, e.g. f0 × 1.3: validation/crossval/_22_dispersive_results/rfx__falsifier_lorentz_f0_x1p3.json::arms.lorentz.mean_dR_gated = 0.093 in R. Scope: normal incidence only, one pole per model (Debye, Lorentz, Drude), no conductivity-plus-pole mix, 4–10 GHz, one slab thickness (10 mm) and one mesh (dx = 1 mm); the documented user path Simulation.add_material(..., debye_poles=/lorentz_poles=) (materials-geometry guide) is NOT exercised — the case drives update_e_debye / update_e_lorentz directly, as 04_multilayer_fresnel drives update_e. The Meep reference (40 px/cm) is converged only one rung under the window: Drude Meep-vs-transfer-matrix validation/crossval/_22_dispersive_results/rfx.json::arms.drude.meep.mean_dT_meep_tmm_gated = 0.0095 against validation/crossval/_22_dispersive_results/rfx.json::arms.drude.meep.mean_window4_T = 0.017 (1.8×); the 20 px/cm rung would fail it. Continuum gate + exact-lattice witness at every rung (docs/design_notes/20260903_lattice_witness_standard.md): the continuum windows above are unchanged; each of the three committed dx rungs additionally passes |rfx − exact 1-D Yee lattice| ≤ W_witness(f), per bin and band-mean, with W_witness derived from the lattice model’s own error budget (record truncation, incident-reference truncation, float32) and never fitted (the float32 term is a statistical estimate, not a bound: it is the √-accumulation size N ε32/√2; the coherent worst case is computed and reported, never gated) — validation/crossval/_22_dispersive_results/lattice_witness.json::verdict.all_rungs_ok, validation/crossval/_22_dispersive_results/lattice_witness.json::verdict.n_rungs = 3, band-mean |rfx − lattice| in R validation/crossval/_22_dispersive_results/lattice_witness.json::rungs.lorentz.mean_dR_lattice_gated = 0.00025 against validation/crossval/_22_dispersive_results/lattice_witness.json::rungs.lorentz.mean_W_witness_R_gated = 0.0026. The lattice model predicts this case’s whole transfer-matrix residual a priori (0.0022 / 0.0028 / 0.00049 against the measured 0.0023 / 0.0028 / 0.00049). On the Debye arm the wrong-model falsifier does not fire — that arm’s record settles only to validation/crossval/_22_dispersive_results/rfx.json::arms.debye.tail.scat_refl_rel = 0.0072, so its truncation term dominates its lattice term and its lattice gate is declared non-discriminating. That limitation is a property of the RUNG, not the arm: §8.1’s pre-declared remedy rung (--settling-bar 3e-4, committed as validation/crossval/_22_dispersive_diag/lattice_witness.json) ran on the 2026-09-03 VESSL leg and came out on the first branch of its own prediction — W_witness,R 0.017 → validation/crossval/_22_dispersive_diag/lattice_witness.json::rungs.debye_tail3e4.mean_W_witness_R_gated = 0.000332, |rfx − lattice| 6.5e-4 → validation/crossval/_22_dispersive_diag/lattice_witness.json::rungs.debye_tail3e4.mean_dR_lattice_gated = 1.06e-05, the wrong-model falsifier now firing at 6.68× on all 229 gated bins — while the arm’s residual against the continuum transfer matrix stayed put at validation/crossval/_22_dispersive_diag/rfx__debye_tail3e4.json::arms.debye.mean_dR_gated = 0.0022. That rung is a committed DIAGNOSTIC in its own directory and changes none of cv22’s three committed rungs. Recorded consequence (note §5.4): the slab family’s continuum gate is one slab’s discretisation error measured inside a window derived from another slab’s discretisation error — re-deriving W_mean,R from the non-lattice part of the same measurement (gate_from_envelope(1.69e-4, quantum=1000) = 0.001) fails 7 of the 12 committed rungs, and every measured residual equals its own lattice term to two significant figures. No window is changed. | python validation/crossval/22_dispersive_slab_fresnel.pypython validation/crossval/22_dispersive_slab_fresnel.py --lattice-witnesspytest -q tests/crossval/test_cv22_dispersive_slab_gates.py tests/crossval/test_cv22_dispersive_eps_mapping.py tests/crossval/test_lattice_witness_gates.py |
| Validated comparison | 23_lossy_slab_fresnel | Lossy (conductive) slab, ε’ = 4, 10 mm, σ from tan δ 0.1 / 1 / 3 at 7 GHz: normal-incidence R(f), T(f) and the absorption A(f) = 1 − R − T, 4–10 GHz, on the 22_dispersive_slab_fresnel rig (tand0p1 / tand1 at dx = 1 mm, tand3 at dx/2). Scope: normal incidence only, one thickness (10 mm), σ-only loss (no pole, no σ + pole mix), 4–10 GHz. | Analytic transfer matrix with the complex ε’ − jσ/(ωε0), gated per bin and band-mean on R, T and A; Meep with D_conductivity = σ·a/(c ε0 ε') (mapping unit-tested to 1e-9 with wrong-scaling falsifiers) at validation/crossval/_23_lossy_results/meep_tand0p1.json::resolution = 80 px/cm (tand0p1) and validation/crossval/_23_lossy_results/meep_tand3.json::resolution = 40 px/cm (tand1, tand3) gated against the transfer matrix and rfx gated against Meep. Two arms use the documented Simulation.add_material(..., sigma=) + add(Box) path (arrays asserted equal to the direct construction); the time stepping is the same update_e in every arm. | Band-mean |ΔR| / |ΔT| / |ΔA| vs the transfer matrix: tand0p1 validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.mean_dR_gated = 0.0039 / validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.mean_dT_gated = 0.0057 / validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.mean_dA_gated = 0.002 against validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.mean_window_R = 0.01 / validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.mean_window_T = 0.017 / validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.mean_window_A = 0.027, tand1 validation/crossval/_23_lossy_results/rfx.json::arms.tand1.mean_dR_gated = 0.0051 / validation/crossval/_23_lossy_results/rfx.json::arms.tand1.mean_dT_gated = 0.0018 / validation/crossval/_23_lossy_results/rfx.json::arms.tand1.mean_dA_gated = 0.0033 against validation/crossval/_23_lossy_results/rfx.json::arms.tand1.mean_window_R = 0.01 / validation/crossval/_23_lossy_results/rfx.json::arms.tand1.mean_window_T = 0.017 / validation/crossval/_23_lossy_results/rfx.json::arms.tand1.mean_window_A = 0.027, tand3 validation/crossval/_23_lossy_results/rfx.json::arms.tand3.mean_dR_gated = 0.0031 / validation/crossval/_23_lossy_results/rfx.json::arms.tand3.mean_dT_gated = 3e-5 / validation/crossval/_23_lossy_results/rfx.json::arms.tand3.mean_dA_gated = 0.0031 against validation/crossval/_23_lossy_results/rfx.json::arms.tand3.mean_window_R = 0.01 / validation/crossval/_23_lossy_results/rfx.json::arms.tand3.mean_window_T = 0.017 / validation/crossval/_23_lossy_results/rfx.json::arms.tand3.mean_window_A = 0.027 — the T gate on tand3 is vacuous as pre-declared (T_TMM ≤ 0.003 < 0.017 everywhere; R and A carry that arm) (windows: the 04_multilayer_fresnel committed envelope × 1.5 plus the σ update’s own temporal-discretization term; A windows are the triangle sums); rfx vs Meep: tand0p1 validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.meep.mean_dR_rfx_meep_gated = 0.0052 / validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.meep.mean_dA_rfx_meep_gated = 0.0028, tand3 validation/crossval/_23_lossy_results/rfx.json::arms.tand3.meep.mean_dR_rfx_meep_gated = 0.0022 / validation/crossval/_23_lossy_results/rfx.json::arms.tand3.meep.mean_dA_rfx_meep_gated = 0.0024; verdict validation/crossval/_23_lossy_results/rfx.json::verdict.exit_code = 0. The exact Yee-lattice solution of the staircase slab reproduces every arm (|rfx − lattice| validation/crossval/_23_lossy_results/rfx.json::arms.tand3.lattice.mean_dR_lattice_gated = 3e-5 on tand3) and is why tand3 runs at dx/2 — resolution, not tolerance; no physics window or gate moved in three rounds (one diagnostic assertion, the tail-fit block count, was relaxed after the measurement and the tail-fit keys post-processed — stated in the note’s §15). Pre-declared in docs/design_notes/20260902_cv23_lossy_slab_predeclaration.md with falsifiers that each exited 1 (σ × 1.5, σ = 0, a gain arm with R + T > 1, Meep σ_D mis-scaled), e.g. σ = 0 on tand3: validation/crossval/_23_lossy_results/rfx__falsifier_tand3_sigma_zero.json::arms.tand3.mean_dA_gated = 0.59 in A. The tand0p1 Meep reference is converged one rung under the window: Meep-vs-transfer-matrix validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.meep.mean_dR_meep_tmm_gated = 0.0078 against validation/crossval/_23_lossy_results/rfx.json::arms.tand0p1.meep.mean_window4_R = 0.01 (1.3×, with a known +a/80 thickness systematic in the reference: Meep minus TMM(d + a/80) is 8.8e-5 in the mean); the 40 px/cm rung would fail it — Meep’s first-order term is a one-cell thickness excess of a Block without eps_averaging (committed ladders and a centre-shift leg at validation/crossval/_23_lossy_results/meep_ladder_summary.json::diagnostics.tand0p1.res40_shift_half.mean_dR_meep_tmm_gated = 0.00024). Continuum gate + exact-lattice witness at every rung (docs/design_notes/20260903_lattice_witness_standard.md): the continuum windows above are unchanged; all nine committed dx rungs (three arms × dx, dx/2, dx/4) additionally pass |rfx − exact 1-D Yee lattice| ≤ W_witness(f) per bin and band-mean, W_witness derived from the lattice model’s own error budget (record truncation, incident-reference truncation, float32 — the float32 term a statistical estimate, not a bound) and never fitted (validation/crossval/_23_lossy_results/lattice_witness.json::verdict.all_rungs_ok, validation/crossval/_23_lossy_results/lattice_witness.json::verdict.n_rungs = 9; worst per-bin ratio over every rung and observable validation/crossval/_23_lossy_results/lattice_witness.json::rungs.tand3_dx4.worst_ratio_T = 0.3). This turns round 3’s reported lattice witness into a gate at every rung, and it un-vacuums the tand3 transmission observable the continuum gate cannot resolve. Recorded consequence (note §5.4): the slab family’s continuum gate is one slab’s discretisation error measured inside a window derived from another slab’s discretisation error — re-deriving W_mean,R from the non-lattice part of the same measurement (gate_from_envelope(1.69e-4, quantum=1000) = 0.001) fails 7 of the 12 committed rungs, and every measured residual equals its own lattice term to two significant figures. No window is changed. | python validation/crossval/23_lossy_slab_fresnel.pypython validation/crossval/23_lossy_slab_fresnel.py --lattice-witnesspytest -q tests/crossval/test_cv23_lossy_slab_gates.py tests/crossval/test_cv23_lossy_eps_mapping.py tests/crossval/test_lattice_witness_gates.py |
| Validated comparison | 04_multilayer_fresnel | Normal-incidence slab R(f) and T(f) | Analytic transfer-matrix/Fresnel solution (Taflove Ch. 5) and, when it is importable, Meep. Both references decide the verdict since #812: before it, Meep’s availability was required for exit 0 but every Meep number was printed and none was gated. | Against the analytic result, mean T error, mean R error, and R+T energy deviation are each < 0.05; every masked bin additionally satisfies |R+T−1| ≤ 0.06; a post-pulse settling-tail witness is gated; and a fringe-resolved gate compares each Fabry–Perot extremum of R(f) one at a time — position within W(f) = 2·(df/2 + Yee dispersion shift) = 59/106/235 MHz at the 3.75/7.50/11.24 GHz fringes, value within 0.04 (11% of the 0.36 fringe contrast). When Meep runs, max|Meep − analytic| ≤ 0.08, max|rfx − Meep| ≤ 0.16, and Meep’s own fringe positions are gated on its native flux grid. A band mean over an interference pattern is blind to shape error: with the mean gates alone the case passed while the true eps was 12.33% away (measured: T mean err 0.0494, R 0.0449, both under 0.05). Continuum gate + exact-lattice witness at every rung (docs/design_notes/20260903_lattice_witness_standard.md): --lattice-witness writes the exact 1-D Yee-lattice prediction of this very slab and the derived W_witness beside the run (validation/crossval/_04_fresnel_results/lattice_witness.json). At this case’s committed 719-step record the lattice witness is REPORTED, not gated — the record does not settle to −40 dB (tails 0.036 / 0.051 of the incident peak), so W_witness is validation/crossval/_04_fresnel_results/lattice_witness.json::rungs.slab_eps4.mean_W_witness_R_gated = 0.0535 in the gated mean, looser than this case’s own band-mean windows. The settled rung for the same slab is committed under cv23 (validation/crossval/_23_lossy_results/rfx__falsifier_tand0p1_sigma_zero.json), where |rfx − lattice| is 1.7e-4 against W_witness 5.5e-3 — cv04’s committed band-mean |ΔR| of 0.0066 is this slab’s Yee-lattice second-order term, not solver error. Scoped to |ΔR|: the lattice term is the same number in R and T (0.00727 over this case’s 170-bin mask; the slab is lossless, so A ≡ 0 and |Δlattice−TMM| is equal in the two), but the committed envelopes are 0.0066 in R (lattice/envelope 1.10) and 0.011 in T (0.66), and at cv04’s own rung |rfx − lattice| is 0.00168 in R against validation/crossval/_04_fresnel_results/lattice_witness.json::rungs.slab_eps4.mean_dT_lattice_gated = 0.00625 in T. More than half of the T envelope is this record’s truncation, so W_mean,T = 0.017 — derived from it — is not a discretisation number. | python validation/crossval/04_multilayer_fresnel.pypython validation/crossval/04_multilayer_fresnel.py --lattice-witnesspytest -q tests/crossval/test_lattice_witness_gates.py |
| Diagnostic only | 05_patch_antenna | 2.4 GHz rectangular patch resonance on FR4 (coarse-mesh integration check) | OpenEMS “Simple Patch Antenna” tutorial and Balanis transmission-line estimate, both reported not gated | Reports a coarse-mesh (dx=1 mm) resonance and exercises the probe-fed patch / finite-GP / CPML / non-uniform-z / openEMS-lumped-port path end to end. It does not gate accuracy; since issue #812 the reported resonance is the mode identified as TM100 of the declared spectrum (a mis-realized 22.0 mm patch is assigned TM110 and fails by name: tests/fixtures/patch_mode_identification/cv05_ringdown_spectra.json::runs.patch_len_22p0mm.modes[0].freq = 2.993459 GHz), an identification window of [−11.11 %, +12.50 %] that is not an accuracy claim — and, fired against the instrument, a 38.0 mm build passes on the committed fixture because a weak extra pole (tests/fixtures/patch_mode_identification/cv05_ringdown_spectra.json::runs.patch_len_38p0mm.modes[1].freq = 2.609612 GHz) inside the window is named TM100; the committed patch evidence lives in tests/locks/test_patch_edgefed_resonance_harminv.py, tests/locks/test_patch_edgefed_s11_passivity.py, tests/oracle/test_patch_cavity_eps_oracle.py, and tests/crossval/test_patch_canonical_farfield_e4.py (committed thirds-rule OpenEMS far-field reference, envelope-locked). These are signed-envelope regression locks and characterizations, not accuracy claims — the harminv gate pins a measured ~−6% offset from the realized-raster Balanis value WITH its sign, and says so in its own docstring; the pre-#702 9.32/9.21 GHz patch numbers are retired (issue #782). | python validation/crossval/05_patch_antenna.pypytest -q -m slow tests/crossval/test_patch_canonical_farfield_e4.py |
| Validated comparison | 06b_msl_notch_filter_uniform | MSL open-stub notch filter using distributed add_msl_port on a uniform mesh | Analytic quarter-wave notch plus a separately gated, committed OpenEMS dx=50 µm result | Script gate (re-derived 2026-09-01, issue #812 P3): notch frequency located by sub-bin log-parabolic vertex, error < 4.0%; −10 dB stopband fractional width within ±20% of the ideal shunt-open-stub closed form 0.210274; half-grid resolution witness < 1.000 bin; median Z₀ ∈ (40, 65) Ω. The < −10 dB depth check is RETAINED as a witness only — the worst sampled minimum on this grid is −31.23 dB, so it cannot fail while a notch exists. Judged on this board on 2026-09-02 (issue #812 round 2, VESSL 369367257702): every gate passed — notch error validation/crossval/_06b_msl_notch_results/cv06b_build_falsifiers_summary.json::criterion_A_baseline.err_pct = 1.453 %, width ratio validation/crossval/_06b_msl_notch_results/cv06b_build_falsifiers_summary.json::criterion_A_baseline.bw_ratio = 0.9684, witness validation/crossval/_06b_msl_notch_results/cv06b_build_falsifiers_summary.json::criterion_A_baseline.witness_bins = 0.3175 bin — and a 5-cell stub fires the width gate while the depth witness stays blind. One pre-declared falsifier FIRED: a one-cell stub-length error moved the refined notch estimate validation/crossval/_06b_msl_notch_results/cv06b_build_falsifiers_summary.json::stub_1cell.refined_delta_pct = 0.145 % against a predicted validation/crossval/_06b_msl_notch_results/cv06b_build_falsifiers_summary.json::stub_1cell.true_shift_pct = 0.532 %, so sub-bin resolution of the notch frequency on this board is not demonstrated (cause not attributed; the 4 % verdict does not depend on it). The committed 2026-08-27 run predates the two new gates (< 15% / < −10 dB were the gates in force then, history not current). Fixture-test gate: both results passive, notch-frequency difference ≤ 7%, off-notch mean/max |S21| difference ≤ 0.13/0.25. | python validation/crossval/06b_msl_notch_filter_uniform.pypytest -q tests/crossval/test_msl_notch_e4_comparison_gates.py |
| Diagnostic only | 07_sheen_lpf | Sheen 1990 stepped-impedance microstrip low-pass filter stopband | OpenEMS matched-geometry run, whose own passivity and Re(Z₀) sanity are gated first, plus a committed Palace FEM conformal-mesh referee (tests/fixtures/sheen_lpf_e4/) | Characterization locks only, no accuracy gate. The referee shows the stopband is a double transmission zero (~7.0 and ~8.0 GHz) that OpenEMS matches to ~0.7%. The rfx leg was regenerated 2026-07-26 with the measured recipe (num_periods = 60, settling witness −72 dB vs the −24.7 dB truncation failure of the original num_periods = 20 leg; n_probe_offset = 30, which removes a measured probe near-field contamination — moving the rfx argmin from 7.218 to 7.874 GHz, the ~8 GHz doublet member, 1.4% from OpenEMS — while keeping the λ_g/4 reflector clearance that offset = 40 violated). Quoted rfx |S| is passivity-enforced (strict ≤ 1, max column power 0.9995); the raw excess is recorded per bin in passivity_correction (0/120 bins > 0.05 on the current leg, worst 0.0145 at 17.38 GHz — the pre-#516 leg’s 84/120 bins, worst 0.365 at 18.0 GHz, was substantially extractor defects, not a coarse-mesh envelope). The fitted passband median Re(Z₀) (50.3 Ω vs OpenEMS’s ~51 Ω) is printed, not gated — the old ~67 Ω fit-conditioning artifact is gone; the in-band median is 52.4 Ω. Gated: OpenEMS reference passive (column power ≤ 1.10), median Re(Z₀) ∈ (40, 65) Ω, passband mean |S21| ≥ 0.85, stopband zero ≤ −20 dB; committed bin counts; the committed argmin nulls to 1%, the rfx passband mean to 1%, and the referee verdict; and the evidence chain (strict bound, settling witness < −40 dB, correction footprint) failing closed when a leg lacks the witness/enforcement fields. Re-gated 2026-09-01 (issue #812 mechanism P3, docs/design_notes/estimator_resolution_regate.md): the argmin null was bin-quantised at 163.866 MHz = 2.081% per bin against a declared 1% window, so erasing one of the two transmission zeros passed all 17 gates. The argmin lock is kept and four families were added — both doublet members per solver located by sub-bin log-parabolic vertex and locked to ±0.50% against the committed referee fixture’s own values; the transmission-zero COUNT (== 2, depth ≤ −20 dB); the −3 dB corner frequency (±0.25%), which this case never gated; and an in-run half-grid resolution witness that a bin-quantised estimator scores exactly 1.0000 bin on and cannot pass. 28/28 gates pass on the committed legs. | python validation/crossval/07_sheen_lpf.pypytest -q tests/crossval/test_sheen_lpf_palace_referee_gates.py |
| Validated comparison | 09_half_symmetric_waveguide | PMC symmetry plane versus a full PEC cavity for TE₁₀₁ | Analytic rectangular-cavity result (Pozar Ch. 6) and full-versus-half self-check | Realized geometry (issue #812): every solved extent, and the realized PMC mirror plane a_eff = 2 × (H_tan wall), within dx/4 = 127 µm of declared; f_full and f_half are each within 10% of analytic; |f_full − f_half|/f_full < (d²/(a²+d²))·(dx/4)/a = 0.356% (tightened from 5%, which tolerated a full cell of mirror-plane error) | python validation/crossval/09_half_symmetric_waveguide.py |
| Diagnostic only | 10_pmc_cpml_half_symmetric | PMC+CPML per-face composition on uniform and nonuniform grids | Peak-stability self-check, bit-exact PMC realization, and a mirrored full-domain image-source amplitude control; no external solver | For each path: peak spread (max−min)/max < 0.02; no NaN or Inf; max|H_tan| on the declared y_lo face exactly 0.0 at every cpml_layers (with a non-degeneracy guard that the field is non-zero somewhere); and the half-domain PMC peak within 2% of the equivalent full-domain in-phase image-pair peak at cpml_layers = 8. The last two exist because the first is a RELATIVE spread that a constant amplitude factor cancels — issue #812 measured that deleting apply_pmc_faces improves it from 0.2487% to 0.0667%. | python validation/crossval/10_pmc_cpml_half_symmetric.pypytest -q tests/crossval/test_crossval10_pmc_regate.py |
| Diagnostic only | 11_waveguide_port_wr90 | WR-90 port results for empty guide, PEC short, and a dielectric slab | Analytic matched-guide, short-circuit, and Airy references are gated; optional Meep, OpenEMS, and Palace result files populate comparison tables | Empty guide: band-mean linear-magnitude difference < 0.02 for S11 and < 0.03 for S21. PEC short: band-mean magnitude difference < 0.05, band-mean round-trip phase difference < 15°, per-frequency |S11| ∈ [0.93, 1.07], and a max|S11| ≤ 1.05 passivity ceiling (a start-up self-test proves the per-frequency gate rejects a synthetic spike). Slab: band-mean magnitude difference < 0.10 for S11 and < 0.07 for S21; masked band-mean phase difference < 60° where |S_ref| ≥ 0.30; maximum complex-S difference < 0.30. | python validation/crossval/11_waveguide_port_wr90.py |
| Validated comparison | 14_rect_cavity_pozar | Rectangular air-filled PEC cavity eigenfrequencies | Exact closed-form Pozar cavity resonance f_mnl = (c/2)·√((m/a)²+(n/b)²+(l/d)²), re-derived in the script; no external solver | Wall registration: the effective separation (n−1)·dx must equal (a,b,d) to within 1e-9 m on every axis; TE₁₀₁ frequency error < 1%; every one of TE₁₀₁/TM₁₁₀/TE₀₁₁/TM₁₁₁/TE₂₀₁/TM₂₁₀/TE₁₀₂ within 2% (max over all seven, and a mode that cannot be extracted fails the gate); and each mode within one tenth of the record’s Fourier bin of the closed-form discrete-Yee eigenvalue of the same box, a second exact oracle also re-derived in the script. For the (a,b,d) = (50,30,40) mm cavity on a dx = 1 mm mesh that divides all three extents exactly. The cavity is closed and lossless, so the ring-down settling witness does not apply and harminv Q is a window-length artifact that is printed but not gated; only the frequency is gated. | python validation/crossval/14_rect_cavity_pozar.py |
| Validated comparison | 24_nu_rect_cavity_pozar | The 14_rect_cavity_pozar cavity on GRADED-z meshes: uniform 1 mm control, one 0.5 mm band, small–large–small–large, and uniform 0.5 mm as the cost control. Scope: one closed lossless PEC cavity, z grading only, chain ratio ≤ 1.4 (the #785 envelope), seven modes below 8.5 GHz; no S-parameters, ports, absorbers, materials or in-plane grading. | Exact closed-form Pozar spectrum (E2) with the exact discrete Yee-lattice eigenfrequency of each arm’s own grid as the a-priori witness, gated to 4 ppm (validation/crossval/_24_nu_cavity_results/rfx.json::declared.w_est = 4e-06); the #810 allowance gate on the equal-dt spatial eigenvalue with the allowance derived from #785’s per-transition reflection budget; mode count 7 by (m, n, l); #785’s energy envelope; two-window stationarity. | Pre-declared before the run (docs/design_notes/20260902_cv24_nu_cavity_predeclaration.md) and measured (VESSL 369367257837, every gate true on every arm, validation/crossval/_24_nu_cavity_results/rfx.json): lattice residual ≤ 0.48 ppm on every arm and mode, ≤ 0.08 ppm on the graded arms (validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.multi_band.rows.TE102.resid_lattice = -0.00001 %); TE101 validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.uniform.rows.TE101.dev_raw = -0.00833 % / validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.single_band.rows.TE101.dev_raw = -0.01497 % / validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.multi_band.rows.TE101.dev_raw = -0.01411 % / validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.uniform_fine.rows.TE101.dev_raw = -0.00205 % and TE102 validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.uniform.rows.TE102.dev_raw = -0.05202 % / validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.single_band.rows.TE102.dev_raw = -0.08214 % / validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.multi_band.rows.TE102.dev_raw = -0.01821 % / validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.uniform_fine.rows.TE102.dev_raw = -0.01294 % vs Pozar on the four arms, each within 0.5 ppm of its prediction; five falsifiers each exit 1 as declared. Read plainly: the graded-z mesh is within the derived allowance of the uniform mesh it refines (equal dt: within 11 ppm on the l = 1 modes, +107 ppm worse on TE102 for the single band, validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.single_band.rows.TE102.dev_spatial = -0.10157 % vs validation/crossval/_24_nu_cavity_results/rfx.json::evaluations.uniform.rows.TE102.dev_spatial = -0.09091 %, and 532 ppm better for the two-band profile; raw, at each arm’s own CFL dt, 60–300 ppm worse on every l ≤ 1 mode) and the grading adds no error beyond the operator’s own metric (lattice residual ≤ 0.08 ppm) — but it is NOT as accurate as uniform-fine: it buys only the z-share of each mode’s error (78 % of uniform-fine’s TE102 improvement at 15 % of the cells, validation/crossval/_24_nu_cavity_results/rfx.json::arms.multi_band.cost.n_cells = 73500 vs validation/crossval/_24_nu_cavity_results/rfx.json::arms.uniform_fine.cost.n_cells = 480000; nothing on the l = 0 modes); the #810 reflection-derived allowance is a loose bound for eigenfrequencies (≤ 17 % used; its bound contains the control’s own dispersion and cannot fail for an in-envelope profile), the lattice gate (4 ppm) carries the whole mesh claim. | python validation/crossval/24_nu_rect_cavity_pozar.py (VESSL: scripts/vessl_cv24_nu_cavity.yaml) |
| Diagnostic only | 15_patch_antenna_rt5880 | Probe-fed 40×50 mm patch on RT/Duroid 5880 (eps_r 2.2, h = 3.175 mm), uniform-mesh integration study | OpenEMS same-geometry run is gated; its canonical “Simple Patch Antenna” tutorial reproduce-check and a Balanis transmission-line estimate are reported, not gated | Same-geometry integration envelope, not a patch-accuracy claim: rfx ring-down f0 versus the OpenEMS S11 dip within 8% (issue #812: blind to the #740 one-plane-ground realization, validation/crossval/_15_patch_results/rfx_one_plane_ground_b29f9de7.json::f_harminv_hz = 2471858108 Hz vs validation/crossval/_15_patch_results/openems.json::f_dip_hz = 2330000000 Hz, +6.09 % — only the build assertion catches it; a ratio-band re-gate was tried and withdrawn as an honest STOP), the −40 dB open-domain settling witness, passivity on both legs (max|S11| ≤ 1.05), and broadside directivity within 3 dB. The substrate is asserted to rasterize to exactly four z-cells on the uniform mesh before any f0 is quoted. Patch evidence remains delegated to 05_patch_antenna and the committed tests it names (regression locks and characterizations, not accuracy claims). | python validation/crossval/15_patch_antenna_rt5880.py |
| Diagnostic only | 16_pec_sphere_mie_ka_sweep | PEC-sphere monostatic RCS versus the exact Mie series, ka 0.5–4.0 | Exact conducting-sphere Mie backscatter series (Ruck 1970), re-implemented twice (the script’s committed self-witnessing oracle, and again in the frozen-fixture gates from scipy.special; a four-way convention-independent check is recorded in the PR #475 review); no external solver | Two-tier posture at a derived CPU-scale operating point (cells-per-radius 6.4 coarse / 12.8 fine — the sphere is never cell-starved at low ka). Gated: coarse ka ≤ 1.25 and the fine rung at ka = 2.0 only, at gate = round-up(measured clearance-scan envelope × 1.5); the envelope population (a 7-point clearance scan plus three domain realizations) is committed and the gate test recomputes it, so gate and envelope cannot drift apart. Reported, not gated: coarse ka ≥ 1.5 and fine ka = 3.0/4.0 — near the deep Mie interference nulls, under a domain-size-only change, the domain-to-domain spread reaches 8.0 dB (coarse, at ka = 1.75) and 14.5 dB (fine, at ka = 3.0 across the committed clearance scan), the worst single-point deviation is 11.1 dB (coarse ka = 3.0) / 9.3 dB (fine ka = 3.0), the local-minimum positions move with domain size, and fine ka = 4.0 fails a 3.5 dB gate at 9 of 13 clearances (the review’s anti-aliasing scan; the original 3-clearance sample happened to hit passing values). Attribution: truncation, domain and clearance witnesses are committed fixture data; the effective-radius and 9.6-rung resolution probes are recorded as provenance of the 2026-07-27 offline session, not data; incident leakage at the monostatic bin is excluded by the documented backscatter leakage null (the formerly-quoted sub-vs-unsub probe was retracted as a same-array tautology — monostatic_rcs is unsubtracted by construction). Extends, does not replace, the committed coarse ladder and fine ka≈1 fixtures. | python validation/crossval/16_pec_sphere_mie_ka_sweep.pypytest -q tests/crossval/test_rcs_mie_ka_sweep_gates.py |
| Diagnostic only | 17_dielectric_sphere_mie | Lossless dielectric sphere (eps_r 2.56) monostatic RCS versus the exact Mie series, ka 0.5–2.5 | Exact Bohren–Huffman dielectric Mie backscatter series, re-implemented twice with self-witnesses (Rayleigh, m→1, convergence, unitarity); no external solver | The material-path twin of the PEC ka-sweep and the first cross-method record of the binary dielectric rasterize path (no sub-cell interface averaging exists — this measures that staircase, it does not certify smooth interfaces). Single gated tier: coarse ka ≤ 1.25 within 6.3 dB = round-up(committed 7-point clearance-scan envelope 4.18 dB × 1.5), a regression lock rather than a converged-accuracy claim. No fine rung is gated — a measured decision: the committed cpr-12.8 witness scan envelopes 3.75/3.04 dB, barely below the coarse 4.18 (the single-clearance apparent convergence was the aliasing class the PEC review exposed, caught here before commit). Reported, not gated: every bin with ka ≥ 1.5 — domain-size-only spreads of 11.7 dB (ka = 1.75) and 29.0 dB (ka = 2.5), worse than the PEC twin; truncation falsified by direct probes; incident leakage at the monostatic bin excluded by the documented backscatter leakage null (monostatic_rcs is unsubtracted by construction, so a sub-vs-unsub probe is a same-array tautology for this observable). Bempp PMCHWT corroboration is an offline follow-up. Issue #812 re-gate: the dB tier is an RCS-agreement envelope, not a permittivity calibration — the 6.3 dB gate accepts a rasterized eps_r across the committed model pass set validation/crossval/_17_dielectric_results/material_blind_window.json::summary.pass_runs_eps[0][0] = 2.0–validation/crossval/_17_dielectric_results/material_blind_window.json::summary.pass_runs_eps[0][1] = 4.5 and validation/crossval/_17_dielectric_results/material_blind_window.json::summary.pass_runs_eps[1][0] = 5.0–validation/crossval/_17_dielectric_results/material_blind_window.json::summary.pass_runs_eps[1][1] = 5.6, with a fail island validation/crossval/_17_dielectric_results/material_blind_window.json::summary.fail_islands_eps[0].eps_r[0] = 4.6–validation/crossval/_17_dielectric_results/material_blind_window.json::summary.fail_islands_eps[0].eps_r[3] = 4.9 (worst bin validation/crossval/_17_dielectric_results/material_blind_window.json::summary.fail_islands_eps[0].max_abs_delta_db = 9.787 dB) on the ka = 1.25 Mie resonance, first failing below at validation/crossval/_17_dielectric_results/material_blind_window.json::summary.first_failing_eps_below = 1.8; live defect runs at 2.0 / 5.5 (pass) and 1.8 / 6.0 (fail) agree with the model, while inside the island the live solver passes at 4.6–4.8 and fails only at 4.9 (validation/crossval/_17_dielectric_results/cv17_permittivity_island.json::summary.n_verdicts_agree_with_model = 3 of 6 — the model over-predicts on the resonance, the live window is wider), because the Mie oracle moves only about 10 dB per unit relative permittivity at the gated ka (9.816/10.202/9.978/7.134 at ka = 0.50/0.75/1.00/1.25) and the gate is already round-up(envelope × 1.5). The permittivity is gated on a channel of its own instead: the value read back out of the rasterized array must sit within 0.5% relative of the declared 2.56 (half the gate’s 0.1 dB reporting quantum divided by that worst slope), and the array must hold exactly two distinct values — the first check anywhere of the binary-rasterize claim this row’s scope rests on. | python validation/crossval/17_dielectric_sphere_mie.pypytest -q tests/crossval/test_rcs_dielectric_sphere_mie_gates.py |
| Diagnostic only | 18_wr90_iris_modematch | Single symmetric inductive iris in WR-90, flux-normalized |S11| versus a mode-matching oracle, 8.2–12.4 GHz (29 points) | TEn0 mode-matching cascade (two H-plane width-step junctions + a below-cutoff section), re-implemented twice with self-witnesses (unitarity 1.1e-16; Marcuvitz cot² thin-limit anchor 10.8%); the pre-merge review independently reproduced it with a formulation-independent 2-D H-plane FDFD to 6e-4; no external solver | The first calibrated evidence for a PEC obstacle in the rectangular-waveguide S-parameter lane (iris/post/septum remain experimental; multi-iris filters stay fenced — this is the stage-1 prerequisite). Gated: fine rung dx = a/60 within 0.04 = round-up(measured envelope 0.0232 × 1.5) across 8 configurations (three apertures × centred/off-centre iris, plus two guide lengths — every configuration lands within 0.023), and the Richardson extrapolation 2·S(a/60) − S(a/30) on the oracle within 0.01 at every one of those pairs, which cross-confirms the oracle together with the first-order discretization attribution (gap ratios 0.527–0.604). Reported, not gated: the coarse rung (0.018–0.043), the raw-extraction record (worse than flux; pointwise difference up to 0.033), and residual detrended ripple, measured against the oracle so its own curvature is not counted (fine ≤ 0.0077, coarse ≤ 0.0158). Three setup defects found during the campaign are each fenced by an assert or a derived setting: a parasitic wall-slot, half-ulp-fragile node-plane box corners, and a fin footprint that made the electrical aperture d + 2·dx — the last, together with a too-thin absorber, had inflated an earlier envelope 4–6×. The earlier modal-extraction fence is retracted: its column-power 1.112–1.164 evidence does not survive the corrected setup (now 1.0013–1.0207, zero extractor warnings), and the withdrawn fence is replaced by a committed witness that records modal accuracy too — modal gaps run 0.0002–0.0055 worse than flux, which is why flux still carries the gate. Issue #812 re-gate: the fine gate is now per configuration — round-up of each configuration’s own envelope × 1.5 at quantum 1000, i.e. 0.019/0.034/0.015/0.022/0.035/0.015/0.034/0.034, with the pooled 0.04 retained unchanged as a ceiling — because the pooled value was set by the worst configuration and then spent at all eight. Aperture resolution is now stated rather than implied, and its numbers are committed in validation/crossval/_18_wr90_iris_results/aperture_resolution.json (re-derived from the committed traces by an independent oracle in the gate test) rather than restated here: a one-cell aperture error at each rung is detected as an over-aperture at every configuration and is not resolved with margin as an under-aperture at any of them. At the strong aperture the under-aperture defect even scores better than the undefected row, because the fine rung’s own effective aperture is half a fine cell wider than nominal — so narrowing the geometry by a cell moves it toward the measured trace rather than away. The Richardson witness is blind to the whole class in both signs by construction — a one-cell-per-rung error is proportional to dx, which is exactly what 2·S(a/60) − S(a/30) removes — so no tightening of its 0.01 gate could catch it. The calibration this stage supplies downstream is therefore aperture-resolved to +1 fine cell, not to −1, and the three declared apertures are pinned as claims (an exact and even cell count at both rungs). | python validation/crossval/18_wr90_iris_modematch.pypytest -q tests/crossval/test_wr90_iris_modematch_gates.py |
| Diagnostic only | 19_wr90_iris_filter_aghanim | A published 4th-order WR-90 inductive-iris bandpass filter (five irises, four coupled cavities), flux-normalized |S11| versus a mode-matching cascade oracle, 10.40–11.70 GHz (131 points at 10 MHz) | TEn0 mode-matching cascade with arbitrary aperture position, carrying the evanescent modes across every cavity. Self-witnessed (unitarity 2.2e-15, reciprocity and mirror symmetry exact, mode convergence 8.8e-4, an L → 0 collapse witness at 2.5e-05 measured), and its N=1 limit reproduces the stage-1 single-iris result to 1.05e-04 in an independent odd-mode formulation — the object a pre-merge review confirmed against a formulation-independent 2-D H-plane FDFD to 6e-4. Those witnesses have known limits: unitarity constrains only the propagating modes, mirror symmetry holds by construction here, and a formulation-independent check of the five-iris cascade now backs the oracle: a 2-D H-plane FDFD sharing only numpy/scipy with it agrees to 1.1 MHz in centre frequency and 1.0 MHz in bandwidth after first-order Richardson extrapolation over three refinement levels whose two extrapolation estimates agree to 0.4 MHz — a same-model consistency check, not an error bound: the three levels fit convergence order ≈1.25 rather than exactly 1, worth ~0.7–1.0 MHz of order-assumption uncertainty in the extrapolate, and f0 and bandwidth derive from the same two band edges, one confirmation rather than two — while rfx differs from both routes by the same ~12–13 MHz, placing that residual on the FDTD side. The published HFSS/CST curves are a digitized anchor, not a solver run — this case does not compare rfx against either commercial code. | The first resonant multi-obstacle case in the rectangular-waveguide lane: unlike a single iris, a per-face geometry error here is a passband shift rather than a magnitude tolerance. Gated: centre frequency within 19 MHz = round-up(envelope 12.1230 × 1.5); the structural reflection-zero count, witnessed invariant across the 8.0–8.5-cell iris-thickness ambiguity band by a committed oracle-side sweep; and passband contiguity as a regression lock (span_holes ≤ 1 — the f0 gate alone cannot see a split passband, since band edges are the outermost crossings), against the oracle on the as-realized geometry — measured +12.08 MHz, 3 zeros versus 3, one interior hole bin. The envelope is a population of nine configurations over four setup axes (guide height, run length, port standoff, absorber depth), each axis carrying an interior sample as well as an endpoint — an endpoint-only design cannot detect non-monotonic sensitivity, which is how PR #475 failed — spanning 12.0605–12.1230 MHz, a 0.06 MHz spread: the residual is a reproducible systematic difference, about 0.12 cell of cavity length at the measured −105 MHz/cell, not a setup artifact. Band edges (+17.08/+7.09 MHz) and bandwidth (−9.99 MHz) are reported, not gated, and they are one fact rather than two since the bandwidth residual is identically the difference of the edge residuals. They are not gated because the oracle is fed the geometry that was built and one leg of that convention is unsettled: the cavity leg (L+1)·dx is confirmed to 0.04–0.17 cell and the transverse aperture leg to better than 0.05 cell, but the iris-thickness leg measures ≈(t−0.68)·dx across four drawn thicknesses, matching neither integer rule, which leaves a half-cell comparator-input uncertainty worth ~40 MHz of bandwidth against f0’s ~2.4 MHz. Handing the oracle the drawn counts instead biases f0 by +107.5 MHz, and the envelope×1.5 rule does not catch that class — it bounds scatter, and that is bias. Also reported: worst in-band return loss, ripple levels, every zero depth, contiguity detail beyond the span_holes lock (the built filter has one 10 MHz bin at 9.80 dB inside its 340 MHz span, longest contiguous run 270 MHz, where the oracle is contiguous), the coarse a/60 rung, and phase. Setup is gated separately from physics — a num_periods doubling with a column-power settling criterion (one witness family, not two: the energy ring-down witness is not populated on the waveguide S-matrix path, #538), feed clearance and absorber depth each hold the edges to one 10 MHz bin. The built structure is a snapped version of the published filter: centre frequency within the reference’s own solver scatter of CST though not of HFSS, one of four structural zeros lost, and worst return loss degraded 13.82 → 10.65 dB by rasterization alone. Multi-iris filters, posts and septa remain experimental; this measures one published design on one mesh and supports no broader claim. | pytest -q tests/crossval/test_wr90_iris_filter_gates.pypython validation/crossval/19_wr90_iris_filter_aghanim.py (≈19 min, excluded from the CPU lane) |
| Diagnostic only | 20_msl_phase_referee | MSL de-embedded phase (angle(S21)) versus an external OpenEMS referee, on a matched-mesh (dx = 50 µm) RO4350B thru line, W = 600 µm, h = 254 µm, L12 = 5 mm | OpenEMS MSL_NotchFilter.py tutorial reproduce-gate (Stage A, against a Hammerstad–Jensen closed-form notch frequency) plus a matched-mesh thru-line comparison against rfx’s own compute_msl_s_matrix (Stage B) | Answers whether rfx’s un-gated MSL phase was a time-convention mismatch or a reference-plane mismatch: both tools’ DFT kernels are identical at the source, so it was reference-plane, resolved by placing OpenEMS’s measurement plane at rfx’s own probe-0 coordinate. Gated: each solver’s own angle(S21) against its own measured beta (self-consistency, a 3° budget from a ±4-cell plane-position allowance, resolving-power tested against a planted referral defect), plus passivity and settling guards, over 3.0–4.5 GHz. Also gated since 2026-09-01 (issue #812): the raw cross-solver angle(S21) difference itself, at 3°, measured validation/crossval/_issue812_phase_identity/regate_evidence.json::cv20.run2_realized_board.cross_solver_max_abs_raw_phase_diff_deg = 0.3418 degrees inside the gated band. It was reported-only before that date, which mattered because the self-consistency leg alone cannot fail on a coherent phase-velocity error — a factor-2 error reads validation/crossval/_issue812_phase_identity/regate_evidence.json::cv20.blindness.audit_construction_e1_max_phase_dev_deg = 0.241 degrees against its 3° gate. The 500 MHz bin is excluded from any claim (the 5 mm line is ~0.03 guide wavelengths there and three-point port extraction is ill-conditioned). Reported, not gated: the dispersion-corrected residual, which is provably blind to that same error class. OpenEMS’s CalcPort reference-plane-referral rotation is unexercised on this fixture (the effective shift measures 0.0 by construction) and remains untested off-grid. | python validation/crossval/20_msl_phase_referee.pypytest -q tests/crossval/test_msl_phase_referee_header.py |
| Diagnostic only | 21_coax_two_port_referee | Coax two-port through-line |S21|, phase, and group delay versus an external OpenEMS referee, on a matched-mesh (dx = 0.37474 mm) SMA-class PTFE coax (a = 0.635 mm, b = 2.055 mm, eps_r = 2.1, L12 = 58.4595 mm) | OpenEMS Coax.m tutorial reproduce-gate (Stage A, against Coax.m’s own closed-form line-impedance check) plus a matched-mesh CoaxialPort thru-line comparison against rfx’s own compute_coaxial_two_port fixture (Stage B, forward + reverse drive, full S-matrix) | Comparator-leg referee — brackets, does not judge rfx’s own numbers; this run was one leg of the evidence chain behind a SEPARATE, later decision (issue #489, 2026-08-06) that lifted compute_coaxial_two_port’s own EXPERIMENTAL label to VALIDATED WITH SCOPE (through-line class, this coax geometry family; see docs/guides/sparameter_support_matrix.md for the full scope statement — coax-to-planar transitions stay EXPERIMENTAL, diagnostic-only). Gated: Stage A reproduces Coax.m’s own documented ZL check (measured 50.432 Ω vs analytic 49.940 Ω, max dev 0.511 Ω); Stage B gates each solver’s own self-consistency — passivity (|S11|² + |S21|² ≤ 1.05, measured ≤ 1.0001), a one-sided matched-through magnitude band ([0.5, 1.1]; the gated central-band leg measured 0.9999966–1.0000282 combined across S21/S12), and a phase/group-delay witness against the port’s own measured beta, not an idealized analytic one: the analytic-beta path fails by 111.38° (>30° tolerance) even though its own magnitude and group-delay legs pass (282.6 ps expected, 35.4 ps deviation, inside a 200 ps tolerance) — switching to the port’s own measured beta collapses the same run-3 data to < 0.44° phase deviation on both S21 and S12. This is staircase dispersion (the referral hypothesis is ruled out by the implied-length mismatch matching no candidate discrete referral-cell count) from this coax’s own coarse a-to-b PTFE annulus (measured/analytic beta ratio ~1.12×, 1.1205–1.1212 in the gated central band; ~3.8 cells across) — now measured by a mesh-refinement convergence witness (issue #489 leg 1, VESSL 369367251845, 2026-08-05, one-sided acceptance band pre-declared before the run): a 1.5× mesh refinement (annulus 3.79 → 5.68 cells) moves the measured/analytic beta ratio from 1.1208 to 1.0662 (implied convergence order p ≈ 1.5 (two-point, from this single 1.5× step — not a multi-level fit), between the naive first-order and second-order bounds — consistent with openEMS’s own material averaging at a dielectric interface, not a fault) — the pre-declared one-sided acceptance band CONFIRMED. The four agreeing beta readings (port1/port2, both drives) at the original mesh density remain one witness, not four (same run, same solver, same CalcPort extractor); the mesh-refinement run above is the independent second axis. Reported, not gated: run-3’s own |S21| ≈ 1.000 (0.99997–1.00003) across the full 4–12 GHz sweep for this independent, differently-discretized geometry — consistent with, not a refutation of, rfx’s own raw 0.960 → 0.737 decline (a different geometry/discretization); reciprocity (|S21| vs |S12|, max magnitude deviation 1.2 × 10⁻⁵, max phase deviation 0.0012° — essentially exact for this passive fixture). Does not characterize a different geometry, mesh density, coax-to-planar transition, AD, or the reverse_drive call-site wiring (the active #489 dev track). | python validation/crossval/21_coax_two_port_referee.pypytest -q tests/crossval/test_coax_two_port_referee_header.py |
Case-specific cautions
05_patch_antenna
The OpenEMS comparison requires CSXCAD and openEMS. If either is unavailable,
the script exits 2: the rfx Harminv and lumped-port \|S11\| self-checks run,
but the OpenEMS comparison does not.
The single-cell lumped port has parasitic reactance. Use the Harminv frequency
as the resonance result. The shallow local \|S11\| dip is only a passivity
and local-dip check; it is not an absolute feed calibration.
The script’s own OpenEMS comparison is coarse (no thirds-rule edge mesh) and is
reported, not gated. A separate committed reference from the canonical
thirds-rule recipe (scripts/diagnostics/patch_tutorial_openems.py; f_res
2.4221 GHz, Q 20.1, broadside D 6.79 dBi, from our source-built openEMS run —
full provenance in the fixture meta block) is locked by
tests/crossval/test_patch_canonical_farfield_e4.py. Those reference numbers are
the 32 mm feed-axis mode, and the rfx side gated against them is that same design
mode — the upper ring-down mode, 2.6954 GHz — not the 40 mm cross mode.
| Gate | Constant | Measured |
|---|---|---|
| broadside directivity | D_ABS_TOL_DB = 1.0 | 0.60 dB at the design-mode bin (7.39 vs 6.79 dBi) |
| E-/H-plane beam peaks | PEAK_ANGLE_TOL_DEG = 15.0 from broadside | 0° / −3° (openEMS 0° / 0°) |
| resonance at dx = 2 mm | F_RES_REL_LO = +0.06, F_RES_REL_HI = +0.16 — a HIGH band, sign locked | +11.3% |
| ring-down settling | SETTLING_BAR_DB = -40.0, asserted before any gated number | the num_periods=90 demo of this fixture reads −36.5 dB, so the test runs 110 |
The frequency band is a discretization-bias regression lock, not an accuracy claim. It was repinned on 2026-08-27 (issue #693): the earlier envelopes — 0.5 dB with a measured 0.08 dB, and a −12% to −5% band with a measured −8.6% — had been pinned against the 40 mm cross mode (2.2147 GHz), the wrong member of the patch’s near-degenerate pair, and the surrounding prose was corrected on 2026-08-31 (#812 Phase 0). Text quoting 0.08 dB or −8.6% is retired.
The sign belongs to this fixture’s sheet realization and mesh, not to a general
“coarse grid reads low” rule. On the same geometry the corrected two_plane
realization measures −4.7% at dx = 2 mm and −1.5% at dx = 1 mm, against +11.3%
for the one-plane sheets gated above, and validation/crossval/manifest.json
records the older cv05 agreement as “a documented z-under-res/staircase error
cancellation”.
06b_msl_notch_filter_uniform
The script applies loose analytic tolerances to the uniform-mesh
add_msl_port result. It runs at dx = 63.5 µm = h_sub/4, chosen so the
254 µm substrate rasterizes exactly; the analytic quarter-wave reference is
evaluated on the realized 635 µm trace width read from the run’s own
fidelity_report(), not on the declared 600 µm.
The committed dx=50 µm rfx/OpenEMS comparison is checked separately by
tests/crossval/test_msl_notch_e4_comparison_gates.py. It covers that geometry and
resolution only, not arbitrary MSL structures or meshes — and it is not the
same board: its rfx leg is generated at dx = 50 µm, where the same 254 µm
substrate rasterizes to 300 µm. Until that generator is regenerated, read the
two as characterizing different boards.
10_pmc_cpml_half_symmetric
This is a regression check with no external solver. It verifies only that far-face absorber thickness does not materially change the interior peak for the scripted uniform and nonuniform cases.
11_waveguide_port_wr90
This script prints a diagnostic report. The
empty-guide and PEC-short magnitude criteria are direct checks. The dielectric
slab phase and complex-S tolerances account for different analytic and
solver-monitor reference planes and therefore must not be read as general
phase-accuracy limits. The regression checks are in
tests/oracle/test_waveguide_port_validation_battery.py.
Missing optional external result files do not skip the analytic checks. Read the per-frequency table before interpreting a passing exit code.
Cases not included
Research-only subgrid scripts are under validation/research/subgrid/. They are
not in the cross-validation manifest and are not documented as supported
reference cases.
Reproduce the CPU subset
Run the CPU-feasible cases with:
PYTHONPATH=. python scripts/run_crossval_cpu.pyThe runner starts each script in a fresh subprocess, applies a per-script timeout, and prints one of these results:
| Result | Meaning |
|---|---|
PASS | The script returned 0 and no failure sentinel was found. |
SELF-CHECK-ONLY | The script returned 2; required external evidence is unavailable, so the result is inconclusive. |
FAIL | The script returned 1, another unexpected exit code, or a contradictory failure sentinel. |
ENV-SKIP | An optional reference solver such as Meep is installed but cannot be imported. This is an environment or packaging problem, not an rfx result. |
TIMEOUT | The script exceeded its configured time limit. |
The runner exits 0 only when no script ends in FAIL or TIMEOUT.
The CPU runner does not attempt these cases:
06b_msl_notch_filter_uniform: GPU only. Measured, not projected: a CPU run of the shipped dx = 63.5 µm mesh was abandoned unfinished at 2 h 52 m, while a single RTX 4090 solves it in 329 s. Do not budget a CPU slot for it.
Cases 01 through 04 require a Meep build compatible with the host NumPy ABI.
If Meep cannot be imported, the runner reports ENV-SKIP instead of FAIL;
cases 01 and 04 still complete their rfx-side self-checks.