Cross-Validation and Accuracy

A finished run, a clean preflight and a smooth plot do not show that a number is right. This page answers the question you should ask before you report an rfx result: how do I know it is correct? It covers what rfx has already checked, what only you can check for your structure, and where the evidence lives.

rfx tests its solver in three independent ways.

The cross-solver cases compare against reference records that are frozen in the repository, so no external solver is needed to re-check rfx against them. Benchmarks lists the cases, what each one shows and how to reproduce it.

Two limits apply to all of this evidence:

• A passing case covers its own setup. The geometry, mesh, band, port family and observable of that case. It does not cover your structure.
• Some features have little or no evidence. The support tables in Support Boundaries say which are recommended, which are limited to stated conditions, and which are outside the documented scope. The maintainer support matrix has the detail for each feature.

rfx cannot validate your structure for you. Before you report a number, work through this list.

1. Preflight. Read every warning from sim.preflight(). Each one names a condition that can make the result wrong without an error: a mesh too coarse for the material, a probe inside the absorber, geometry that lands off the grid, a lossless resonator.
2. Run length. Check result.settling_witness["status"]. It is pass when, at every frequency the run’s spectral results read, the part of the value still ringing past the record’s end is at or below −40 dB (settling_db is the worst of these). fail, undetermined and absent are not passes; above the bar, spectra, S-parameters and far fields still carry a cut-off transient. For gradients, the record must be longer still: rfx.gradient_record_length_witness checks that a gradient does not change when the record is doubled.
3. Mesh convergence. Refine the mesh and report how far your number moves, not only the finest answer. Convergence Study walks through one.
4. Physical sanity. A passive structure has |S| ≤ 1. A reciprocal one has S21 = S12. A lossless one conserves power, |S11|² + |S21|² = 1 or R + T = 1. A breach means the extraction or the setup is wrong, not that the device is active.
5. An independent reference. Compare with a closed form where one exists, otherwise with another solver or a measurement of your structure.
6. Stay inside the support boundaries. Use the calculator that matches your port family, within its stated limits. See Support Boundaries.

A closed PEC box has an exact resonance, so it shows two of the checks above in a few seconds. The TM110 mode of an a × b × d box with d the shortest side resonates at f = (c/2)·√((1/a)² + (1/b)²):

import numpy as np
from rfx import Simulation, GaussianPulse

C0 = 299_792_458.0
a, b, d = 0.024, 0.020, 0.010              # box size in metres
f_exact = C0 / 2 * np.hypot(1 / a, 1 / b)  # TM110, about 9.756 GHz


def tm110(dx):
    sim = Simulation(freq_max=15e9, domain=(a, b, d), dx=dx, boundary="pec")
    sim.add_source((0.30 * a, 0.35 * b, d / 2), "ez",
                   waveform=GaussianPulse(f0=f_exact, bandwidth=0.8),
                   amplitude_kind="current")
    sim.add_probe((0.65 * a, 0.60 * b, d / 2), "ez")
    # The same physical record length on every mesh.
    result = sim.run(n_steps=round(3000 * 2e-3 / dx))
    modes = result.find_resonances(freq_range=(0.8 * f_exact, 1.2 * f_exact))
    return min(modes, key=lambda m: abs(m.freq - f_exact)).freq


for dx in (2e-3, 1e-3, 0.5e-3):
    f = tm110(dx)
    print(f"dx = {dx * 1e3:.1f} mm: {f / 1e9:.4f} GHz, "
          f"error {100 * (f / f_exact - 1):+.3f} %")

On a CPU this prints errors of about −0.13 %, −0.03 % and −0.01 %. Each halving of the cell cuts the error by about four, the second-order convergence the Yee scheme should show. Your own study works the same way. Pick one number you care about, refine the mesh, and check that the number settles toward a limit.

The box is lossless, so its physical Q is infinite. The frequency is the result here; a Q from this record would only measure the record length. A closed box never rings down, so the run uses a fixed n_steps rather than until_decay.

Port results need finer meshes than field pictures. These are measured starting points for a convergence study. They are not a substitute for one.

The microstrip row’s cross-check, an open-stub notch filter against the openEMS tutorial (tests/crossval/msl_notch_filter/), is pending for 2.0 (tracker 1471): it is being re-measured under the current settling check, and no agreement figure is quoted until that run is in the repository.

• An exported file is not evidence. Touchstone files, plots, HDF5 snapshots and reports record what a run produced. Pair them with the checks above before citing them.
• An optimizer’s objective is a proxy. A design that minimizes a differentiable objective still needs its final RF result checked on its own, with a converged forward run.
• A regression test is not an accuracy claim. A test that pins a number guards against change at its own mesh and band. It does not say the number is right for yours.