Validation: Eddy3D against the measurements
Each case is solved through Eddy3D’s production case writer (build-case → run → probe). The probe sits at the tunnel’s own measurement coordinates.
Cases
Each case’s Validation page has its measurement locations, every engine’s results and a leaderboard of its runs.
- Case A: 1:1:2 building
- Case B: 1:4:4 slab
- Case C: 3 × 3 block array
- Case D: high-rise among city blocks
- Case E: Niigata district
- Case F: Shinjuku district
- Case G: flow behind a tree belt
- Case H: 1:1:2 building with a floor point source
- Case I: cube with a roof point source
- Case J: 1:1:2 building, non-isothermal
- Case K: 9 x 9 cube array, point source
- Case L: 9 x 14 cube array, line source, non-isothermal
- Case M: TPU Atsugi campus
Cases C and D by configuration
Every case and engine
| Case | OpenFOAM 12 | OpenLB | FluidX3D | ML surrogate |
|---|---|---|---|---|
| Case A | scored R +0.48 … +0.96 FAC2 0.35 … 0.82 | scored R +0.16 … +0.80 FAC2 0.00 … 0.65 | scored R +0.59 … +0.85 FAC2 0.22 … 0.60 | scored R -0.13 … +0.43 FAC2 0.26 … 0.62 |
| Case B | scored R +0.40 … +0.96 FAC2 0.28 … 0.70 | scored R +0.45 … +0.93 FAC2 0.05 … 0.61 | scored R +0.85 … +0.95 FAC2 0.48 … 0.64 | scored R -0.24 … +0.01 FAC2 0.36 … 0.46 |
| Case C | scored R +0.74 … +0.91 FAC2 0.55 … 0.98 | not comparable | pending | scored R +0.07 … +0.21 FAC2 0.44 … 0.88 |
| Case D | scored R +0.78 … +0.95 FAC2 0.48 … 0.92 | not comparable | pending | scored R +0.22 … +0.59 FAC2 0.34 … 0.64 |
| Case E | scored R +0.69 … +0.85 FAC2 0.74 … 0.95 | not comparable | pending | scored R +0.23 … +0.43 FAC2 0.22 … 0.84 |
| Case F compared with the wind tunnel and the field | scored R +0.79 … +1.00 FAC2 0.85 … 1.00 | pending | pending | pending |
| Case G compared with the field | scored R +0.21 … +0.50 FAC2 0.68 … 1.00 | pending | pending | pending |
| Case H | scored R +0.11 … +0.95 FAC2 0.31 … 0.69 | pending | pending | pending |
| Case I | scored R +0.90 … +1.00 FAC2 0.36 … 0.92 | pending | pending | pending |
| Case J | scored R +0.32 … +0.92 FAC2 0.29 … 1.00 | pending | pending | pending |
| Case K | scored R -0.15 … +0.96 FAC2 0.04 … 0.80 | pending | pending | pending |
| Case L | scored R -0.04 … +0.94 FAC2 0.18 … 1.00 | pending | pending | pending |
| Case M compared with the wind tunnel and the field | scored R +0.18 … +0.73 FAC2 0.07 … 0.73 | pending | pending | pending |
Nomenclature: scores
Scores
- \(n\) — number of pairs of a measured value \(o_i\) and a modelled value \(p_i\) at one point
- \(R\) — Pearson correlation coefficient of the pairs
- \(R^2\) — square of \(R\): share of the measured variance a linear fit to the model explains
- \(\mathrm{FAC2}\) — fraction of pairs with \(0.5 \le p_i/o_i \le 2\)
- \(\mathrm{FB}\) — fractional bias \(2(\overline{o}-\overline{p})/(\overline{o}+\overline{p})\), positive when the model is low
- \(\mathrm{HR}\) (hit rate) — fraction of pairs with \(\lvert p_i-o_i\rvert/\lvert o_i\rvert \le \delta_{\mathrm{rel}}\) or \(\lvert p_i-o_i\rvert \le \delta_{\mathrm{abs}}\), where \(\delta_{\mathrm{rel}} = 0.25\) and \(\delta_{\mathrm{abs}} = 0.05\thinspace U_H\) for a velocity component
Quantities scored
- \(U\), \(V\) and \(W\) (mean velocity components) over \(U_H\)
- \(k\) (turbulent kinetic energy) over \(U_H^2\)
- the speed \(\lvert\mathbf{U}\rvert\) and \(\sqrt{\lvert\mathbf{U}\rvert^2+2k}\), against a thermistor’s \(V_{\mathrm{s}}\)
- \(c^\ast = C/c_0\) (dimensionless concentration) at turbulent Schmidt numbers \(\mathrm{Sc_t}\) of 0.3, 0.5 and 0.7
- \(\theta\) (floor-referenced temperature)
R, FAC2: range over the engine’s headline scores, negative controls excluded. The method defines every metric.
| Status | Meaning |
|---|---|
| scored | A validation run exists and its scores are in cases/{id}/validation.json. |
| running | The solve is in progress; no scores yet. |
| pending | Not run yet. |
| not comparable | The engine cannot produce the measured quantity at the measured location: OpenLB at 2 m on Cases C to E, where its empty-domain control misses the bar. The measurement behind it is on the OpenLB and FluidX3D page. |
- A dash in the matrix: the engine does not apply to the case.
- OpenFOAM 12 on Cases J and L is urbanMicroclimateFoam (UMF), the OpenFOAM 12 solver Eddy3D uses for non-isothermal flow; the isothermal solver was not run on them.
- API engine ids:
openfoam12-realizableKE(alsoopenfoam12-kEpsilonandopenfoam12-kOmegaSST, Case I only),umf-realizableKE(alsoumf-buoyantKEpsilon, Case J only),openlb,fluidx3dandml-surrogate.
In this section
- Method and metrics - how each case is scored, the metrics, the inflow, the two systematic differences
- Inlet contract - the approach flow a submission should use, per case: the measured profile, and the inlet this repository recommends
- OpenLB and FluidX3D - the two lattice-Boltzmann engines: how they are run, and why they are not scored on C to E
- ML surrogate - how the Wind Predictor is run on Cases A to E, and what its scores grade