Case F: validation
Contents
Engine status: OpenFOAM 12 scored · OpenLB pending · FluidX3D pending · ML surrogate pending
Nomenclature: symbols, columns and scores
Geometry, reference scales and release
- \(H\), \(b\), \(D\) (m) — height of the building, cube, tower or tree belt; width of the building; edge of the cubes of Case C: the lengths the cases are drawn and normalised by
- \(U_H\) (m/s) — reference speed: the mean approach-flow speed at the height \(H\) (each Data page gives the value used)
- \(U_R\), \(H_R\), \(Z_R\) (m/s, m) — reference speed and height of a workbook: Case E, \(U_R\) at \(Z_R\) = 250 m full scale (1 m in the model); Case M, \(U_R\) and \(H_R\) at the reference point of the tunnel and of each field run
- \(U_{\mathrm{ref}}\), \(L_{\mathrm{ref}}\) (m/s, m) — reference speed and length of a case’s normalisation: \(U_H\) and \(H\) (\(U_R\) and \(H_R\) in Case M); for the velocity ratios the approach speed at 15.9 m (Case E) or at 500 m (Case F)
- \(U_b\) (m/s) — Case G: mean approach-flow speed at 9 m, the normalisation of its approach-flow table
- \(q\), \(c_{\mathrm{gas}}\) (m³/s, ppm) — tracer release of the experiment: volume flow of the released gas, and its tracer content (\(10^6\) ppm when pure); \(Q\) is the source strength given to a solve
- \(R_b\) (dimensionless) — bulk Richardson number of the heated-floor cases J and L (negative: unstable stratification)
- \(\mathrm{Re_H}\), \(\mathrm{Re_D}\) (dimensionless) — Reynolds numbers \(U_H H/\nu\) and \(U_H D/\nu\) (\(\nu\): kinematic viscosity of air)
- \(u_\ast\), \(z_0\), \(\kappa\) (m/s, mm, dimensionless) — friction velocity and roughness length of the log law fitted to the approach flow, and the von Kármán constant (0.41)
- \(\mathrm{WD}\) (column
wind_direction_deg; °) — wind direction (each frame says how it is counted); \(\mathrm{WD_R}\) is the mean direction of a field run at its reference point (Case M) - \(x\), \(y\), \(z\) (columns
x,y,z; m) — coordinates in the case’s frame (\(+x\), \(+y\) and \(+z\) are the positive directions; each Data page says which way \(+x\) points), \(z\) up from the floor of the tunnel or from the ground - \(\mathrm{d}x\) (m) — lattice spacing of the OpenLB and FluidX3D runs, in metres or as a fraction of \(H\) (\(H/64\))
- \(L_r\) (m) — reattachment length behind the building, read from a wake line at ground level; reported as \(L_r/H\)
Velocity and turbulence
- \(U\), \(V\), \(W\) (columns
U,V,W; m/s) — mean velocity along \(+x\), \(+y\), \(+z\); \(V\) is positive toward \(+y\) in every case - \(\lvert\mathbf{U}\rvert\) (m/s) — magnitude of the mean velocity vector, \(\sqrt{U^2+V^2+W^2}\)
- \(V_{\mathrm{s}}\) (column
Vs; m/s) — mean omnidirectional scalar speed of a thermistor (never a vector magnitude) - \(\overline{u'u'}\), \(\overline{v'v'}\), \(\overline{w'w'}\) (columns
uu,vv,ww; m²/s²) — velocity variances (overbar: time average, prime: fluctuation about it) - \(\overline{u'v'}\), \(\overline{u'w'}\), \(\overline{v'w'}\) (columns
uv,uw,vw; m²/s²) — Reynolds shear stresses - \(k = \tfrac{1}{2}\left(\overline{u'u'}+\overline{v'v'}+\overline{w'w'}\right)\) (column
k; m²/s²) — turbulent kinetic energy - \(U/U_{\mathrm{ref}}\) (column
velocity_ratio; dimensionless) — mean speed over the case’s reference speed (\(V_{\mathrm{s}}/U_{\mathrm{ref}}\) for a thermistor) - \(\sigma_{U/U_{\mathrm{ref}}}\) (column
velocity_ratio_sd; dimensionless) — standard deviation ofvelocity_ratioover the observations behind a field value (Case F) - \(k/U_{\mathrm{ref}}^2\) (column
k_ratio; dimensionless) — turbulent kinetic energy over the square of the reference speed, where only normalised values are published (Case G)
Concentration and temperature
- \(C\) (column
C; ppm) — mean tracer-gas concentration as published (volume fraction \(\times 10^6\) of the released gas diluted in air) - \(c_{\mathrm{rms}}\) (column
c_rms; ppm) — root mean square of the concentration fluctuation - \(c_0\) (ppm) — reference concentration, defined for each case on its Data page: \(c_0 = \dfrac{c_{\mathrm{gas}}\thinspace q}{U_H H^2}\) (\(U_R H_R^2\) in the denominator in Case M)
- \(c^\ast\) (column
concentration_ratio; dimensionless) — dimensionless concentration \(c^\ast = C/c_0\); the value of a solve is \(c\thinspace U_H H^2/Q\) for its mean concentration \(c\) and its source strength \(Q\) - \(\mathrm{Sc_t}\) (dimensionless) — turbulent Schmidt number of a solve (0.3, 0.5 and 0.7)
- \(T\), \(T_{\mathrm{rms}}\) (columns
T,T_rms; °C, K) — mean air temperature as published (Cases J and L) and the root mean square of its fluctuation - \(\overline{u'c'}\), \(\overline{v'c'}\), \(\overline{w'c'}\) (columns
uc,vc,wc; ppm·m/s) — turbulent concentration fluxes - \(\overline{u'\theta'}\), \(\overline{v'\theta'}\), \(\overline{w'\theta'}\) (columns
uT,vT,wT; K·m/s) — turbulent heat fluxes (\(\theta'\): the temperature fluctuation) - \(\theta = \dfrac{T_f - T}{T_f - T_H}\) (dimensionless) — floor-referenced temperature the solves of Cases J and L are scored on: 0 at the floor, 1 in the approach air at \(H\); \(T_f\) is the floor temperature and \(T_H\) the temperature of the approach flow at \(H\)
Scores
- \(n\) — number of pairs of a measured value \(o_i\) and a modelled value \(p_i\) at one point
- \(R\) — Pearson correlation coefficient of the pairs
- \(R^2\) — square of \(R\): share of the measured variance a linear fit to the model explains
- \(\mathrm{FAC2}\) — fraction of pairs with \(0.5 \le p_i/o_i \le 2\)
- \(\mathrm{FB}\) — fractional bias \(2(\overline{o}-\overline{p})/(\overline{o}+\overline{p})\), positive when the model is low
- \(\mathrm{HR}\) (hit rate) — fraction of pairs with \(\lvert p_i-o_i\rvert/\lvert o_i\rvert \le \delta_{\mathrm{rel}}\) or \(\lvert p_i-o_i\rvert \le \delta_{\mathrm{abs}}\), where \(\delta_{\mathrm{rel}} = 0.25\) and \(\delta_{\mathrm{abs}} = 0.05\thinspace U_H\) for a velocity component
- \(\mathrm{NMSE} = \dfrac{\overline{(o-p)^2}}{\overline{o}\thinspace \overline{p}}\) — normalised mean square error
- \(\mathrm{NAD} = \dfrac{\overline{\lvert o-p\rvert}}{\overline{o}+\overline{p}}\) — normalised absolute difference
- \(\mathrm{MG} = \exp\overline{(\ln o-\ln p)}\) — geometric mean bias
- \(\mathrm{VG} = \exp\overline{(\ln o-\ln p)^2}\) — geometric variance
- The ratio of \(\mathrm{FAC2}\) keeps its sign, so a model of the other sign than the measurement is a miss; \(\mathrm{FB}\), \(\mathrm{NMSE}\) and \(\mathrm{NAD}\) divide by the means, so they are undefined (shown as —) for a signed component unless the measured and the modelled mean are both positive
- VDI 3783-9 calls the hit rate \(q\) and the tolerances \(D\) and \(W\); on this site those letters keep their other meanings (\(q\) the tracer release rate, \(D\) the block size of Case C, \(W\) the vertical velocity)
- The detection limit of a concentration is one value per case, \(10^{-3}\) of the largest measured value. It sets \(\delta_{\mathrm{abs}}\) of the hit rate and the both-below rule of \(\mathrm{FAC2}\), and floors the values in \(\mathrm{MG}\) and \(\mathrm{VG}\).
- The scores \(\mathrm{FB}\), \(\mathrm{FAC2}\), \(\mathrm{NMSE}\), \(\mathrm{NAD}\), \(\mathrm{MG}\) and \(\mathrm{VG}\) follow Chang & Hanna (2004); the hit rate follows VDI 3783-9.
Quantities scored
- \(U\), \(V\) and \(W\) (mean velocity components) over \(U_H\)
- \(k\) (turbulent kinetic energy) over \(U_H^2\)
- the speed \(\lvert\mathbf{U}\rvert\) and \(\sqrt{\lvert\mathbf{U}\rvert^2+2k}\), against a thermistor’s \(V_{\mathrm{s}}\)
- \(c^\ast = C/c_0\) (dimensionless concentration) at turbulent Schmidt numbers \(\mathrm{Sc_t}\) of 0.3, 0.5 and 0.7
- \(\theta\) (floor-referenced temperature)
Measurement locations
Explore the points
OpenFOAM 12
Status: scored
The Shinjuku sub-centre of Tokyo: a 1978 high-rise district, thermistor speed in a 1:800 tunnel and a year of field cup anemometers. An Eddy3D production run solved on 2026-10-18, realizable k-ε at numerics level 3, one steady solve on one mesh for the wind from NNW only (2,143,488 cells, 2000 iterations). It stopped at the iteration limit with the SIMPLE residual control unmet; the initial residual of epsilon at the last iteration was \(1.0 \times 10^{-3}\) (\(U_z\) \(2.6 \times 10^{-3}\)).
- Scored in the benchmark’s own ratio, \(U/U(500\text{ m})\). The tunnel’s thermistor and the field’s cup anemometer read a scalar speed, so the model is scored as \(\lvert\mathbf{U}\rvert\) and as \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\); both are in the tables. The model is divided by its own approach speed at 500 m, 9.862 m/s (probed 1100 m upwind of the centre, 0.986 of the imposed 10 m/s); over the imposed 10 m/s the scores are in the files. The tunnel measured at 5, 9 and 20 m above ground at 13 points (7, 13 and 13 of them have a value for NNW) and the field at 13 points, 3 to 11 m up.
- Two vertical frames. The readme says z is measured from the tunnel floor and the ground is z − 10 m, but the STL’s own ground under all 13 points is 5 m. Literal probes the model at the published z in the STL’s frame, which is 10, 14 and 25 m above the STL’s ground for the three planes; agl probes the STL’s ground plus 5, 9 and 20 m. The files do not say which one the tunnel meant, so both are scored, as two runs of one engine (
configurationliteral and agl).
Data notes: Scores here and Eddy3D's own
The tunnel’s values are identical to the CSV’s. The field scores differ in the third decimal because point 14’s 16 published field values are shifted by one direction in the CSV and realigned with the workbook in the cleaned table (clean/QUESTIONS.md QF1).
Measured against modelled
\(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) over \(U(500\text{ m})\):
| Points | Frame | R | FAC2 | FB | Hit rate | n |
|---|---|---|---|---|---|---|
| Tunnel, 9 m plane | literal | +0.935 | 1.000 | +0.036 | 0.846 | 13 |
| Tunnel, 9 m plane | agl | +0.945 | 1.000 | +0.126 | 0.769 | 13 |
| Tunnel, 5 m plane | literal | +0.995 | 1.000 | +0.127 | 1.000 | 7 |
| Tunnel, 5 m plane | agl | +0.918 | 1.000 | +0.128 | 0.571 | 7 |
| Tunnel, 20 m plane | literal | +0.796 | 0.923 | +0.071 | 0.769 | 13 |
| Tunnel, 20 m plane | agl | +0.807 | 1.000 | +0.208 | 0.462 | 13 |
| Tunnel, all points | literal | +0.912 | 0.970 | +0.067 | 0.848 | 33 |
| Tunnel, all points | agl | +0.898 | 1.000 | +0.165 | 0.606 | 33 |
| Field | literal | +0.873 | 1.000 | +0.027 | 0.769 | 13 |
| Field | agl | +0.889 | 1.000 | +0.147 | 0.692 | 13 |
\(\lvert\mathbf{U}\rvert\) alone:
| Points | Frame | R | FAC2 | FB | Hit rate | n |
|---|---|---|---|---|---|---|
| Tunnel, 9 m plane | literal | +0.928 | 1.000 | +0.134 | 0.615 | 13 |
| Tunnel, 9 m plane | agl | +0.935 | 0.923 | +0.242 | 0.538 | 13 |
| Tunnel, 5 m plane | literal | +0.988 | 0.857 | +0.307 | 0.571 | 7 |
| Tunnel, 5 m plane | agl | +0.923 | 0.857 | +0.259 | 0.571 | 7 |
| Tunnel, 20 m plane | literal | +0.788 | 0.923 | +0.112 | 0.692 | 13 |
| Tunnel, 20 m plane | agl | +0.801 | 0.846 | +0.269 | 0.308 | 13 |
| Tunnel, all points | literal | +0.906 | 0.939 | +0.149 | 0.636 | 33 |
| Tunnel, all points | agl | +0.894 | 0.879 | +0.258 | 0.455 | 33 |
| Field | literal | +0.860 | 1.000 | +0.126 | 0.308 | 13 |
| Field | agl | +0.866 | 0.846 | +0.266 | 0.308 | 13 |
Controls:
| Control | Frame | R | FAC2 | FB | Hit rate | n |
|---|---|---|---|---|---|---|
| 9 m plane over the imposed \(U(500\text{ m})\) | literal | +0.935 | 1.000 | +0.050 | 0.846 | 13 |
| 9 m plane over the imposed \(U(500\text{ m})\) | agl | +0.945 | 1.000 | +0.139 | 0.692 | 13 |
| Tunnel’s SSE column (opposite wind) | literal | +0.077 | 0.615 | +0.327 | 0.077 | 13 |
| Tunnel’s SSE column (opposite wind) | agl | +0.102 | 0.385 | +0.413 | 0.000 | 13 |
| Tunnel’s 9 m column as a model of the field | literal | +0.933 | 1.000 | −0.015 | 0.692 | 13 |
| Tunnel’s 9 m column as a model of the field | agl | +0.933 | 1.000 | −0.015 | 0.692 | 13 |
FB is positive when the model is low.
- 9 m plane and all points, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\). At the 9 m plane R is 0.935 (literal) and 0.945 (agl) with FAC2 1.00; over all 33 tunnel points R is 0.912 and 0.898, FAC2 0.970 and 1.00. The 20 m plane has the lowest R (0.796 and 0.807); the 5 m plane (7 points) the highest in the literal frame (0.995) and 0.918 in the agl frame. The mean of the modelled scalar speed at the 9 m plane is 0.252 (literal) and 0.230 (agl) against 0.261 measured, FB +0.04 and +0.13; for \(\lvert\mathbf{U}\rvert\) alone FB is +0.13 and +0.24, the model’s mean 0.228 and 0.205.
- Frames. The published z sits 5 m higher above the STL’s ground than the instrument heights. FB, literal against agl: +0.04 and +0.13 at the 9 m plane, +0.07 and +0.21 at the 20 m plane; R differs by at most 0.02 at those planes. Nothing in the files says which datum the tunnel meant. At the 4 m surface cells the lowest probes of the agl frame (the 7 points of the 5 m plane, 1.25 cells up, and the field’s point at 3 m) sit in the first cell.
- Point by point. At the 9 m plane, literal frame, the model-to-tunnel ratio of \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) runs from 0.70 (point 8) to 1.51 (point 3): the model is faster than the tunnel at points 3, 5, 10 and 14 and slower at points 1, 2, 8, 9 and 13, within 5 % at points 6, 7, 11 and 15. In the agl frame the ratio runs from 0.65 (point 1) to 1.31 (point 3), slower at points 1, 2, 6, 8, 9 and 13 (0.65 to 0.80) and faster at points 3, 5, 10 and 14 (1.08 to 1.31). Over all 33 tunnel points one is outside a factor of two in the literal frame (FAC2 0.970), none in the agl frame. The plan is in Figure 3.
- Field. R is 0.873 and 0.889 and FAC2 1.00 for \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) (\(\lvert\mathbf{U}\rvert\): R 0.860 and 0.866, FB +0.13 and +0.27). The standard deviation of the 10-minute means behind a field value is 0.075 to 0.15 of \(U(500\text{ m})\), 0.20 to 0.78 of the value itself. The tunnel’s own 9 m column scored as a model of the field gives R 0.933, FAC2 1.00, FB −0.015; the model’s R against the field is 0.04 to 0.06 under it.
- Opposite wind. The solved field correlates R 0.935 with the tunnel’s 9 m column for the wind from NNW and R 0.077 with its column for the opposite wind, SSE (agl frame 0.945 and 0.102). The cleaned release’s direction table marks its NNW rows corroborated by a solved field; the other directions stay unverified until one is solved (clean/QUESTIONS.md QF9).
Interpretation: Case F frames, field and wind direction
- The pattern follows the tunnel in both frames and the frame changes the level; the literal frame has the smaller bias at the 9 m and 20 m planes. The offset is the datum, which the files do not fix.
- At the lowest probes of the agl frame the point interpolation is pulled toward the wall’s zero.
- A steady solve is one state for one direction; a field value averages a sector 22.5° wide over a year of strong winds.
- The tunnel’s own 9 m column reaches R 0.933 as a model of the field, the level a model of the tunnel can be expected to reach.
- The compass sense is corroborated at this direction. A RANS solution knows no compass, so the correlation with the NNW column and the lack of it with the SSE one support reading the direction labels as the direction the wind comes from, which the cleaned release had marked unverified.
Field maps
The solved velocity on the measured planes, every point drawn on it as a dot coloured by its measured value on the same scale: a dot that stands out from the colour around it is a point the model misses. The wind blows across the map, so (a) and (b) show the component along the wind.
OpenFOAM 12, realizable k-epsilon (Eddy3D Outdoor), configuration literal
Quantity: the thermistor's (tunnel) and the cup anemometer's (field) mean speed over \(U(500\text{ m})\) against the model's \(\lvert\mathbf{U}\rvert\) and \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) over its own approach \(U(500\text{ m})\) = 9.862 m/s (probed 1100 m upwind of the centre; the imposed \(U(500\text{ m})\) is 10 m/s), at the 33 tunnel points (planes at 5, 9 and 20 m) and the 13 field observations of wind from NNW; frame literal.
All 13 headline scores
| Score | R | \(R^2\) | FAC2 | FB | Hit rate | n |
|---|---|---|---|---|---|---|
| Tunnel, 9 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.935 | 0.874 | 1.000 | +0.036 | 0.846 | 13 |
| Tunnel, 5 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.995 | 0.990 | 1.000 | +0.127 | 1.000 | 7 |
| Tunnel, 20 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.796 | 0.633 | 0.923 | +0.071 | 0.769 | 13 |
| Tunnel, all 33 points, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.912 | 0.832 | 0.970 | +0.067 | 0.848 | 33 |
| Tunnel, 9 m plane, \(\lvert\mathbf{U}\rvert\) | +0.928 | 0.862 | 1.000 | +0.134 | 0.615 | 13 |
| Tunnel, 5 m plane, \(\lvert\mathbf{U}\rvert\) | +0.988 | 0.977 | 0.857 | +0.307 | 0.571 | 7 |
| Tunnel, 20 m plane, \(\lvert\mathbf{U}\rvert\) | +0.788 | 0.620 | 0.923 | +0.112 | 0.692 | 13 |
| Tunnel, all 33 points, \(\lvert\mathbf{U}\rvert\) | +0.906 | 0.822 | 0.939 | +0.149 | 0.636 | 33 |
| Field, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.873 | 0.762 | 1.000 | +0.027 | 0.769 | 13 |
| Field, \(\lvert\mathbf{U}\rvert\) | +0.860 | 0.740 | 1.000 | +0.126 | 0.308 | 13 |
| Tunnel, 9 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) over the imposed \(U(500\text{ m})\) = 10 m/s | +0.935 | 0.874 | 1.000 | +0.050 | 0.846 | 13 |
| Tunnel's 9 m column as a model of the field (ceiling control) | +0.933 | 0.870 | 1.000 | -0.015 | 0.692 | 13 |
| Tunnel's SSE column, the opposite wind, against the solved NNW field (compass control) | +0.077 | 0.006 | 0.615 | +0.327 | 0.077 | 13 |
- Reproduce: Eddy3D fixture Eddy3DCli.Tests/Validation/TestAijCaseFContainerE2E (EDDY3D_AIJ_F_DIRECTIONS=NNW, the default) (Shinjuku district, 1:800, wind from NNW, frame literal; OpenFOAM 12 realizable k-ε, numerics level 3, 2000 iterations, 2,143,488 cells, solved 2026-10-18); figures validation/plot_cases_fm_openfoam.py
Notes:
- Frame 'literal': the model is probed at the published z in the STL's frame. The readme says z is measured from the tunnel floor and the ground is z - 10 m, but the STL's own ground under all 13 points is 5 m, so the 5, 9 and 20 m planes are probed 10, 14 and 25 m above the STL's ground (height_above_stl_ground in the pairs). The other frame, 'agl' (the STL's ground plus 5, 9 and 20 m), is the other run of this case. The files do not say which frame the tunnel meant: the 9 m plane's \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) scores R 0.93 and FB +0.04 in the literal frame and R 0.94 and FB +0.13 in the agl frame.
- Both sides are \(U/U(500\text{ m})\): the tunnel's ratio is as published, the model's is over its own approach \(U(500\text{ m})\) = 9.862 m/s, probed 1100 m upwind of the centre (0.986 of the imposed 10 m/s). Over the imposed 10 m/s the 9 m plane's \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) scores R 0.93 and FB +0.05.
OpenFOAM 12, realizable k-epsilon (Eddy3D Outdoor), configuration agl
Quantity: the thermistor's (tunnel) and the cup anemometer's (field) mean speed over \(U(500\text{ m})\) against the model's \(\lvert\mathbf{U}\rvert\) and \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) over its own approach \(U(500\text{ m})\) = 9.862 m/s (probed 1100 m upwind of the centre; the imposed \(U(500\text{ m})\) is 10 m/s), at the 33 tunnel points (planes at 5, 9 and 20 m) and the 13 field observations of wind from NNW; frame agl.
All 13 headline scores
| Score | R | \(R^2\) | FAC2 | FB | Hit rate | n |
|---|---|---|---|---|---|---|
| Tunnel, 9 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.945 | 0.893 | 1.000 | +0.126 | 0.769 | 13 |
| Tunnel, 5 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.918 | 0.843 | 1.000 | +0.128 | 0.571 | 7 |
| Tunnel, 20 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.807 | 0.651 | 1.000 | +0.208 | 0.462 | 13 |
| Tunnel, all 33 points, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.898 | 0.806 | 1.000 | +0.165 | 0.606 | 33 |
| Tunnel, 9 m plane, \(\lvert\mathbf{U}\rvert\) | +0.935 | 0.875 | 0.923 | +0.242 | 0.538 | 13 |
| Tunnel, 5 m plane, \(\lvert\mathbf{U}\rvert\) | +0.923 | 0.853 | 0.857 | +0.259 | 0.571 | 7 |
| Tunnel, 20 m plane, \(\lvert\mathbf{U}\rvert\) | +0.801 | 0.642 | 0.846 | +0.269 | 0.308 | 13 |
| Tunnel, all 33 points, \(\lvert\mathbf{U}\rvert\) | +0.894 | 0.800 | 0.879 | +0.258 | 0.455 | 33 |
| Field, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | +0.889 | 0.791 | 1.000 | +0.147 | 0.692 | 13 |
| Field, \(\lvert\mathbf{U}\rvert\) | +0.866 | 0.750 | 0.846 | +0.266 | 0.308 | 13 |
| Tunnel, 9 m plane, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) over the imposed \(U(500\text{ m})\) = 10 m/s | +0.945 | 0.893 | 1.000 | +0.139 | 0.692 | 13 |
| Tunnel's 9 m column as a model of the field (ceiling control) | +0.933 | 0.870 | 1.000 | -0.015 | 0.692 | 13 |
| Tunnel's SSE column, the opposite wind, against the solved NNW field (compass control) | +0.102 | 0.010 | 0.385 | +0.413 | 0.000 | 13 |
- Reproduce: Eddy3D fixture Eddy3DCli.Tests/Validation/TestAijCaseFContainerE2E (EDDY3D_AIJ_F_DIRECTIONS=NNW, the default) (Shinjuku district, 1:800, wind from NNW, frame agl; OpenFOAM 12 realizable k-ε, numerics level 3, 2000 iterations, 2,143,488 cells, solved 2026-10-18); figures validation/plot_cases_fm_openfoam.py
Notes:
- Frame 'agl': the model is probed at the STL's own ground under each point plus the instrument height (5, 9 and 20 m for the tunnel, 3 to 11 m for the field). The readme says z is measured from the tunnel floor and the ground is z - 10 m, but the STL's ground under all 13 points is 5 m, so the published z would sit 5 m higher; that frame is the other run of this case, 'literal'. The files do not say which frame the tunnel meant: the 9 m plane's \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) scores R 0.93 and FB +0.04 in the literal frame and R 0.94 and FB +0.13 in the agl frame.
- At the 4 m surface cells of the mesh the lowest probes of this frame (the 7 points of the 5 m plane, 1.25 cells above the ground, and the field's point at 3 m) sit in the first cell, where the point interpolation is pulled toward the wall's zero.
- Both sides are \(U/U(500\text{ m})\): the tunnel's ratio is as published, the model's is over its own approach \(U(500\text{ m})\) = 9.862 m/s, probed 1100 m upwind of the centre (0.986 of the imposed 10 m/s). Over the imposed 10 m/s the 9 m plane's \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) scores R 0.94 and FB +0.14.
Not run
- A second mesh or iteration count.
- The other eleven of the tunnel’s twelve directions, and the field’s other fifteen. Which vertical datum the tunnel meant is not settled (both frames are published).
- The published STL is open along the rim of its disc (812 boundary edges) and the mesh fails one checkMesh check (43 highly skew faces, maximum skewness 14.8), so the solve ran on Eddy3D’s hardened numerics. The rooftop anemometers’ modelled speeds are in the results file and are not scored.
- OpenLB, FluidX3D, UMF and the ML surrogate.
Interpretation: Limits of this run
- With one mesh and the residual control unmet, none of these numbers is mesh- or iteration-independent.
- The open STL rim and the failed checkMesh check were not varied, so their effect on the scores is unknown.
Notes on every run
- Normalisation: the thermistor's (tunnel) and the cup anemometer's (field) mean speed over \(U(500\text{ m})\), the benchmark's own ratio, against the model's \(\lvert\mathbf{U}\rvert\) and \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) over its own approach \(U(500\text{ m})\) = 9.862 m/s (the imposed \(U(500\text{ m})\) is 10 m/s); hit rate \(\delta_{\mathrm{rel}} = 0.25\), \(\delta_{\mathrm{abs}} = 0.05\) in those units; 33 tunnel points and 13 field observations of wind from NNW; FB is positive when the model is low
- Per-point values:
/api/cases/F/pairs/openfoam12-realizableKE.csv
Notes:
- One direction of the tunnel's twelve was solved: wind from NNW, which with ENE has the most tunnel rows (33). The other eleven tunnel directions and the field's other fifteen were not run.
- The thermistor reads a scalar speed and so does the cup anemometer: the model is scored as \(\lvert\mathbf{U}\rvert\), and as \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\), the speed of a velocity with the modelled turbulent kinetic energy in all three components.
- The field is a year of 10-minute means above 5 m/s at a rooftop reference, over a sector 22.5° wide, at 3 to 11 m above ground; the standard deviation of the means behind each value is 0.075 to 0.15 of \(U(500\text{ m})\), 0.20 to 0.78 of the value itself (the horizontal bars of the figure). The tunnel's 9 m column scored as a model of the field, point for point, gives R 0.93, FAC2 1.00, FB -0.01.
- The field values of point 14 are the cleaned table's, realigned with the workbook: the published RS-FO_caseF.csv holds them shifted by one direction (clean/QUESTIONS.md QF1). The field scores here therefore differ from Eddy3D's own results file in the third decimal (\(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) in the literal frame: R 0.8729 here, 0.8731 there; in the agl frame 0.8893 and 0.8872); the tunnel scores are identical to it.
- The published STL is open along the rim of its 725 m disc (812 boundary edges, one shell, wound inward), which the canvas's pre-meshing pass cannot close (it removed 3 degenerate faces). The mesh has 2,143,488 cells and checkMesh fails one check (43 highly skew faces, maximum skewness 14.8, maximum non-orthogonality 64.8°), so the solve started on the hardened numerics of Eddy3D's mesh-quality tier.
- The solve ran to its 2000-iteration limit with the residualControl unmet (no 'solution converged' line); the initial residuals at the last iteration were \(U_x\) 1.0e-04, \(U_y\) 5.9e-05, \(U_z\) 2.6e-03, \(p\) 3.0e-05, \(\varepsilon\) 1.0e-03 and \(k\) 1.4e-04, and the log has 1,988 'bounding \(k\)' lines.
- The solved field correlates R 0.93 with the tunnel's 9 m column for the wind from NNW and R 0.08 with its column for the opposite wind, SSE (agl frame: 0.94 and 0.10). A RANS field knows no compass, so this supports reading the direction labels as the direction the wind comes from (clean/QUESTIONS.md QF9), at this one direction.
- The model's \(U/U(500\text{ m})\) at the workbook's two rooftop anemometers (SMB 0.85 for \(\lvert\mathbf{U}\rvert\), KDD 0.45) is in the results file and not scored: the tunnel's anchors for them exist for the north and south tests only.
Interpretation: Case F scalar speed and field data
The modelled \(\lvert\mathbf{U}\rvert\) is the lower bound. \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) is the upper bracket (as in Cases C to E). A RANS field is one steady state for one exact direction. The tunnel's 9 m column as a model of the field is the bar a model of the tunnel is not expected to clear.
How the scores are computed
- Scores recomputed from the pairs by validation/dispersion_metrics.py; the SHA-256 of every file the snapshot was read from is in its results file, under
sources.
Leaderboard
| Rank | Run | Source | Hit rate, \(\sqrt{\lvert\mathbf{U}\rvert^2 + 2k}\) | Hit rate, \(\lvert\mathbf{U}\rvert\) | Hit rate | FAC2 | R | \(R^2\) |
|---|---|---|---|---|---|---|---|---|
| 1 | OpenFOAM 12 · literal | Eddy3D | 0.846 | 0.547 | 0.696 | 0.963 | +0.895 | 0.807 |
| 2 | OpenFOAM simpleFoam (Eddy3D Outdoor case writer) (eddy3d-of12-nutcap-lifted) | community | 0.659 | 0.582 | 0.621 | 0.934 | +0.866 | 0.755 |
| 3 | OpenFOAM 12 · agl | Eddy3D | 0.624 | 0.431 | 0.527 | 0.934 | +0.886 | 0.787 |
All runs scored by validation/score_submissions.py. Ranked by mean hit rate (VDI 3783) over the tasks; bold: best per column. Not ranked: the ML surrogate (no point ids) and negative controls. Community rows are self-reported (Validate your own solver).