Case A: validation

Contents
  1. Measurement locations
  2. Explore the points
  3. OpenFOAM 12
    1. Measured against modelled
    2. Field maps
    3. Scores
  4. OpenLB
    1. Scores
  5. FluidX3D
    1. Scores
  6. ML surrogate
    1. Scores
  7. Grid uncertainty
  8. Notes on every run
  9. Leaderboard

Engine status: OpenFOAM 12 scored · OpenLB scored · FluidX3D scored · ML surrogate scored

Nomenclature: symbols, columns and scores

Geometry, reference scales and release

  • \(H\), \(b\), \(D\) (m) — height of the building, cube, tower or tree belt; width of the building; edge of the cubes of Case C: the lengths the cases are drawn and normalised by
  • \(U_H\) (m/s) — reference speed: the mean approach-flow speed at the height \(H\) (each Data page gives the value used)
  • \(U_R\), \(H_R\), \(Z_R\) (m/s, m) — reference speed and height of a workbook: Case E, \(U_R\) at \(Z_R\) = 250 m full scale (1 m in the model); Case M, \(U_R\) and \(H_R\) at the reference point of the tunnel and of each field run
  • \(U_{\mathrm{ref}}\), \(L_{\mathrm{ref}}\) (m/s, m) — reference speed and length of a case’s normalisation: \(U_H\) and \(H\) (\(U_R\) and \(H_R\) in Case M); for the velocity ratios the approach speed at 15.9 m (Case E) or at 500 m (Case F)
  • \(U_b\) (m/s) — Case G: mean approach-flow speed at 9 m, the normalisation of its approach-flow table
  • \(q\), \(c_{\mathrm{gas}}\) (m³/s, ppm) — tracer release of the experiment: volume flow of the released gas, and its tracer content (\(10^6\) ppm when pure); \(Q\) is the source strength given to a solve
  • \(R_b\) (dimensionless) — bulk Richardson number of the heated-floor cases J and L (negative: unstable stratification)
  • \(\mathrm{Re_H}\), \(\mathrm{Re_D}\) (dimensionless) — Reynolds numbers \(U_H H/\nu\) and \(U_H D/\nu\) (\(\nu\): kinematic viscosity of air)
  • \(u_\ast\), \(z_0\), \(\kappa\) (m/s, mm, dimensionless) — friction velocity and roughness length of the log law fitted to the approach flow, and the von Kármán constant (0.41)
  • \(\mathrm{WD}\) (column wind_direction_deg; °) — wind direction (each frame says how it is counted); \(\mathrm{WD_R}\) is the mean direction of a field run at its reference point (Case M)
  • \(x\), \(y\), \(z\) (columns x, y, z; m) — coordinates in the case’s frame (\(+x\), \(+y\) and \(+z\) are the positive directions; each Data page says which way \(+x\) points), \(z\) up from the floor of the tunnel or from the ground
  • \(\mathrm{d}x\) (m) — lattice spacing of the OpenLB and FluidX3D runs, in metres or as a fraction of \(H\) (\(H/64\))
  • \(L_r\) (m) — reattachment length behind the building, read from a wake line at ground level; reported as \(L_r/H\)

Velocity and turbulence

  • \(U\), \(V\), \(W\) (columns U, V, W; m/s) — mean velocity along \(+x\), \(+y\), \(+z\); \(V\) is positive toward \(+y\) in every case
  • \(\lvert\mathbf{U}\rvert\) (m/s) — magnitude of the mean velocity vector, \(\sqrt{U^2+V^2+W^2}\)
  • \(V_{\mathrm{s}}\) (column Vs; m/s) — mean omnidirectional scalar speed of a thermistor (never a vector magnitude)
  • \(\overline{u'u'}\), \(\overline{v'v'}\), \(\overline{w'w'}\) (columns uu, vv, ww; m²/s²) — velocity variances (overbar: time average, prime: fluctuation about it)
  • \(\overline{u'v'}\), \(\overline{u'w'}\), \(\overline{v'w'}\) (columns uv, uw, vw; m²/s²) — Reynolds shear stresses
  • \(k = \tfrac{1}{2}\left(\overline{u'u'}+\overline{v'v'}+\overline{w'w'}\right)\) (column k; m²/s²) — turbulent kinetic energy
  • \(U/U_{\mathrm{ref}}\) (column velocity_ratio; dimensionless) — mean speed over the case’s reference speed (\(V_{\mathrm{s}}/U_{\mathrm{ref}}\) for a thermistor)
  • \(\sigma_{U/U_{\mathrm{ref}}}\) (column velocity_ratio_sd; dimensionless) — standard deviation of velocity_ratio over the observations behind a field value (Case F)
  • \(k/U_{\mathrm{ref}}^2\) (column k_ratio; dimensionless) — turbulent kinetic energy over the square of the reference speed, where only normalised values are published (Case G)

Concentration and temperature

  • \(C\) (column C; ppm) — mean tracer-gas concentration as published (volume fraction \(\times 10^6\) of the released gas diluted in air)
  • \(c_{\mathrm{rms}}\) (column c_rms; ppm) — root mean square of the concentration fluctuation
  • \(c_0\) (ppm) — reference concentration, defined for each case on its Data page: \(c_0 = \dfrac{c_{\mathrm{gas}}\thinspace q}{U_H H^2}\) (\(U_R H_R^2\) in the denominator in Case M)
  • \(c^\ast\) (column concentration_ratio; dimensionless) — dimensionless concentration \(c^\ast = C/c_0\); the value of a solve is \(c\thinspace U_H H^2/Q\) for its mean concentration \(c\) and its source strength \(Q\)
  • \(\mathrm{Sc_t}\) (dimensionless) — turbulent Schmidt number of a solve (0.3, 0.5 and 0.7)
  • \(T\), \(T_{\mathrm{rms}}\) (columns T, T_rms; °C, K) — mean air temperature as published (Cases J and L) and the root mean square of its fluctuation
  • \(\overline{u'c'}\), \(\overline{v'c'}\), \(\overline{w'c'}\) (columns uc, vc, wc; ppm·m/s) — turbulent concentration fluxes
  • \(\overline{u'\theta'}\), \(\overline{v'\theta'}\), \(\overline{w'\theta'}\) (columns uT, vT, wT; K·m/s) — turbulent heat fluxes (\(\theta'\): the temperature fluctuation)
  • \(\theta = \dfrac{T_f - T}{T_f - T_H}\) (dimensionless) — floor-referenced temperature the solves of Cases J and L are scored on: 0 at the floor, 1 in the approach air at \(H\); \(T_f\) is the floor temperature and \(T_H\) the temperature of the approach flow at \(H\)

Scores

  • \(n\) — number of pairs of a measured value \(o_i\) and a modelled value \(p_i\) at one point
  • \(R\) — Pearson correlation coefficient of the pairs
  • \(R^2\) — square of \(R\): share of the measured variance a linear fit to the model explains
  • \(\mathrm{FAC2}\) — fraction of pairs with \(0.5 \le p_i/o_i \le 2\)
  • \(\mathrm{FB}\) — fractional bias \(2(\overline{o}-\overline{p})/(\overline{o}+\overline{p})\), positive when the model is low
  • \(\mathrm{HR}\) (hit rate) — fraction of pairs with \(\lvert p_i-o_i\rvert/\lvert o_i\rvert \le \delta_{\mathrm{rel}}\) or \(\lvert p_i-o_i\rvert \le \delta_{\mathrm{abs}}\), where \(\delta_{\mathrm{rel}} = 0.25\) and \(\delta_{\mathrm{abs}} = 0.05\thinspace U_H\) for a velocity component
  • \(\mathrm{NMSE} = \dfrac{\overline{(o-p)^2}}{\overline{o}\thinspace \overline{p}}\) — normalised mean square error
  • \(\mathrm{NAD} = \dfrac{\overline{\lvert o-p\rvert}}{\overline{o}+\overline{p}}\) — normalised absolute difference
  • \(\mathrm{MG} = \exp\overline{(\ln o-\ln p)}\) — geometric mean bias
  • \(\mathrm{VG} = \exp\overline{(\ln o-\ln p)^2}\) — geometric variance
  • The ratio of \(\mathrm{FAC2}\) keeps its sign, so a model of the other sign than the measurement is a miss; \(\mathrm{FB}\), \(\mathrm{NMSE}\) and \(\mathrm{NAD}\) divide by the means, so they are undefined (shown as —) for a signed component unless the measured and the modelled mean are both positive
  • VDI 3783-9 calls the hit rate \(q\) and the tolerances \(D\) and \(W\); on this site those letters keep their other meanings (\(q\) the tracer release rate, \(D\) the block size of Case C, \(W\) the vertical velocity)
  • The detection limit of a concentration is one value per case, \(10^{-3}\) of the largest measured value. It sets \(\delta_{\mathrm{abs}}\) of the hit rate and the both-below rule of \(\mathrm{FAC2}\), and floors the values in \(\mathrm{MG}\) and \(\mathrm{VG}\).
  • The scores \(\mathrm{FB}\), \(\mathrm{FAC2}\), \(\mathrm{NMSE}\), \(\mathrm{NAD}\), \(\mathrm{MG}\) and \(\mathrm{VG}\) follow Chang & Hanna (2004); the hit rate follows VDI 3783-9.

Quantities scored

  • \(U\), \(V\) and \(W\) (mean velocity components) over \(U_H\)
  • \(k\) (turbulent kinetic energy) over \(U_H^2\)
  • the speed \(\lvert\mathbf{U}\rvert\) and \(\sqrt{\lvert\mathbf{U}\rvert^2+2k}\), against a thermistor’s \(V_{\mathrm{s}}\)
  • \(c^\ast = C/c_0\) (dimensionless concentration) at turbulent Schmidt numbers \(\mathrm{Sc_t}\) of 0.3, 0.5 and 0.7
  • \(\theta\) (floor-referenced temperature)

Measurement locations

Measurement lines and planes, Case A
Figure 1. Case A, measurement lines and planes: A vertical profiles, \(y/b = 0\); B plane \(z/b = 0.125\); C plane \(z/b = 1.25\). Positions in multiples of \(b\).

Explore the points

model lowmodel high
Loading the points… Hover a point to see where it sits in 3D and how the model differs from the measurement; click to pin it. This needs JavaScript; the figures on this page show the same scores.

OpenFOAM 12

Status: scored

Measured against modelled

Measured vs modelled, Case A
Figure 2. Case A, measured vs modelled \(U\), \(V\), \(W\) and \(k\), with the 1:1 line and, where the quantity does not change sign, the factor-of-two band. The dots take the colour of the line or plane they were measured on, as in Figure 1.
  • Normalised by \(U_H\). FB is positive when the model is low.
  • Reattachment length behind the building: \(L_r/H\) = 1.49.
  • \(V\) and \(W\) change sign. The mean measured \(W\) is ~0.01 m/s.
  • The scores table shows the served steady run (RANS) beside the same solver run transient (URANS: SST k-omega, PIMPLE, 1000 s of flow time on the measured inlet, scored on the time means with \(k\) as the modelled mean plus the resolved part). The URANS run reattaches at \(L_r/H\) = 1.29 and resolves 27 % of the wake’s \(k\) (3 % upstream); its inlet is the tunnel’s measured profile, the RANS run’s the fitted log law.
RANS and URANS scores, Case A
Figure 3. Case A, the scores of the table, one panel per metric and one row per quantity: the served steady run (RANS, hollow circle) and the transient run (URANS, filled square), the dashed line where a perfect model scores. FB of \(V\) is not defined for the URANS run.
Centre-plane profiles, Case A
Figure 4. Case A, vertical profiles of \(U/U_H\) on the centre plane at the first three measured stations behind the building, offset per station.
Interpretation: Case A scores and profiles
  • The streamwise field is right: \(R = 0.96\), FAC2 0.82.
  • \(k\) is under-predicted (FB +0.48, \(R = 0.48\)); Cases B and H show the same under-prediction.
  • \(V\) and \(W\) change sign, so factor-of-two agreement is a harsh measure near zero and FAC2 is low; R and hit rate carry the comparison there.
  • \(W\)’s FB of −1.50 and its NMSE follow from a near-zero mean; the correlation is \(R = +0.71\).
  • The model follows the measured profile closely above roof height and in the recovering wake; the near-ground deficit immediately behind the building is where the closure does least well.

Field maps

The solved velocity on the measured planes, every point drawn on it as a dot coloured by its measured value on the same scale: a dot that stands out from the colour around it is a point the model misses.

Solved field and measured points, Case A, plan
Figure 5. Case A, the horizontal plane \(z = 0.01\thinspace\mathrm{m}\): (a) \(u\) over \(U_H\); (b) the same with iso-lines every 0.1 and the edge of the reversed flow, \(u = 0\), dashed; (c) the in-plane speed with streamlines.
Solved field and measured points, Case A, section
Figure 6. Case A, the centre plane \(y = 0\): (a) \(u\) over \(U_H\); (b) the same with iso-lines every 0.1 and the edge of the reversed flow, \(u = 0\), dashed; (c) the in-plane speed with streamlines.

Scores

Reattachment length \(L_r/H\) = 1.49.

Score Run R \(R^2\) FAC2 FB Hit rate
\(U\) RANS +0.955 0.912 0.817 +0.185 0.586
  URANS +0.953 0.908 0.763 +0.181 0.683
\(V\) RANS +0.961 0.923 0.349 -0.207 0.892
  URANS +0.953 0.909 0.360 — 0.817
\(W\) RANS +0.708 0.502 0.398 -1.501 0.801
  URANS +0.675 0.456 0.414 -1.352 0.699
\(k\) RANS +0.480 0.230 0.667 +0.478 0.177
  URANS +0.626 0.392 0.694 +0.453 0.392

Every score is over the same 186 points.

OpenLB

Status: scored

OpenLB (Eddy3D’s eddy3dWind: Smagorinsky, third-order regularised D3Q19, image 1.9.3-r6, CPU build) solved the experiment with the same fitted inflow as OpenFOAM 12 and probed the same measured points, at \(\mathrm{d}x = 1.6\thinspace\mathrm{m}\) (\(H/10\)), 1.3 million cells, 13 min on 24 threads of one node (54 MLUPS). \(H/20\) costs about 16 times the cell-steps.

Measured vs modelled U and V, Case A, one row per engine
Figure 7. Case A, measured vs modelled \(U\) and \(V\) on the same points, one row per engine: OpenFOAM 12, OpenLB and FluidX3D. FluidX3D’s hollow points are the near-ground heights it is not scored on.
Quantity n FAC2 FB hit rate R OpenFOAM FAC2 / hit rate / R
\(U/U_H\), all points 186 0.47 +0.38 0.38 0.78 0.82 / 0.59 / 0.96
\(U/U_H\), points ≥ 2 cells above the floor 121 0.65 +0.22 0.56 0.80 —
\(V/U_H\) 186 0.25 — 0.66 0.77 0.35 / 0.89 / 0.96
\(W/U_H\) 186 0.18 — 0.56 0.57 0.40 / 0.80 / 0.71
  • OpenLB meets the ground with a plain no-slip (bounce-back) wall at \(z = 0\) and carries the roughness only in the inlet profile. In the undisturbed column at the windward face its \(U\) is 0.60 of the inlet log law at the first node (half a cell up), 0.88 one cell up, 1.14 at 1.5, 1.10 at 2, 1.04 at 3 and 1.01 from 4. Case A’s lowest plane sits 0.6 cells up.
  • The building’s wall faces sit at \(z = 0\) and \(16\thinspace\mathrm{m}\) (250 solid nodes). Until image 1.9.3-r6 the solver’s lattice started at the domain’s floor, which put the wall half a cell up and the roof a row short (14.4 m); this run replaces the one made with that lattice.
  • The resolved \(k\) is 1 % of the measured one; \(k\) is not compared.
  • Reattachment length: \(L_r = 2.80\thinspace H\) against OpenFOAM’s 1.49; the field fits best sampled \(H/2\) downstream (R 0.81 against 0.78).
  • Control: with the lateral sign flipped, \(V\)’s R becomes −0.77; \(U\) scrambled across the points drops R to 0.10.
Interpretation: OpenLB on Case A
  • The near-zero model \(U\) at the floor in the OpenLB row of Figure 7 is the no-slip wall; above two cells FAC2 rises to 0.65 and FB falls to +0.22.
  • The solve is nearly steady: a steady log-law inflow at \(H/10\) resolves almost no turbulence, so \(k\) is not compared, and with nothing to entrain the wake runs long.
  • Its frame is right.

Scores

Reattachment length \(L_r/H\) = 2.80.

Score R \(R^2\) FAC2 FB Hit rate n
\(U\) +0.782 0.611 0.468 +0.378 0.376 186
\(U\), points ≥ 2 dx +0.800 0.640 0.653 +0.219 0.562 121
\(V\) +0.767 0.588 0.253 -0.162 0.661 186
\(W\) +0.574 0.329 0.177 -1.711 0.559 186
\(k\) (resolved) +0.162 0.026 0.000 +1.958 0.000 186
\(V\), lateral sign flipped (control) -0.767 0.588 0.124 -24.677 0.554 186
  • Per-point values: /api/cases/A/pairs/openlb.csv
  • Reproduce: Eddy3D fixture TestAijCaseAOpenLbContainerE2E (OpenLB eddy3dWind, image 1.9.3-r6, CPU build, \(\mathrm{d}x = 1.6\thinspace\mathrm{m}\)); figure validation/plot_cases_ab_engines.py

FluidX3D

Status: scored

FluidX3D (Eddy3D’s GPU lattice-Boltzmann engine: D3Q19 with Smagorinsky LES) solved the experiment at \(\mathrm{d}x = H/64\) (2.5 mm, 117 million cells) with the same fitted log-law inflow, averaging over 200 \(H/U_H\) after 40 of spin-up. The run has an empty-domain control on the same lattice, and a point is scored only at a height where that control holds the imposed log law within 5 %: 116 of the 186 points. Its row is in Figure 7.

Quantity, comparable heights n FAC2 FB hit rate R OpenFOAM on the same points: FAC2 / hit rate / R
\(U/U_H\) 116 0.60 +0.41 0.57 0.84 0.85 / 0.73 / 0.95
\(V/U_H\) 116 0.26 — 0.75 0.85 0.27 / 0.90 / 0.95
\(W/U_H\) 116 0.22 — 0.41 0.59 0.49 / 0.79 / 0.93
  • The run meets the ground with FluidX3D’s bounce-back wall; it predates Eddy3D’s opt-in ground wall function. In the empty domain the flow at the lowest plane (\(z = 0.01\thinspace\mathrm{m}\), 4 cells up) reads 0.52 of the imposed log law on average, and 0.86 at \(z = 0.04\thinspace\mathrm{m}\). Those points are scored separately (velocity.*.excluded) and drawn hollow in Figure 7.
  • Empty-domain control on the comparable points, \(U\): FAC2 0.77, FB −0.26, hit rate 0.59, R 0.51. The solved field: FAC2 0.60, FB +0.41, hit rate 0.57, R 0.84.
  • Control: with the lateral sign flipped, the \(V\) correlation becomes −0.85.
Interpretation: FluidX3D on Case A
  • The excluded points are the floor again.
  • The approach flow alone scores better on \(U\) (FAC2, FB and hit rate); only the correlation shows the building’s effect.
  • Its frame is right.

Scores

Reattachment length \(L_r/H\) = 0.05.

Score R \(R^2\) FAC2 FB Hit rate n
\(U\), comparable heights +0.840 0.705 0.603 +0.405 0.569 116
\(V\), comparable heights +0.845 0.714 0.259 -0.692 0.750 116
\(W\), comparable heights +0.591 0.349 0.224 -1.678 0.414 116
\(U\), all points +0.806 0.650 0.597 +0.372 0.452 186
\(U\), approach flow alone (control) +0.510 0.260 0.767 -0.256 0.586 116
  • Per-point values: /api/cases/A/pairs/fluidx3d.csv
  • Reproduce: Eddy3D fixture TestAijCaseAFluidX3DE2E (Eddy3DCli.Tests/Validation/TestAijVelocityFluidX3DE2E; FluidX3D GPU lattice Boltzmann, D3Q19 Smagorinsky LES, \(\mathrm{d}x = H/64\), 40 + 200 \(H/U_H\) spin-up + averaging, with an empty-domain control on the same lattice); figure validation/plot_cases_ab_engines.py

ML surrogate

Status: scored

Eddy3D’s Wind Predictor (the public Yel-2.0 network) predicts \(\lvert\mathbf{U_{\mathrm{mean}}}\rvert/U_{\mathrm{ref}}\) at one height, 1.8 m, so the case is scaled ×180 to put its measurement plane there; the prediction is read at the tunnel’s own sensor coordinates. How the surrogate is run is on the ML surrogate page.

Measured vs predicted, ML surrogate, Case A
Figure 8. Case A, measured vs predicted pedestrian-height speed, ML surrogate, with the 1:1 line and the factor-of-two band.
Quantity n FB FAC2 R hit rate OpenFOAM, same points and quantity
\(\lvert\mathbf{U}\rvert/U_H\), \(z = 0.01\thinspace\mathrm{m}\) plane 65 +0.77 0.26 −0.13 0.09 FB +0.27, FAC2 0.85, R +0.90, hit rate 0.45
  • FB is positive when the model is low. Reversing the wind changes the predicted field by at most \(1 \times 10^{-4}\) \(U_{\mathrm{ref}}\): the prediction does not depend on the wind direction.

Scores

Score R \(R^2\) FAC2 FB Hit rate
\(\lvert\mathbf{U}\rvert\) (\(z = 0.01\thinspace\mathrm{m}\) plane) -0.125 0.016 0.262 +0.772 0.092
\(\lvert\mathbf{U}\rvert\), lattice cut at ±250 m +0.434 0.188 0.615 +0.401 0.338
wind reversed (control) -0.125 0.016 0.262 +0.772 0.092

Every score is over the same 65 points.

Grid uncertainty

OpenFOAM on four meshes (edge ratio \(\sqrt{2}\), 151,210 to 1,411,566 cells), eddy-viscosity cap lifted as in eddy3d-of12-nutcap-lifted. The served run keeps the cap.

Grid uncertainty, OpenFOAM, Case A
Figure 9. Case A, second mesh of four: measured vs modelled, bar = grid uncertainty. Hollow: no band.
  • Median band: \(U\) 0.011 \(U_H\) (p90 0.051); \(V\), \(W\), \(k\) about 0.001. 140 of 186 points within the hit rate’s 0.05 \(U_H\); 30 without a band.
  • Across the meshes: \(U\) FAC2 0.79–0.82, hit rate 0.58–0.63; \(k\) \(R\) 0.28–0.44.
  • \(L_r/H\) 1.49, 1.44, 1.50, 1.51; extrapolated 1.51, measured 0.38–0.75. The long wake is the closure, not the mesh.
  • Bands: grid/openfoam12-realizableKE.csv. Method: Grid uncertainty.

Notes on every run

  • Normalisation: velocities by \(U_H\), \(k\) by \(U_H^2\) (the benchmark's own convention; Case E by \(U\) at 15.9 m, as it is published); hit rate \(\delta_{\mathrm{rel}} = 0.25\), \(\delta_{\mathrm{abs}} = 0.05\) in those units

Leaderboard

Rank Run Source Hit rate, \(U\) Hit rate, \(V\) Hit rate, \(W\) Hit rate, \(\lvert\mathbf{U}\rvert\) Hit rate FAC2 R \(R^2\)
1 OpenFOAM simpleFoam (Eddy3D Outdoor case writer) (eddy3d-of12-mesh2x-measured-inlet) community 0.651 0.887 0.833 0.677 0.762 0.603 +0.900 0.819
2 OpenFOAM simpleFoam (Eddy3D Outdoor case writer) (eddy3d-of12-nutcap-lifted) community 0.613 0.903 0.833 0.667 0.754 0.613 +0.891 0.805
3 OpenFOAM simpleFoam (Eddy3D Outdoor case writer) (eddy3d-of12-measured-inlet) community 0.624 0.876 0.828 0.667 0.749 0.610 +0.894 0.810
4 OpenFOAM pimpleFoam (Eddy3D Outdoor case writer) (eddy3d-of12-sst-urans) community 0.683 0.817 0.699 0.720 0.730 0.590 +0.877 0.782
5 OpenFOAM 12 Eddy3D 0.586 0.892 0.801 0.634 0.728 0.610 +0.891 0.805
6 OpenFOAM simpleFoam (Eddy3D Outdoor case writer) (example-openfoam12) example submission 0.586 0.892 0.801 0.634 0.728 0.610 +0.891 0.805
7 FluidX3D Eddy3D 0.452 0.618 0.441 0.554 0.516 0.462 +0.717 0.546
8 OpenLB Eddy3D 0.376 0.661 0.559 0.409 0.501 0.353 +0.728 0.538

All runs scored by validation/score_submissions.py. Ranked by mean hit rate (VDI 3783) over the tasks; bold: best per column. Not ranked: the ML surrogate (no point ids) and negative controls. Community rows are self-reported (Validate your own solver).