Error analysis

Global RMSE averages over very different pixels. This page breaks error down to show where the model struggles.

By height band

RMSE per height band, with TTA. Error grows with height for every model. Tall structures are rare and have the largest absolute errors.Source: Model_Traning/Logs/v3 (2)/metrics.json · Model_Traning/V4_modal/Output/metrics.json

Bias per height band, with TTA. Every model over-predicts low objects and under-predicts tall ones; the sign flips between the 2–5 m and 5–10 m bands. This is regression towards the mean in a long-tailed target. On the test split, the 20+ m band is under-predicted by 4.78 m.Source: Model_Traning/Logs/v3 (2)/metrics.json · Model_Traning/V4_modal/Output/metrics.json

Evaluation	0-2 m	2-5 m	5-10 m	10-20 m	20 m+
v3 · val (TTA)	1.565	2.651	2.837	4.795	5.638
v4-modal · val (TTA)	2.968	3.294	3.337	4.586	5.553
v4-modal · test (TTA)	1.833	3.023	2.936	4.598	10.389

RMSE (m) per height band, TTA. Source: Model_Traning/Logs/v3 (2)/metrics.json · Model_Traning/V4_modal/Output/metrics.json

Tall and flat bias

Model / split	Tall bias (> 15 m)	Flat bias (< 1 m)
v3 · val	−2.12 m	+0.38 m
v4-modal · val	−1.96 m	+0.96 m
v2 · val (post-mortem)	−5.3 m (structures > 15 m)	flat ground within 0.48 m

v3 has the best flat-ground behaviour. v4 traded flat ground for tall structures, which is the trade-off analysed in Design findings.

By landscape and domain

v4-modal RMSE by landscape on the GAMUS test split (TTA), and on two other domains. Sparse scenes are easiest. Urban and forested scenes are equally hard. The out-of-domain satellite set is hardest.Source: Model_Traning/V4_modal/Output/metrics.json; Research-Paper/main.tex

On DFC23, structures above 15 m reach 7.97 m RMSE. A different sensor and continent shift both the colour statistics and the building typology.

Error sources, summarised

Source of error	Evidence	Mitigation
Regression to the mean on rare tall structures	Bias −2 to −5 m above 15 m	stratum balancer; adaptive-bin head; more tall data (US3D, MVS3DM)
Phantom height on flat ground	Flat bias +0.4 to +1.0 m	flatness loss; β = 0.5
Canopy under-estimated or flattened	NEON forest 4.11 m; v4 flattened trees	NEON data; forest-weighted sampling and selection
Over-smoothing at edges	Gradient ratio 0.23 on MVS3DM	detail branch; stronger gradient and normal losses
Sensor shift	DFC23 4.97 m vs GAMUS 3.32 m	per-scene stretch; photometric and Cartosat augmentation
Wrong GSD	Heights scale with the GSD error	read GSD from the GeoTIFF; warn when it is assumed