LiDAR validation

The IARPA Multi-View Stereo 3D Mapping (MVS3DM) benchmark pairs WorldView-3 imagery of San Fernando, Argentina with airborne LiDAR. DepthWizard v5 was run on a pansharpened true-ortho of one scene (0.3 m, 5.4° off-nadir) and compared with a LiDAR nDSM at the 8 official benchmark sites. These sites form the test split in tools/pack_mvs3dm.py, and tiles within a buffer around them are dropped, so they were never trained on.

Per-pixel RMSE

3.38m

pooled over 13.1 M pixels · bias +0.35

30 m-cell RMSE

1.35m

1,488 cells · bias +0.36

Median ground error

≤ 0.11m

on LiDAR ground pixels, every site

Per site

RMSE per site at three levels: every pixel, confident pixels only (the more certain half, by predicted σ), and 30 m cell averages. Averaging to 30 m, the resolution of public DEMs, removes most of the edge and alignment error.Source: cartosat_2S_Sample/mvs3dm_results/summary.txt

Site	px RMSE	Bias	r	Conf. RMSE	30 m RMSE	30 m bias	Median gnd. err.
MasterSequestered1	2.59	1.17	0.65	1.71	1.32	1.17	0.11
MasterSequestered2	2.45	0.64	0.68	1.62	0.93	0.66	0.09
MasterSequestered3	4.83	-0.06	0.84	2.10	1.93	0.11	0.04
MasterSequesteredPark	3.23	0.09	0.84	1.16	1.17	0.05	0.02
Explorer	2.55	0.16	0.76	0.90	0.89	0.14	0.00
MasterProvisional1	2.54	1.02	0.68	1.51	1.26	1.02	0.04
MasterProvisional2	3.31	0.80	0.85	1.29	1.41	0.81	0.01
MasterProvisional3	6.67	-0.37	0.82	1.60	2.60	-0.23	0.00

Source: cartosat_2S_Sample/mvs3dm_results/summary.txt

Examples

Explorer · RGB — Explorer site: 2.55 m per-pixel RMSE, 0.89 m per 30 m cell. Height colour scale 0–20 m (turbo). In the error map, red is over-prediction and blue under-prediction.Imagery: IARPA MVS3DM, courtesy of DigitalGlobe. Approved for public release.

LiDAR nDSM — Explorer site: 2.55 m per-pixel RMSE, 0.89 m per 30 m cell. Height colour scale 0–20 m (turbo). In the error map, red is over-prediction and blue under-prediction.Imagery: IARPA MVS3DM, courtesy of DigitalGlobe. Approved for public release.

Sequestered3 · RGB — MasterSequestered3: dense high-rise blocks. 4.83 m per pixel (r 0.84); most error sits on building edges and in the tallest towers.Imagery: IARPA MVS3DM, courtesy of DigitalGlobe. Approved for public release.