Geospatial Sciences shows up in a lot of AI-adjacent work without being labeled that way: satellite imagery, LiDAR, photogrammetry, spatial analysis. My own geospatial exposure is real but comes in through a specific door, wind resource assessment in the renewable energy sector, where met-mast siting and offshore layout work is geospatial analysis wearing a different name. That's adjacent to hands-on remote-sensing and LiDAR work, not the same thing, so I built the actual pipeline: real point clouds, real satellite imagery, real ground truth, real metrics, rather than leaning on the overlap.
This is the write-up. The live demo is linked at the bottom, and the repository is public.
Picking data that's actually real, not a Kaggle CSV
A colleague's original proposal for this pointed at SpaceNet 6 (SAR plus optical imagery over Rotterdam, ~48,000 building footprints) paired with PNOA, Spain's national LiDAR programme via IGN. Both are excellent datasets and neither made it into the final build, for reasons worth stating plainly rather than glossing over: SpaceNet 6 is tens of gigabytes, more than a demonstrator needs to make its point, and PNOA's download portal is a session-based web form, not something you can point a script at reliably. I swapped in three sources that are just as real and actually scriptable with no account, no API key, and no registration:
The area of interest is a rowhouse block on Capitol Hill in Washington, DC, chosen because it's where all three sources actually overlap: 3DEP LiDAR coverage from a 2014 flight, a near-cloud-free Sentinel-2 scene (0.003% cloud cover), and dense OpenStreetMap building coverage to use as ground truth.
flowchart LR
lidar["USGS 3DEP\nLAZ point cloud"] --> dtm["DTM / DSM\n(laspy, ground-classified)"]
dtm --> ndsm["nDSM\nDSM minus DTM"]
s2["Sentinel-2 L2A\nB04 / B08 COGs"] --> ndvi["NDVI"]
osm["OpenStreetMap\nOverpass API"] --> mask["building mask\n(ground truth)"]
ndsm --> features["feature stack\nnDSM + NDVI + slope + texture"]
ndvi --> features
features --> rf["RandomForest\nbuilding classifier"]
mask -.train/validate.-> rf
rf --> pixel["pixel metrics\nF1, IoU"]
rf --> vector["vectorize\n(rasterio.features.shapes)"]
vector --> polygon["polygon metrics\ndetection rate, area error"]
mask -.-> polygon
Why a RandomForest, not a CNN
The feature stack per grid cell is four numbers: canopy/building height above ground (nDSM), vegetation index (NDVI), local terrain slope, and a texture measure (local standard deviation of nDSM). That's a tabular problem, not an image problem, over a single ~700m AOI with no need for a GPU. A scikit-learn RandomForestClassifier trains on it in seconds, and unlike a CNN it hands back a feature-importance ranking for free, which turned out to matter for the story this project ended up telling.
The result that actually justifies "multimodal"
It's easy to claim combining sensors helps. Here it's checked against two single-feature baselines, evaluated on the same held-out region as the full model:
| Model | Precision | Recall | F1 | IoU |
|---|---|---|---|---|
| NDVI-only (low NDVI → building) | 77.3% | 43.7% | 55.8% | 38.7% |
| nDSM-only (height > 2.5m → building) | 35.7% | 99.7% | 52.5% | 35.6% |
| RandomForest, nDSM+NDVI+slope+texture | 79.3% | 80.2% | 79.7% | 66.3% |
NDVI alone is precise but blind to anything that isn't vegetation, roads and parking lots get called "building" too. Height alone catches almost every raised structure (99.7% recall) but drags every tree canopy along with it. Neither failure mode is fixable by tuning a threshold; they're complementary blind spots, and the combined model (feature importance: NDVI 45.2%, nDSM 39.8%, texture 11.8%, slope 3.2%) is the one that actually closes both.
Where pixel accuracy stops telling the truth
79.7% pixel F1 looks like a strong result. A GIS reviewer doesn't score pixels, they score buildings, so the prediction gets vectorized and matched polygon-by-polygon against the 74 OpenStreetMap footprints in the test region.
The gap isn't a bug, it's what the block actually looks like. Capitol Hill rowhouses share party walls, so three or four attached units present one continuous rooftop to the nDSM, with no height discontinuity between them for the classifier to notice. It correctly labels the whole roofline "building" at the pixel level; the vectorizer then merges those units into a single polygon, which counts as one match against several ground-truth parcels. Mean matched-polygon IoU stayed a respectable 0.61, mean area error 36.1%, mean centroid displacement 6.5m, but the detection-rate number is the one that would actually surface in a real QA pass, and it's exactly the kind of instance-segmentation gap that pixel accuracy is structurally unable to show you. This is the difference the mentor's original proposal was pointing at between "ran a CNN on satellite images" and demonstrating the full geospatial ML evaluation lifecycle, and it only shows up if you do the polygon-level check.
Turning the AI-evaluation angle on itself
GIS and remote sensing is a domain where an AI can sound confident and still be subtly wrong, so this project turns that lens on itself: 30 domain questions (CRS, raster vs. vector, DSM/DTM/nDSM, SAR vs. optical, orthorectification, spatial indexing, and more) put to Amazon Nova Lite via AWS Bedrock. The questions this pipeline has direct ground truth for, DSM/DTM/nDSM and LiDAR returns specifically, get graded against that ground truth rather than taken on faith. One answer came back imprecise on exactly that check, which is the point: knowing enough about the domain to catch a plausible-sounding wrong answer is the actual skill being demonstrated, not the trivia itself.
A second pass fed Nova three real misclassifications from the held-out region, each with its actual measured nDSM/NDVI/slope values, and asked for a domain explanation. On a false positive at nDSM 9.0m / NDVI 0.21 / slope 1.0°, it correctly reasoned the height sits near the classifier's decision boundary rather than inventing a cause. That's a small thing, but it's the difference between an AI system that reasons from the numbers in front of it and one that free-associates a plausible-sounding paragraph.
Two things that broke, both worth knowing about
The LiDAR vendor's ASPRS classification codes were the first: a large fraction of points in an area with zero bridges were labeled class 17, "bridge deck," and genuinely extreme outliers (elevations up to 716m in a neighborhood that tops out around 30m) were labeled class 18, "high noise," which sounds like exactly the label you'd want to trust for outlier rejection. Trusting either label for the DSM would have produced a garbage surface. The fix was a plain elevation sanity clip on the raw returns instead of trusting the vendor's classification, a reminder that "classified point cloud" doesn't mean the classification is reliable for every code, and checking the actual elevation distribution before believing a semantic label is worth the extra five minutes.
The second was a Bedrock region gotcha: Amazon Nova Lite's plain on-demand model ID isn't invokable directly in eu-west-1 (this account's standing default region for new AWS usage), only through the cross-region inference profile, eu.amazon.nova-lite-v1:0. Same model, different identifier, and the error message for the wrong one doesn't say "use the inference profile instead," it just declines the request.
The stack, briefly
What it costs to run
Effectively nothing. Every data source is a public, no-sign-request S3 bucket or a free, keyless API, there's no persistent server, and the only metered cost is roughly 30 short Bedrock Nova Lite prompts (about 1,900 input / 2,300 output tokens total), a fraction of a cent. The demo itself is a static page synced onto infrastructure that already runs everything else on this domain.
Try it
The live demo at forwardforecasting.eu/gis-lidar has the full pixel and polygon metrics, the nDSM/NDVI/prediction overlays, the baseline comparison, the three grounded error explanations, and the full 30-question benchmark with the ground-truth-checked answers flagged. The code, including the classification-code gotcha and the region-specific Bedrock model ID, is public.
See it live
Real LiDAR, real satellite imagery, real ground truth, a real limitation found and explained, not a slide deck.
Open the demo → View on GitHub →About the cover image: the gold grid traces the property lines between attached roofs — the exact geometry that turned out to matter most, a shared-wall block where the nDSM sees one continuous rooftop and the polygon-level detection rate drops to 44.6% as a result. ImageModelstability.stable-image-core-v1:1 · AWS Bedrock (us-west-2)Prompt“Editorial illustration, aerial satellite view of a row of attached townhouse rooftops, a glowing gold LiDAR point-cloud scan grid overlaying the rooflines, deep navy background, minimalist flat vector illustration style, clean geometric linework, no text, no logos”