Blog

Validating a satellite-derived feature layer against field survey points

A dry, scrub-covered hillside landscape under a clear sky, the kind of terrain a field crew might walk.

You ran the field day. Crew walked the transects, dropped GPS points on every instance of whatever you're mapping, logged attributes in the field app. Now you've got a satellite-derived layer back: output from a classifier, a vendor deliverable, or a workflow that learned the feature from a handful of labeled examples. Either way, someone's going to ask how you know it's right, and "the vendor said so" isn't an answer that survives a QA review.

There's no published accuracy standard for a feature class that doesn't exist in NLCD or any other off-the-shelf product. Your field points become the benchmark, built fresh for this one deliverable.

Setting up the comparison

Start by holding back a subset of field points that had no role in producing the layer, whether that means they weren't used to train a classifier or weren't shown to whoever digitized the output. If every point you have was already baked into the layer, you're not validating anything, you're just checking your own homework.

For each held-back point, check whether a mapped feature falls within a reasonable distance. "Reasonable" depends on your GPS unit's accuracy and the imagery's ground sample distance, not on a number that feels right. A handheld GPS with a few meters of drift plus 1-2 m imagery means your tolerance buffer should be somewhere in that combined range, not an arbitrary 10 m because it's easier to hit.

Build a simple error matrix from the comparison:

  • True positive: field point has a corresponding mapped feature within tolerance
  • False negative (omission): field point exists, nothing was mapped there
  • False positive (commission): a mapped feature has no field point near it, and you've confirmed on the ground or via high-res imagery that it shouldn't be there

Commission errors are easier to run down later from the desk. Omission errors are the ones to chase first, because they tell you something the layer is systematically missing, whether that's a spectral confusion with a similar surface, canopy shadow at edge of plot, or a sub-pixel feature the sensor just can't resolve at that GSD.

Reading the pattern, not just the count

A raw tally of hits and misses tells you less than where the misses cluster. Pull up the false negatives on the imagery and look for what they share. If every missed point sits under partial canopy, log it as a known detection limit in your QA memo. If the misses are scattered with no pattern, that's a different problem, and probably a harder one to fix by re-running the model.

This is also where field notes earn their keep. A GPS point with no attribute context is just a dot. A point tagged "small patch, under 2 m across, partial dieback" tells you why it might not show up in a layer built from satellite imagery, and whether that's an acceptable gap for the project's purpose or a disqualifying one.

Document the method itself in whatever accompanies the deliverable: how many points, how they were split between training and validation if applicable, what buffer tolerance you used and why, and the resulting omission and commission counts. A reviewer or regulator reading the metadata should be able to reconstruct your QA approach without calling you.

If the error pattern points back to the source data rather than the comparison method, raise it before the layer goes into a report. A one-off feature class built from a handful of labeled examples is only as good as the loop between what the field crew confirms and what gets corrected in the next pass, and a validation step that catches the gap early saves a rerun after the deliverable's already been cited somewhere.

If you've got field points for a feature nobody's built a classifier for yet, Few-Shot Mapping is worth a look before you start digitizing it by hand.

Request early access

← Back to the blog