geointel/docs/18-ultra-prep/QA_QC_MATCHING_ALGORITHM.md

QA/QC Matching Algorithm

Goal

Compare AI detections/segmentations with a reference layer such as GRB buildings.

Inputs

• predicted polygons
• reference polygons
• class filter
• IoU threshold, default 0.5

Steps

1. Validate CRS alignment.
2. Reproject to analytical CRS.
3. Build spatial indexes.
4. For every predicted polygon, find candidate reference polygons by bbox intersection.
5. Compute IoU for candidates.
6. Assign best match above threshold greedily, one reference per prediction.
7. Count true positives, false positives, false negatives.
8. Compute precision, recall, F1, mean IoU.
9. Produce unmatched prediction layer and unmatched reference layer.

Metrics

precision = TP / (TP + FP)
recall = TP / (TP + FN)
f1 = 2 * precision * recall / (precision + recall)
IoU = intersection_area / union_area

Output Layers

• matched predictions
• false positives
• false negatives
• low IoU matches

Edge Cases

• empty predictions and empty reference: score should be explicit no_objects not perfect success;
• empty predictions with reference: recall 0;
• predictions with empty reference: precision 0;
• invalid polygons must be repaired or excluded with warning.