Initial public release

docs/QA_QC_SPECIFICATION.md

@@ -0,0 +1,176 @@
+# GeoIntel Kempen — QA/QC Specification v1.0
+
+QA/QC is a core showcase feature. It must compare model-derived geospatial outputs against reference data such as GRB or user-supplied labels.
+
+## Primary use case
+
+Compare AI building detections or segmentations against GRB building footprints.
+
+## Inputs
+
+### Prediction layer
+
+- Polygons converted from object detections or segmentation masks.
+- Required fields: `id`, `class_name`, `confidence`, `geometry`, `analysis_run_id`.
+
+### Reference layer
+
+- GRB building polygons, OSM buildings, or user annotation polygons.
+- Required fields: `id`, `class_name` or canonical class, `geometry`, `source`.
+
+## Geometry preparation
+
+1. Reproject both layers to the metric CRS.
+2. Repair invalid geometries.
+3. Clip both layers to the analysis area.
+4. Optionally simplify only for visualization, not for metric calculation.
+5. Filter classes if class-specific QA is requested.
+
+## Matching strategy
+
+Default matching is polygon IoU-based matching.
+
+### IoU
+
+```text
+IoU = area(intersection(prediction, reference)) / area(union(prediction, reference))
+```
+
+### Match threshold
+
+Default: `IoU >= 0.50`.
+
+Additional thresholds for reporting:
+
+- lenient: `0.30`
+- standard: `0.50`
+- strict: `0.75`
+
+### Matching algorithm
+
+1. Build spatial index.
+2. For every prediction, find intersecting reference candidates.
+3. Calculate IoU for candidates.
+4. Select the highest IoU candidate.
+5. Enforce one-to-one matching: one reference can match only one prediction.
+6. Resolve conflicts by highest IoU, then highest confidence.
+
+## Classification of outcomes
+
+| Outcome | Definition |
+|---|---|
+| True Positive | prediction matched to a reference above threshold |
+| False Positive | prediction not matched to any reference |
+| False Negative | reference not matched by any prediction |
+| Low-IoU Match | prediction overlaps reference but below selected threshold |
+| Class Mismatch | geometry match but class differs |
+
+## Metrics
+
+```text
+precision = TP / (TP + FP)
+recall = TP / (TP + FN)
+f1 = 2 * precision * recall / (precision + recall)
+mean_iou = average IoU of matched pairs
+```
+
+Also calculate:
+
+- false positive count
+- false negative count
+- low confidence false positives
+- confidence distribution by outcome
+- area-weighted recall
+
+## Outputs
+
+### QA summary JSON
+
+```json
+{
+  "threshold": 0.5,
+  "true_positive_count": 120,
+  "false_positive_count": 8,
+  "false_negative_count": 13,
+  "precision": 0.9375,
+  "recall": 0.9022,
+  "f1": 0.9195,
+  "mean_iou": 0.72,
+  "class_name": "building"
+}
+```
+
+### QA layers
+
+- `qa_true_positives`
+- `qa_false_positives`
+- `qa_false_negatives`
+- `qa_low_iou_matches`
+- `qa_match_lines` connecting prediction centroids to reference centroids
+
+### QA dashboard cards
+
+- Precision
+- Recall
+- F1
+- Mean IoU
+- False positives
+- False negatives
+- Top issue areas
+
+## UI requirements
+
+The QA/QC page must show:
+
+1. Reference layer selector.
+2. Prediction layer selector.
+3. Class filter.
+4. IoU threshold slider.
+5. Run QA button.
+6. Metrics cards.
+7. Map with color-coded TP/FP/FN layers.
+8. Table of individual findings.
+9. Export buttons for GeoJSON and CSV.
+
+## V1 implementation target
+
+V1 must support polygon-vs-polygon QA for building detections against GRB or OSM building footprints. Raster mask QA and pixel-level IoU can be added later.
+
+## Sprint 12 golden QA/QC benchmark
+
+Sprint 12 adds a deterministic benchmark package for regression detection:
+
+- `fixtures/golden/reference_buildings.geojson`
+- `fixtures/golden/predicted_buildings.geojson`
+- `fixtures/golden/expected_qa_metrics.json`
+- `scripts/run_golden_qa_benchmark.py`
+
+The benchmark uses two reference building polygons and two candidate building polygons:
+
+- one candidate polygon matches one reference polygon at IoU `0.8339768339761133`;
+- one candidate polygon is a false positive;
+- one reference polygon is a false negative.
+
+Expected baseline at IoU threshold `0.5`:
+
+```json
+{
+  "matches": 1,
+  "false_positive_count": 1,
+  "false_negative_count": 1,
+  "precision": 0.5,
+  "recall": 0.5,
+  "f1": 0.5,
+  "mean_iou": 0.8339768339761133
+}
+```
+
+Run from the repository root:
+
+```bash
+python scripts/run_golden_qa_benchmark.py
+```
+
+The script uses the existing `QaService` and `QualityService` paths. It fails if metrics drift outside the documented tolerance and verifies that one `QualityCheck` plus metric rows for precision, recall, F1, mean IoU, false positives and false negatives are produced.
+
+This benchmark is explicit fixture/demo data only. It does not require live providers, real AI models, Docker or PostGIS.