geointel/docs/BELGIUM_BUILDING_TRAINING_LOOP.md

Belgian building detector: closed training loop

Meaning of complete

100% trained means that every frozen release gate below passes. It does not mean a fabricated 100% precision, recall or mAP score. A model that memorises a small test set is not complete.

The loop is:

1. provision new, spatially independent AOIs from governed official services;
2. freeze imagery, labels, metadata and checksums into a new corpus version;
3. reject invalid, duplicate and sub-resolution labels and run spatial-leakage checks;
4. train only on the train split with CUDA on the Tower NVIDIA GPU;
5. use validation for early stopping and calibration only for threshold choice;
6. evaluate the fixed threshold once on regional test and background-test data;
7. attribute false positives and false negatives to a region, AOI and context;
8. add new training-only examples for the observed failure modes and repeat;
9. stop only when every objective gate passes; request human review afterward.

Protected calibration, test and background-test AOIs never become training data. A new iteration adds independent training AOIs instead.

Frozen release gates

Area	Gate
Runtime	CUDA required; NVIDIA device visible; no CPU fallback
Corpus	Immutable manifest and artifacts with SHA-256 evidence
Geographic composition	Each land region has at least 15 train, 2 val, 3 calibration, 3 test and 2 background-test AOIs
Contexts	Dense urban, suburban, rural, industrial and difficult negative contexts represented
Leakage	No intersecting AOIs across protected split roles
Label integrity	No malformed rows; sub-resolution labels explicitly rejected
Temporal truth	Unknown per-pixel dates remain unknown; acquisition dates may not masquerade as observation dates
Threshold selection	Calibration set only; maximise the worst regional F1 before aggregate F1
Test aggregate	F1 at least 0.55 at the frozen footprint/detection match IoU 0.25 contract
Regional test	Every region: F1 at least 0.45, precision at least 0.50 and recall at least 0.40
Pure background	Zero detections on every pure-empty tile at the selected threshold
Production	Exact candidate checksum and fail-closed promotion report required
Final review	Human accepts every queued AOI contact sheet after all automated gates pass

These are minimum release gates, not performance targets. Raising a confidence threshold until detections disappear cannot pass because regional recall is a simultaneous gate.

Current gap inventory

The v3 corpus closes basic composition and leakage gaps with 60 independent AOIs and 10,262 accepted building labels. It adds coastal, ribbon-development, farmland, park, forest and additional urban contexts. Its remaining known gaps are:

• all rolling regional mosaics have an unknown exact per-pixel observation date; this is recorded honestly and must be resolved through dated provider products or change-aware label review, never inferred from download time;
• GRB/PICC/UrbIS describe ground footprints, whereas visible roofs can remain displaced. The existing detector QA contract therefore uses IoU 0.25; the threshold is frozen and cannot be relaxed per candidate;
• the first loop candidate generalises poorly in Flanders and Wallonia, especially Mechelen, Sint-Niklaas, Leuven, Mons and dense PICC areas;
• sparse hard contexts pass the empty-image test more easily than dense urban recall, so both gates must remain independent;
• building boxes are a valid first detector contract, but footprint-perfect geometry ultimately requires a separately validated segmentation model.

The active production model remains unchanged while any gate fails.

Reproducible evidence

• corpus assembler: scripts/assemble_belgium_building_corpus.py;
• corpus auditor: scripts/audit_belgium_building_corpus.py;
• tile exporter/auditor/contact sheets: the operator_yolo scripts;
• per-AOI evaluator: scripts/evaluate_belgium_building_candidate.py;
• calibration-only selection and release gates: scripts/assess_belgium_building_training_iteration.py.

Every failed assessment returns continue_training_loop. Only a report with training_complete may proceed to final human review and guarded activation.