Gate filtered YOLO candidate and harden provenance
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-12 23:03:55 +02:00
parent c9b22f8f8b
commit 8daaa071b0
10 changed files with 112 additions and 7 deletions
+9
View File
@@ -7,6 +7,15 @@
# Changelog
## Sprint 169 Filtered YOLO candidate gate and operator hardening (2026-07-12)
- Trained and fully gated inactive `geointel-building-yolov8s-aoi1024cleanpx12vis035lowvar512e50-pt` against seven positive AOIs and split pure-empty/sparse-context backgrounds.
- Rejected the candidate for default promotion: best mean positive F1 was approximately `0.154`, below the `0.25` gate; threshold `0.05` also produced one pure-empty background detection.
- Preserved the current active `aoi1024bg512r3e50` model and all runtime defaults.
- Added SHA256 provenance to future YOLO training summaries.
- Improved long project/dataset/AOI readability with matching tooltips and compact two-line readiness values.
- No API contract, migration, provider fetching, fake output, model download or automatic model activation changed.
## Sprint 164 Tower AI deploy env hardening (2026-07-11)
- Fixed Tower deploy automation so `scripts/deploy_tower.sh` and `scripts/deploy_tower.ps1` source the remote `.env` before building the all-in-one image.
@@ -70,5 +70,9 @@ def test_operator_yolo_train_smoke_script_contract() -> None:
assert "DejaVuSans.ttf" in script
assert "Arial.ttf" in script
assert "training_summary.json" in script
assert '"dataset_yaml_sha256"' in script
assert '"dataset_summary_sha256"' in script
assert '"base_model_sha256"' in script
assert '"trained_model_sha256"' in script
assert "download" not in script.lower()
assert "fixture_mode" not in script
@@ -0,0 +1,22 @@
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
def test_long_context_names_remain_compact_and_inspectable() -> None:
app = (ROOT / "frontend" / "src" / "App.tsx").read_text(encoding="utf-8")
status = (
ROOT / "frontend" / "src" / "components" / "WorkbenchStatusStrip.tsx"
).read_text(encoding="utf-8")
css = (ROOT / "frontend" / "src" / "styles" / "app.css").read_text(
encoding="utf-8"
)
assert "const projectContextLabel = selectedProject?.name ?? 'No project'" in app
assert "const datasetContextLabel = selectedDataset?.name ?? (datasets.length > 0 ? 'Select dataset' : 'No dataset')" in app
assert "<strong title={projectContextLabel}>{projectContextLabel}</strong>" in app
assert "<strong title={datasetContextLabel}>{datasetContextLabel}</strong>" in app
assert "<strong title={item.value}>{item.value}</strong>" in status
assert ".status-tile > strong" in css
assert "-webkit-line-clamp: 2;" in css
+29
View File
@@ -1,3 +1,32 @@
## Sprint 169 Filtered YOLO candidate gate and operator hardening (2026-07-12)
Changed:
- Trained inactive local model asset `geointel-building-yolov8s-aoi1024cleanpx12vis035lowvar512e50-pt` from the visually audited AOI1024 dataset after low-variance negative filtering.
- Added SHA256 provenance fields to future `training_summary.json` output for `dataset.yaml`, the dataset summary, the local base model and the copied trained model.
- Kept long workbench context names compact with matching native tooltips and clamped readiness values to two lines.
- Archived the training, positive-AOI, split-background and promotion evidence under `artifacts/model-review/aoi1024cleanpx12vis035lowvar512e50` locally and matching Tower artifact directories.
Runtime evidence:
- Training completed for 50 CPU epochs with YOLOv8s, image size `512`, batch `4`; best-model validation ended at precision `0.454`, recall `0.491`, mAP50 `0.368` and mAP50-95 `0.149`.
- Model catalog SHA256: `75345767b51cc66692a9d2c2cd6577b7feecaf5971c8762365e8b610b9dfde8e`; catalog status `available`, `active=false`, `will_download_models=false`.
- Local model-load preflight passed with Torch `2.13.0` and Ultralytics `8.4.92`; CUDA is unavailable and no model download occurred.
- Seven-AOI persisted QA matrix produced 28 runs. Best single result was Westerlo at threshold `0.15`, F1 `0.3002114164904862`.
- Mean positive F1 was `0.153943` at threshold `0.05`, `0.153872` at `0.15`, `0.128891` at `0.25` and `0.099011` at `0.35`.
- Strict pure-empty background evidence was clean at thresholds `0.15`, `0.25` and `0.35`; threshold `0.05` produced one Postel-bos detection.
- Split-aware promotion report recommended `none`: every threshold failed the positive mean-F1 gate, and `0.05` also failed background false-positive pressure.
- The existing active `geointel-building-yolov8s-aoi1024bg512r3e50-pt` remains materially stronger at its promoted `0.35` profile with mean F1 `0.320866`; no model default or `.env` value was changed.
Tested:
- Red step: `python -m pytest backend/tests/test_sprint169_long_context_name_readability.py -q` failed before context tooltips and readiness clamping were present.
- `python -m pytest backend/tests/test_sprint169_long_context_name_readability.py backend/tests/test_sprint22_workbench_status_strip.py backend/tests/test_sprint161_widescreen_workbench.py -q` (`5 passed`).
- Red step: `python -m pytest backend/tests/test_sprint129_operator_yolo_training_dataset.py::test_operator_yolo_train_smoke_script_contract -q` failed before training hashes were recorded.
- `python -m pytest backend/tests/test_sprint129_operator_yolo_training_dataset.py -q` (`3 passed`).
- `bash -n scripts/train_operator_yolo_detector.sh` and frontend typecheck passed.
- Full readiness: `bash scripts/run_readiness_check.sh` (`461 passed`; one Alembic head; frontend typecheck/build and shell syntax gates passed).
Next:
- Do not retrain the same architecture blindly. Inspect per-AOI false-negative evidence and improve label geometry/class balance or add targeted positive samples for the weakest AOIs before the next candidate.
## Sprint 164 Tower AI deploy env hardening (2026-07-11)
Changed:
+3 -1
View File
@@ -133,7 +133,9 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Add deterministic visual YOLO label QA contact sheets before spending more CPU on another training run.
- [x] Filter no-data/low-variance pure-empty negative tiles from operator YOLO exports before the next training run.
- [x] Regenerate the AOI1024 cleanpx YOLO dataset with low-variance negative filtering and rerun visual contact-sheet QA before training.
- [ ] Train one inactive candidate from the filtered AOI1024 cleanpx YOLO dataset and gate it through the positive-AOI plus split-background promotion workflow.
- [x] Train one inactive candidate from the filtered AOI1024 cleanpx YOLO dataset and gate it through the positive-AOI plus split-background promotion workflow; reject it because mean positive F1 remains below gate.
- [x] Add deterministic dataset/base/trained-model SHA256 provenance to future operator training summaries.
- [ ] Review per-AOI false-negative evidence for the weakest AOIs and improve positive sample/label geometry coverage before another training candidate.
- [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke.
## Sprint 8 status
+8 -4
View File
@@ -631,6 +631,10 @@ function App(): JSX.Element {
target: 'exports',
},
]
const projectContextLabel = selectedProject?.name ?? 'No project'
const areaContextLabel = selectedArea?.name ?? (areas.length > 0 ? 'Select area' : 'No AOI')
const datasetContextLabel = selectedDataset?.name ?? (datasets.length > 0 ? 'Select dataset' : 'No dataset')
const layerContextLabel = mapFeatureCollection ? `${mapFeatureCount} features` : 'No active layer'
return (
<div className="app-shell workbench-shell">
@@ -645,19 +649,19 @@ function App(): JSX.Element {
<div className="context-bar" aria-label="Active workbench context">
<div>
<span>Project</span>
<strong>{selectedProject?.name ?? 'No project'}</strong>
<strong title={projectContextLabel}>{projectContextLabel}</strong>
</div>
<div>
<span>AOI</span>
<strong>{selectedArea?.name ?? (areas.length > 0 ? 'Select area' : 'No AOI')}</strong>
<strong title={areaContextLabel}>{areaContextLabel}</strong>
</div>
<div>
<span>Dataset</span>
<strong>{selectedDataset?.name ?? (datasets.length > 0 ? 'Select dataset' : 'No dataset')}</strong>
<strong title={datasetContextLabel}>{datasetContextLabel}</strong>
</div>
<div>
<span>Layer</span>
<strong>{mapFeatureCollection ? `${mapFeatureCount} features` : 'No active layer'}</strong>
<strong title={layerContextLabel}>{layerContextLabel}</strong>
</div>
</div>
</header>
@@ -122,7 +122,7 @@ export function WorkbenchStatusStrip({
<span>{item.label}</span>
<span className="status-pill">{item.state}</span>
</div>
<strong>{item.value}</strong>
<strong title={item.value}>{item.value}</strong>
<p>{item.detail}</p>
</div>
))}
+8
View File
@@ -4554,6 +4554,14 @@ section {
line-height: 1.2;
}
.status-tile > strong {
display: -webkit-box;
overflow: hidden;
overflow-wrap: anywhere;
-webkit-box-orient: vertical;
-webkit-line-clamp: 2;
}
.status-tile p {
margin-top: 0.18rem;
font-size: 0.72rem;
+3 -1
View File
@@ -337,7 +337,9 @@ docker exec \
The training wrapper is intentionally outside the product UI. It runs
Ultralytics from the existing runtime, copies the best trained artifact to
`TRAIN_MODEL_OUTPUT_PATH` and writes `training_summary.json`. Afterward, treat
`TRAIN_MODEL_OUTPUT_PATH` and writes `training_summary.json`. The summary records
SHA256 provenance for `dataset.yaml`, the available YOLO dataset summary, the
local base model and the copied trained model. Afterward, treat
the resulting `.pt` file like any other local model asset: verify preflight,
run the real-data matrix and compare persisted QA/QC metrics before activating
it as a useful default.
+25
View File
@@ -82,12 +82,21 @@ mkdir -p "${TRAIN_OUTPUT_DIR}" "$(dirname "${TRAIN_MODEL_OUTPUT_PATH}")"
"${PYTHON_BIN}" - <<'PY'
from __future__ import annotations
import hashlib
import json
import os
import shutil
from pathlib import Path
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def seed_ultralytics_font() -> None:
font_candidates = [
Path("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf"),
@@ -141,6 +150,17 @@ if not best_path.exists():
raise SystemExit(f"Expected trained model artifact was not created: {best_path}")
shutil.copy2(best_path, trained_model_output_path)
dataset_summary_path = next(
(
candidate
for candidate in (
dataset_yaml.parent / "yolo_tile_dataset_summary.json",
dataset_yaml.parent / "yolo_dataset_summary.json",
)
if candidate.is_file()
),
None,
)
summary = {
"status": "ok",
"dataset_yaml": str(dataset_yaml),
@@ -149,6 +169,11 @@ summary = {
"train_run_name": run_name,
"trained_model_path": str(trained_model_output_path),
"best_artifact_path": str(best_path),
"dataset_yaml_sha256": sha256_file(dataset_yaml),
"dataset_summary_path": str(dataset_summary_path) if dataset_summary_path else None,
"dataset_summary_sha256": sha256_file(dataset_summary_path) if dataset_summary_path else None,
"base_model_sha256": sha256_file(base_model_path),
"trained_model_sha256": sha256_file(trained_model_output_path),
"epochs": epochs,
"image_size": image_size,
"batch_size": batch_size,