Gate filtered YOLO candidate and harden provenance
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-12 23:03:55 +02:00
parent c9b22f8f8b
commit 8daaa071b0
10 changed files with 112 additions and 7 deletions
+9
View File
@@ -7,6 +7,15 @@
# Changelog # Changelog
## Sprint 169 Filtered YOLO candidate gate and operator hardening (2026-07-12)
- Trained and fully gated inactive `geointel-building-yolov8s-aoi1024cleanpx12vis035lowvar512e50-pt` against seven positive AOIs and split pure-empty/sparse-context backgrounds.
- Rejected the candidate for default promotion: best mean positive F1 was approximately `0.154`, below the `0.25` gate; threshold `0.05` also produced one pure-empty background detection.
- Preserved the current active `aoi1024bg512r3e50` model and all runtime defaults.
- Added SHA256 provenance to future YOLO training summaries.
- Improved long project/dataset/AOI readability with matching tooltips and compact two-line readiness values.
- No API contract, migration, provider fetching, fake output, model download or automatic model activation changed.
## Sprint 164 Tower AI deploy env hardening (2026-07-11) ## Sprint 164 Tower AI deploy env hardening (2026-07-11)
- Fixed Tower deploy automation so `scripts/deploy_tower.sh` and `scripts/deploy_tower.ps1` source the remote `.env` before building the all-in-one image. - Fixed Tower deploy automation so `scripts/deploy_tower.sh` and `scripts/deploy_tower.ps1` source the remote `.env` before building the all-in-one image.
@@ -70,5 +70,9 @@ def test_operator_yolo_train_smoke_script_contract() -> None:
assert "DejaVuSans.ttf" in script assert "DejaVuSans.ttf" in script
assert "Arial.ttf" in script assert "Arial.ttf" in script
assert "training_summary.json" in script assert "training_summary.json" in script
assert '"dataset_yaml_sha256"' in script
assert '"dataset_summary_sha256"' in script
assert '"base_model_sha256"' in script
assert '"trained_model_sha256"' in script
assert "download" not in script.lower() assert "download" not in script.lower()
assert "fixture_mode" not in script assert "fixture_mode" not in script
@@ -0,0 +1,22 @@
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
def test_long_context_names_remain_compact_and_inspectable() -> None:
app = (ROOT / "frontend" / "src" / "App.tsx").read_text(encoding="utf-8")
status = (
ROOT / "frontend" / "src" / "components" / "WorkbenchStatusStrip.tsx"
).read_text(encoding="utf-8")
css = (ROOT / "frontend" / "src" / "styles" / "app.css").read_text(
encoding="utf-8"
)
assert "const projectContextLabel = selectedProject?.name ?? 'No project'" in app
assert "const datasetContextLabel = selectedDataset?.name ?? (datasets.length > 0 ? 'Select dataset' : 'No dataset')" in app
assert "<strong title={projectContextLabel}>{projectContextLabel}</strong>" in app
assert "<strong title={datasetContextLabel}>{datasetContextLabel}</strong>" in app
assert "<strong title={item.value}>{item.value}</strong>" in status
assert ".status-tile > strong" in css
assert "-webkit-line-clamp: 2;" in css
+29
View File
@@ -1,3 +1,32 @@
## Sprint 169 Filtered YOLO candidate gate and operator hardening (2026-07-12)
Changed:
- Trained inactive local model asset `geointel-building-yolov8s-aoi1024cleanpx12vis035lowvar512e50-pt` from the visually audited AOI1024 dataset after low-variance negative filtering.
- Added SHA256 provenance fields to future `training_summary.json` output for `dataset.yaml`, the dataset summary, the local base model and the copied trained model.
- Kept long workbench context names compact with matching native tooltips and clamped readiness values to two lines.
- Archived the training, positive-AOI, split-background and promotion evidence under `artifacts/model-review/aoi1024cleanpx12vis035lowvar512e50` locally and matching Tower artifact directories.
Runtime evidence:
- Training completed for 50 CPU epochs with YOLOv8s, image size `512`, batch `4`; best-model validation ended at precision `0.454`, recall `0.491`, mAP50 `0.368` and mAP50-95 `0.149`.
- Model catalog SHA256: `75345767b51cc66692a9d2c2cd6577b7feecaf5971c8762365e8b610b9dfde8e`; catalog status `available`, `active=false`, `will_download_models=false`.
- Local model-load preflight passed with Torch `2.13.0` and Ultralytics `8.4.92`; CUDA is unavailable and no model download occurred.
- Seven-AOI persisted QA matrix produced 28 runs. Best single result was Westerlo at threshold `0.15`, F1 `0.3002114164904862`.
- Mean positive F1 was `0.153943` at threshold `0.05`, `0.153872` at `0.15`, `0.128891` at `0.25` and `0.099011` at `0.35`.
- Strict pure-empty background evidence was clean at thresholds `0.15`, `0.25` and `0.35`; threshold `0.05` produced one Postel-bos detection.
- Split-aware promotion report recommended `none`: every threshold failed the positive mean-F1 gate, and `0.05` also failed background false-positive pressure.
- The existing active `geointel-building-yolov8s-aoi1024bg512r3e50-pt` remains materially stronger at its promoted `0.35` profile with mean F1 `0.320866`; no model default or `.env` value was changed.
Tested:
- Red step: `python -m pytest backend/tests/test_sprint169_long_context_name_readability.py -q` failed before context tooltips and readiness clamping were present.
- `python -m pytest backend/tests/test_sprint169_long_context_name_readability.py backend/tests/test_sprint22_workbench_status_strip.py backend/tests/test_sprint161_widescreen_workbench.py -q` (`5 passed`).
- Red step: `python -m pytest backend/tests/test_sprint129_operator_yolo_training_dataset.py::test_operator_yolo_train_smoke_script_contract -q` failed before training hashes were recorded.
- `python -m pytest backend/tests/test_sprint129_operator_yolo_training_dataset.py -q` (`3 passed`).
- `bash -n scripts/train_operator_yolo_detector.sh` and frontend typecheck passed.
- Full readiness: `bash scripts/run_readiness_check.sh` (`461 passed`; one Alembic head; frontend typecheck/build and shell syntax gates passed).
Next:
- Do not retrain the same architecture blindly. Inspect per-AOI false-negative evidence and improve label geometry/class balance or add targeted positive samples for the weakest AOIs before the next candidate.
## Sprint 164 Tower AI deploy env hardening (2026-07-11) ## Sprint 164 Tower AI deploy env hardening (2026-07-11)
Changed: Changed:
+3 -1
View File
@@ -133,7 +133,9 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Add deterministic visual YOLO label QA contact sheets before spending more CPU on another training run. - [x] Add deterministic visual YOLO label QA contact sheets before spending more CPU on another training run.
- [x] Filter no-data/low-variance pure-empty negative tiles from operator YOLO exports before the next training run. - [x] Filter no-data/low-variance pure-empty negative tiles from operator YOLO exports before the next training run.
- [x] Regenerate the AOI1024 cleanpx YOLO dataset with low-variance negative filtering and rerun visual contact-sheet QA before training. - [x] Regenerate the AOI1024 cleanpx YOLO dataset with low-variance negative filtering and rerun visual contact-sheet QA before training.
- [ ] Train one inactive candidate from the filtered AOI1024 cleanpx YOLO dataset and gate it through the positive-AOI plus split-background promotion workflow. - [x] Train one inactive candidate from the filtered AOI1024 cleanpx YOLO dataset and gate it through the positive-AOI plus split-background promotion workflow; reject it because mean positive F1 remains below gate.
- [x] Add deterministic dataset/base/trained-model SHA256 provenance to future operator training summaries.
- [ ] Review per-AOI false-negative evidence for the weakest AOIs and improve positive sample/label geometry coverage before another training candidate.
- [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke. - [ ] Apply promoted V1 default building detector only after explicit operator review of the emitted `.env` updates, followed by rebuild/restart and browser/runtime smoke.
## Sprint 8 status ## Sprint 8 status
+8 -4
View File
@@ -631,6 +631,10 @@ function App(): JSX.Element {
target: 'exports', target: 'exports',
}, },
] ]
const projectContextLabel = selectedProject?.name ?? 'No project'
const areaContextLabel = selectedArea?.name ?? (areas.length > 0 ? 'Select area' : 'No AOI')
const datasetContextLabel = selectedDataset?.name ?? (datasets.length > 0 ? 'Select dataset' : 'No dataset')
const layerContextLabel = mapFeatureCollection ? `${mapFeatureCount} features` : 'No active layer'
return ( return (
<div className="app-shell workbench-shell"> <div className="app-shell workbench-shell">
@@ -645,19 +649,19 @@ function App(): JSX.Element {
<div className="context-bar" aria-label="Active workbench context"> <div className="context-bar" aria-label="Active workbench context">
<div> <div>
<span>Project</span> <span>Project</span>
<strong>{selectedProject?.name ?? 'No project'}</strong> <strong title={projectContextLabel}>{projectContextLabel}</strong>
</div> </div>
<div> <div>
<span>AOI</span> <span>AOI</span>
<strong>{selectedArea?.name ?? (areas.length > 0 ? 'Select area' : 'No AOI')}</strong> <strong title={areaContextLabel}>{areaContextLabel}</strong>
</div> </div>
<div> <div>
<span>Dataset</span> <span>Dataset</span>
<strong>{selectedDataset?.name ?? (datasets.length > 0 ? 'Select dataset' : 'No dataset')}</strong> <strong title={datasetContextLabel}>{datasetContextLabel}</strong>
</div> </div>
<div> <div>
<span>Layer</span> <span>Layer</span>
<strong>{mapFeatureCollection ? `${mapFeatureCount} features` : 'No active layer'}</strong> <strong title={layerContextLabel}>{layerContextLabel}</strong>
</div> </div>
</div> </div>
</header> </header>
@@ -122,7 +122,7 @@ export function WorkbenchStatusStrip({
<span>{item.label}</span> <span>{item.label}</span>
<span className="status-pill">{item.state}</span> <span className="status-pill">{item.state}</span>
</div> </div>
<strong>{item.value}</strong> <strong title={item.value}>{item.value}</strong>
<p>{item.detail}</p> <p>{item.detail}</p>
</div> </div>
))} ))}
+8
View File
@@ -4554,6 +4554,14 @@ section {
line-height: 1.2; line-height: 1.2;
} }
.status-tile > strong {
display: -webkit-box;
overflow: hidden;
overflow-wrap: anywhere;
-webkit-box-orient: vertical;
-webkit-line-clamp: 2;
}
.status-tile p { .status-tile p {
margin-top: 0.18rem; margin-top: 0.18rem;
font-size: 0.72rem; font-size: 0.72rem;
+3 -1
View File
@@ -337,7 +337,9 @@ docker exec \
The training wrapper is intentionally outside the product UI. It runs The training wrapper is intentionally outside the product UI. It runs
Ultralytics from the existing runtime, copies the best trained artifact to Ultralytics from the existing runtime, copies the best trained artifact to
`TRAIN_MODEL_OUTPUT_PATH` and writes `training_summary.json`. Afterward, treat `TRAIN_MODEL_OUTPUT_PATH` and writes `training_summary.json`. The summary records
SHA256 provenance for `dataset.yaml`, the available YOLO dataset summary, the
local base model and the copied trained model. Afterward, treat
the resulting `.pt` file like any other local model asset: verify preflight, the resulting `.pt` file like any other local model asset: verify preflight,
run the real-data matrix and compare persisted QA/QC metrics before activating run the real-data matrix and compare persisted QA/QC metrics before activating
it as a useful default. it as a useful default.
+25
View File
@@ -82,12 +82,21 @@ mkdir -p "${TRAIN_OUTPUT_DIR}" "$(dirname "${TRAIN_MODEL_OUTPUT_PATH}")"
"${PYTHON_BIN}" - <<'PY' "${PYTHON_BIN}" - <<'PY'
from __future__ import annotations from __future__ import annotations
import hashlib
import json import json
import os import os
import shutil import shutil
from pathlib import Path from pathlib import Path
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def seed_ultralytics_font() -> None: def seed_ultralytics_font() -> None:
font_candidates = [ font_candidates = [
Path("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf"), Path("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf"),
@@ -141,6 +150,17 @@ if not best_path.exists():
raise SystemExit(f"Expected trained model artifact was not created: {best_path}") raise SystemExit(f"Expected trained model artifact was not created: {best_path}")
shutil.copy2(best_path, trained_model_output_path) shutil.copy2(best_path, trained_model_output_path)
dataset_summary_path = next(
(
candidate
for candidate in (
dataset_yaml.parent / "yolo_tile_dataset_summary.json",
dataset_yaml.parent / "yolo_dataset_summary.json",
)
if candidate.is_file()
),
None,
)
summary = { summary = {
"status": "ok", "status": "ok",
"dataset_yaml": str(dataset_yaml), "dataset_yaml": str(dataset_yaml),
@@ -149,6 +169,11 @@ summary = {
"train_run_name": run_name, "train_run_name": run_name,
"trained_model_path": str(trained_model_output_path), "trained_model_path": str(trained_model_output_path),
"best_artifact_path": str(best_path), "best_artifact_path": str(best_path),
"dataset_yaml_sha256": sha256_file(dataset_yaml),
"dataset_summary_path": str(dataset_summary_path) if dataset_summary_path else None,
"dataset_summary_sha256": sha256_file(dataset_summary_path) if dataset_summary_path else None,
"base_model_sha256": sha256_file(base_model_path),
"trained_model_sha256": sha256_file(trained_model_output_path),
"epochs": epochs, "epochs": epochs,
"image_size": image_size, "image_size": image_size,
"batch_size": batch_size, "batch_size": batch_size,