geointel/scripts/README.md

# Scripts

Setup-, import-, demo- en maintenance-scripts voor GeoIntel.

## Runtime verification

Audit the active backend route surface against `docs/API_CONTRACTS.md`:

```bash
python scripts/audit_api_contracts.py
```

The audit imports the FastAPI app, compares implemented `GET`/`POST`/`PATCH`/
`DELETE` routes with active API contract headings and tracks the explicit
non-envelope exceptions (`/health` and export downloads). It fails when a route
exists without docs or when docs claim an endpoint that is not implemented.

Verify the browser-facing Docker/LAN runtime:

```bash
bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202
bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202
```

Verify the explicit demo workflow plus export artifact path:

```bash
bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202
```

The demo/export smoke is intentionally mutating and idempotent: it seeds the
offline fixture demo if needed, verifies the project area GeoJSON, fixture
datasets, vector FeatureCollection content, vector feature summary, persisted
QA/QC metrics, creates metadata/report/vector GeoJSON exports, lists exports
and downloads the JSON/GeoJSON/HTML artifacts through the frontend proxy. The
persisted QA/QC result is compared against `fixtures/golden/expected_qa_metrics.json`
so runtime demo precision, recall, F1, mean IoU and false-positive/negative
counts cannot drift silently.

Verify the explicit demo raster workflow:

```bash
bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202
```

The raster smoke is intentionally mutating and idempotent enough for local
runtime checks: it seeds the offline demo workflow, validates the
`demo_context_raster.tif` fixture dataset, then exercises raster inspect,
preview, stats and one small tile/manifest generation through canonical
`data` envelopes. It does not run AI inference or fetch external imagery.

Verify that the browser-facing workbench can populate the default demo start
state through the frontend proxy:

```bash
bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202
```

This smoke is dependency-light and intentionally idempotent: it seeds the
offline demo workflow, then verifies that `GeoIntel Demo - Building QA` exposes
the `Demo AOI - Geel buildings` map geometry, `3/3 ready` demo datasets
(candidate vector, reference vector and raster fixture) and a persisted QA/QC
result through canonical `data.items` envelopes. Pair it with a Codex/browser
screenshot pass when checking visual layout or overflow.

Verify the backing state for the core workbench interactions:

```bash
bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202
```

This smoke validates the state behind project switching, AOI/map selection,
dataset selection, QA refresh and export refresh through the same frontend
proxy used by the browser. The frontend also exposes stable `data-testid`
anchors for Codex/browser click checks on those controls.

Verify the browser click handoff from raster tiling into Detection and
Segmentation Lab:

```bash
bash scripts/verify_ai_handoff_interactions.sh http://192.168.10.150:1202
```

The AI handoff smoke seeds the explicit offline demo workflow, generates a
small raster tile manifest, opens the workbench in Chromium, clicks the raster
inspector `Use in Detection Lab` and `Use in Segmentation Lab` actions, and
verifies that the selected raster dataset plus manifest path are populated in
the AI workspace. Playwright/Chromium must be available in the runner
environment; GeoIntel does not add Playwright as a frontend dependency by
default. The main readiness gate checks this script's syntax only.

Capture visual regression handoff screenshots for the workbench:

```bash
bash scripts/capture_workbench_screenshots.sh http://192.168.10.150:1202
CAPTURE_MOBILE=0 bash scripts/capture_workbench_screenshots.sh http://192.168.10.150:1202 /tmp/geointel-screens
```

The capture script seeds the explicit offline demo workflow, opens each main
workspace, captures viewport desktop screenshots and, by default, viewport
mobile screenshots.
It writes PNG files plus `manifest.json` under `artifacts/screenshots/...` or a
caller-provided output directory. Playwright/Chromium must be available in the
runner environment; GeoIntel does not add Playwright as a frontend dependency
by default. The main readiness gate checks script syntax only.

Verify the deterministic QA/QC golden benchmark:

```bash
bash scripts/verify_golden_qa_benchmark.sh
python scripts/run_golden_qa_benchmark.py --json
```

The benchmark uses only explicit local fixtures under `fixtures/golden`,
executes the existing QA/QC matching logic, verifies the expected precision,
recall, F1, mean IoU and false-positive/false-negative counts, and checks that
`QualityCheck` plus `Metric` rows would be persisted. Scenarios are listed in
`fixtures/golden/golden_qa_benchmarks.json` and currently cover partial match,
perfect match, no-overlap and MultiPolygon building comparisons. The main
readiness gate runs this benchmark so QA metric drift fails before a release.

Verify a configured local YOLO model without running inference:

```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json --check-model-load --json
```

Against the Docker runtime:

```bash
docker compose exec -T backend python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json --check-model-load --json
```

The model-load smoke is opt-in, requires real optional AI dependencies, refuses
`--assume-dependencies`, loads only the supplied local file and does not download
weights or run prediction.

Verify the full configured-YOLO model asset workflow against a running runtime:

```bash
bash scripts/verify_model_asset_detection_workflow.sh http://192.168.10.150:1202
```

This smoke is intentionally mutating and requires a real AI-enabled runtime with
at least one mounted local model asset. It seeds the explicit offline demo
workflow, generates a small raster tile manifest, selects the active local model
asset from `GET /api/v1/detection/model-assets`, validates read-only YOLO
preflight, runs `POST /api/v1/detection/run`, and verifies the persisted
AnalysisRun, Detection list and Detection GeoJSON endpoints. A zero detection
count is allowed because the demo raster is a synthetic runtime fixture; the
script validates the operational path and provenance, not production model
quality. The main readiness gate checks this script's syntax only.

Verify the full operator-provided raster/reference detection and QA path:

```bash
REAL_RASTER_PATH=/mnt/user/appdata/geointel/data/orthophoto.tif \
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/data/reference-buildings.geojson \
bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202
```

The current Tower operator sample is available at:

```bash
REAL_RASTER_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_orthophoto_wms_512.tif \
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_grb_gbg_buildings.geojson \
bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202
```

Those files are runtime artifacts generated from Digitaal Vlaanderen's
OMWRGBMRVL WMS `Ortho` layer and GRB OGC API Features `GBG` building collection
for a small Geel AOI. They are intentionally not repository fixtures.

To prepare the documented operator samples reproducibly inside the all-in-one
runtime container, run:

```bash
docker exec -it geointel python3 /app/scripts/prepare_operator_real_data_samples.py
```

This writes GeoTIFF/GeoJSON pairs and `operator_samples_manifest.json` under
`/app/storage/operator-data` inside the container, which maps to
`storage/operator-data` in the Tower appdata checkout. The default corpus
contains reference AOIs for Geel, Mol, Turnhout, Herentals, Balen, Retie and
Westerlo plus background candidates for Postel-bos, Lommel-heide,
Kasterlee-bos, Dessel-heide, Ravels-bos, Meerhout-bos, Geel-Bel,
Arendonk-heide and Herenthout-bos. Normal reference AOIs still fail when GRB
returns no buildings; background candidates are explicitly marked with
`sample_role` and may write an empty reference FeatureCollection for
negative-tile training. The helper fetches only the explicit documented AOIs,
records Digitaal Vlaanderen attribution and reuses existing files by default.
Use `--force` only when the local runtime artifacts should be regenerated.

The real-data smoke is intentionally mutating and refuses to run without
operator-supplied files. Current V1 upload support expects a georeferenced
`.tif`, `.tiff` or `.geotiff` raster and a `.geojson` or `.json` reference
building vector. The script creates a project, uploads the raster as a source
dataset, uploads the vector as a `reference` dataset, validates raster/vector
metadata, tiles the raster, selects a mounted local model asset, verifies
read-only YOLO preflight, runs configured YOLO detection, runs detection QA
against persisted `vector_features`, and exports the detection run as GeoJSON.
It does not seed demo data, enable fixture detections, fetch external data or
download model weights. Configured-YOLO model class labels are normalized to
lowercase for filtering and persisted detections, while the original model label
is retained in detection provenance. Raster tile manifests generated by the
workflow include source CRS metadata so persisted detection GeoJSON coordinates
can be transformed to WGS84. A zero detection count is accepted
operationally only when the selected model genuinely returns no usable
detections after class filtering; it must be interpreted as model/data quality
evidence rather than as a successful building extraction result.

Run a confidence-threshold calibration sweep against the same real-data path:

```bash
REAL_RASTER_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_orthophoto_wms_512.tif \
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_grb_gbg_buildings.geojson \
CALIBRATION_THRESHOLDS="0.50 0.35 0.25 0.15" \
bash scripts/run_detection_calibration_sweep.sh http://192.168.10.150:1202
```

The sweep reuses `verify_real_data_detection_qa_workflow.sh` once per
threshold, so every row is backed by persisted Project, Dataset, AnalysisRun,
Detection, QualityCheck, Metric and export records. It writes per-threshold
logs plus `calibration_summary.json` under
`artifacts/detection-calibration/<timestamp>` unless
`CALIBRATION_OUTPUT_DIR` is set. This is a calibration/benchmarking tool only:
it does not seed demo data, enable fixture detections, fetch external data or
download model weights.
Per-threshold summaries include persisted detection count, raw candidate count
before GeoIntel duplicate suppression, suppressed duplicate count and the
configured duplicate IoU threshold.

Run a broader model/tile/threshold quality matrix when multiple local model
assets or tile settings need to be compared:

```bash
REAL_RASTER_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_orthophoto_wms_512.tif \
REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_grb_gbg_buildings.geojson \
QUALITY_MODEL_ASSET_IDS="yolov8n-building-segmentation-pt yolov8n-pt" \
QUALITY_TILE_SIZES="512 640" \
QUALITY_TILE_OVERLAPS="64" \
QUALITY_THRESHOLDS="0.50 0.15" \
bash scripts/run_detection_quality_matrix.sh http://192.168.10.150:1202
```

The quality matrix repeats the same real-data upload, tiling, configured-YOLO,
QA/QC and export workflow for every model/tile/threshold row. It writes per-run
logs plus `quality_matrix_summary.json` under
`artifacts/detection-quality-matrix/<timestamp>` unless `QUALITY_OUTPUT_DIR` is
set. The summary ranks `best_by_score`, `best_by_recall` and
`best_by_precision` so the next model decision is based on persisted
`QualityCheck`/`Metric` evidence rather than visual guesses. It does not create
provider data, use fixtures or download model weights.

Run the same matrix across every prepared operator sample:

```bash
OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json \
QUALITY_MODEL_ASSET_IDS="yolov8n-building-segmentation-pt yolov8n-pt" \
QUALITY_TILE_SIZES="512 640" \
QUALITY_TILE_OVERLAPS="64" \
QUALITY_THRESHOLDS="0.50 0.15" \
bash scripts/run_multi_sample_detection_quality_matrix.sh http://192.168.10.150:1202
```

The multi-sample wrapper writes one per-sample `quality_matrix_summary.json`
plus a combined `multi_sample_quality_summary.json` with
`best_overall_by_score`, `best_overall_by_recall`,
`best_overall_by_precision` and `best_by_sample` rankings. It resolves
container-style `/app/storage/...` manifest paths to repo-relative
`storage/...` paths when run from the Tower host checkout.

Export the same operator samples to a local YOLO detection dataset when the
public model candidates are not strong enough for the target imagery:

```bash
docker exec -it geointel python3 /app/scripts/export_operator_yolo_dataset.py \
  --manifest-path /app/storage/operator-data/operator_samples_manifest.json \
  --output-dir /app/storage/operator-data/yolo-building-dataset \
  --val-samples turnhout \
  --force
```

The exporter writes `dataset.yaml`, `images/train`, `labels/train`,
`images/val`, `labels/val` and `yolo_dataset_summary.json`. It uses only the
explicit operator sample manifest and GRB building references where
`source_name=grb` and `reference_layer_name=buildings`. It does not call
GeoIntel APIs, create provider data, run inference or train a model.

Run a small local training smoke only in an AI-enabled runtime with an existing
local base model file:

```bash
docker exec \
  -e OPERATOR_YOLO_DATASET_DIR=/app/storage/operator-data/yolo-building-dataset \
  -e YOLO_BASE_MODEL_PATH=/app/models/yolov8n.pt \
  -e TRAIN_MODEL_OUTPUT_PATH=/app/models/geointel-building-detector.pt \
  -e TRAIN_EPOCHS=8 \
  -e TRAIN_IMGSZ=512 \
  -e TRAIN_BATCH=2 \
  -e TRAIN_WORKERS=0 \
  -e TRAIN_DEVICE=cpu \
  -e PYTHON_BIN=python3 \
  geointel bash /app/scripts/train_operator_yolo_detector.sh
```

The training wrapper is intentionally outside the product UI. It runs
Ultralytics from the existing runtime, copies the best trained artifact to
`TRAIN_MODEL_OUTPUT_PATH` and writes `training_summary.json`. Afterward, treat
the resulting `.pt` file like any other local model asset: verify preflight,
run the real-data matrix and compare persisted QA/QC metrics before activating
it as a useful default.

When whole-image training does not improve QA/QC, export a tile-level dataset
with overlapping raster windows:

```bash
docker exec -it geointel python3 /app/scripts/export_operator_yolo_tile_dataset.py \
  --manifest-path /app/storage/operator-data/operator_samples_manifest.json \
  --output-dir /app/storage/operator-data/yolo-building-tile-expanded160 \
  --tile-size 160 \
  --stride 80 \
  --negative-keep-ratio 1.0 \
  --min-label-visible-ratio 0.25 \
  --val-samples turnhout,retie,kasterlee_bos \
  --force
```

The tile exporter clips GRB building bounding boxes into each tile, writes
YOLO labels beside each tile image, keeps a deterministic ratio of empty
negative tiles, and records `yolo_tile_dataset_summary.json` with
`positive_tile_count`, `negative_tile_count` and skipped negative tile counts.
`--min-label-visible-ratio` drops labels where only a small clipped fragment of
the original building bbox is visible inside the tile; this reduces noisy
tile-edge labels in overlapping-tile datasets. Use `0` for legacy behavior.
It remains operator tooling only: no provider fetch, no API mutation and no
automatic model training.

Audit the generated tile dataset before spending another long training run:

```bash
python scripts/audit_operator_yolo_dataset_quality.py \
  --summary-path /mnt/user/appdata/geointel/storage/operator-data/yolo-building-tile-hardneg160r8/yolo_tile_dataset_summary.json \
  --output-dir /mnt/user/appdata/geointel/artifacts/operator-yolo-dataset-audit/hardneg160r8
```

The audit reads the tile summary and YOLO label files, then writes
`operator_yolo_dataset_quality_audit.json` and
`operator_yolo_dataset_quality_audit.md`. It reports positive/background sample
coverage, train/validation split coverage, repeated hard-negative pressure,
minimum visible label ratio, missing or invalid label rows and normalized
box-area signals. Treat
`needs_attention` as a dataset-design warning, not as a runtime failure: the
next action is usually more positive AOIs, better validation coverage or more
unique hard negatives rather than simply extending epochs.

Current Tower audit status:

- `yolo-building-tile-expanded160`: clean baseline; no missing/invalid labels.
- `yolo-building-tile-hardneg160r4` and `yolo-building-tile-hardneg160r8`:
  repeat-heavy hard-negative variants; useful evidence, but add more unique
  background AOIs before training another hard-negative-balanced candidate.
- Regenerate `operator_samples_manifest.json` after pulling Sprint 147+ so the
  expanded unique background AOI set is available for the next tile export.
- `yolo-building-tile-uniquehardneg160`: clean expanded-background baseline;
  576 tiles, 346 positive, 230 negative, 11,757 labels, 0 invalid labels and
  0 repeated background negatives in the first Tower audit.

After rebuilding the all-in-one image, the operator scripts are available inside
the container at `/app/scripts/...`. Before rebuilding, use the host checkout or
temporarily copy scripts into the running container for one-off data prep.

For hard-negative-balanced experiments, repeat only train-split negative tiles
from samples marked `sample_role=background_candidate`:

```bash
docker exec -it geointel python3 /app/scripts/export_operator_yolo_tile_dataset.py \
  --manifest-path /app/storage/operator-data/operator_samples_manifest.json \
  --output-dir /app/storage/operator-data/yolo-building-tile-hardneg160r8 \
  --tile-size 160 \
  --stride 80 \
  --negative-keep-ratio 1.0 \
  --background-negative-repeat 8 \
  --val-samples turnhout,retie,kasterlee_bos \
  --force
```

The repeat option can also be set with
`OPERATOR_YOLO_BACKGROUND_NEGATIVE_REPEAT`. It does not duplicate validation
tiles, positive tiles or normal reference-sample negatives. Repeated background
tiles receive deterministic `_hnXX` filenames and tile metadata records
`sample_role`, `repeat_index` and `is_repeated_background_negative`.

Train against the tile dataset by pointing the existing wrapper at the tile
output directory:

```bash
docker exec \
  -e OPERATOR_YOLO_DATASET_DIR=/app/storage/operator-data/yolo-building-tile-expanded160 \
  -e YOLO_BASE_MODEL_PATH=/app/models/yolov8n.pt \
  -e TRAIN_OUTPUT_DIR=/app/storage/training/operator-yolo \
  -e TRAIN_RUN_NAME=geointel-building-yolov8n-expanded160e50 \
  -e TRAIN_MODEL_OUTPUT_PATH=/app/models/geointel-building-yolov8n-expanded160e50.pt \
  -e TRAIN_EPOCHS=50 \
  -e TRAIN_IMGSZ=256 \
  -e TRAIN_BATCH=8 \
  -e TRAIN_WORKERS=0 \
  -e TRAIN_DEVICE=cpu \
  -e PYTHON_BIN=python3 \
  geointel bash /app/scripts/train_operator_yolo_detector.sh
```

Benchmark any trained candidate through the same persisted QA/QC matrix before
using it operationally:

```bash
OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json \
OPERATOR_SAMPLE_SLUGS="geel mol turnhout retie kasterlee_bos" \
QUALITY_MODEL_ASSET_IDS="geointel-building-yolov8n-expanded160e50-pt geointel-building-yolov8n-tile30-pt yolov8s-building-segmentation-pt" \
QUALITY_TILE_SIZES="640" \
QUALITY_TILE_OVERLAPS="64" \
QUALITY_THRESHOLDS="0.25 0.15 0.05" \
MULTI_SAMPLE_OUTPUT_DIR=artifacts/detection-quality-matrix/multi-sample/expanded160e50-live \
bash scripts/run_multi_sample_detection_quality_matrix.sh http://192.168.10.150:1202
```

The expanded 50-epoch candidate improved dense Geel/Mol/Turnhout/Retie scores,
but the sparse Kasterlee-bos run still showed too many false positives. Treat it
as the best current experimental dense-AOI candidate, not as a V1 default.

Run a dedicated hard-negative matrix against documented background candidates
before changing model defaults:

```bash
OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json \
OPERATOR_BACKGROUND_SAMPLE_SLUGS="postel_bos lommel_heide kasterlee_bos dessel_heide ravels_bos meerhout_bos geel_bel arendonk_heide herenthout_bos" \
QUALITY_MODEL_ASSET_IDS="geointel-building-yolov8n-expanded160e50-pt geointel-building-yolov8n-tile30-pt yolov8s-building-segmentation-pt" \
QUALITY_TILE_SIZES="640" \
QUALITY_TILE_OVERLAPS="64" \
QUALITY_THRESHOLDS="0.25 0.15 0.05" \
HARD_NEGATIVE_OUTPUT_DIR=artifacts/detection-hard-negatives/expanded160e50-live \
bash scripts/run_operator_hard_negative_detection_matrix.sh http://192.168.10.150:1202
```

The hard-negative matrix uploads only the background raster, generates tiles,
runs configured-YOLO detection and counts persisted detections as
`false_positive_pressure`. It does not upload a reference vector and does not
run QA/QC, because empty or sparse background AOIs do not have a meaningful
precision/recall target. In the first live run, `geointel-building-yolov8n-expanded160e50-pt`
was clean on Postel-bos and Lommel-heide at thresholds `0.25` and `0.15`, but
produced 38 detections on Kasterlee-bos even at `0.25`. That blocks it from
becoming a V1 default until a hard-negative-balanced candidate improves.

The hard-negative-balanced `geointel-building-yolov8n-hardneg160r8e40-pt`
candidate reduced Kasterlee-bos false-positive pressure to 5/9/25 detections
at thresholds `0.25`/`0.15`/`0.05` and stayed at 0 detections on Postel-bos and
Lommel-heide across all tested thresholds. It also regressed dense-AOI F1
against `geointel-building-yolov8n-expanded160e50-pt`, so it is useful model
quality evidence but not a V1 default.

An external remote-sensing YOLOv8l candidate was also benchmarked as an
operator-provided local model asset:

```bash
mkdir -p models
curl -L --fail \
  -o models/yolo-remote-sensing-photovoltaic-v8l-detect-1000.pt \
  https://huggingface.co/agademer/yolo-remote-sensing-photovoltaic/resolve/main/yolo-remote-sensing-photovoltaic-v8l-solar-farms-and-cities-v20260331-detect-1000_epochs.pt
```

GeoIntel exposed the file as
`yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt` with SHA256
`242ff4ab889569278f0eb9fcd22eb2c4bf2a52e48d05d89cc7cfa7941165d203`, and
YOLO preflight loaded it without downloads. On the live dense matrix it missed
most buildings and scored far below `geointel-building-yolov8n-expanded160e50-pt`
on Geel, Mol, Turnhout and Retie. On Kasterlee-bos it was clean and precise,
but that sparse-AOI behavior is not enough for V1 extraction. Keep it as
benchmark evidence only, not as a default model.

Export calibration QA evidence for visual review:

```bash
CALIBRATION_SUMMARY_PATH=/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_summary.json \
bash scripts/export_detection_calibration_evidence.sh http://192.168.10.150:1202
```

Browser Detection Lab calibration summary exports are supported too:

```bash
bash scripts/export_detection_calibration_evidence.sh http://192.168.10.150:1202 ./detection-calibration-summary.json
```

Run the local browser-summary evidence bundle smoke without touching live
application data:

```bash
bash scripts/smoke_detection_calibration_evidence_bundle.sh
```

The smoke creates a temporary Detection Lab-style calibration summary, mocks
the canonical persisted QA evidence endpoint responses, runs the real evidence
exporter and verifies that `calibration_evidence.geojson`,
`calibration_evidence_summary.json` and `calibration_evidence_review.html` are
written correctly.

Assemble multiple AOI evidence bundles into one model-review portfolio:

```bash
bash scripts/assemble_detection_calibration_evidence_portfolio.sh \
  http://192.168.10.150:1202 \
  ./calibration-evidence-portfolio-manifest.json
```

Example `calibration-evidence-portfolio-manifest.json`:

```json
{
  "portfolio_name": "Kempen building model calibration",
  "model_asset_id": "geointel-building-yolov8s-hardneg160r4e50-pt",
  "model_sha256": "optional-model-checksum",
  "notes": "Operator comparison notes.",
  "samples": [
    {
      "sample_slug": "geel",
      "aoi_label": "Geel center",
      "summary_path": "/path/to/detection-calibration-summary.json",
      "operator_notes": "Dense urban validation sample."
    }
  ]
}
```

The portfolio assembler copies each summary into a deterministic sample folder,
runs the existing evidence exporter per AOI and writes
`calibration_evidence_portfolio.json` plus
`calibration_evidence_portfolio.md`. It is evidence packaging only: it does not
run inference, create QA checks or mutate application data.

The evidence export reads each persisted `quality_check_id`, calls the existing
QA evidence GeoJSON endpoint, writes `calibration_evidence.geojson`,
`calibration_evidence_summary.json` and a standalone
`calibration_evidence_review.html` with an SVG overview of matched detections,
matched references, false positives and false negatives. Set
`CALIBRATION_EVIDENCE_MODE=best` to export only the `best_by_score` run.

Docker images install only the GIS runtime by default. To build a local/Tower
image with PyTorch/Ultralytics available for the configured-YOLO preflight and
runtime path, set:

```bash
GEOINTEL_INSTALL_AI=true
```

For Unraid/all-in-one deployments, place model files under
`GEOINTEL_MODELS_PATH` so they appear in the container under `/app/models`, then
set `YOLO_ENABLED=true`, `YOLO_MODELS_DIR=/app/models` and
`YOLO_MODEL_PATH=/app/models/<model>.pt`.

Configure the Unraid/Tower env file from an existing local model without
downloading weights or running inference:

```bash
python scripts/configure_yolo_model.py \
  --models-dir /mnt/user/appdata/geointel/models \
  --env-file /mnt/user/appdata/geointel/.env
```

If exactly one supported model file (`.pt`, `.onnx` or `.engine`) is present,
apply the env update explicitly:

```bash
python scripts/configure_yolo_model.py \
  --models-dir /mnt/user/appdata/geointel/models \
  --env-file /mnt/user/appdata/geointel/.env \
  --apply
```

The configurator refuses to proceed when no model exists or when multiple model
files are present without `--model-file`. It writes only
`GEOINTEL_INSTALL_AI=true`, `YOLO_ENABLED=true`, `YOLO_MODELS_DIR=/app/models`
and the mounted `YOLO_MODEL_PATH`.

Tower-local model evaluation status:

- `geointel-building-yolov8s-hardneg160r4e50.pt` is available as an evaluated
  local runtime artifact after the hard-negative YOLOv8s training pass.
- The live catalog id is `geointel-building-yolov8s-hardneg160r4e50-pt`.
- The model SHA256 is
  `9bf71ad4742048ac77f07060b677bacd9757b8d310497fcada334d543e320d19`.
- The current safest observed threshold is `0.25`, but the model remains an
  evaluation candidate because one hard-negative forest sample still produced
  false detections at that threshold.
- Do not silently activate this model as a default. Apply it only as an explicit
  operator choice until the model catalog/threshold workflow is hardened.

Build a model promotion decision report from an existing positive-AOI evidence
portfolio and one or more hard-negative/background summaries:

```bash
python scripts/build_detection_model_promotion_report.py \
  --positive-portfolio /mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/positive-aoi-expanded-20260708/output/calibration_evidence_portfolio.json \
  --hard-negative-summary /mnt/user/appdata/geointel/artifacts/detection-hard-negatives/hardneg160r8e40-live/hard_negative_matrix_summary.json \
  --output-dir /mnt/user/appdata/geointel/artifacts/detection-model-promotion/expanded-positive-vs-hard-negative-20260708
```

The report writes `detection_model_promotion_report.json` and
`detection_model_promotion_report.md`. It groups candidates by
`model_asset_id`, tile size, tile overlap and confidence threshold, then applies
explicit gates for positive-AOI sample count, background sample count, mean F1
and maximum background detections per sample. It is evidence/report tooling
only: it does not run inference, mutate application data, download models or
change the active YOLO configuration.

If a legacy positive evidence portfolio records `model_asset_id` at portfolio
level but does not include per-run tile size/overlap, pass explicit tile
defaults instead of letting the report guess:

```bash
python scripts/build_detection_model_promotion_report.py \
  --positive-portfolio /mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/uniquehardneg160e50-positive/calibration_evidence_portfolio.json \
  --hard-negative-summary /mnt/user/appdata/geointel/artifacts/detection-hard-negatives/uniquehardneg160e50/hard_negative_matrix_summary.json \
  --output-dir /mnt/user/appdata/geointel/artifacts/detection-model-promotion/uniquehardneg160e50-positive-vs-hard-negative \
  --min-positive-samples 7 \
  --min-background-samples 9 \
  --min-mean-f1 0.25 \
  --max-background-detections-per-sample 0 \
  --default-positive-tile-size 640 \
  --default-positive-tile-overlap 64
```

Clean old offline demo export artifacts without touching uploaded source data:

```bash
python scripts/cleanup_demo_artifacts.py
python scripts/cleanup_demo_artifacts.py --keep-latest 10 --export-type project_report_html
python scripts/cleanup_demo_artifacts.py --keep-latest 10 --max-delete 100 --apply
```

Against the Docker runtime, run the backend-container entrypoint:

```bash
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py --keep-latest 10 --export-type project_report_html
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py --keep-latest 10 --max-delete 100 --apply
```

The cleanup script is dry-run by default. It only targets the explicit
`GeoIntel Demo - Building QA` project unless `--project-name` is provided, keeps
the newest exports per matching project, deletes only `exports` rows/files when
`--apply` is set, and refuses to remove files outside the configured
`STORAGE_ROOT`. `--max-delete` defaults to 25 and blocks large cleanup runs until
the operator raises it after reviewing dry-run output. Repeat `--export-type` to
limit cleanup to specific artifact kinds such as `project_report_html` or
`project_metadata_json`.

Verify the cleanup path against a running backend without deleting anything:

```bash
bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=compose bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=container CLEANUP_CONTAINER=geointel bash scripts/verify_demo_cleanup_dry_run.sh
```

The smoke runs the cleanup command without `--apply`, expects `dry_run=true`,
expects `deleted_export_count=0`, verifies candidate fields are present and
prints the matched/type-filtered/selected counts. Use `KEEP_LATEST`,
`MAX_DELETE` and `EXPORT_TYPE` environment variables to adjust the dry-run
thresholds without changing the script. The main readiness gate checks this
script's syntax; run it explicitly against Docker/PostGIS when validating a
live deployment.

## Tower deployment

Push the local branch to Gitea, then rebuild the Unraid/Tower Docker runtime:

```bash
bash scripts/deploy_tower.sh
```

From the Codex Windows workspace, use the PowerShell wrapper:

```powershell
.\scripts\deploy_tower.ps1
```

For the first deployment into an existing non-Git appdata folder, bootstrap the
checkout explicitly:

```bash
DEPLOY_BOOTSTRAP=1 bash scripts/deploy_tower.sh
```

```powershell
.\scripts\deploy_tower.ps1 -Bootstrap
```

Useful overrides:

```bash
REMOTE_HOST=root@192.168.10.150
REMOTE_PATH=/mnt/user/appdata/geointel
REMOTE_REPO=gitea-widefrog:NuklearRabbit/geointel.git
FRONTEND_URL=http://192.168.10.150:1202
```