Fix runtime GIS uploads for operator QA
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-09 14:03:38 +02:00
parent a1b33555b9
commit 7bf0757470
9 changed files with 75 additions and 5 deletions
+10
View File
@@ -7,6 +7,16 @@
# Changelog
## Sprint 151 runtime GIS upload and AOI1024 YOLO candidate (2026-07-09)
- Fixed the operator YOLO training wrapper so the all-in-one runtime defaults to `/opt/geointel/venv/bin/python` when present, while still falling back to `python3` for local shells.
- Raised the Nginx upload limit to `250m` in both the compose frontend proxy and Unraid all-in-one proxy after a live 1024px GeoTIFF QA upload hit `413 Request Entity Too Large`.
- Prepared a larger Tower operator sample manifest at `/app/storage/operator-data/operator-samples-1024` using explicit `1024x1024` rasters and doubled AOI half-size.
- Exported and audited `/app/storage/operator-data/yolo-building-aoi1024-visible025`: 144 tiles, 117 positive tiles, 27 negative tiles, 15,079 labels and `min_label_visible_ratio=0.25`; audit remains `needs_attention` because median label area is still below gate.
- Trained inactive local model asset `geointel-building-yolov8s-aoi1024visible025e50-pt` from the AOI1024 dataset. Ultralytics validation ended at approximately precision `0.275`, recall `0.331`, mAP50 `0.188` and mAP50-95 `0.0716`.
- The candidate remains inactive and must pass persisted detection QA/QC plus background/hard-negative promotion gates before default activation.
- No API contract, migration, product feature, provider fetching, fake detection data or active model default changed.
## Sprint 150 YOLO label visible-ratio gate (2026-07-09)
- Added `--min-label-visible-ratio` / `OPERATOR_YOLO_MIN_LABEL_VISIBLE_RATIO` to the operator YOLO tile dataset exporter.
@@ -118,6 +118,14 @@ def test_frontend_uses_same_origin_api_proxy_by_default() -> None:
assert "try_files $uri $uri/ /index.html" in nginx_config
def test_nginx_runtime_allows_real_gis_upload_payloads() -> None:
frontend_nginx = (ROOT / "frontend" / "nginx.conf").read_text(encoding="utf-8")
all_in_one_nginx = (ROOT / "deploy" / "unraid" / "nginx-all-in-one.conf").read_text(encoding="utf-8")
assert "client_max_body_size 250m;" in frontend_nginx
assert "client_max_body_size 250m;" in all_in_one_nginx
def test_compose_does_not_publish_postgis_on_default_host_port() -> None:
compose = (ROOT / "docker-compose.yml").read_text(encoding="utf-8")
@@ -60,6 +60,8 @@ def test_operator_yolo_train_smoke_script_contract() -> None:
assert "TRAIN_MODEL_OUTPUT_PATH" in script
assert "TRAIN_EPOCHS" in script
assert "TRAIN_IMGSZ" in script
assert "/opt/geointel/venv/bin/python" in script
assert "PYTHON_BIN=\"python3\"" in script
assert "dataset.yaml" in script
assert "from ultralytics import YOLO" in script
assert "model.train" in script
+1
View File
@@ -1,6 +1,7 @@
server {
listen 80;
server_name _;
client_max_body_size 250m;
root /usr/share/nginx/html;
index index.html;
+30
View File
@@ -5937,3 +5937,33 @@ Open:
## Next recommended pass
- Use `yolo-building-tile-uniquehardneg160` as the next safer hard-negative training dataset candidate. Benchmark after training before changing defaults.
# Sprint 151 - Runtime GIS upload and AOI1024 YOLO candidate
## What changed
- Fixed `scripts/train_operator_yolo_detector.sh` so the all-in-one image uses `/opt/geointel/venv/bin/python` by default when that AI venv exists. Explicit `PYTHON_BIN` still wins, and local shells still fall back to `python3`.
- Raised the Nginx request body limit to `250m` in both `frontend/nginx.conf` and `deploy/unraid/nginx-all-in-one.conf` after the live 1024px GeoTIFF upload path returned `413 Request Entity Too Large`.
- Kept the change runtime-only: no API contract, persistence model, migration, model-download behavior or default model selection changed.
## What was tested
- Red/green TDD guard for the training wrapper fallback:
- `python -m pytest backend\tests\test_sprint129_operator_yolo_training_dataset.py -q`
- Red/green TDD guard for real GIS upload payload support:
- `python -m pytest backend\tests\test_docker_runtime_config.py::test_nginx_runtime_allows_real_gis_upload_payloads -q`
- `bash -n scripts/train_operator_yolo_detector.sh`
- Tower live model training on `/app/storage/operator-data/yolo-building-aoi1024-visible025`:
- output model: `/app/models/geointel-building-yolov8s-aoi1024visible025e50.pt`
- model asset id: `geointel-building-yolov8s-aoi1024visible025e50-pt`
- final Ultralytics validation: precision approximately `0.275`, recall `0.331`, mAP50 `0.188`, mAP50-95 `0.0716`
## Known limitations
- The AOI1024 tile audit is still `needs_attention`: median normalized box area is below gate and small-box share remains high.
- Several dense 1024 GRB reference exports reached the current 1000-feature source cap. Treat those samples as useful but potentially reference-capped until the provider query path supports paging or smaller dense AOIs are chosen.
- The trained model is intentionally inactive. It needs persisted detection QA/QC matrix evidence and background/hard-negative evidence before default promotion.
## Next recommended pass
- Rebuild/deploy the runtime upload-limit fix, rerun the four-sample AOI1024 persisted QA matrix, then decide whether label/source paging or additional AOI quality work comes before another training run.
+5 -1
View File
@@ -464,4 +464,8 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Prepare the new samples on Tower and build a fresh hard-negative tile dataset.
- [x] Rebuild Tower all-in-one image so the newly copied operator scripts are available inside `/app/scripts` without `docker cp`.
- [x] Fix YOLO preflight CLI so it respects Tower `.env` runtime configuration.
- [ ] Train a new candidate from `yolo-building-tile-uniquehardneg160` and run the positive/background promotion gates before activating it.
- [x] Train a new inactive AOI1024 YOLOv8s candidate with visible-label filtering.
- [x] Fix the all-in-one/compose Nginx upload limit after live 1024px GeoTIFF uploads hit `413 Request Entity Too Large`.
- [ ] Rerun persisted QA/QC matrix for `geointel-building-yolov8s-aoi1024visible025e50-pt` after redeploying the upload-limit fix.
- [ ] Add GRB paging or smaller dense AOI sampling before trusting 1000-feature-capped dense reference exports as full ground truth.
- [ ] Keep every local YOLO candidate inactive until positive-AOI and hard-negative promotion reports recommend default activation.
+1
View File
@@ -1,6 +1,7 @@
server {
listen 80;
server_name _;
client_max_body_size 250m;
root /usr/share/nginx/html;
index index.html;
+9 -2
View File
@@ -318,7 +318,6 @@ docker exec \
-e TRAIN_BATCH=2 \
-e TRAIN_WORKERS=0 \
-e TRAIN_DEVICE=cpu \
-e PYTHON_BIN=python3 \
geointel bash /app/scripts/train_operator_yolo_detector.sh
```
@@ -328,6 +327,9 @@ Ultralytics from the existing runtime, copies the best trained artifact to
the resulting `.pt` file like any other local model asset: verify preflight,
run the real-data matrix and compare persisted QA/QC metrics before activating
it as a useful default.
Inside the all-in-one image the wrapper prefers
`/opt/geointel/venv/bin/python` when that AI runtime exists. Set `PYTHON_BIN`
only when intentionally overriding the interpreter.
When whole-image training does not improve QA/QC, export a tile-level dataset
with overlapping raster windows:
@@ -383,6 +385,12 @@ Current Tower audit status:
- `yolo-building-tile-uniquehardneg160`: clean expanded-background baseline;
576 tiles, 346 positive, 230 negative, 11,757 labels, 0 invalid labels and
0 repeated background negatives in the first Tower audit.
- `yolo-building-aoi1024-visible025`: larger AOI candidate baseline; 144
tiles, 117 positive tiles, 27 negative tiles, 15,079 labels and
`min_label_visible_ratio=0.25`. Its audit remains `needs_attention` because
the median normalized box area is still small. The trained
`geointel-building-yolov8s-aoi1024visible025e50-pt` asset is inactive until
persisted QA/QC and hard-negative promotion gates pass.
After rebuilding the all-in-one image, the operator scripts are available inside
the container at `/app/scripts/...`. Before rebuilding, use the host checkout or
@@ -424,7 +432,6 @@ docker exec \
-e TRAIN_BATCH=8 \
-e TRAIN_WORKERS=0 \
-e TRAIN_DEVICE=cpu \
-e PYTHON_BIN=python3 \
geointel bash /app/scripts/train_operator_yolo_detector.sh
```
+9 -2
View File
@@ -21,7 +21,8 @@ Environment variables:
TRAIN_BATCH Batch size. Default: 2
TRAIN_WORKERS Data-loader workers. Default: 0
TRAIN_DEVICE Device passed to Ultralytics. Default: cpu
PYTHON_BIN Python executable. Default: python3
PYTHON_BIN Python executable. Default: /opt/geointel/venv/bin/python
when present, otherwise python3.
This helper is an operator/runtime smoke wrapper. It requires an existing local
base model and an existing local dataset.yaml. It does not create app features.
@@ -43,7 +44,13 @@ TRAIN_IMGSZ="${TRAIN_IMGSZ:-512}"
TRAIN_BATCH="${TRAIN_BATCH:-2}"
TRAIN_WORKERS="${TRAIN_WORKERS:-0}"
TRAIN_DEVICE="${TRAIN_DEVICE:-cpu}"
PYTHON_BIN="${PYTHON_BIN:-python3}"
if [[ -z "${PYTHON_BIN:-}" ]]; then
if [[ -x "/opt/geointel/venv/bin/python" ]]; then
PYTHON_BIN="/opt/geointel/venv/bin/python"
else
PYTHON_BIN="python3"
fi
fi
DATASET_YAML="${OPERATOR_YOLO_DATASET_DIR%/}/dataset.yaml"
SUMMARY_PATH="${TRAIN_OUTPUT_DIR%/}/${TRAIN_RUN_NAME}/training_summary.json"