Files
geointel/scripts/README.md
T
Codex c85953f389
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled
Expand golden QA benchmark scenarios
2026-06-18 21:46:19 +02:00

184 lines
7.1 KiB
Markdown

# Scripts
Setup-, import-, demo- en maintenance-scripts voor GeoIntel.
## Runtime verification
Audit the active backend route surface against `docs/API_CONTRACTS.md`:
```bash
python scripts/audit_api_contracts.py
```
The audit imports the FastAPI app, compares implemented `GET`/`POST`/`PATCH`/
`DELETE` routes with active API contract headings and tracks the explicit
non-envelope exceptions (`/health` and export downloads). It fails when a route
exists without docs or when docs claim an endpoint that is not implemented.
Verify the browser-facing Docker/LAN runtime:
```bash
bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202
bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202
```
Verify the explicit demo workflow plus export artifact path:
```bash
bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202
```
The demo/export smoke is intentionally mutating and idempotent: it seeds the
offline fixture demo if needed, verifies the project area GeoJSON, fixture
datasets, vector FeatureCollection content, vector feature summary, persisted
QA/QC metrics, creates metadata/report/vector GeoJSON exports, lists exports
and downloads the JSON/GeoJSON/HTML artifacts through the frontend proxy. The
persisted QA/QC result is compared against `fixtures/golden/expected_qa_metrics.json`
so runtime demo precision, recall, F1, mean IoU and false-positive/negative
counts cannot drift silently.
Verify that the browser-facing workbench can populate the default demo start
state through the frontend proxy:
```bash
bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202
```
This smoke is dependency-light and intentionally idempotent: it seeds the
offline demo workflow, then verifies that `GeoIntel Demo - Building QA` exposes
the `Demo AOI - Geel buildings` map geometry, `2/2 ready` demo datasets and a
persisted QA/QC result through canonical `data.items` envelopes. Pair it with a
Codex/browser screenshot pass when checking visual layout or overflow.
Verify the backing state for the core workbench interactions:
```bash
bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202
```
This smoke validates the state behind project switching, AOI/map selection,
dataset selection, QA refresh and export refresh through the same frontend
proxy used by the browser. The frontend also exposes stable `data-testid`
anchors for Codex/browser click checks on those controls.
Capture visual regression handoff screenshots for the workbench:
```bash
bash scripts/capture_workbench_screenshots.sh http://192.168.10.150:1202
CAPTURE_MOBILE=0 bash scripts/capture_workbench_screenshots.sh http://192.168.10.150:1202 /tmp/geointel-screens
```
The capture script seeds the explicit offline demo workflow, opens each main
workspace, captures viewport desktop screenshots and, by default, viewport
mobile screenshots.
It writes PNG files plus `manifest.json` under `artifacts/screenshots/...` or a
caller-provided output directory. Playwright/Chromium must be available in the
runner environment; GeoIntel does not add Playwright as a frontend dependency
by default. The main readiness gate checks script syntax only.
Verify the deterministic QA/QC golden benchmark:
```bash
bash scripts/verify_golden_qa_benchmark.sh
python scripts/run_golden_qa_benchmark.py --json
```
The benchmark uses only explicit local fixtures under `fixtures/golden`,
executes the existing QA/QC matching logic, verifies the expected precision,
recall, F1, mean IoU and false-positive/false-negative counts, and checks that
`QualityCheck` plus `Metric` rows would be persisted. Scenarios are listed in
`fixtures/golden/golden_qa_benchmarks.json` and currently cover partial match,
perfect match, no-overlap and MultiPolygon building comparisons. The main
readiness gate runs this benchmark so QA metric drift fails before a release.
Verify a configured local YOLO model without running inference:
```bash
python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json --check-model-load --json
```
Against the Docker runtime:
```bash
docker compose exec -T backend python scripts/yolo_preflight.py --model-path /absolute/path/to/model.pt --tile-manifest-path /absolute/path/to/manifest.json --check-model-load --json
```
The model-load smoke is opt-in, requires real optional AI dependencies, refuses
`--assume-dependencies`, loads only the supplied local file and does not download
weights or run prediction.
Clean old offline demo export artifacts without touching uploaded source data:
```bash
python scripts/cleanup_demo_artifacts.py
python scripts/cleanup_demo_artifacts.py --keep-latest 10 --export-type project_report_html
python scripts/cleanup_demo_artifacts.py --keep-latest 10 --max-delete 100 --apply
```
Against the Docker runtime, run the backend-container entrypoint:
```bash
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py --keep-latest 10 --export-type project_report_html
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py --keep-latest 10 --max-delete 100 --apply
```
The cleanup script is dry-run by default. It only targets the explicit
`GeoIntel Demo - Building QA` project unless `--project-name` is provided, keeps
the newest exports per matching project, deletes only `exports` rows/files when
`--apply` is set, and refuses to remove files outside the configured
`STORAGE_ROOT`. `--max-delete` defaults to 25 and blocks large cleanup runs until
the operator raises it after reviewing dry-run output. Repeat `--export-type` to
limit cleanup to specific artifact kinds such as `project_report_html` or
`project_metadata_json`.
Verify the cleanup path against a running backend without deleting anything:
```bash
bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=compose bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=container CLEANUP_CONTAINER=geointel bash scripts/verify_demo_cleanup_dry_run.sh
```
The smoke runs the cleanup command without `--apply`, expects `dry_run=true`,
expects `deleted_export_count=0`, verifies candidate fields are present and
prints the matched/type-filtered/selected counts. Use `KEEP_LATEST`,
`MAX_DELETE` and `EXPORT_TYPE` environment variables to adjust the dry-run
thresholds without changing the script. The main readiness gate checks this
script's syntax; run it explicitly against Docker/PostGIS when validating a
live deployment.
## Tower deployment
Push the local branch to Gitea, then rebuild the Unraid/Tower Docker runtime:
```bash
bash scripts/deploy_tower.sh
```
From the Codex Windows workspace, use the PowerShell wrapper:
```powershell
.\scripts\deploy_tower.ps1
```
For the first deployment into an existing non-Git appdata folder, bootstrap the
checkout explicitly:
```bash
DEPLOY_BOOTSTRAP=1 bash scripts/deploy_tower.sh
```
```powershell
.\scripts\deploy_tower.ps1 -Bootstrap
```
Useful overrides:
```bash
REMOTE_HOST=root@192.168.10.150
REMOTE_PATH=/mnt/user/appdata/geointel
REMOTE_REPO=gitea-widefrog:NuklearRabbit/geointel.git
FRONTEND_URL=http://192.168.10.150:1202
```