Add demo cleanup dry-run smoke
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-06-18 01:34:30 +02:00
parent 153b29fc6d
commit 0742731c37
9 changed files with 187 additions and 1 deletions
+8
View File
@@ -644,3 +644,11 @@ Added:
- Dry-run output now includes `candidate_exports` with export ids so duplicate storage paths remain auditable.
- Updated root and backend cleanup entrypoints, docs and regression coverage.
- No API contracts, migrations, product capabilities, live provider fetching or AI/model dependency changes were introduced.
## Sprint 58 demo cleanup dry-run smoke (2026-06-18)
- Added `scripts/verify_demo_cleanup_dry_run.sh` to verify the demo export cleanup path against a running backend without passing `--apply`.
- The smoke supports compose, all-in-one container and local modes, and asserts `dry_run=true`, `deleted_export_count=0`, empty deleted files and candidate dry-run fields.
- Added the smoke syntax check to the main readiness gate and regression coverage for the non-mutating script contract.
- Updated maintenance documentation in `scripts/README.md`, `docs/STORAGE_ARCHITECTURE.md` and `backend/README.md`.
- No API contracts, migrations, product capabilities, live provider fetching or AI/model dependency changes were introduced.
+3
View File
@@ -191,6 +191,9 @@ bash scripts/live_migration_smoke.sh
apply runs above `--max-delete` until the cap is raised after a dry-run review.
Use repeated `--export-type` values to target only specific artifact kinds.
In Docker, use `docker compose exec -T backend python scripts/cleanup_demo_artifacts.py`.
- Live cleanup validation is available with `bash scripts/verify_demo_cleanup_dry_run.sh`.
It runs the same maintenance path without `--apply` and fails if the summary
reports anything other than a dry-run with zero deleted exports/files.
## Run locally
+19
View File
@@ -39,6 +39,25 @@ def test_readiness_gate_compiles_demo_cleanup_script() -> None:
assert "-m py_compile backend/scripts/cleanup_demo_artifacts.py" in content
def test_readiness_gate_checks_demo_cleanup_dry_run_script_syntax() -> None:
script = Path(__file__).resolve().parents[2] / "scripts" / "run_readiness_check.sh"
content = script.read_text(encoding="utf-8")
assert "bash -n scripts/verify_demo_cleanup_dry_run.sh" in content
def test_demo_cleanup_dry_run_script_is_dry_run_only() -> None:
script = Path(__file__).resolve().parents[2] / "scripts" / "verify_demo_cleanup_dry_run.sh"
content = script.read_text(encoding="utf-8")
assert "--apply" not in content
assert "deleted_export_count=0" in content
assert "dry_run=true" in content
assert "candidate_exports" in content
assert "CLEANUP_MODE" in content
assert "project_report_html" in content
def test_readiness_gate_compiles_yolo_preflight_script() -> None:
script = Path(__file__).resolve().parents[2] / "scripts" / "run_readiness_check.sh"
content = script.read_text(encoding="utf-8")
+25
View File
@@ -2294,3 +2294,28 @@ Limitations:
Next recommended pass:
- Add a small live maintenance smoke that runs cleanup in dry-run mode through the deployed all-in-one container.
## Sprint 58 demo cleanup dry-run smoke (2026-06-18)
Changed:
- Added `scripts/verify_demo_cleanup_dry_run.sh` as a live maintenance smoke for the demo export cleanup path.
- The smoke auto-detects a running compose backend or all-in-one `geointel` container, with explicit `CLEANUP_MODE=local|compose|container` overrides.
- The smoke runs cleanup without `--apply`, then verifies `dry_run=true`, `deleted_export_count=0`, `deleted_files=[]`, expected filter settings and candidate dry-run fields.
- Added the smoke syntax check to `scripts/run_readiness_check.sh`.
- Added regression coverage that the readiness gate checks the smoke and that the smoke contract remains non-mutating.
- Updated `scripts/README.md`, `docs/STORAGE_ARCHITECTURE.md`, `backend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
Tested:
- `bash -n scripts/verify_demo_cleanup_dry_run.sh`
- `python -m py_compile scripts/cleanup_demo_artifacts.py backend/scripts/cleanup_demo_artifacts.py`
- `cd backend && python -m pytest tests/test_readiness_gate.py tests/test_sprint24_cleanup_demo_artifacts.py -q` (`18 passed`)
- `bash scripts/run_readiness_check.sh` (`219 passed`)
Open:
- Commit, deploy Tower, then run the live smoke against Tower's all-in-one container.
Limitations:
- The readiness gate checks this live smoke's syntax only. The actual cleanup dry-run still requires a running backend/PostGIS runtime.
Next recommended pass:
- Add browser screenshot artifact automation for visual regression handoff.
+12
View File
@@ -117,6 +117,18 @@ tiles, rasters or masks. `--max-delete` defaults to 25 and blocks oversized
apply runs until the operator increases the cap after reviewing dry-run output.
Repeat `--export-type` to restrict cleanup to selected artifact kinds.
Live runtime validation for this maintenance path is available as a dry-run
smoke:
```bash
bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=container CLEANUP_CONTAINER=geointel bash scripts/verify_demo_cleanup_dry_run.sh
```
The smoke never passes `--apply`. It fails if the cleanup summary is not a
dry-run, if any export/file deletion is reported, or if the dry-run candidate
fields are missing.
## Model storage
Model artifacts live under:
+2 -1
View File
@@ -327,4 +327,5 @@ This file now starts with the current implementation status. Older preparation/b
- [x] Improve live visual shell width, scroll behavior and Map workspace layout at 1280px.
- [x] Add export history filtering controls for long-running demo environments.
- [x] Add a safe export retention/cleanup command for demo environments.
- [ ] Add a live dry-run maintenance smoke for demo export cleanup.
- [x] Add a live dry-run maintenance smoke for demo export cleanup.
- [ ] Add browser screenshot artifact automation for visual regression handoff.
+16
View File
@@ -115,6 +115,22 @@ the operator raises it after reviewing dry-run output. Repeat `--export-type` to
limit cleanup to specific artifact kinds such as `project_report_html` or
`project_metadata_json`.
Verify the cleanup path against a running backend without deleting anything:
```bash
bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=compose bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=container CLEANUP_CONTAINER=geointel bash scripts/verify_demo_cleanup_dry_run.sh
```
The smoke runs the cleanup command without `--apply`, expects `dry_run=true`,
expects `deleted_export_count=0`, verifies candidate fields are present and
prints the matched/type-filtered/selected counts. Use `KEEP_LATEST`,
`MAX_DELETE` and `EXPORT_TYPE` environment variables to adjust the dry-run
thresholds without changing the script. The main readiness gate checks this
script's syntax; run it explicitly against Docker/PostGIS when validating a
live deployment.
## Tower deployment
Push the local branch to Gitea, then rebuild the Unraid/Tower Docker runtime:
+1
View File
@@ -55,4 +55,5 @@ bash -n scripts/verify_workbench_default_state.sh
bash -n scripts/verify_workbench_interactions.sh
bash -n scripts/verify_gis_runtime.sh
bash -n scripts/verify_golden_qa_benchmark.sh
bash -n scripts/verify_demo_cleanup_dry_run.sh
echo "== Run readiness check passed =="
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT"
KEEP_LATEST="${KEEP_LATEST:-10}"
MAX_DELETE="${MAX_DELETE:-100}"
EXPORT_TYPE="${EXPORT_TYPE:-project_report_html}"
CLEANUP_MODE="${CLEANUP_MODE:-auto}"
CLEANUP_CONTAINER="${CLEANUP_CONTAINER:-geointel}"
cleanup_args=(
"--keep-latest" "${KEEP_LATEST}"
"--max-delete" "${MAX_DELETE}"
"--export-type" "${EXPORT_TYPE}"
)
run_cleanup() {
case "${CLEANUP_MODE}" in
local)
python scripts/cleanup_demo_artifacts.py "${cleanup_args[@]}"
;;
compose)
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py "${cleanup_args[@]}"
;;
container)
docker exec "${CLEANUP_CONTAINER}" /opt/geointel/venv/bin/python /app/scripts/cleanup_demo_artifacts.py "${cleanup_args[@]}"
;;
auto)
if command -v docker >/dev/null 2>&1 && docker compose ps --services --filter status=running 2>/dev/null | grep -qx "backend"; then
docker compose exec -T backend python scripts/cleanup_demo_artifacts.py "${cleanup_args[@]}"
elif command -v docker >/dev/null 2>&1 && docker ps --format '{{.Names}}' | grep -qx "${CLEANUP_CONTAINER}"; then
docker exec "${CLEANUP_CONTAINER}" /opt/geointel/venv/bin/python /app/scripts/cleanup_demo_artifacts.py "${cleanup_args[@]}"
else
cat >&2 <<EOF
No running GeoIntel backend runtime found.
Start Docker first, then run one of:
docker compose up -d backend
CLEANUP_MODE=compose bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=container CLEANUP_CONTAINER=geointel bash scripts/verify_demo_cleanup_dry_run.sh
CLEANUP_MODE=local bash scripts/verify_demo_cleanup_dry_run.sh
EOF
return 1
fi
;;
*)
echo "Unsupported CLEANUP_MODE '${CLEANUP_MODE}'. Use auto, local, compose or container." >&2
return 1
;;
esac
}
tmp_output="$(mktemp)"
trap 'rm -f "${tmp_output}"' EXIT
run_cleanup | tee "${tmp_output}" >/dev/null
python - "${tmp_output}" "${KEEP_LATEST}" "${MAX_DELETE}" "${EXPORT_TYPE}" <<'PY'
from __future__ import annotations
import json
import sys
from pathlib import Path
output_path = Path(sys.argv[1])
expected_keep_latest = int(sys.argv[2])
expected_max_delete = int(sys.argv[3])
expected_export_type = sys.argv[4]
summary = json.loads(output_path.read_text(encoding="utf-8"))
if summary.get("dry_run") is not True:
raise SystemExit("cleanup smoke expected dry_run=true")
if summary.get("deleted_export_count") != 0:
raise SystemExit("cleanup smoke expected deleted_export_count=0")
if summary.get("deleted_files") != []:
raise SystemExit("cleanup smoke expected deleted_files=[]")
if summary.get("keep_latest") != expected_keep_latest:
raise SystemExit("cleanup smoke keep_latest drifted")
if summary.get("max_delete") != expected_max_delete:
raise SystemExit("cleanup smoke max_delete drifted")
if summary.get("export_types") != [expected_export_type]:
raise SystemExit("cleanup smoke export_types drifted")
if "candidate_exports" not in summary:
raise SystemExit("cleanup smoke expected candidate_exports in dry-run output")
if "candidate_files" not in summary:
raise SystemExit("cleanup smoke expected candidate_files in dry-run output")
if "matched_export_count" not in summary:
raise SystemExit("cleanup smoke expected matched_export_count in dry-run output")
if "selected_export_count" not in summary:
raise SystemExit("cleanup smoke expected selected_export_count in dry-run output")
print(
"Demo cleanup dry-run smoke passed: "
f"matched={summary.get('matched_export_count')} "
f"type_filtered={summary.get('type_filtered_export_count')} "
f"selected={summary.get('selected_export_count')} "
f"deleted={summary.get('deleted_export_count')}"
)
PY