11161 lines
770 KiB
Markdown
11161 lines
770 KiB
Markdown
## 2026-07-19 - GeoIntel 1.0.0 final release closeout
|
|
|
|
- Promoted the current clean `main` revision to semantic version `1.0.0` in
|
|
`VERSION`, backend/frontend package metadata, health/API examples and the
|
|
immutable Docker runtime identity. Historical RC entries remain unchanged.
|
|
- Re-ran the final repository release gate after the version promotion:
|
|
backend compile/tests, frontend unit tests/typecheck/build, readiness,
|
|
Alembic single-head/offline SQL and live-smoke syntax all pass.
|
|
- The final Tower image is tied to commit
|
|
`04b8373d7ece0f5eb29674507d718f0ea51ed5c4` and passed health, PostGIS
|
|
migration, proxy and browser runtime checks.
|
|
- Final live browser acceptance selected the Walloon golden area, activated
|
|
Waterbodem, analysed the full area and displayed real SPW mDNG metrics:
|
|
mean `74.55 m`, measured surface `22.69 ha`, coverage `7.07%`, and explicit
|
|
no-depth/no-volume limitations. No unrelated on-demand source acquisition
|
|
was triggered.
|
|
- Remaining boundaries are deliberate V1 limitations: MDK North Sea
|
|
analytical bathymetry remains not configured, water volume requires a
|
|
governed surface/datum/uncertainty contract, and YOLO remains human-reviewed.
|
|
|
|
## Autonomous RC program for Belgium and the Belgian North Sea (2026-07-17)
|
|
|
|
### Post-RC national data federation (2026-07-19)
|
|
|
|
- Continued the national federation pass with official history and
|
|
bathymetry. Statbel's real 2021 geometry release exposed two documented
|
|
packaging differences: the ZIP member omits the repeated `31370` token and
|
|
`dt_situation` uses `YYYY/MM/DD`. The preflight now normalizes only those
|
|
forms and retains every existing fail-closed content check.
|
|
- The corrected Tower fetch-only pass validated national 2021-2024 snapshots,
|
|
each with 19,795 retained sector geometries, before any API persistence.
|
|
- Downloaded the fixed official SPW Geoportail bathymetry release
|
|
(`0a544b42-0b30-4c8e-85e7-38149b99eae0`) to persistent operator evidence.
|
|
SHA-256 is
|
|
`04122a1c5cecc7b77be025b580995d024545e82d8310e158b4c112f7fa5f8a6e`.
|
|
GDAL verified a 456,893 by 179,437 Float32 EPSG:3812 source at 0.5 m,
|
|
nodata `-9999`, with mDNG waterbed elevations.
|
|
- Added `import_spw_bathymetry.py`, bounded COG persistence, source-specific
|
|
coverage resolution, raster selection/image APIs and frontend Waterbodem
|
|
integration. The contract exposes bed elevation, measured-cell hectares and
|
|
coverage only; current depth, volume and datum conversion remain blocked.
|
|
- Added the bathymetry PNG route to the explicit non-envelope API audit set
|
|
after the first full gate correctly rejected its untracked binary response.
|
|
- The complete repository gate passed 1,052 backend tests, 22 frontend tests,
|
|
frontend typecheck/build, one Alembic head and all readiness checks before
|
|
live deployment.
|
|
- Live browser acceptance exposed an existing orchestration problem: one
|
|
selected theme also launched every unrelated not-yet-materialized regional
|
|
product. The selection flow still summarizes every persisted theme, but now
|
|
acquires only the actively selected on-demand theme so one slow provider
|
|
cannot block the requested result.
|
|
- Added explicit administrative and maritime themes for persisted NGI/RBINS
|
|
reference data and made readiness depend on analyzable themes rather than a
|
|
raw Dataset count.
|
|
- Generalized the governed Statbel population scripts to the Belgium boundary
|
|
and complete municipality inventory while retaining all release-promotion
|
|
safety gates.
|
|
- Added allowlisted bounded SPW/PICC ArcGIS REST and UrbIS WFS adapters with
|
|
source-specific coverage, CRS, identity, paging, geometry, licence,
|
|
provenance and metric contracts. Output still enters PostGIS only through
|
|
`DatasetService.import_vector_bytes`.
|
|
- Made the frontend resolve actual coverage zones before choosing GRB,
|
|
SPW/PICC or UrbIS and retain separate source results for mixed-zone
|
|
selections.
|
|
- Added SPW/UrbIS runtime controls to both Compose variants and the editable
|
|
Unraid/Dockerman deployment path.
|
|
- Rechecked MDK bathymetry. Strict hostname validation still fails because the
|
|
configured official hostname presents a `*.l27powered.eu` certificate, so
|
|
acquisition remains truthfully disabled.
|
|
- Live national Statbel staging exposed and fixed a release-review blocker:
|
|
`catalog_checked_at` was incorrectly compared as publication identity.
|
|
Reviews now tolerate a later probe timestamp while still blocking any
|
|
identifier, release URL/version or catalog content-hash change.
|
|
- Live coverage verification then exposed a cross-theme materialization leak:
|
|
the Statbel population Dataset also appeared under `admin`. The coverage
|
|
registry now enforces each source's `operational_themes`, leaving NGI as the
|
|
administrative authority and Statbel as population evidence only.
|
|
- Bounded provider materialization now also checks its persisted
|
|
`bbox_epsg4326`; one small PICC/UrbIS/GRB/raster acquisition can no longer
|
|
claim that an entire region is locally loaded.
|
|
- The live governed Statbel apply materialized the reviewed 2025 `NEW` edition
|
|
as Dataset `ee0a46e5-6139-4b0a-93b5-9dabb3b3dfc7`: 20,781 statistical
|
|
sectors, 565 municipalities, four repaired geometries and a reconciled
|
|
national total of 11,825,551 inhabitants (11,817,897 spatially located plus
|
|
7,654 explicitly unlocated).
|
|
- Live bounded PICC and UrbIS persistence proved the same service path:
|
|
54 Walloon building geometries retained in Dataset
|
|
`0c3c0a11-c00e-4c84-87be-8ffa64302dec` and 47 Brussels building geometries
|
|
in Dataset `de53ccc0-a494-4c4e-bd0d-16d807bfefe7`.
|
|
- Corrected the map catalog precedence so reusable persisted datasets win over
|
|
on-demand placeholders. National Statbel 2025 population is now presented
|
|
as the active source, while selection-bounded PICC/UrbIS acquisitions remain
|
|
available only through an exact bounded request.
|
|
- Live browser verification found that a changed work area was fitted together
|
|
with the complete active national layer. Area changes now fit the selected
|
|
geography first and retain that viewport when a complete national layer is
|
|
loaded on the following render or the map container resizes.
|
|
- The final browser audit then exposed a responsive layout defect rather than
|
|
a CRS defect: at 1,264 pixels wide the theme list stretched the MapLibre
|
|
canvas to 723 by 1,891 pixels, so a correct fit placed Belgium mostly below
|
|
the visible fold. The first explorer grid row is now bounded independently,
|
|
the theme list scrolls internally and the live canvas measures 723 by 496
|
|
pixels at the same viewport.
|
|
- Map updates no longer depend on transient `isStyleLoaded()` results after
|
|
the initial `style.load`; Area, dataset, selection, raster and QA overlay
|
|
changes are retained while base-map tiles load.
|
|
- Live browser acceptance switched Belgium land, the North Sea golden zone,
|
|
Brussels and Wallonia without manual zoom or console warnings. Persisted
|
|
sources remained preferred, while PICC/UrbIS stayed selection-bounded.
|
|
- `scripts/run_rc8_release_journeys.sh` passed against port 1202 and reused all
|
|
seven golden Areas. The retained manifest records PostGIS 3.6, migration
|
|
`202607160001`, clean console/request audits, a 49-feature persisted export,
|
|
exact 2013-2025 forest change, a grounded 656-character Ollama answer with
|
|
36 metrics, and a successful configured-YOLO Job/AnalysisRun map handoff.
|
|
- The deployed AI runtime imports PyTorch `2.13.0+cpu` and Ultralytics
|
|
`8.4.99`; `/api/v1/system/capabilities` and the detection model registry
|
|
report configured local YOLO truthfully.
|
|
- The final local readiness gate passed with 1,042 backend tests and 20
|
|
frontend tests, frontend typecheck and production build, a single Alembic
|
|
head, the complete offline `upgrade head --sql` chain and live-smoke shell
|
|
syntax. The Windows runner has no local Docker CLI; compose, PostGIS and
|
|
migration runtime evidence therefore comes from the successful Tower deploy.
|
|
- P5 is complete. MDK remains the only deliberately blocked source in this
|
|
board because strict TLS/capabilities evidence does not pass; no bathymetry
|
|
or water-volume value was invented.
|
|
|
|
### RC-4 national and maritime coverage foundation
|
|
|
|
- Added an independent coverage registry so the exact Sprint 7
|
|
`grb|osm|manual|fixture` provider contract remains unchanged.
|
|
- Added `GET /api/v1/external/coverage/catalog` and
|
|
`POST /api/v1/external/coverage/resolve` with canonical envelopes, normalized
|
|
themes and explicit project-materialization checks.
|
|
- Added `provision_belgium_north_sea_scope.py`. It accepts only the official
|
|
NGI AdminVector archive, RBINS marine reporting units and the exact 15-layer
|
|
`imsp26` WFS allowlist. It verifies archive/response limits, safe extraction,
|
|
layer counts, pagination, geometries and SHA256 evidence.
|
|
- Verified live source contracts from the Codex host: all 15 MSP layers were
|
|
reachable and returned 101 features in total; the current RBINS reporting
|
|
units yielded BPNS, territorial sea, EEZ and continental-shelf geometries.
|
|
- The territorial sea is derived only by unioning official 0-1 nm and 1-12 nm
|
|
units. EEZ and continental shelf retain the same official offshore geometry
|
|
as separate records with different legal-domain provenance.
|
|
- Frontend startup now prefers a materialized
|
|
`Belgium and North Sea Workbench`; drawn bboxes receive a compact,
|
|
zone-split coverage matrix. Existing Mol/Kempen workspaces remain available.
|
|
- Local validation passed 986 backend tests, frontend typecheck/build, the
|
|
full readiness gate, Alembic head `202607160001`, complete offline migration
|
|
SQL and live-smoke shell syntax.
|
|
- Tower fetch-only, persistence, live coverage API and browser acceptance are
|
|
the remaining RC-4 exit evidence.
|
|
- The first Tower fetch-only run failed closed before API persistence because
|
|
AdminVector exposes `modifdate` as a Pandas `Timestamp`. GeoJSON
|
|
serialization was corrected to explicit string conversion and guarded by a
|
|
regression assertion; downloaded source artifacts remain operator cache
|
|
only.
|
|
- The corrected Tower fetch-only run validated all seven generated artifacts
|
|
and 101 current MSP 2026 features. The first API persistence run then exposed
|
|
a separate live-only PostGIS constraint: AdminVector AOI boundaries carry a
|
|
zero-valued Z coordinate while `areas.geometry` is intentionally 2D.
|
|
Shared AOI normalization now removes Z before validation and persistence.
|
|
- Live API persistence subsequently completed with one national workspace,
|
|
eight scope areas and six ready authoritative datasets. A second operator
|
|
run reused the same project and dataset identifiers. Browser acceptance then
|
|
exposed and closed a map guard that had prevented coverage-only selection
|
|
when the chosen theme was honestly unavailable.
|
|
- RC-5 deployment hardening started by centralizing Tower release execution,
|
|
moving build identity behind expensive dependency layers, retaining
|
|
immutable commit images plus a `previous` rollback target, adding automatic
|
|
rollback and an isolated fresh-install smoke, and failing closed on default
|
|
database secrets or inconsistent upload limits.
|
|
- Tower proved cached apt/PyTorch layers, a fresh install on isolated paths and
|
|
a healthy rollback against the retained database/storage. The release tag
|
|
contract was then tightened to immutable commit-plus-profile tags so repeated
|
|
deploys reuse rather than overwrite the same build identity.
|
|
- A repeated no-op deploy reused the existing profile-tagged image and
|
|
preserved the actual previous image. The Unraid template now exposes every
|
|
operator-owned acquisition, upload, local-model and assistant setting as an
|
|
editable field, with only deployment bridge variables kept internal.
|
|
- Added an explicit release upgrade verifier that composes the checksum-backed
|
|
isolated restore drill with the deployed image's Alembic chain and destroys
|
|
only the generated verification database.
|
|
- Tower release `944269c25b1d7647c2ef468df75e195d045d8c0c` completed that
|
|
isolated upgrade proof against backup
|
|
`rc-belgium-north-sea-fc42ea9-secure`: checksums and retained counts matched,
|
|
PostGIS 3.6 and Alembic `202607160001` passed, the temporary database was
|
|
removed and the evidence records the production database as untouched.
|
|
|
|
- Froze the RC geography as all Belgian land plus the separately labelled
|
|
territorial sea, EEZ and continental shelf.
|
|
- Retained Mol and the Kempen as validated regression areas instead of the
|
|
final product boundary.
|
|
- Added `docs/RC_SCOPE_FREEZE_BELGIUM_NORTH_SEA.md`.
|
|
- Added `docs/RC_ROADMAP_BELGIUM_NORTH_SEA.md` with autonomous stop rules,
|
|
exact phase gates, evidence requirements and RC-0 through RC-11.
|
|
- Removed the need for a separate RC-12 phase. Fresh-install, upgrade,
|
|
rollback and runtime evidence remain mandatory inside RC-5 and RC-11.
|
|
- Verified the official source families for NGI/IGN, Statbel, SPW Wallonia,
|
|
UrbIS Brussels, federal marine planning, RBINS/BMDC and MDK before freezing
|
|
the coverage architecture.
|
|
- Started RC-0 implementation; the release-evidence command is the first code
|
|
gate.
|
|
- Implemented and tested `scripts/capture_release_evidence.py`. The first
|
|
local/live baseline recorded commit `9513f86`, dirty-state evidence, one
|
|
Alembic head `202607160001`, dependency/configuration checksums and reachable
|
|
Tower health/capability routes without storing secret values.
|
|
- Replaced the obsolete pre-implementation `docs/BUILD_STATUS.md` with the
|
|
current RC state and moved runtime release evidence outside Git.
|
|
- RC-0 is complete. RC-1 backup/restore safety is active.
|
|
- Added atomic custom-format PostgreSQL backup, read-only verification and
|
|
generated-database restore-smoke scripts. Their focused safety/syntax tests
|
|
pass; Tower backup and isolated restore execution remain the RC-1 live gate.
|
|
- Implemented RC-2 truthful runtime state: independent liveness, fail-closed
|
|
DB/PostGIS/Alembic/storage readiness, runtime-derived YOLO capability,
|
|
request correlation IDs, exception trace logging, SQL logging at WARNING
|
|
and all-in-one startup reconciliation for orphaned `running` jobs/runs.
|
|
- Docker, Nginx, API contracts and health documentation now use
|
|
`/health/ready`; `/health` remains a compatibility readiness alias.
|
|
- Validation passed: targeted Ruff, backend compile, 963 backend tests,
|
|
frontend typecheck/build, 122-route contract audit, one Alembic head,
|
|
complete offline migration SQL and the full readiness gate.
|
|
- Local Docker CLI is unavailable on the Windows Codex host. Docker config,
|
|
image build, live health truthfulness, backup and restore are therefore
|
|
scheduled on the Docker-enabled Tower after push.
|
|
- Pushed and deployed commit `9c402e0`; subsequently pushed the cwd-independent
|
|
backup fix `38a3bd0` and secret rotation command `fc42ea9`.
|
|
- Tower Docker config, all-in-one build, live migration smoke and browser
|
|
runtime smoke passed. `/health/live` and `/health/ready` report `ok`,
|
|
PostGIS 3.6, Alembic `202607160001`, writable storage and configured YOLO.
|
|
- Startup reconciliation changed five impossible orphaned jobs and two
|
|
analysis runs to terminal `failed` state with `PROCESS_INTERRUPTED`; no
|
|
`running` rows remain.
|
|
- Created the full SHA-256 inventory backup
|
|
`rc-belgium-north-sea-38a3bd0`, verified it read-only and restored it in an
|
|
isolated temporary database. The first restore attempt exposed stale glibc
|
|
collation metadata in `template1`/`postgres`; those empty system databases
|
|
were reindexed and refreshed after the backup, then restore passed.
|
|
- Rotated the production PostgreSQL password with a generated 256-bit value
|
|
without logging it, atomically updated the Tower `.env` and recreated the
|
|
healthy container.
|
|
- Created the current secure backup
|
|
`rc-belgium-north-sea-fc42ea9-secure` (1.4 GiB), verified all checksums and
|
|
repeated isolated restore. PostGIS/Alembic and all retained critical table
|
|
counts match; the temporary database was removed.
|
|
- RC-1 and RC-2 are complete. RC-3 is active.
|
|
|
|
## Sprint 229 - Governed ALZ definitive release promotion (2026-07-17)
|
|
|
|
Implemented:
|
|
- Added `scripts/manage_alz_agriculture_release.py` with separate read-only
|
|
`plan`, filesystem-only `stage`, named `review` and exact-hash `apply`
|
|
actions for the approved Kempen regional scope.
|
|
- Derived the only accepted future archive from the allowlisted definitive
|
|
`YYYY-v3` campaign and publication date exposed by the existing source-
|
|
catalog probe. Provisional v1/v2 snapshots remain visible but non-importable;
|
|
arbitrary URLs, current/older releases and catalog drift fail closed.
|
|
- Extended the existing agricultural provisioner to accept exactly one
|
|
explicitly governed future edition per run. Final response URLs and streamed
|
|
archive size are bounded, retained editions cannot be overridden and all
|
|
workspace API collections are read with stable-total pagination.
|
|
- Bound archive, normalized GeoJSON, GeoPackage schema/CRS, complete crop-code
|
|
list, scope counts/area and previous-definitive-edition manifest/deltas into
|
|
the staged plan. ZIP member/extracted size is bounded. Review and apply
|
|
require exact SHA-256 values; apply still uses DatasetService and preserves
|
|
all historical snapshots.
|
|
- Packaged the manager in the Unraid image/readiness gate and updated source,
|
|
data, API, persistence, storage and operator documentation. No endpoint,
|
|
migration, scheduler, browser fetch or frontend behavior changed.
|
|
|
|
Validation:
|
|
- 43 tests passed across the original ALZ importer, official catalog probe and
|
|
new release manager; 58 passed together with the Statbel release suite.
|
|
- Target Python compilation and Ruff pass. Coverage includes future release
|
|
identity, v1/v2 exclusion, pagination drift, download bounds, release
|
|
ordering, stage/apply separation, source/schema/codelist/baseline evidence,
|
|
path confinement, byte tampering, named review, catalog drift, full mocked
|
|
apply, current-edition refusal and runtime packaging.
|
|
- Complete readiness passed with 850 backend tests, 110 documented routes,
|
|
one Alembic head `202607160001`, frontend typecheck and production build.
|
|
Static full-chain Alembic SQL and shell syntax checks also passed. Local
|
|
Docker is not installed in the Codex Windows environment; Compose and live
|
|
PostGIS validation are therefore deferred to the Tower deployment gate.
|
|
- Tower was rebuilt from commit `ad2c348`. Compose validated, the all-in-one
|
|
container became healthy on port 1202, PostGIS 3.6 and all required schema
|
|
objects answered, Alembic remained at `202607160001` and the frontend/API/
|
|
icon browser proxy smoke passed.
|
|
- The packaged manager's live plan reported local `2025-definitive`, official
|
|
`2025-v3`, provisional non-importable `2026-v1` and decision `current`, with
|
|
publication-page hash
|
|
`16f513d46d16940a2970599ec34013d5ff5727277b021024aa4a320671627664`.
|
|
A deliberate deployed stage attempt exited 1, Dataset count remained 2,429
|
|
and no 2025 `staged-plan.json` was written.
|
|
- Live visual verification loaded the 28-municipality workspace, official map
|
|
boundary and all available themes, including 54,071 agriculture features.
|
|
The OpenStreetMap basemap rendered and the browser reported no warnings or
|
|
errors.
|
|
|
|
Boundary:
|
|
- The official page currently advertises definitive `2025-v3` plus
|
|
provisional `2026-v1`. No newer definitive edition exists, so live stage,
|
|
review and apply must remain blocked; only a read-only plan and deliberate
|
|
current-edition refusal may be exercised after deployment.
|
|
|
|
## Sprint 228 - Governed Statbel population release promotion (2026-07-17)
|
|
|
|
Implemented:
|
|
- Added `scripts/manage_statbel_population_release.py` with four separate
|
|
operator actions for the approved Kempen scope. `plan` is read-only; `stage`
|
|
requires exact year/layout confirmation and performs bounded fresh download
|
|
plus preflight only; `review` requires a named explicit approval; `apply`
|
|
requires the exact staged-plan and review-evidence SHA-256 values.
|
|
- Derived the only accepted population and matching sector-geometry URLs from
|
|
the allowlisted catalog year/layout contract. The coordinator accepts no
|
|
arbitrary source URL or process, and current, older, unavailable, ambiguous
|
|
or catalog-drifted releases are not stageable.
|
|
- Bound catalog identity, source archives, preflight manifest, derived
|
|
snapshot, scope/national accounting, ZZZZ totals, baseline trend and geometry
|
|
repairs into `staged-plan.json`. Bound the named human decision to that plan
|
|
in `review-evidence.json`; successful apply records the immutable Dataset in
|
|
`applied-evidence.json` without deleting prior snapshots.
|
|
- Extended `provision_mol_population_history.py` with a complete all-or-none
|
|
release config for one future year. Existing 2021-2025 arguments remain
|
|
compatible. Source downloads now enforce response and streaming byte bounds,
|
|
validate final official URLs and discover the latest retained baseline from
|
|
actual snapshot files rather than a hardcoded year list.
|
|
- Replaced the population operator's 200-row workspace lookup with complete,
|
|
total-consistent pagination. This keeps repeated apply idempotent in the live
|
|
project with 2,429 Datasets.
|
|
- Added Docker/readiness packaging and updated source, API, persistence,
|
|
storage and operator documentation. No API route, migration, scheduler,
|
|
automatic fetch or frontend behavior changed.
|
|
|
|
Validation:
|
|
- 15 focused Sprint 228 tests and 38 combined Sprint 194/227/228 tests passed.
|
|
Coverage includes release ordering, future URL/layout validation, dynamic
|
|
baseline discovery, bounded downloads, >200-row pagination, stage/apply
|
|
command separation, plan/review hashes, source/review tampering, evidence-root
|
|
confinement, catalog drift, named review, apply evidence and current-edition
|
|
refusal.
|
|
- Complete readiness passed with 830 backend tests, 110 documented routes,
|
|
one Alembic head `202607160001`, frontend typecheck and production build.
|
|
Static Alembic SQL, shell syntax, target Ruff and diff checks passed.
|
|
- A compatibility run against the live Tower API used the temporary candidate
|
|
scripts before deployment. The official Statbel catalog reported
|
|
`NodeID6475`, remote/local edition `2025`, layout `new`, catalog hash
|
|
`64b17ce059a9c2f936d4b5741b20aed4b178409d8b5be81be86e6fdc9fe7c9d9` and
|
|
decision `current`; `plan` wrote no evidence. A deliberate `stage` attempt
|
|
for that current edition exited 1 with `not safely stageable: current` and
|
|
left the database Dataset count unchanged at 2,429.
|
|
- Tower was rebuilt and deployed at commit `d0a8a3a`. The all-in-one container
|
|
became healthy on port 1202, PostGIS 3.6 answered, Alembic remained at the
|
|
single `202607160001` head and the frontend/API proxy plus icon smoke passed.
|
|
- The packaged manager repeated the live `plan --refresh-catalog` result as
|
|
`current` for official edition 2025. Its deliberate deployed `stage` attempt
|
|
again exited 1, Dataset count remained 2,429, the health endpoint reported a
|
|
working database and no 2025 `staged-plan.json` was written.
|
|
|
|
Boundary:
|
|
- No newer population edition is currently advertised, so no real stage,
|
|
review or apply was executed. Their complete state machine is fixture-tested;
|
|
the first future release must still pass all four explicit operator phases.
|
|
|
|
## Sprint 227 - Statbel population import compatibility preflight (2026-07-16)
|
|
|
|
Implemented:
|
|
- Added `scripts/statbel_population_preflight.py` as a local, fail-closed gate
|
|
between staged official Statbel archives and the existing population
|
|
provisioner. It performs exact source/edition allowlisting, bounded ZIP
|
|
inspection, population and geometry schema checks, EPSG:31370 and situation
|
|
date validation, geometry validity/area checks, sector joins, explicit
|
|
municipality reconciliation, national/scope total reconciliation and an
|
|
annualized baseline-change guard.
|
|
- Removed the invalid historical assumption that `CD_SECTOR[:5]` always equals
|
|
the current `CD_REFNIS`. The 2025 REDEGEO layout can retain old sector codes
|
|
after a municipal merger; population and geometry must instead agree on
|
|
their explicit current municipality fields.
|
|
- Kept official `ZZZZ` population rows in national and scope accounting while
|
|
excluding them honestly from map geometry. Bounded `make_valid` repair is
|
|
allowed only when the result remains polygonal, valid and area-preserving;
|
|
repaired sector codes are recorded in the manifest.
|
|
- Hardened `provision_mol_population_history.py` so a new fetch is preflighted
|
|
before source retention or derivation, source ZIPs and the derived GeoJSON
|
|
are written atomically, and all three SHA-256 values are rechecked before
|
|
upload. Existing immutable Datasets remain idempotent; a legacy cached file
|
|
cannot create a new Dataset without `--force` restaging evidence.
|
|
- Packaged the script in the all-in-one image, added it to readiness compile
|
|
checks and documented command, storage layout, evidence and limitations.
|
|
No API route, migration, scheduler, background fetch or existing Dataset was
|
|
changed.
|
|
|
|
Validation:
|
|
- Focused preflight/regional-time-series suite: 23 passed. It covers source and
|
|
member identity, missing schema, duplicate sectors, invalid totals, archive
|
|
traversal, CRS/date mismatch, explicit municipality mismatch, unexpected
|
|
non-spatial rows, bounded topology repair, excessive trend change, atomic
|
|
staging, source/snapshot tampering and legacy parser strictness.
|
|
- Complete readiness passed with 815 backend tests, 110 documented routes,
|
|
one Alembic head `202607160001`, frontend typecheck and production build.
|
|
Static Alembic SQL, shell syntax, target Ruff and diff checks passed. A
|
|
repository-wide Ruff audit still reports 17 pre-existing warnings outside
|
|
this change; no unrelated refactor was performed.
|
|
- Live compatibility-only execution inside the healthy Tower container used
|
|
locally staged official archives and made no API/database call:
|
|
- 2024 standard layout: 646 spatial Kempen sectors, spatial population
|
|
503,405, 28 `ZZZZ` rows / 276 unlocated inhabitants and accounted total
|
|
503,681 against the retained 2023 baseline.
|
|
- 2025 new REDEGEO layout: 733 spatial Kempen sectors, spatial population
|
|
506,473, 26 `ZZZZ` rows / 294 unlocated inhabitants and accounted total
|
|
506,767 against the retained 2024 baseline; annualized change 0.6094%.
|
|
- National 2025 accounting reconciled 21,183 population rows and 20,781
|
|
geometries to 11,825,551 inhabitants, including 402 `ZZZZ` rows / 7,654
|
|
unlocated inhabitants. Four official self-intersection cases were repaired
|
|
without area change and recorded by sector code.
|
|
- The complete real 2025 operator staging path retained both source archives,
|
|
generated a 733-feature scoped GeoJSON and authorized source/snapshot hashes
|
|
in an isolated `/tmp/statbel-stage` directory. It did not import or replace a
|
|
Dataset.
|
|
- Gitea commit `4f4d467` rebuilt the Tower all-in-one image with AI extras.
|
|
The container became healthy on port 1202, live migration smoke confirmed
|
|
PostGIS 3.6 and Alembic head `202607160001`, and the browser proxy/icon smoke
|
|
passed. The preflight was then rerun from its deployed `/app/scripts` path
|
|
against the official 2025 archives with the same 506,473 + 294 = 506,767
|
|
scope accounting. Database dataset count remained exactly 2,429 before and
|
|
after. All temporary validation archives and manifests were removed.
|
|
|
|
Boundary:
|
|
- A passed preflight proves technical compatibility only. Adding a future
|
|
edition to `POPULATION_URLS` and replacing or importing any immutable Dataset
|
|
remains an explicit reviewed operator action.
|
|
|
|
## Sprint 226 - Governed Statbel population edition probe (2026-07-16)
|
|
|
|
Implemented:
|
|
- Extended the explicit source-catalog audit with the official Statbel DCAT
|
|
Turtle catalog. The parser selects one latest Dutch population-by-sector
|
|
release and requires consistent period evidence, identifier, canonical
|
|
landing page, CC BY 4.0 license and allowlisted distribution identities.
|
|
- Added strict endpoint and final-redirect validation plus a separate 5 MiB
|
|
response limit. The catalog probe never follows a ZIP/XLSX link, imports a
|
|
Dataset, writes PostGIS or enables background polling.
|
|
- Kept population reference year, sector-geometry edition and REDEGEO layout
|
|
distinct. The 2025 NEW distribution is current; OLD is transition evidence,
|
|
and the separate 2026 sector geometry is not presented as 2026 population.
|
|
- Reused the canonical source-catalog response, cache, operator CLI and Status
|
|
UI. Propagated the fixed URL and size bound through local, Compose and Unraid
|
|
configuration and added RDFLib as a pure-Python core parser dependency.
|
|
|
|
Validation:
|
|
- The bounded parser read the live official DCAT catalog and identified
|
|
`NodeID6475`, population edition `2025`, four distributions, the new REDEGEO
|
|
layout, legacy transition availability, CC BY 4.0 and catalog date
|
|
`2026-07-07` without requesting a distribution.
|
|
- All 24 focused source-catalog tests passed, including hostile host, redirect,
|
|
configured URL, duplicate release, conflicting year, missing license,
|
|
missing current distribution and oversized-response failures.
|
|
- Complete readiness passed with 801 backend tests, 110 documented routes, one
|
|
Alembic head, frontend typecheck and production build. Static Alembic SQL,
|
|
Ruff, diff checks and shell syntax also passed. Local Docker validation was
|
|
unavailable on Windows and is completed through the Tower deployment gate.
|
|
- Tower rebuilt commit `da2371d`; server-side Compose validation passed, the
|
|
all-in-one container became healthy on port 1202, PostGIS 3.6 and required
|
|
schema objects passed, Alembic remained at `202607160001`, and the browser
|
|
proxy/icon smoke passed. The runtime reports RDFLib `7.6.0` and the exact
|
|
configured Statbel URL with its 5 MiB limit.
|
|
- The live regional endpoint reported `4/4` providers available and zero
|
|
edition differences. Statbel returned `NodeID6475`, local/remote `2025`, all
|
|
three evidence markers, four advertised distributions and catalog date
|
|
`2026-07-07`.
|
|
- Browser verification showed the four cards in a balanced two-column
|
|
1920-pixel layout, no horizontal overflow, the full REDEGEO transition copy
|
|
and zero console warnings/errors.
|
|
|
|
Boundary:
|
|
- This is publication evidence only. A future population edition still needs
|
|
a governed schema, sector-geometry and totals review before the existing
|
|
operator may create a new immutable Dataset.
|
|
|
|
## Sprint 225 - Governed ALZ edition probe (2026-07-16)
|
|
|
|
Implemented:
|
|
- Extended the existing explicit source-catalog audit with the exact official
|
|
ALZ agricultural-use parcel publication page. The parser is bounded and
|
|
fail-closed to the canonical HTTPS release path plus exact
|
|
`agpa_<year>_<date>_public.zip` link identities.
|
|
- Kept the edition model honest: the retained local `2025-definitive` Dataset
|
|
compares with official `2025-v3`; the current `2026-v1` snapshot is visible
|
|
as provisional and cannot trigger or justify historical replacement.
|
|
- Reused the canonical source-catalog API envelope, cache, operator command and
|
|
Status surface. No archive, feature, raster or model is fetched and no
|
|
Dataset/PostGIS mutation, scheduler or migration was introduced.
|
|
- Propagated the ALZ release URL through backend settings, Compose, the Unraid
|
|
environment/template/launcher and operator documentation. An odd final
|
|
source-status card spans the full grid width for a balanced desktop layout.
|
|
|
|
Validation:
|
|
- The live official page probe returned `available`, definitive `2025-v3`,
|
|
current provisional `2026-v1`, both required evidence markers and 19
|
|
validated publication links without requesting an archive.
|
|
- Focused source-catalog coverage passed with 13 tests, including release-page
|
|
redirects, untrusted archive hosts, missing current snapshot, definitive
|
|
year ordering, cache bypass and disabled/network-free behavior.
|
|
- Complete readiness passed with 790 backend tests, 110 documented routes, one
|
|
Alembic head, frontend typecheck and production build. Static Alembic SQL and
|
|
live-migration script syntax also passed.
|
|
- Tower was rebuilt from commit `0a23e34`; the all-in-one container became
|
|
healthy on port 1202, PostGIS 3.6 answered, required schema objects passed
|
|
and Alembic remained at `202607160001`.
|
|
- The live regional API reported three available providers and no edition
|
|
differences. Browser verification showed `3/3 bereikbaar`, ALZ
|
|
`bereikbaar / zelfde editie`, `Publicaties: 2/2 bevestigd`, the provisional
|
|
warning and zero console warnings/errors.
|
|
|
|
Boundary:
|
|
- The probe is release evidence only. A future definitive campaign still
|
|
requires human confirmation and the existing governed ALZ operator; no
|
|
automatic ALZ refresh has been enabled.
|
|
|
|
## Sprint 218 Regional terrain and flood completion (2026-07-16)
|
|
|
|
Changed:
|
|
- Added the resumable 28-municipality DHMV DTM/DSM operator and executed all
|
|
56 governed acquisitions through the canonical API.
|
|
- Executed all 336 governed VMM mechanism/climate/return-period combinations
|
|
across the same 28 persisted municipality Areas.
|
|
- Diagnosed Retie's official WCS integer-grid edge rounding in both providers.
|
|
Accepted only bounded 5%/0.25 m edge drift, harmonized accepted tiles to the
|
|
exact requested grid and retained source resolutions, tile indexes and
|
|
method in provenance. A 4.5 m regression fixture remains rejected.
|
|
- Added exact partitioned terrain and flood selection routes for bounded
|
|
rectangles that cross municipality boundaries. They use only persisted
|
|
GeoTIFFs, calculate global statistics from combined cells and retain the
|
|
contributing Dataset ids in the response.
|
|
- Made the complete Kempen Area expose DHMV/VMM as logical regional MapLibre
|
|
layers while retaining municipality-linked storage and twelve distinct VMM
|
|
scenario identities.
|
|
|
|
Live evidence:
|
|
- VMM: 336 ready Datasets, 28 Areas, 12 products, 336 DatasetVersions,
|
|
165,277,992 bytes, zero duplicate Area/product pairs and zero missing or
|
|
size-mismatched files.
|
|
- DHMV: 56 ready Datasets, 28 Areas, two products, 56 DatasetVersions,
|
|
282,645,991 bytes, zero duplicate Area/product pairs and zero missing or
|
|
size-mismatched files.
|
|
- Every VMM Dataset has null `observed_at` and explicit false flags for
|
|
bathymetry, permanent depth/volume and concurrent volume. A repeated Retie
|
|
run reused all twelve immutable Dataset ids.
|
|
- Tower commit `153cff0` passed container health, PostGIS 3.6, Alembic head
|
|
`202607160001` and frontend/API/icon proxy checks before the regional UI
|
|
follow-up.
|
|
|
|
Validation evidence:
|
|
- Focused DHMV/VMM and regional explorer backend suites passed, including
|
|
exact adjacent-partition percentile and area calculations.
|
|
- The pre-UI-fix release gate passed 746 tests, backend compilation, API
|
|
contract checks, one Alembic head, frontend typecheck and production build.
|
|
- The final local release gate passed all 752 backend tests, backend
|
|
compilation, 107 documented API route checks, Alembic head `202607160001`,
|
|
frontend typecheck and the production build. Live deployment and browser
|
|
evidence are recorded after the Tower rollout.
|
|
- Tower deployment of commit `45dc730` passed container health, PostGIS 3.6,
|
|
the live migration/schema smoke and the frontend/API/icon proxy checks.
|
|
- One live 0.15 by 0.13 degree cross-boundary rectangle opened seven persisted
|
|
DHMV and seven persisted VMM partitions. It returned 6,056,943 valid terrain
|
|
cells at 99.9719% coverage and 651.56 ha of modeled pluvial current-climate
|
|
T100 positive-depth cells without contacting either WCS.
|
|
- Browser verification showed both regional themes as available, twelve
|
|
deduplicated VMM scenarios, one 28-municipality MapLibre layer and zero
|
|
console warnings/errors. A follow-up status-label fix makes a visible raster
|
|
report `Rasterlaag actief` instead of `Geen actieve laag`.
|
|
- Interactive browser drawing over the central Kempen returned all fifteen
|
|
theme summaries, including 619.71 ha modeled pluvial T100 area, 18.26 m TAW
|
|
mean terrain height and 56,310 inhabitants. The boundary probe also exposed
|
|
and closed a frontend inconsistency: every selection and derive path now
|
|
forwards the active Area id for authoritative backend clipping.
|
|
|
|
Next:
|
|
- Use the now-complete regional current-state layers as the baseline for a
|
|
governed refresh/change scheduler. Keep source-specific publication dates
|
|
and scenario semantics; do not turn VMM scenarios into a historical water
|
|
level series.
|
|
|
|
## Sprint 217 Regional DOV soil coverage (2026-07-16)
|
|
|
|
Changed:
|
|
- Generalized the explicit DOV `bodemkaart:bodemtypes` operator from Mol to the
|
|
approved 28-member Kempen scope without changing source or persistence
|
|
semantics.
|
|
- Added independently reusable municipality partitions, deterministic gzip raw
|
|
responses, boundary/output checksums, unique NIS-suffixed persisted ids and
|
|
one regional snapshot manifest.
|
|
- Packaged and release-checked the operator in the all-in-one image.
|
|
- Kept regional and municipality metric queries on exact PostGIS intersections
|
|
instead of asserting a preclipped fast path across CRS round trips.
|
|
|
|
Validation evidence:
|
|
- The live operator completed 28/28 partitions, 62 WFS responses, 46,134
|
|
bounded source features and 27,200 persisted valid EPSG:4326 polygons.
|
|
- PostGIS found zero invalid/empty/wrong-SRID geometries, zero duplicate source
|
|
ids, complete municipality/NIS identity and 139,871.44 ha mapped area.
|
|
- A strict storage-boundary audit measured only 355.15 m2 of coordinate-rounding
|
|
slivers across 579 boundary features (about 0.000025%); exact selection
|
|
intersections prevent those slivers from bypassing the selected Area.
|
|
- A repeat operator run reused every verified partition and the existing
|
|
Dataset.
|
|
- Browser validation showed Bodem available for the complete Kempen and for
|
|
Arendonk, returned 139,871.44 ha and 5,514.99 ha respectively, rendered the
|
|
MapLibre layer at detail zoom and produced no browser warnings or errors.
|
|
- The pre-deployment readiness gate passed 738 backend tests, compilation,
|
|
contract checks, one Alembic head, frontend typecheck and production build.
|
|
|
|
Next:
|
|
- Fill the remaining region-wide current-state gaps shown in the Map workspace:
|
|
VMM flood-hazard coverage and DHMV terrain. Preserve their separate modeled
|
|
scenario and elevation semantics; do not infer current water volume.
|
|
|
|
## Sprint 216 Regional thematic coverage (2026-07-16)
|
|
|
|
Changed:
|
|
- Ran the governed thematic-raster operator for every one of the 28 persisted
|
|
municipality Areas in `Kempen Regional Workbench` after a successful
|
|
Arendonk smoke. All five allowlisted products completed for every Area.
|
|
- Added five separately persisted snapshots for the complete `Vervoerregio
|
|
Kempen - officiele operationele grens`, enabling free selections that cross
|
|
municipality boundaries while preserving exact Area clipping.
|
|
- Increased only `THEMATIC_RASTER_MAX_SIDE_M` to 60,000 and
|
|
`THEMATIC_RASTER_MAX_PIXELS` to 30,000,000. The service still rejects larger
|
|
requests and reads the official WCS in fixed 10 km tiles.
|
|
- Retried transient WCS failures at most three times with a bounded delay.
|
|
Interrupted responses are discarded and permanent failures remain explicit.
|
|
- Required the local assistant to keep each metric's year, source and
|
|
measurement quality together when a theme has several official datasets.
|
|
|
|
Validation evidence:
|
|
- The final inventory contains 145 ready thematic Datasets across 29 Areas and
|
|
five product keys, with zero missing metadata/storage paths and zero duplicate
|
|
Area/product pairs. All 145 TIFF files are non-empty and occupy 56 MB.
|
|
- A second complete-region operator run reused all five existing Datasets.
|
|
- Browser validation on `http://192.168.10.150:1202` showed every regional
|
|
thematic source as available, rendered the MapLibre overlay and calculated
|
|
the complete Kempen result without browser warnings or errors.
|
|
- The full readiness gate passed 733 backend tests, compilation, fixture and
|
|
API-contract audits, one Alembic head, frontend typecheck, production build
|
|
and shell checks.
|
|
- Live PostGIS reports version 3.6 and the all-in-one container remained
|
|
healthy throughout acquisition and analysis.
|
|
|
|
Next:
|
|
- Generalize the separate DOV soil-map operator from Mol to all 28 municipality
|
|
partitions. Keep the historical 1949-1971 survey semantics and raw source
|
|
manifest explicit; do not merge soil provenance into the raster operator.
|
|
|
|
## Sprint 215 Grounded assistant cross-domain reliability (2026-07-16)
|
|
|
|
Changed:
|
|
- Measured the exact live Mol assistant context at 12 current themes, 36
|
|
semantic metrics, six temporal series and 17,302 compact JSON characters.
|
|
- Increased the bounded output allowance to 1,200 tokens and instructed the
|
|
local model to cover every explicitly listed theme, omit unrelated themes
|
|
and avoid repeated limitations.
|
|
- Kept `think=false`, deterministic temperature, the 16,384-token context and
|
|
fail-closed rejection of `done_reason=length` unchanged.
|
|
- Added the output limit to the editable Unraid template and aligned all
|
|
deployment defaults and documentation.
|
|
- Browser QA in the central 28-municipality workbench exposed that an explicit
|
|
six-theme question still summarized every current vector theme. Agricultural
|
|
subclass intersections pushed context construction past 110 seconds.
|
|
- Added deterministic Dutch theme selection before GIS calculation. Explicit
|
|
questions now calculate only named themes; general summaries and source
|
|
inventory questions deliberately retain full context.
|
|
- The optimized live response completed in 12.3 seconds but exposed literal
|
|
Markdown and raw floating-point tails in the plain-text chat renderer. Added
|
|
deterministic Markdown normalization and unit-aware context rounding while
|
|
retaining the unrounded metrics in the canonical API response.
|
|
- The next live answer still called an area-weighted Statbel population value
|
|
an official count before acknowledging it as an estimate. The deterministic
|
|
estimate guard now removes that contradictory label whenever persisted
|
|
population context is estimated.
|
|
- Preserved sentence casing and natural Dutch word order in that deterministic
|
|
rewrite after the browser regression exposed an awkward replacement phrase.
|
|
- A 390 px browser pass exposed a CSS-specificity conflict: the generic
|
|
`section > div` grid overruled the explorer's responsive `display: block`.
|
|
The geographic layout is now excluded from that generic rule and has a
|
|
regression assertion for its mobile stacking contract.
|
|
- A final accuracy audit compared the DOV inventory and assistant wording
|
|
directly with PostGIS. All 1,159 soil features are non-empty, valid and
|
|
intersect Mol, and their transformed area is 11,448.35 ha. The model still
|
|
mistyped the supporting count as 1,150 in prose even though its canonical
|
|
context contained 1,159.
|
|
- Added compound-word theme recognition so `bodemdetails` now selects only the
|
|
soil context instead of the complete project portfolio. Supporting object
|
|
counts are retained in `context_metrics` but withheld from the model prompt
|
|
when semantic measurements exist, eliminating that irrelevant failure mode.
|
|
|
|
Validation evidence:
|
|
- A read-only live production-chain probe with the 1,200-token setting
|
|
completed without truncation against the full persisted Mol context.
|
|
- The focused assistant suite passed 14 tests.
|
|
- The complete readiness gate passed 713 backend tests, backend compilation,
|
|
105 documented API routes with three explicit binary/non-envelope routes,
|
|
one Alembic head and the frontend TypeScript and production build.
|
|
- After deterministic theme filtering was added, the complete readiness gate
|
|
passed 720 backend tests plus all compile, contract, Alembic, frontend and
|
|
shell gates.
|
|
- After plain-text normalization and semantic prompt rounding, the final
|
|
readiness rerun passed 725 backend tests and the same complete gate set.
|
|
- After hardening estimated-population wording, the final complete readiness
|
|
rerun passed 726 backend tests and all remaining gates unchanged.
|
|
- The regional thematic operator dry-run resolved exactly 28 official
|
|
municipality Areas. The live Mol run in `Kempen Regional Workbench` imported
|
|
all five products with complete source coverage; the DOV operator imported
|
|
1,159 exact Mol soil polygons into the same project.
|
|
- Browser validation then showed all six new themes as available and returned
|
|
3,638.41 ha space occupation, 31.76% share and the full 15-theme Mol summary
|
|
from persisted data.
|
|
- The final Tower deployment at commit `43dd535` passed container health,
|
|
browser proxy/icon checks, PostGIS 3.6 schema validation and Alembic head
|
|
`202607160001`.
|
|
- The exact six-theme Mol assistant regression returned 2,575 readable
|
|
characters covering every requested theme, with no Markdown, contradictory
|
|
official-count wording or unrelated theme. The 390 px explorer measured
|
|
317 px for theme, map and results panels with no page overflow; desktop and
|
|
widescreen map checks remained clean and browser logs contained no warnings.
|
|
- The focused live `bodemdetails` regression at commit `59194e9` selected one
|
|
DOV dataset and six canonical metrics, retained the exact 1,159 object count
|
|
in API evidence and returned only the five source-correct hectare metrics
|
|
plus the 1949-1971/1:20,000 limitation in end-user prose.
|
|
- The final complete readiness gate passed 730 backend tests, compilation,
|
|
documentation/fixture/contract audits, one Alembic head, frontend typecheck,
|
|
production build and shell-script checks.
|
|
|
|
Next:
|
|
- Roll out the five governed thematic rasters municipality by municipality to
|
|
the remaining 27 Kempen Areas, with storage/runtime monitoring and an audit
|
|
after each batch. Keep DOV soil expansion separate until its operator is
|
|
generalized beyond the current explicit Mol boundary.
|
|
|
|
## Sprint 213-214 Cross-domain thematic rasters and DOV soil map (2026-07-16)
|
|
|
|
Changed:
|
|
- Added a governed five-product MercatorNet thematic-raster registry covering
|
|
space, people, accessibility and services with fixed coverage ids, native
|
|
resolutions, units, years, legends and limitations.
|
|
- Added safe WCS tiling, exact Area masking, canonical raster persistence,
|
|
MapLibre rendering, selection metrics and persisted-metric Ollama context.
|
|
- Added an explicit DOV soil-map operator for Mol with deterministic WFS 2.0
|
|
pagination, exact EPSG:31370 clipping, raw checksums and canonical vector
|
|
upload. No source writes directly to PostGIS.
|
|
- Added end-user themes for space occupation, open space, population,
|
|
accessibility, services and soil. Current-only official states no longer
|
|
make Evolution appear available without at least two observations.
|
|
|
|
Validated during implementation:
|
|
- Focused thematic and soil suites passed with 14 tests.
|
|
- Frontend TypeScript typecheck passed after map and API integration.
|
|
- Live source contracts were checked against MercatorNet WCS and the DOV
|
|
production WFS; live runtime provisioning follows after deployment.
|
|
|
|
Next:
|
|
- Run the complete readiness gate, deploy Tower, provision all six Mol layers,
|
|
verify live PostGIS metrics and inspect the map at desktop, widescreen and
|
|
mobile widths before starting regional rollout.
|
|
|
|
## Sprint 195 Guided raster-to-detection workflow (2026-07-14)
|
|
|
|
Changed:
|
|
- Connected the existing Dataset upload, `raster/tile`, YOLO preflight, configured detection, persisted result listing and MapLibre GeoJSON overlay into one guided frontend action.
|
|
- Added explicit GeoTIFF input inside Detection Lab so users no longer need to navigate to dataset management before starting image analysis.
|
|
- Uses fixed safe tiling defaults of 512 px with 64 px overlap, reuses the current manifest and estimates the tile count from canonical raster inspection before writing tiles; the backend remains authoritative for tile-limit, manifest, dependency and local-model validation.
|
|
- Moved manifest paths, direct-manifest execution and calibration under management disclosures while returning persisted reference QA to the main result flow.
|
|
- Kept every run on the existing `Dataset -> Job -> AnalysisRun -> Detection -> QualityCheck/Metric` chain. No migration, API route, external fetch, model download or synthetic inference was added.
|
|
|
|
Validated locally:
|
|
- `bash scripts/run_readiness_check.sh` passed with `575 passed`, backend compile, one Alembic head (`202607140001`), frontend typecheck/build and live-smoke syntax validation.
|
|
- `python -m alembic upgrade head --sql` generated the complete 26,972-byte PostgreSQL/PostGIS migration plan successfully.
|
|
- Focused guided-flow, stale-map-state and direct-upload contracts passed. The Windows workstation has no Docker CLI; live container/PostGIS validation therefore remains part of the Tower deployment pass.
|
|
|
|
Next:
|
|
- Provision one existing real Mol operator orthophoto and matching official GRB reference into the regional workbench through the canonical upload API, then execute the guided action and verify the persisted map/QA result in the browser.
|
|
|
|
## Sprint 194 Regional official time series and full-Area performance (2026-07-14)
|
|
|
|
Changed:
|
|
- Generalized the explicit Statbel operator to the approved 28-municipality scope and added one coordinator for official 2021-2025 population plus 2013-2025 forest snapshots.
|
|
- Added municipality-partitioned official 10 m WCS retrieval after the upstream response-size limit rejected a single full-region request; source partitions are resumable and mosaicked locally before exact region clipping.
|
|
- Added a provenance-gated full-Area selection path for trusted pre-clipped operator datasets. Free rectangles, unrelated Areas and ordinary uploads continue to execute the normal PostGIS intersection path.
|
|
- Grouped older dated snapshots behind `Historische meetmomenten`, replaced raw provider/type labels in the primary catalog and translated the remaining visible temporal, download, platform-status and detection-evaluation language.
|
|
|
|
Live regional evidence:
|
|
- Population datasets: five exact dated snapshots, 3,317 persisted `vector_features`; 2021 `489,927` inhabitants and 2025 `506,473`, a measured increase of `16,546` (`3.377%`).
|
|
- Forest datasets: five exact dated snapshots, 273,669 persisted `vector_features`; 2013 `27,659.976 ha` and 2025 `27,410.687 ha`, a source-resolution change of `-249.289 ha` (`-0.9013%`).
|
|
- All ten snapshots have one immutable dataset version and zero invalid, empty, non-4326 or source-ID-missing geometries. A repeated synchronization reused the same ten Dataset IDs without duplicates.
|
|
- Complete-Kempen current analysis returns 506,473 inhabitants, 27,410.69 ha forest, 466,078 buildings, 88,332 water features, 84,504 roads and 415,288 parcels over 1,399.25 km2.
|
|
|
|
Performance and runtime evidence:
|
|
- Population full-Area API selection completed in `1.738 s`; forest completed in `1.083 s`. The browser rendered the full six-theme analysis and 2021-2025 comparison within the 3.5-second verification window, replacing the previous roughly 92-second forest-heavy workflow.
|
|
- Tower health, PostGIS 3.6, required schema objects, Alembic head `202607140001`, frontend, API proxy and icon checks passed after deployment.
|
|
- PyTorch `2.13.0+cpu`, Ultralytics `8.4.95` and the active local model file load successfully. The catalog exposes 24 local assets; the active seven-AOI profile remains review-required at F1 `0.5825` and was not blindly retrained.
|
|
|
|
Tested:
|
|
- `bash scripts/run_readiness_check.sh` passed with `571 passed`, backend compile, one Alembic head, frontend typecheck/build and live-smoke syntax validation.
|
|
- Focused regional, temporal and end-user contract checks passed. The browser confirmed the complete regional default, six current themes, five population moments, exact temporal totals and no required project/region selection step.
|
|
|
|
Next:
|
|
- Complete the 48 false-positive and 48 false-negative operator review decisions, then use those decisions to build a justified next training corpus and expose the existing raster-to-tiles-to-detection workflow as one guided end-user action.
|
|
|
|
## Sprint 193 End-user regional workbench simplification (2026-07-14)
|
|
|
|
Changed:
|
|
- Made `Kempen Regional Workbench` the automatic complete data context before the municipality fallback and removed the primary map's project-level region selector.
|
|
- Kept the 28 municipality Areas plus the complete region available as one spatial work-area selector over the already loaded regional datasets.
|
|
- Simplified the visible shell and moved technical project runs, raw source metadata, QA internals, provider capabilities, export history and AI diagnostics into explicit advanced disclosures.
|
|
- Defaulted Detection Lab to `yolo-configured`, threshold `0.15`, the active local asset and automatic non-mutating preflight checks.
|
|
- Added end-user workbench contract tests without changing backend API contracts, migrations, persistence or model execution.
|
|
|
|
AI evidence:
|
|
- The configured Tower runtime reports PyTorch `2.13.0+cpu`, Ultralytics `8.4.95`, a valid local active model file and a successful model-load preflight; CUDA is not available.
|
|
- The retained active profile records precision `0.5898`, recall `0.5770` and F1 `0.5825` over seven positive AOIs with zero detections in the empty-background control.
|
|
- No automatic download or blind retraining was introduced. The 48 false-positive and 48 false-negative review decisions remain a required gate before constructing another training corpus.
|
|
|
|
Tested before deployment:
|
|
- `bash scripts/run_readiness_check.sh` passed with `560 passed`, backend compile, one Alembic head (`202607140001`), frontend typecheck/build and live-smoke syntax validation.
|
|
- `python -m alembic upgrade head --sql` generated a 30,166-byte offline migration plan successfully; `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- The false-positive review validator returned the expected `review_required` gate with all 48 selected decisions still `unreviewed`; no unreviewed QA mismatch was promoted into training data.
|
|
- Local Docker validation was unavailable because the Windows workstation has no Docker CLI; live container/PostGIS validation remains part of the Tower deployment pass.
|
|
|
|
Live browser follow-up:
|
|
- Tower rebuilt the AI-enabled all-in-one image successfully; container health, PostGIS 3.6, required schema objects, Alembic head `202607140001`, frontend, API proxy and icon checks passed.
|
|
- The map opened directly on the complete regional workspace with 29 work Areas, no region selector, the full region selected and 466,078 buildings, 88,332 water objects, 84,504 roads and 415,288 parcels available.
|
|
- The browser audit found four remaining presentation leaks: the legacy Mol-only project was still prominent, boundary datasets showed filenames, profile names were English and an empty quality status said `waiting`. These were corrected and protected by the now `561`-test suite.
|
|
- A final visible-language sweep translated the export preview, details control and provider layer chips; the complete readiness gate remained green with `561 passed`.
|
|
|
|
Next:
|
|
- Extend official population and modern land-use time series from Mol to all 28 municipalities, then expose a guided raster-import and configured-detection run over a drawn map selection.
|
|
|
|
## Sprint 191 Regional Kempen GRB context foundation (2026-07-14)
|
|
|
|
Changed:
|
|
- Added `scripts/provision_regional_grb_context.py` for bounded, independently resumable `roads`, `water` and `parcels` source snapshots across the approved 28-municipality scope.
|
|
- Reused geographic-scope artifacts and the existing DatasetService/VectorFeatureService partition import boundary; no direct SQL, migration, API contract or interactive provider fetch was introduced.
|
|
- Preserved line and polygon source dimensions and collection-qualified identities, with length-based or area-based deterministic ownership for boundary-crossing features.
|
|
- Added Docker packaging, readiness compilation, focused source-geometry/service tests and operator/source documentation.
|
|
|
|
Tested before deployment:
|
|
- `python -m py_compile scripts/provision_regional_grb_context.py`.
|
|
- `python -m pytest backend/tests/test_sprint191_regional_grb_context.py backend/tests/test_sprint190_regional_grb_buildings.py backend/tests/test_sprint106_map_bbox_extract.py -q` (`19 passed`).
|
|
- `bash scripts/run_readiness_check.sh` (`553 passed`; one Alembic head; frontend typecheck/build and script syntax gates passed).
|
|
|
|
Runtime evidence:
|
|
- Pushed commits `0ed214d` and `7af47ab`, rebuilt the AI-enabled all-in-one Tower runtime and passed PostGIS 3.6/Alembic head `202607140001` plus browser-proxy verification.
|
|
- Roads: 84,504 unique `Wegsegment` features, 124,721,658-byte artifact, Dataset `715b24f1-148a-4d50-9c40-d5468f670ffb`.
|
|
- Water: 88,332 unique features (25,648 `WTZ`, 4,605 `WLAS`, 58,079 `WGR`), 122,737,017-byte artifact, Dataset `689bb65d-0b03-4c5c-8f42-b5d2d899d8ca`.
|
|
- Parcels: 415,288 unique `ADP` features, 525,243,364-byte artifact, Dataset `3cf5e8da-9ab0-49ef-96e4-f74b7557e0a4`.
|
|
- All three repeat runs returned `reused=true`; manifest, metadata, PostGIS row and distinct source-ID counts match exactly. There are no missing IDs, invalid/empty geometries or non-4326 rows, and every dataset has one immutable version.
|
|
- Canonical Area selections returned the full regional counts and Mol counts of 8,444 roads, 3,669 water objects and 33,038 parcels. The browser exposed all four available regional themes and honestly left population and forest unavailable.
|
|
|
|
Next:
|
|
- Extend official population and modern land-use time series from Mol to the approved regional scope, retaining source-specific methodology and bounded municipality partitions.
|
|
|
|
## Sprint 192 Regional map state correctness (2026-07-14)
|
|
|
|
Changed:
|
|
- Cleared spatial selection and multi-theme result state before changing the selected work Area.
|
|
- Invalidated in-flight single-theme and multi-theme selection requests when their context is reset.
|
|
- Made large-vector viewport guidance name the active reference layer instead of always saying `buildings`.
|
|
|
|
Tested:
|
|
- `python -m pytest backend/tests/test_sprint192_regional_map_state.py backend/tests/test_sprint186_map_first_geographic_explorer.py backend/tests/test_sprint190_regional_grb_buildings.py -q` (`14 passed`).
|
|
- Frontend typecheck and production build passed.
|
|
- `bash scripts/run_readiness_check.sh` (`556 passed`; one Alembic head; frontend typecheck/build and script syntax gates passed).
|
|
|
|
Runtime evidence:
|
|
- Pushed commit `7997a72`, rebuilt the all-in-one Tower runtime and passed container health, PostGIS 3.6, Alembic head `202607140001` and browser-proxy verification.
|
|
- Repeated the exact defect path: full regional parcels returned 415,288 objects over 1,399.25 km2; switching the work Area to Mol immediately cleared the old result and selection.
|
|
- A fresh Mol full-Area run returned 33,038 parcels over 114.55 km2 plus 36,941 buildings, 3,669 water objects and 8,444 roads from the regional datasets.
|
|
- Viewport guidance said `parcels`; zooming below the display cap loaded 525 visible PostGIS features over the OpenStreetMap basemap.
|
|
- In-app browser checks at 1280x720 and 3440x1440 found no horizontal overflow, blank canvas, console warning or console error. The temporary ultrawide override was reset.
|
|
|
|
Next:
|
|
- Extend official population and land-use time series from Mol to the approved regional scope.
|
|
|
|
## Sprint 169 Filtered YOLO candidate gate and operator hardening (2026-07-12)
|
|
|
|
Changed:
|
|
- Trained inactive local model asset `geointel-building-yolov8s-aoi1024cleanpx12vis035lowvar512e50-pt` from the visually audited AOI1024 dataset after low-variance negative filtering.
|
|
- Added SHA256 provenance fields to future `training_summary.json` output for `dataset.yaml`, the dataset summary, the local base model and the copied trained model.
|
|
- Kept long workbench context names compact with matching native tooltips and clamped readiness values to two lines.
|
|
- Archived the training, positive-AOI, split-background and promotion evidence under `artifacts/model-review/aoi1024cleanpx12vis035lowvar512e50` locally and matching Tower artifact directories.
|
|
|
|
Runtime evidence:
|
|
- Training completed for 50 CPU epochs with YOLOv8s, image size `512`, batch `4`; best-model validation ended at precision `0.454`, recall `0.491`, mAP50 `0.368` and mAP50-95 `0.149`.
|
|
- Model catalog SHA256: `75345767b51cc66692a9d2c2cd6577b7feecaf5971c8762365e8b610b9dfde8e`; catalog status `available`, `active=false`, `will_download_models=false`.
|
|
- Local model-load preflight passed with Torch `2.13.0` and Ultralytics `8.4.92`; CUDA is unavailable and no model download occurred.
|
|
- Seven-AOI persisted QA matrix produced 28 runs. Best single result was Westerlo at threshold `0.15`, F1 `0.3002114164904862`.
|
|
- Mean positive F1 was `0.153943` at threshold `0.05`, `0.153872` at `0.15`, `0.128891` at `0.25` and `0.099011` at `0.35`.
|
|
- Strict pure-empty background evidence was clean at thresholds `0.15`, `0.25` and `0.35`; threshold `0.05` produced one Postel-bos detection.
|
|
- Split-aware promotion report recommended `none`: every threshold failed the positive mean-F1 gate, and `0.05` also failed background false-positive pressure.
|
|
- The existing active `geointel-building-yolov8s-aoi1024bg512r3e50-pt` remains materially stronger at its promoted `0.35` profile with mean F1 `0.320866`; no model default or `.env` value was changed.
|
|
- Pushed implementation commit `8daaa07` and redeployed the AI-enabled Tower all-in-one container on `http://192.168.10.150:1202`.
|
|
- Post-deploy live migration smoke passed with PostGIS `3.6`, required schema objects and Alembic head `202606120900`.
|
|
- In-app browser verification found no console warnings/errors, confirmed matching full-value context tooltips and a two-line readiness value, and showed both the inactive low-variance candidate and the unchanged active `aoi1024bg512r3e50` asset in Detection Lab.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint169_long_context_name_readability.py -q` failed before context tooltips and readiness clamping were present.
|
|
- `python -m pytest backend/tests/test_sprint169_long_context_name_readability.py backend/tests/test_sprint22_workbench_status_strip.py backend/tests/test_sprint161_widescreen_workbench.py -q` (`5 passed`).
|
|
- Red step: `python -m pytest backend/tests/test_sprint129_operator_yolo_training_dataset.py::test_operator_yolo_train_smoke_script_contract -q` failed before training hashes were recorded.
|
|
- `python -m pytest backend/tests/test_sprint129_operator_yolo_training_dataset.py -q` (`3 passed`).
|
|
- `bash -n scripts/train_operator_yolo_detector.sh` and frontend typecheck passed.
|
|
- Full readiness: `bash scripts/run_readiness_check.sh` (`461 passed`; one Alembic head; frontend typecheck/build and shell syntax gates passed).
|
|
|
|
Next:
|
|
- Do not retrain the same architecture blindly. Inspect per-AOI false-negative evidence and improve label geometry/class balance or add targeted positive samples for the weakest AOIs before the next candidate.
|
|
|
|
## Sprint 164 Tower AI deploy env hardening (2026-07-11)
|
|
|
|
Changed:
|
|
- Hardened `scripts/deploy_tower.sh` and `scripts/deploy_tower.ps1` so the remote Tower `.env` is sourced before building the all-in-one image.
|
|
- Updated the PowerShell deploy wrapper to stream the remote script through `bash -s`, matching the Bash deploy path and preserving Bash variable expansion during `.env`-driven builds.
|
|
- The PowerShell wrapper now writes a UTF-8-without-BOM temporary script, copies it with `scp`, runs it with `bash` on Tower and removes the remote temp file while preserving the deploy exit code.
|
|
- `GEOINTEL_INSTALL_AI=true` in `/mnt/user/appdata/geointel/.env` now drives the automatic image build by default; explicit local overrides remain possible for one-off deploys.
|
|
- Documented the deploy behavior in `deploy/unraid/README.md`.
|
|
|
|
Why:
|
|
- A manual `docker compose up -d --build` against the multi-container compose file failed on Tower because Docker had exhausted default bridge address pools. The healthy runtime is the Unraid all-in-one container, which should be redeployed through the Dockerman-native scripts instead.
|
|
- The previous deploy script path could build a GIS-only image while the remote runtime `.env` enabled YOLO, leaving the configured detector in `dependency_unavailable`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint31_unraid_template.py::test_tower_deploy_build_uses_remote_env_ai_setting_by_default` failed because the deploy scripts did not source remote `.env` before `docker build`.
|
|
- `python -m pytest backend/tests/test_sprint31_unraid_template.py::test_tower_deploy_build_uses_remote_env_ai_setting_by_default backend/tests/test_sprint31_unraid_template.py::test_tower_deploy_uses_single_container_unraid_compose backend/tests/test_docker_runtime_config.py::test_unraid_deploy_passes_ai_build_arg_and_yolo_runtime_env` (`3 passed`).
|
|
- Red step: `python -m pytest backend/tests/test_sprint31_unraid_template.py::test_powershell_tower_deploy_streams_remote_script_to_bash -q` failed because the PowerShell wrapper passed the remote script as an SSH command argument instead of streaming it to `bash -s`.
|
|
- `python -m pytest backend/tests/test_sprint31_unraid_template.py::test_powershell_tower_deploy_streams_remote_script_to_bash backend/tests/test_sprint31_unraid_template.py::test_tower_deploy_build_uses_remote_env_ai_setting_by_default backend/tests/test_docker_runtime_config.py::test_unraid_deploy_passes_ai_build_arg_and_yolo_runtime_env -q` (`3 passed`).
|
|
- Red step: the same PowerShell deploy transport test failed until the wrapper wrote a `[System.Text.UTF8Encoding]::new($false)` temp script, copied it via `scp`, ran it through remote `bash` and propagated the remote exit code.
|
|
|
|
## Sprint 150 YOLO label visible-ratio gate (2026-07-09)
|
|
|
|
Changed:
|
|
- Added `--min-label-visible-ratio` / `OPERATOR_YOLO_MIN_LABEL_VISIBLE_RATIO` to `scripts/export_operator_yolo_tile_dataset.py`.
|
|
- The tile exporter now computes the visible share of each original building bbox inside a tile and can drop labels below the configured ratio.
|
|
- Default remains `0` for legacy behavior; use `0.25` for the next overlap-heavy operator dataset experiment.
|
|
- Tile dataset summaries include `min_label_visible_ratio`.
|
|
- `scripts/audit_operator_yolo_dataset_quality.py` now reports `min_label_visible_ratio` in JSON and Markdown.
|
|
- Added operator-only `--width`, `--height` and `--half-size-scale` options to `scripts/prepare_operator_real_data_samples.py`; generated raster names now include the requested width.
|
|
- Updated operator script documentation.
|
|
|
|
Why:
|
|
- The current rejected AOI512 candidate still shows low precision/recall after max-det and duplicate suppression hardening.
|
|
- A likely label-quality issue is that overlapping tile export can create many small clipped edge labels for buildings mostly outside a tile.
|
|
- This pass improves the next training dataset gate without activating a model, faking detections, fetching providers or changing persistence.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` failed because the exporter lacked `min_label_visible_ratio`, CLI help and visible-fragment filtering.
|
|
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` (`6 passed`)
|
|
- Red step: `python -m pytest backend\tests\test_sprint146_operator_yolo_dataset_quality_audit.py -q` failed because the audit report did not expose `min_label_visible_ratio`.
|
|
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py backend\tests\test_sprint146_operator_yolo_dataset_quality_audit.py -q` (`7 passed`)
|
|
- Red step: `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint131_operator_sample_expansion.py -q` failed because sample prep lacked larger-AOI options.
|
|
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint131_operator_sample_expansion.py -q` (`8 passed`)
|
|
- Full readiness: `bash scripts/run_readiness_check.sh` (`427 passed`, frontend typecheck/build passed).
|
|
- Tower deploy: first AI rebuild failed with Docker storage full; after Docker build cache cleanup `/var/lib/docker` had 98G free and redeploy passed live migration smoke and browser runtime verification on `http://192.168.10.150:1202`.
|
|
- Tower dataset audit: exported `/app/storage/operator-data/yolo-building-aoi512-visible025` with `min_label_visible_ratio=0.25`; audit returned `needs_attention` because the current 512x512 source rasters still produce only 16 tiles and median normalized box area remains below gate.
|
|
|
|
Next:
|
|
- Prepare a larger explicit operator sample manifest, for example `/app/storage/operator-data/operator-samples-1024` with `--width 1024 --height 1024 --half-size-scale 2`, then export/audit `yolo-building-aoi1024-visible025` before another CPU training candidate.
|
|
|
|
## Sprint 149 YOLO duplicate suppression evidence (2026-07-09)
|
|
|
|
Changed:
|
|
- Added configured-YOLO cross-tile duplicate suppression after pixel boxes are converted to EPSG:4326 geometries and before `Detection` rows are persisted.
|
|
- Added backend setting `YOLO_DUPLICATE_IOU_THRESHOLD` with default `0.5`; `0` disables the GeoIntel-side pass for debugging.
|
|
- Detection run `result_json` now records:
|
|
- `raw_detection_count`
|
|
- `suppressed_detection_count`
|
|
- `duplicate_iou_threshold`
|
|
- Updated `.env.example`, Docker Compose, Unraid env examples and the Dockerman run script.
|
|
- Updated calibration and quality matrix scripts to fetch detection run details and include raw/suppressed counts in per-run and aggregate summaries.
|
|
- Updated backend/API/AI/pipeline documentation.
|
|
|
|
Why:
|
|
- Dense overlapping tile inference can produce duplicate candidate buildings, which inflates persisted false positives before QA/QC.
|
|
- The previous Sprint 148 cap fix allowed dense AOIs to persist more candidates, but made duplicate pressure more visible.
|
|
- This pass keeps all outputs honest: no model activation, no fake detections and no migration. It only removes lower-confidence same-class geometric duplicates before persistence.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py::test_yolo_run_suppresses_cross_tile_duplicate_detections -q` failed with `detection_count == 2`.
|
|
- `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py::test_yolo_run_suppresses_cross_tile_duplicate_detections -q` (`1 passed`)
|
|
- Red step: Docker runtime config tests failed before `.env.example` and Unraid runner exposed `YOLO_DUPLICATE_IOU_THRESHOLD`.
|
|
- `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py backend\tests\test_docker_runtime_config.py::test_env_example_uses_runtime_env_names_read_by_backend_and_frontend backend\tests\test_docker_runtime_config.py::test_unraid_deploy_passes_ai_build_arg_and_yolo_runtime_env -q` (`17 passed`)
|
|
- Red step: calibration/matrix script tests failed before detection-run raw/suppressed metadata was included.
|
|
- `python -m pytest backend\tests\test_sprint124_detection_calibration_sweep.py backend\tests\test_sprint126_detection_quality_matrix.py backend\tests\test_sprint127_operator_sample_quality_matrix.py::test_multi_sample_detection_quality_matrix_runs_existing_matrix_for_each_sample -q` (`3 passed`)
|
|
- `bash -n scripts/run_detection_calibration_sweep.sh`
|
|
- `bash -n scripts/run_detection_quality_matrix.sh`
|
|
- `bash -n scripts/run_multi_sample_detection_quality_matrix.sh`
|
|
- `bash scripts/run_readiness_check.sh` (`426 passed`, frontend typecheck/build passed)
|
|
- Redeployed Tower all-in-one image with AI dependencies and verified `YOLO_MAX_DETECTIONS=1000` plus `YOLO_DUPLICATE_IOU_THRESHOLD=0.5` in the live container.
|
|
- Tower live migration smoke and browser runtime verification passed.
|
|
- Live Westerlo calibration with `geointel-building-yolov8s-aoi512e80-pt`:
|
|
- `0.25`: 202 persisted / 270 raw / 68 suppressed, F1 `0.2537313432835821`
|
|
- `0.15`: 328 persisted / 523 raw / 195 suppressed, F1 `0.22054380664652568`
|
|
- `0.05`: 586 persisted / 1000 raw / 414 suppressed, F1 `0.17173913043478262`
|
|
- Live Turnhout calibration with `geointel-building-yolov8s-aoi512e80-pt`:
|
|
- `0.25`: 559 persisted / 822 raw / 263 suppressed, F1 `0.14114114114114112`
|
|
- `0.15`: 659 persisted / 1000 raw / 341 suppressed, F1 `0.14385474860335196`
|
|
- `0.05`: 659 persisted / 1000 raw / 341 suppressed, F1 `0.14385474860335196`
|
|
|
|
Conclusion:
|
|
- Cross-tile duplicate suppression reduces false-positive pressure and improves F1 versus the Sprint 148 uncapped baseline on the checked dense AOIs.
|
|
- The current AOI512 YOLOv8s candidate remains rejected for default use because recall/precision are still too low after post-processing.
|
|
- Next model work should focus on stronger training data/label strategy and a new candidate gate, not further default threshold lowering.
|
|
|
|
## Sprint 148 YOLO max-detection cap hardening (2026-07-09)
|
|
|
|
Changed:
|
|
- Added backend setting `YOLO_MAX_DETECTIONS` / `Settings.yolo_max_detections`.
|
|
- `YoloDetectionAdapter` now forwards the value to Ultralytics as `max_det`.
|
|
- Default is `1000` instead of relying on Ultralytics' upstream default of 300.
|
|
- Added Docker/Unraid/runtime wiring:
|
|
- `.env.example`
|
|
- `docker-compose.yml`
|
|
- `docker-compose.unraid.yml`
|
|
- `deploy/unraid/geointel.env.example`
|
|
- `deploy/unraid/run-dockerman-container.sh`
|
|
- Updated backend/API/AI environment documentation.
|
|
|
|
Why:
|
|
- Real Kempen building AOIs often contain more than 300 reference buildings.
|
|
- The previous configured-YOLO path could saturate at 300 detections before QA/QC, capping recall independently of model quality.
|
|
- This does not activate a model and does not fake detections; it removes an inference runtime cap so persisted QA/QC can measure candidate models honestly.
|
|
|
|
Tested:
|
|
- Red step: targeted YOLO adapter tests failed because `max_det` was not passed to the model.
|
|
- `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py::test_yolo_adapter_converts_single_band_tiles_to_rgb_before_prediction backend\tests\test_sprint8b_yolo_foundation.py::test_yolo_adapter_uses_configured_max_detections -q` (`2 passed`)
|
|
- Red step: Docker runtime config tests failed before `.env.example` and Unraid runner exposed `YOLO_MAX_DETECTIONS`.
|
|
- `python -m pytest backend\tests\test_docker_runtime_config.py::test_env_example_uses_runtime_env_names_read_by_backend_and_frontend backend\tests\test_docker_runtime_config.py::test_unraid_deploy_passes_ai_build_arg_and_yolo_runtime_env backend\tests\test_sprint8b_yolo_foundation.py -q` (`16 passed`)
|
|
- `bash scripts/run_readiness_check.sh` (`425 passed`, frontend typecheck/build passed)
|
|
- Redeployed Tower all-in-one image with AI dependencies and verified `YOLO_MAX_DETECTIONS=1000` in the live container.
|
|
- Tower live migration smoke passed against embedded PostGIS.
|
|
- Live Westerlo calibration with `geointel-building-yolov8s-aoi512e80-pt`:
|
|
- `0.25`: 270 detections, F1 `0.23509933774834438`
|
|
- `0.15`: 523 detections, F1 `0.19603267211201864`
|
|
- `0.05`: 1000 detections, F1 `0.13193403298350823`
|
|
- Live Turnhout calibration with `geointel-building-yolov8s-aoi512e80-pt`:
|
|
- `0.25`: 822 detections, F1 `0.1304075235109718`
|
|
- `0.15`: 1000 detections, F1 `0.13085166384658772`
|
|
- `0.05`: 1000 detections, F1 `0.13085166384658772`
|
|
|
|
Conclusion:
|
|
- The former 300-detection runtime cap is removed; dense AOIs can now persist more candidates.
|
|
- The current AOI512 YOLOv8s candidate remains rejected for operational use because precision/recall quality is still too low and low thresholds saturate the configured `1000` cap.
|
|
- Next model work should focus on training data coverage, label strategy and post-processing/NMS behavior rather than only threshold lowering.
|
|
|
|
## Sprint 147 AOI512 YOLOv8s scale-match candidate gate (2026-07-09)
|
|
|
|
Changed:
|
|
- Built an alternate operator YOLO dataset at AOI scale instead of small 160px tiles:
|
|
- output `/app/storage/operator-data/yolo-building-aoi512-uniquehardneg`
|
|
- tile size `512`, stride `512`
|
|
- validation samples `turnhout`, `retie`, `westerlo`, `arendonk_heide`
|
|
- 16 total images, 13 positive images, 3 negative images, 3172 labels
|
|
- Audited the AOI512 dataset:
|
|
- output `/mnt/user/appdata/geointel/artifacts/operator-yolo-dataset-audits/aoi512-uniquehardneg/operator_yolo_dataset_quality_audit.json`
|
|
- status `ok`
|
|
- no missing label files
|
|
- no invalid label rows
|
|
- 13 positive samples
|
|
- 3 positive validation samples
|
|
- median normalized box area `0.000793456875`
|
|
- Trained a Tower-local YOLOv8s scale-match candidate:
|
|
- dataset `/app/storage/operator-data/yolo-building-aoi512-uniquehardneg/dataset.yaml`
|
|
- base model `/app/models/yolov8s.pt`
|
|
- 80 CPU epochs
|
|
- image size `512`
|
|
- batch `4`
|
|
- artifact `/app/models/geointel-building-yolov8s-aoi512e80.pt`
|
|
- model asset id `geointel-building-yolov8s-aoi512e80-pt`
|
|
- SHA256 `b796284a13358498c296fa41270dd234a81718a76cdafe62490a5206bb86ac5b`
|
|
- The training validation improved versus the previous 160px candidate but remained weak:
|
|
- precision `0.404`
|
|
- recall `0.270`
|
|
- mAP50 `0.163`
|
|
- mAP50-95 `0.0505`
|
|
|
|
Tested:
|
|
- Ran 7-AOI positive calibration sweeps for Geel, Mol, Turnhout, Herentals, Balen, Retie and Westerlo:
|
|
- output root `/mnt/user/appdata/geointel/artifacts/detection-calibration/aoi512e80-positive`
|
|
- best sample: Westerlo threshold `0.25`, F1 `0.23509933774834438`, precision `0.26296296296296295`, recall `0.2125748502994012`, detections `270`
|
|
- most other AOIs remained around F1 `0.10` to `0.14`
|
|
- Ran 9-sample hard-negative/background matrix:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/aoi512e80/hard_negative_matrix_summary.json`
|
|
- threshold `0.25`: total background detections `105`, max sample detections `56`
|
|
- threshold `0.15`: total background detections `181`, max sample detections `100`
|
|
- threshold `0.05`: total background detections `503`, max sample detections `278`
|
|
- Assembled positive evidence portfolio:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/aoi512e80-positive/calibration_evidence_portfolio.json`
|
|
- sample count `7`
|
|
- evidence features `17156`
|
|
- Ran promotion report:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/aoi512e80-positive-vs-hard-negative/detection_model_promotion_report.json`
|
|
- threshold `0.05`: rejected for `positive_mean_f1_below_gate` and `background_false_positive_pressure`, mean F1 `0.1335249717919908`, max background detections `278`
|
|
- threshold `0.15`: rejected for the same reasons, mean F1 `0.1335249717919908`, max background detections `100`
|
|
- threshold `0.25`: rejected for the same reasons, mean F1 `0.13511851520077328`, max background detections `56`
|
|
- recommended candidate `none`
|
|
|
|
Open:
|
|
- Do not activate `geointel-building-yolov8s-aoi512e80-pt` as the V1 default.
|
|
- AOI-scale training improves the Ultralytics validation curve but does not improve persisted QA/QC enough for operational use.
|
|
- The recurring failure mode is overproduction near the 300-detection cap with low recall and too many false positives.
|
|
- Next recommended pass: add/export more diverse positive AOIs and improve label quality/geometry-to-box strategy before training another higher-capacity model. A pure scale or epoch change is not enough.
|
|
|
|
## Sprint 146 Unique hard-negative YOLOv8s candidate gate (2026-07-09)
|
|
|
|
Changed:
|
|
- Fixed the all-in-one Docker runtime so `/app/scripts/train_operator_yolo_detector.sh` is copied into the image and made executable.
|
|
- Added regression coverage in `backend/tests/test_docker_runtime_config.py` so the all-in-one image must include the operator YOLO training wrapper.
|
|
- Hardened `scripts/build_detection_model_promotion_report.py` with explicit `--default-positive-tile-size` and `--default-positive-tile-overlap` options for older positive evidence portfolios that record the model at portfolio level but omit per-run tile provenance.
|
|
- Added regression coverage in `backend/tests/test_sprint143_detection_model_promotion_report.py` for portfolio-level model fallback plus explicit positive tile defaults.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_for_runtime_use -q` failed because the all-in-one Dockerfile did not copy the training wrapper.
|
|
- `python -m pytest backend\tests\test_docker_runtime_config.py -q` (`22 passed`)
|
|
- `bash scripts/run_readiness_check.sh` (`423 passed`; frontend typecheck/build passed; Alembic head `202606120900`)
|
|
- Pushed commit `afd2cba` and redeployed Tower with `.\scripts\deploy_tower.ps1 -InstallAi true`; browser runtime verification and live migration smoke passed.
|
|
- Verified the wrapper in the live container: `/app/scripts/train_operator_yolo_detector.sh` exists and is executable.
|
|
- Trained `geointel-building-yolov8s-uniquehardneg160e50-pt` on Tower from `/app/storage/operator-data/yolo-building-tile-uniquehardneg160/dataset.yaml` using local `/app/models/yolov8s.pt`, 50 CPU epochs, image size `160`, batch `8`.
|
|
- Training completed with final validation precision `0.38`, recall `0.365`, mAP50 `0.278`, mAP50-95 `0.0803`.
|
|
- Live model asset:
|
|
- `/app/models/geointel-building-yolov8s-uniquehardneg160e50.pt`
|
|
- model asset id `geointel-building-yolov8s-uniquehardneg160e50-pt`
|
|
- SHA256 `c4e480273d3da5fc27532cd8bdc3fa7786582e06848ea11b56714ab8cb1750b3`
|
|
- Ran 7-AOI positive calibration sweeps for Geel, Mol, Turnhout, Herentals, Balen, Retie and Westerlo:
|
|
- output root `/mnt/user/appdata/geointel/artifacts/detection-calibration/uniquehardneg160e50-positive`
|
|
- best observed AOI result: Westerlo threshold `0.25`, F1 `0.384180790960452`, precision `0.5177664974619289`, recall `0.30538922155688625`, detections `197`
|
|
- other positive AOIs remained weak, with best F1 roughly `0.10` to `0.17`.
|
|
- Assembled positive evidence portfolio:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/uniquehardneg160e50-positive/calibration_evidence_portfolio.json`
|
|
- sample count `7`
|
|
- evidence features `17008`
|
|
- role counts `false_negative=9710`, `false_positive=4734`, `match_candidate=1282`, `match_reference=1282`
|
|
- Ran 9-sample hard-negative/background matrix:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/uniquehardneg160e50/hard_negative_matrix_summary.json`
|
|
- threshold `0.25`: total background detections `98`, max sample detections `58`
|
|
- threshold `0.15`: total background detections `137`, max sample detections `85`
|
|
- threshold `0.05`: total background detections `276`, max sample detections `172`
|
|
- Red step: `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py -q` failed because the promotion report could not yet accept explicit positive tile defaults.
|
|
- `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py -q` (`2 passed`)
|
|
- Rebuilt the promotion report using explicit positive tile defaults:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/uniquehardneg160e50-positive-vs-hard-negative-v2/detection_model_promotion_report.json`
|
|
- threshold `0.05`: rejected for `positive_mean_f1_below_gate` and `background_false_positive_pressure`, mean F1 `0.15797188547918842`, max background detections `172`
|
|
- threshold `0.15`: rejected for the same reasons, mean F1 `0.1598974047548654`, max background detections `85`
|
|
- threshold `0.25`: rejected for the same reasons, mean F1 `0.15967766715169612`, max background detections `58`
|
|
- recommended candidate `none`
|
|
|
|
Open:
|
|
- Do not activate `geointel-building-yolov8s-uniquehardneg160e50-pt` as the V1 default.
|
|
- The candidate improves some individual AOIs but still combines low mean positive F1 with unacceptable false-positive pressure on background samples.
|
|
- Next model work should focus on materially better training data/model strategy, not another default activation attempt with this artifact.
|
|
|
|
## Sprint 145 YOLOv8s hardneg r8 e60 full candidate evaluation (2026-07-08)
|
|
|
|
Changed:
|
|
- Verified that the Tower-local YOLOv8s hardneg r8 training run completed all 60 requested CPU epochs.
|
|
- Finalized the completed model artifact:
|
|
- `/app/models/geointel-building-yolov8s-hardneg160r8e60.pt`
|
|
- model asset id `geointel-building-yolov8s-hardneg160r8e60-pt`
|
|
- SHA256 `d7daea04bd51a54a06944f0d4bf1961fe453dceb1fb34ef51daa73f6901fca81`
|
|
- Wrote `/app/storage/training/operator-yolo/geointel-building-yolov8s-hardneg160r8e60/training_summary.json`.
|
|
|
|
Tested:
|
|
- Training summary reports `status=ok`, requested epochs `60`, completed epochs `60`.
|
|
- Ran 7-AOI positive matrix:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/yolov8s-hardneg160r8e60-positive-20260708/multi_sample_quality_summary.json`
|
|
- sample count `7`
|
|
- run count `21`
|
|
- best result: Westerlo threshold `0.15`, F1/score `0.20588235294117646`, precision `0.26666666666666666`, recall `0.16766467065868262`, detections `210`, false positives `154`, false negatives `278`.
|
|
- Ran hard-negative matrix:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/yolov8s-hardneg160r8e60-live/hard_negative_matrix_summary.json`
|
|
- Postel-bos detections `0/0/0` at thresholds `0.05/0.15/0.25`
|
|
- Lommel-heide detections `0/0/0`
|
|
- Kasterlee-bos detections `30/11/9`
|
|
- Assembled evidence portfolio:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/yolov8s-hardneg160r8e60-positive-20260708/output/calibration_evidence_portfolio.json`
|
|
- sample count `7`
|
|
- evidence features `15714`
|
|
- role counts `false_negative=10468`, `false_positive=4198`, `match_candidate=524`, `match_reference=524`
|
|
- Ran promotion report:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/yolov8s-hardneg160r8e60-20260708/detection_model_promotion_report.json`
|
|
- evaluated `3` candidate thresholds
|
|
- recommended candidate `none`
|
|
- threshold `0.05`: mean F1 `0.07825931710275633`, max background detections `30`, rejected for `positive_mean_f1_below_gate` and `background_false_positive_pressure`
|
|
- threshold `0.15`: mean F1 `0.07870592446136859`, max background detections `11`, rejected for the same reasons
|
|
- threshold `0.25`: mean F1 `0.06111741186503092`, max background detections `9`, rejected for the same reasons
|
|
- Comparison baseline: `geointel-building-yolov8n-expanded160e50-pt|640|64|0.15` had mean F1 `0.19718007234060198` and max background detections `46`.
|
|
|
|
Open:
|
|
- The e60 YOLOv8s hardneg r8 candidate improves hard-negative behavior at threshold `0.15` compared with `expanded160e50`, but its positive-AOI mean F1 is too low for operational use.
|
|
- Do not activate this model as default.
|
|
- Next model pass should focus on improving positive recall/fit without losing the hard-negative gains, likely through better labels, more positive AOIs, or a different architecture/training strategy rather than simply extending this same run.
|
|
|
|
## Sprint 144 YOLOv8s hardneg r8 partial candidate evaluation (2026-07-08)
|
|
|
|
Changed:
|
|
- Started a Tower-local YOLOv8s training run using:
|
|
- dataset `/app/storage/operator-data/yolo-building-tile-hardneg160r8/dataset.yaml`
|
|
- base model `/app/models/yolov8s.pt`
|
|
- requested epochs `60`
|
|
- image size `640`
|
|
- batch `2`
|
|
- device `cpu`
|
|
- The Codex command reached its 1-hour timeout after 12 completed epochs; the run had produced `weights/best.pt` and `weights/last.pt`.
|
|
- Preserved the partial best artifact as `/app/models/geointel-building-yolov8s-hardneg160r8e12partial.pt`.
|
|
- Wrote `/app/storage/training/operator-yolo/geointel-building-yolov8s-hardneg160r8e60/training_summary_partial_e12.json`.
|
|
- Treated the artifact explicitly as a partial evaluation candidate, not as a completed 60-epoch model.
|
|
|
|
Tested:
|
|
- Live model catalog listed `geointel-building-yolov8s-hardneg160r8e12partial-pt` with SHA256 `0246202cddc47eb994a0afc9ee10d56b72298bd1cdc0b72b75e12b28e2202330`.
|
|
- Ran 7-AOI positive matrix:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/yolov8s-hardneg160r8e12partial-positive-20260708/multi_sample_quality_summary.json`
|
|
- sample count `7`
|
|
- run count `21`
|
|
- best result: Westerlo threshold `0.05`, F1/score `0.14826498422712936`, precision `0.15666666666666668`, recall `0.1407185628742515`, detections `300`, false positives `253`, false negatives `287`.
|
|
- Ran hard-negative matrix:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/yolov8s-hardneg160r8e12partial-live/hard_negative_matrix_summary.json`
|
|
- Postel-bos detections `0/0/0` at thresholds `0.05/0.15/0.25`
|
|
- Lommel-heide detections `0/0/0`
|
|
- Kasterlee-bos detections `18/1/0`
|
|
- Assembled evidence portfolio:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/yolov8s-hardneg160r8e12partial-positive-20260708/output/calibration_evidence_portfolio.json`
|
|
- sample count `7`
|
|
- evidence features `12908`
|
|
- role counts `false_negative=10861`, `false_positive=1785`, `match_candidate=131`, `match_reference=131`
|
|
- Ran promotion report:
|
|
- output `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/yolov8s-hardneg160r8e12partial-20260708/detection_model_promotion_report.json`
|
|
- evaluated `3` candidate thresholds
|
|
- recommended candidate `none`
|
|
- threshold `0.05` rejected for `positive_mean_f1_below_gate` and `background_false_positive_pressure`, with mean F1 `0.05026994383963278` and max background detections `18`
|
|
- threshold `0.15` rejected for the same reasons, with mean F1 `0.0022606965174129354` and max background detections `1`
|
|
- threshold `0.25` rejected for insufficient positive evidence and positive F1 below gate.
|
|
|
|
Open:
|
|
- The partial YOLOv8s r8 candidate is materially worse than the existing `expanded160e50` positive-AOI baseline and must not be activated.
|
|
- CPU-only training is too slow for a complete 60-epoch YOLOv8s pass inside a 1-hour interactive command window.
|
|
- Next pass should either resume/finish long-running training outside the interactive timeout or use GPU/accelerated runtime; only then rerun the same positive, hard-negative, evidence portfolio and promotion gates.
|
|
|
|
## Sprint 143 Detection model promotion decision report (2026-07-08)
|
|
|
|
Changed:
|
|
- Added `scripts/build_detection_model_promotion_report.py` as operator-only evidence tooling.
|
|
- The script combines a positive-AOI `calibration_evidence_portfolio.json` with one or more `hard_negative_matrix_summary.json` files.
|
|
- Candidate rows are grouped by `model_asset_id`, `tile_size`, `tile_overlap` and `threshold`.
|
|
- Promotion gates are explicit:
|
|
- minimum positive sample count
|
|
- minimum background sample count
|
|
- minimum mean positive F1
|
|
- maximum background detections per sample
|
|
- Added readiness `py_compile` coverage for the new script.
|
|
- Documented the Tower command in `scripts/README.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py -q` failed because the report script did not exist.
|
|
- `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py -q` (`1 passed`)
|
|
- `python -m pytest backend\tests\test_sprint143_detection_model_promotion_report.py backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` (`2 passed`)
|
|
- `python -m py_compile scripts\build_detection_model_promotion_report.py`
|
|
- `python scripts\smoke_docs.py`
|
|
- `bash scripts/run_readiness_check.sh` (`419 passed`; frontend typecheck/build passed; Alembic head `202606120900`)
|
|
- Tower pulled commit `c2fba67` and generated:
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/positive-expanded-vs-hard-negative-20260708/detection_model_promotion_report.json`
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/positive-expanded-vs-hard-negative-20260708/detection_model_promotion_report.md`
|
|
- Live promotion report evaluated 15 model/tile/threshold candidates and recommended `none`.
|
|
- Best current positive candidate `geointel-building-yolov8n-expanded160e50-pt|640|64|0.15` was rejected by gates:
|
|
- `positive_mean_f1_below_gate`
|
|
- `background_false_positive_pressure`
|
|
- positive samples `7`
|
|
- background samples `3`
|
|
- mean F1 `0.19718007234060198`
|
|
- max background detections `46`
|
|
- total background detections `46`
|
|
- Tower container remained healthy on `0.0.0.0:1202->80/tcp`.
|
|
|
|
Open:
|
|
- No evaluated model/threshold is ready for default promotion.
|
|
- Next model work should improve positive recall/F1 while preserving a strict hard-negative false-positive gate.
|
|
|
|
## Sprint 141 Expanded positive-AOI matrix and portfolio metadata hardening (2026-07-08)
|
|
|
|
Changed:
|
|
- Ran a fresh Tower multi-sample quality matrix for additional positive AOIs `balen`, `herentals` and `westerlo` using:
|
|
- `geointel-building-yolov8n-expanded160e50-pt`
|
|
- `geointel-building-yolov8n-hardneg160r8e40-pt`
|
|
- tile size `640`, overlap `64`, thresholds `0.15` and `0.05`.
|
|
- Assembled a broader positive-AOI evidence portfolio across 7 AOIs:
|
|
- Geel, Mol, Turnhout and Retie from `expanded160e50-live`.
|
|
- Balen, Herentals and Westerlo from the fresh `balen-herentals-westerlo-live-20260708` run.
|
|
- Hardened `scripts/export_detection_calibration_evidence.sh` so evidence bundles preserve calibration provenance fields in run summaries and GeoJSON properties:
|
|
- `model_asset_id`
|
|
- `model_request`
|
|
- `tile_size`
|
|
- `tile_overlap`
|
|
- Extended `backend/tests/test_sprint139_multi_aoi_calibration_evidence_portfolio.py` to assert portfolio best-run model/tile provenance is retained.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` failed with `KeyError: 'model_asset_id'` while bundle summaries dropped model provenance.
|
|
- `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` (`1 passed`)
|
|
- `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py backend\tests\test_sprint138_calibration_evidence_bundle_smoke.py backend\tests\test_sprint137_browser_calibration_summary_evidence_script.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py -q` (`4 passed`)
|
|
- `bash -n scripts/export_detection_calibration_evidence.sh`
|
|
- `bash -n scripts/assemble_detection_calibration_evidence_portfolio.sh`
|
|
- `python -m compileall backend/app`
|
|
- `bash scripts/run_readiness_check.sh` (`418 passed`; frontend typecheck/build passed; Alembic head `202606120900`; shell syntax gates passed)
|
|
- Tower fresh matrix completed for Balen, Herentals and Westerlo.
|
|
- Tower positive-AOI portfolio assembly completed for 7 AOIs.
|
|
- Tower runtime remained healthy on `0.0.0.0:1202->80/tcp`.
|
|
|
|
Evidence:
|
|
- Fresh Balen/Herentals/Westerlo output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/balen-herentals-westerlo-live-20260708/multi_sample_quality_summary.json`.
|
|
- Expanded 7-AOI portfolio output: `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/positive-aoi-expanded-20260708/output/calibration_evidence_portfolio.json`.
|
|
- Expanded positive-AOI portfolio sample count: `7`.
|
|
- Expanded positive-AOI evidence features: `12438`.
|
|
- Expanded positive-AOI role counts: `false_negative=9200`, `false_positive=2270`, `match_candidate=484`, `match_reference=484`.
|
|
- Best fresh positive-AOI result: Westerlo with `geointel-building-yolov8n-expanded160e50-pt`, threshold `0.05`, F1/score `0.3659305993690852`, precision `0.38666666666666666`, recall `0.3473053892215569`.
|
|
- Fresh Balen best: expanded160e50 threshold `0.05`, F1/score `0.16091954022988506`, precision `0.16333333333333333`, recall `0.15857605177993528`.
|
|
- Fresh Herentals best: expanded160e50 threshold `0.05`, F1/score `0.14093264248704665`, precision `0.22666666666666666`, recall `0.10225563909774436`.
|
|
- Hard-negative context from existing matrices:
|
|
- expanded160e50 at `0.05`: Kasterlee-bos `76`, Lommel-heide `10`, Postel-bos `1` detections.
|
|
- hardneg160r8e40 at `0.05`: Kasterlee-bos `25`, Lommel-heide `0`, Postel-bos `0` detections.
|
|
|
|
Open:
|
|
- expanded160e50 is stronger on positive AOIs, especially Westerlo, but still produces many false positives and misses many references.
|
|
- hardneg160r8e40 is cleaner on hard-negative AOIs but materially weaker on the fresh positive AOIs.
|
|
- No current candidate should be promoted blindly as V1 default without a combined positive/hard-negative decision rule.
|
|
|
|
Limitations:
|
|
- This pass ran additional live inference/QA workflows and produced operator artifacts, but did not change backend APIs, migrations, frontend runtime behavior, model weights, provider fetching or Docker runtime configuration.
|
|
- The 7-AOI portfolio initially generated before the metadata fix lacked model/tile provenance in `best_run_by_score`; it should be regenerated after Tower pulls this commit.
|
|
|
|
Next recommended pass:
|
|
- Add a model promotion decision report that combines positive-AOI F1/recall and hard-negative false-positive pressure into one explicit accept/reject table per model and threshold.
|
|
|
|
## Sprint 140 Live multi-AOI calibration portfolio run (2026-07-08)
|
|
|
|
Changed:
|
|
- Created a Tower-local `calibration-evidence-portfolio-manifest.json` for existing persisted multi-sample quality matrix summaries:
|
|
- Geel: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/geel/quality_matrix_summary.json`
|
|
- Mol: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/mol/quality_matrix_summary.json`
|
|
- Turnhout: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/turnhout/quality_matrix_summary.json`
|
|
- Ran `scripts/assemble_detection_calibration_evidence_portfolio.sh` against the live Tower app at `http://127.0.0.1:1202`.
|
|
- Produced the first real multi-AOI calibration evidence handoff:
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/live-20260708/output/calibration_evidence_portfolio.json`
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/live-20260708/output/calibration_evidence_portfolio.md`
|
|
- per-sample `calibration_evidence.geojson`, `calibration_evidence_summary.json` and `calibration_evidence_review.html` artifacts for Geel, Mol and Turnhout.
|
|
|
|
Tested:
|
|
- Tower manifest validation passed with `python3 -m json.tool`.
|
|
- Live portfolio assembly passed against persisted QA evidence from the running app.
|
|
- Output validation passed: portfolio JSON/Markdown and all per-sample evidence GeoJSON/summary/review files exist and are non-empty.
|
|
- Tower runtime remained healthy on `0.0.0.0:1202->80/tcp`.
|
|
|
|
Evidence:
|
|
- Portfolio sample count: `3`.
|
|
- Total evidence features: `5509`.
|
|
- Combined role counts: `false_negative=5239`, `false_positive=164`, `match_candidate=53`, `match_reference=53`.
|
|
- Best sample by score: `mol`.
|
|
- Best per-sample scores:
|
|
- Geel: threshold `0.15`, F1/score `0.0187207488299532`, precision `0.25`, recall `0.009724473257698542`, evidence features `1885`.
|
|
- Mol: threshold `0.15`, F1/score `0.04195804195804196`, precision `0.16363636363636364`, recall `0.02406417112299465`, evidence features `1223`.
|
|
- Turnhout: threshold `0.15`, F1/score `0.03597122302158273`, precision `0.2459016393442623`, recall `0.019404915912031046`, evidence features `2401`.
|
|
|
|
Open:
|
|
- The live portfolio confirms the evidence pipeline works, but the evaluated model/threshold set is not yet operational-quality for building extraction due to very low recall and a dominant false-negative count.
|
|
|
|
Limitations:
|
|
- This pass produced operator artifacts only. It did not rerun inference, mutate application data, change database state, add endpoints, change frontend runtime behavior, download models, train models or rebuild the Docker app.
|
|
- The portfolio input came from the existing `20260707T025932Z` quality matrix; the next model review should use fresh matrices when new AOIs or model candidates are added.
|
|
|
|
Next recommended pass:
|
|
- Add more positive and hard-negative AOIs, then run a fresh multi-sample quality matrix and portfolio for the next model candidate. Do not promote the current evaluated model as a V1 default.
|
|
|
|
## Sprint 139 Multi-AOI calibration evidence portfolio (2026-07-08)
|
|
|
|
Changed:
|
|
- Added `scripts/assemble_detection_calibration_evidence_portfolio.sh` for packaging multiple AOI calibration summaries and their persisted QA evidence bundles into one model-review portfolio.
|
|
- The assembler reads `calibration-evidence-portfolio-manifest.json`, copies each AOI summary into a sample folder, runs the existing `scripts/export_detection_calibration_evidence.sh` exporter per sample and writes:
|
|
- `calibration_evidence_portfolio.json`
|
|
- `calibration_evidence_portfolio.md`
|
|
- Added optional `CURL_BIN` support to `scripts/export_detection_calibration_evidence.sh` so operator smokes/tests can inject a deterministic endpoint mock while defaulting to normal `curl`.
|
|
- Added readiness syntax coverage and operator docs for the manifest convention.
|
|
- Updated `scripts/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint139_multi_aoi_calibration_evidence_portfolio.py`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` failed while `scripts/assemble_detection_calibration_evidence_portfolio.sh` was absent.
|
|
- `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` (`1 passed`)
|
|
- `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py backend\tests\test_sprint138_calibration_evidence_bundle_smoke.py backend\tests\test_sprint137_browser_calibration_summary_evidence_script.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py -q` (`4 passed`)
|
|
- `bash -n scripts/assemble_detection_calibration_evidence_portfolio.sh`
|
|
- `bash scripts/assemble_detection_calibration_evidence_portfolio.sh --help`
|
|
- `bash -n scripts/export_detection_calibration_evidence.sh`
|
|
- `bash scripts/export_detection_calibration_evidence.sh --help`
|
|
- `python -m compileall backend/app`
|
|
- `bash scripts/run_readiness_check.sh` (`418 passed`; frontend typecheck/build passed; Alembic head `202606120900`; shell syntax gates passed)
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is local/operator evidence packaging only. It does not run inference, call live production data by itself, mutate application data, add backend endpoints, change migrations, create QA metrics, promote thresholds, download models, add provider fetching or change frontend runtime behavior.
|
|
- The regression test uses mocked canonical QA evidence responses; real persisted QA evidence remains validated by running the portfolio assembler against live Detection Lab or calibration-sweep summaries.
|
|
|
|
Next recommended pass:
|
|
- Run the portfolio assembler against the existing Tower calibration summaries for at least two real AOIs, then use the portfolio JSON/Markdown as the first model-review handoff artifact before any further training or threshold promotion.
|
|
|
|
## Sprint 138 Browser calibration evidence bundle smoke (2026-07-08)
|
|
|
|
Changed:
|
|
- Added `scripts/smoke_detection_calibration_evidence_bundle.sh` as a local operator smoke for the Detection Lab `detection-calibration-summary.json` to QA evidence bundle path.
|
|
- The smoke creates a temporary browser-style calibration summary, injects a temporary mock `curl` for canonical QA evidence endpoint responses, runs the real `scripts/export_detection_calibration_evidence.sh` exporter and validates the generated GeoJSON, summary JSON and HTML review artifacts.
|
|
- Added readiness syntax coverage for the smoke script.
|
|
- Updated `scripts/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint138_calibration_evidence_bundle_smoke.py`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint138_calibration_evidence_bundle_smoke.py -q` failed while `scripts/smoke_detection_calibration_evidence_bundle.sh` was absent.
|
|
- `python -m pytest backend\tests\test_sprint138_calibration_evidence_bundle_smoke.py -q` (`1 passed`)
|
|
- `python -m pytest backend\tests\test_sprint138_calibration_evidence_bundle_smoke.py backend\tests\test_sprint137_browser_calibration_summary_evidence_script.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py -q` (`3 passed`)
|
|
- `bash -n scripts/smoke_detection_calibration_evidence_bundle.sh`
|
|
- `bash scripts/smoke_detection_calibration_evidence_bundle.sh --help`
|
|
- `python -m compileall backend/app`
|
|
- `bash scripts/run_readiness_check.sh` (`417 passed`; frontend typecheck/build passed; Alembic head `202606120900`; shell syntax gates passed)
|
|
- Pushed commit `47d3587` to `main`.
|
|
- Tower repo fast-forwarded to `47d3587`; `bash -n scripts/smoke_detection_calibration_evidence_bundle.sh` passed on Tower.
|
|
- Tower operator smoke passed: `bash scripts/smoke_detection_calibration_evidence_bundle.sh` wrote mocked evidence artifacts under `/mnt/user/appdata/geointel/artifacts/detection-calibration-smoke/20260708T115724Z/evidence` with 4 evidence features across matched candidate, matched reference, false positive and false negative roles.
|
|
- Tower runtime remained healthy on `0.0.0.0:1202->80/tcp`; no app rebuild was required because this pass changed operator scripts/docs/tests only.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is local/operator evidence tooling only. It does not call live production data, mutate application data, add backend endpoints, change migrations, rerun inference, create QA metrics, promote thresholds, download models, add provider fetching or change frontend runtime behavior.
|
|
- The smoke uses mocked canonical evidence responses by design; real persisted QA evidence is still validated by running `export_detection_calibration_evidence.sh` against a live browser or sweep summary.
|
|
|
|
Next recommended pass:
|
|
- Add a real multi-AOI calibration evidence capture convention: one folder per AOI/model/threshold matrix with browser summary, evidence bundle and operator notes, so model promotion decisions are based on comparable persisted artifacts rather than isolated runs.
|
|
|
|
## Sprint 137 Browser calibration summary evidence bundle handoff (2026-07-08)
|
|
|
|
Changed:
|
|
- Extended `scripts/export_detection_calibration_evidence.sh` so it can consume Detection Lab `detection-calibration-summary.json` browser exports as well as the existing operator `calibration_summary.json` format.
|
|
- Added summary normalization for browser-exported `rows`, root `project_id`, persisted `quality_check_id` values and `CALIBRATION_EVIDENCE_MODE=best` fallback selection.
|
|
- Updated `scripts/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint137_browser_calibration_summary_evidence_script.py`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint137_browser_calibration_summary_evidence_script.py -q` failed while browser summary support was absent.
|
|
- `python -m pytest backend\tests\test_sprint137_browser_calibration_summary_evidence_script.py backend\tests\test_sprint136_calibration_summary_export_ui.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py -q` (`3 passed`)
|
|
- `bash -n scripts/export_detection_calibration_evidence.sh`
|
|
- `python -m compileall backend/app`
|
|
- `bash scripts/run_readiness_check.sh` (`416 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy from commit `673f6c6` completed with `GEOINTEL_INSTALL_AI=true`; the all-in-one container is healthy and published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 4; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon after one readiness retry.
|
|
- Live container check passed: `docker ps --filter name=geointel` reported `geointel` as healthy on `0.0.0.0:1202->80/tcp`.
|
|
- Live YOLO preflight passed with local model configured, dependencies available, `torch_version=2.12.1`, `ultralytics_version=8.4.90`, `will_download_models=false`, `will_run_inference=false` and `status=manifest_unavailable` because no tile manifest was supplied.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is operator evidence tooling only. It does not add backend endpoints, change migrations, rerun inference, create new QA metrics, promote thresholds, mutate model configuration, download models, add provider fetching or change frontend runtime behavior.
|
|
|
|
Next recommended pass:
|
|
- Add a tiny local fixture smoke for the evidence bundle script that uses a saved browser-style summary plus mocked canonical evidence responses, so the bundle renderer itself is tested beyond static contract checks.
|
|
|
|
## Sprint 136 Guided calibration summary export (2026-07-08)
|
|
|
|
Changed:
|
|
- Added a `Download calibration summary` action to the guided Detection Lab calibration progress table.
|
|
- The client-side JSON export includes calibration thresholds, persisted `analysis_run_id`, `job_id`, `quality_check_id`, metric values and QA evidence GeoJSON URLs.
|
|
- Added compact action-row styling and regression coverage in `backend/tests/test_sprint136_calibration_summary_export_ui.py`.
|
|
- Updated `CHANGELOG.md` and `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint136_calibration_summary_export_ui.py -q` failed while the summary export helpers and button were absent.
|
|
- `python -m pytest backend\tests\test_sprint136_calibration_summary_export_ui.py backend\tests\test_sprint135_calibration_evidence_handoff.py backend\tests\test_sprint134_guided_detection_calibration_runner.py -q` (`3 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`415 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy from commit `8089df3` completed with `GEOINTEL_INSTALL_AI=true`; the all-in-one container is published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live bundle check passed: `Download calibration summary` is present in `assets/index-5CClxMR_.js`.
|
|
- Live YOLO preflight passed with local model configured, dependencies available, `will_download_models=false`, `will_run_inference=false` and `status=manifest_unavailable` because no tile manifest was supplied.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is a browser-side summary export only. It does not create server-side export records, rerun inference, create new QA metrics, promote thresholds, mutate model configuration, download models, add provider fetching or change API/database contracts.
|
|
|
|
Next recommended pass:
|
|
- Add a small import/consume path for downloaded calibration summaries in the existing operator evidence bundle script, so browser-exported summary JSON can be used directly from an operator workstation.
|
|
|
|
## Sprint 135 Calibration evidence handoff (2026-07-08)
|
|
|
|
Changed:
|
|
- Added an `Open evidence map` action to successful guided detection calibration rows.
|
|
- Wired the Detection Lab action to the existing `openQualityEvidenceOnMap` flow, which loads persisted QA/QC evidence GeoJSON and opens the Map workspace overlay.
|
|
- Added compact table action styling and regression coverage in `backend/tests/test_sprint135_calibration_evidence_handoff.py`.
|
|
- Updated `CHANGELOG.md` and `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint135_calibration_evidence_handoff.py -q` failed while the Detection Lab evidence handoff prop was absent.
|
|
- `python -m pytest backend\tests\test_sprint135_calibration_evidence_handoff.py backend\tests\test_sprint134_guided_detection_calibration_runner.py backend\tests\test_sprint133_detection_threshold_calibration_ux.py backend\tests\test_sprint112_qa_evidence_overlay.py -q` (`7 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`414 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy from commit `0c826ef` completed with `GEOINTEL_INSTALL_AI=true`; the all-in-one container is published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This pass adds review handoff only. It does not create new QA metrics, promote thresholds, mutate model configuration, download models, add provider fetching or change API/database contracts.
|
|
|
|
Next recommended pass:
|
|
- Add an operator-facing calibration evidence summary/export shortcut once the map handoff has been used on real AOI runs.
|
|
|
|
## Sprint 134 External remote-sensing YOLO candidate benchmark (2026-07-07)
|
|
|
|
Changed:
|
|
- Added no production code and no repository-stored model weights.
|
|
- Evaluated the Hugging Face `agademer/yolo-remote-sensing-photovoltaic` model card as a candidate because it explicitly targets remote-sensing imagery and includes `building` as a class.
|
|
- Downloaded the YOLOv8l detection checkpoint manually as an operator/runtime asset:
|
|
- source model: `agademer/yolo-remote-sensing-photovoltaic`
|
|
- source file: `yolo-remote-sensing-photovoltaic-v8l-solar-farms-and-cities-v20260331-detect-1000_epochs.pt`
|
|
- runtime file: `/app/models/yolo-remote-sensing-photovoltaic-v8l-detect-1000.pt`
|
|
- catalog asset: `yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt`
|
|
- SHA256: `242ff4ab889569278f0eb9fcd22eb2c4bf2a52e48d05d89cc7cfa7941165d203`
|
|
- Updated `scripts/README.md`, `docs/TODO.md`, `docs/CODEX_EXECUTION_LOG.md` and `CHANGELOG.md` with benchmark evidence and the decision not to promote this model.
|
|
|
|
Tested:
|
|
- Live API preflight passed for `yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Live 45-run multi-sample QA matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/remote-sensing-v8l1000-live/multi_sample_quality_summary.json`
|
|
- command compared `yolo-remote-sensing-photovoltaic-v8l-detect-1000-pt`, `geointel-building-yolov8n-expanded160e50-pt` and `geointel-building-yolov8n-hardneg160r8e40-pt` over Geel, Mol, Turnhout, Retie and Kasterlee-bos with tile `640`, overlap `64`, thresholds `0.25`/`0.15`/`0.05`.
|
|
- dense-sample results for the external candidate:
|
|
- Geel: best F1 `0.0`
|
|
- Mol: best F1 `0.010582010582010581`
|
|
- Turnhout: best F1 `0.019070321811680575`
|
|
- Retie: best F1 `0.0`
|
|
- Kasterlee-bos sparse result: best F1 `0.25` with 1 detection, 1 match and 0 false positives at thresholds `0.25`/`0.15`.
|
|
- dense-sample winner remained `geointel-building-yolov8n-expanded160e50-pt`.
|
|
- Live 27-run hard-negative matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/remote-sensing-v8l1000-live/hard_negative_matrix_summary.json`
|
|
- Postel-bos: external candidate produced 0/1/3 detections at thresholds `0.25`/`0.15`/`0.05`.
|
|
- Lommel-heide: external candidate produced 0/0/0 detections.
|
|
- Kasterlee-bos: external candidate produced 1/2/5 detections, cleaner than expanded160e50 and hardneg160r8e40 on that sparse AOI.
|
|
|
|
Open:
|
|
- None for this benchmark pass.
|
|
|
|
Limitations:
|
|
- The external model is very conservative on the current Kempen orthophoto/GRB benchmark and misses most dense buildings.
|
|
- It may be useful as evidence for a high-precision/sparse-review mode, but it is not a viable V1 default extraction model.
|
|
- The runtime `.pt` remains outside Git and must be managed as an operator asset.
|
|
|
|
Next recommended pass:
|
|
- Train a higher-capacity local detector from a stronger base model using the existing operator tile dataset, then gate it with both dense QA and hard-negative matrices before any model default changes.
|
|
|
|
## Sprint 133 Hard-negative-balanced YOLO candidate (2026-07-07)
|
|
|
|
Changed:
|
|
- Hardened `scripts/export_operator_yolo_tile_dataset.py` with deterministic train-only background-negative repetition through `--background-negative-repeat` and `OPERATOR_YOLO_BACKGROUND_NEGATIVE_REPEAT`.
|
|
- Background-negative repetition applies only when `is_negative=true`, `sample_role=background_candidate` and `split=train`; validation tiles, positive tiles and normal reference samples are not duplicated.
|
|
- Added tile-level provenance fields `sample_role`, `repeat_index` and `is_repeated_background_negative`.
|
|
- Added regression coverage in `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`.
|
|
- Updated `scripts/README.md`, `docs/TODO.md`, `docs/CODEX_EXECUTION_LOG.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- RED: `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` failed before `--background-negative-repeat` and `background_negative_repeat_count` existed.
|
|
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` passed.
|
|
- `python -m py_compile scripts\export_operator_yolo_tile_dataset.py` passed.
|
|
- `python scripts\export_operator_yolo_tile_dataset.py --help` passed.
|
|
- Live Tower hard-negative-balanced tile export passed:
|
|
- dataset: `/app/storage/operator-data/yolo-building-tile-hardneg160r8`
|
|
- source samples: 10
|
|
- tile size: `160`
|
|
- stride: `80`
|
|
- negative keep ratio: `1.0`
|
|
- background negative repeat: `8`
|
|
- exported tiles: `864`
|
|
- positive tiles: `260`
|
|
- negative tiles: `604`
|
|
- labels: `11213`
|
|
- train tiles: `756`
|
|
- validation tiles: `108`
|
|
- Live Tower 40-epoch CPU training passed:
|
|
- output model: `/app/models/geointel-building-yolov8n-hardneg160r8e40.pt`
|
|
- catalog asset: `geointel-building-yolov8n-hardneg160r8e40-pt`
|
|
- SHA256: `7a77bd9f68e4c3927ffc8a8cd978a81067b02f42cffe77ada5334b5f8dbb6b50`
|
|
- final validation: precision `0.403`, recall `0.378`, mAP50 `0.301`, mAP50-95 `0.0944`
|
|
- Live API preflight passed for `geointel-building-yolov8n-hardneg160r8e40-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Live 60-run multi-sample QA matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/hardneg160r8e40-live/multi_sample_quality_summary.json`
|
|
- command compared `geointel-building-yolov8n-hardneg160r8e40-pt`, `geointel-building-yolov8n-expanded160e50-pt`, `geointel-building-yolov8n-tile30-pt` and `yolov8s-building-segmentation-pt` over Geel, Mol, Turnhout, Retie and Kasterlee-bos with tile `640`, overlap `64`, thresholds `0.25`/`0.15`/`0.05`.
|
|
- best overall score and recall remained Geel with `geointel-building-yolov8n-expanded160e50-pt`, precision `0.30333333333333334`, recall `0.14748784440842788`, F1 `0.1984732824427481`.
|
|
- hardneg160r8e40 dense F1 lagged expanded160e50 on Geel (`0.14394765539803708` vs `0.1984732824427481`), Mol (`0.11572700296735906` vs `0.1651651651651652`), Turnhout (`0.14911463187325258` vs `0.1938490214352283`) and Retie (`0.10538116591928251` vs `0.1569506726457399`).
|
|
- Live 36-run hard-negative matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/hardneg160r8e40-live/hard_negative_matrix_summary.json`
|
|
- Postel-bos: hardneg160r8e40 produced 0/0/0 detections at thresholds `0.25`/`0.15`/`0.05`; expanded160e50 produced 0/0/1.
|
|
- Lommel-heide: hardneg160r8e40 produced 0/0/0 detections; expanded160e50 produced 0/0/10.
|
|
- Kasterlee-bos: hardneg160r8e40 produced 5/9/25 detections; expanded160e50 produced 38/46/76.
|
|
|
|
Open:
|
|
- None for the hard-negative-balanced tile export contract itself.
|
|
|
|
Limitations:
|
|
- `geointel-building-yolov8n-hardneg160r8e40-pt` reduced false-positive pressure but regressed dense-AOI recall/F1. It should not become the V1 default.
|
|
- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
|
|
|
|
Next recommended pass:
|
|
- Train or import a materially stronger aerial/Kempen building model candidate, then benchmark it against the same dense QA and hard-negative matrices before changing default model selection.
|
|
|
|
## Sprint 132 Operator hard-negative detection matrix (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/run_operator_hard_negative_detection_matrix.sh`.
|
|
- The script reads `operator_samples_manifest.json`, selects samples marked `background_candidate` or `allow_empty_reference`, uploads only the raster, generates a tile manifest, checks configured-YOLO preflight, runs `POST /api/v1/detection/run` and counts persisted detections.
|
|
- It intentionally does not upload reference vectors and does not call detection QA/QC endpoints, because background AOIs have no meaningful reference target.
|
|
- Added readiness shell-syntax coverage for the new script.
|
|
- Added regression coverage in `backend/tests/test_sprint132_operator_hard_negative_matrix.py`.
|
|
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- RED: `python -m pytest backend\tests\test_sprint132_operator_hard_negative_matrix.py -q` failed while `scripts/run_operator_hard_negative_detection_matrix.sh` did not exist.
|
|
- `python -m pytest backend\tests\test_sprint132_operator_hard_negative_matrix.py -q` passed.
|
|
- `bash -n scripts/run_operator_hard_negative_detection_matrix.sh` passed.
|
|
- Live Tower 27-run hard-negative matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/expanded160e50-live/hard_negative_matrix_summary.json`
|
|
- samples: Postel-bos, Lommel-heide and Kasterlee-bos
|
|
- models: `geointel-building-yolov8n-expanded160e50-pt`, `geointel-building-yolov8n-tile30-pt`, `yolov8s-building-segmentation-pt`
|
|
- tile size: `640`
|
|
- overlap: `64`
|
|
- thresholds: `0.25`, `0.15`, `0.05`
|
|
- Live false-positive pressure results:
|
|
- Postel-bos: expanded160e50 produced 0 detections at `0.25`/`0.15`, 1 at `0.05`; tile30 produced 0/0/1; yolov8s produced 0/3/6.
|
|
- Lommel-heide: expanded160e50 produced 0 detections at `0.25`/`0.15`, 10 at `0.05`; tile30 produced 0/0/3; yolov8s produced 0/0/0.
|
|
- Kasterlee-bos: expanded160e50 produced 38/46/76 detections at `0.25`/`0.15`/`0.05`; tile30 produced 15/22/42; yolov8s produced 5/6/7.
|
|
|
|
Open:
|
|
- None for the hard-negative matrix tooling itself.
|
|
|
|
Limitations:
|
|
- Background matrix scores false-positive pressure from detection counts only. It does not calculate precision/recall/F1 because background candidates intentionally do not provide a full reference target.
|
|
- Kasterlee-bos still has 7 GRB features and is best interpreted as a sparse/hard-negative AOI, not a purely empty background tile.
|
|
- `geointel-building-yolov8n-expanded160e50-pt` should not be promoted to default model while Kasterlee-bos false-positive pressure remains high.
|
|
|
|
Next recommended pass:
|
|
- Train a hard-negative-balanced candidate: oversample sparse/background tiles, lower the dense-AOI max-detection bias, and rerun both dense QA matrix and hard-negative matrix before changing any default model selection.
|
|
|
|
## Sprint 131 Operator sample expansion and negative-tile YOLO candidate (2026-07-07)
|
|
|
|
Changed:
|
|
- Extended `scripts/prepare_operator_real_data_samples.py` with `sample_role` and `allow_empty_reference`.
|
|
- Added reference AOIs for Herentals, Balen, Retie and Westerlo.
|
|
- Added background-candidate AOIs for Postel-bos, Lommel-heide and Kasterlee-bos. Background candidates can persist empty GRB FeatureCollections for negative-tile training, while normal reference samples still fail on empty GRB results.
|
|
- Added regression coverage in `backend/tests/test_sprint131_operator_sample_expansion.py`.
|
|
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- RED: `python -m pytest backend\tests\test_sprint131_operator_sample_expansion.py -q` failed before the new sample metadata and background candidates existed.
|
|
- `python -m pytest backend\tests\test_sprint131_operator_sample_expansion.py -q` passed.
|
|
- `python -m py_compile scripts\prepare_operator_real_data_samples.py` passed.
|
|
- `python scripts\prepare_operator_real_data_samples.py --help` passed.
|
|
- Live Tower operator sample preparation passed:
|
|
- manifest: `/app/storage/operator-data/operator_samples_manifest.json`
|
|
- samples: Geel `617`, Mol `374`, Turnhout `773`, Herentals `665`, Balen `309`, Retie `592`, Westerlo `334`, Postel-bos `0`, Lommel-heide `0`, Kasterlee-bos `7` reference features.
|
|
- Live Tower expanded tile export passed:
|
|
- dataset: `/app/storage/operator-data/yolo-building-tile-expanded160`
|
|
- tile size: `160`
|
|
- stride: `80`
|
|
- exported tiles: `360`
|
|
- positive tiles: `260`
|
|
- negative tiles: `100`
|
|
- labels: `11213`
|
|
- train tiles: `252`
|
|
- validation tiles: `108`
|
|
- Live Tower 50-epoch CPU training passed:
|
|
- output model: `/app/models/geointel-building-yolov8n-expanded160e50.pt`
|
|
- catalog asset: `geointel-building-yolov8n-expanded160e50-pt`
|
|
- SHA256: `bf6a5e8d25a62d784ee53764ea11d7ce89c4e7aeeac7588010e497b8d7dafb2b`
|
|
- final validation: precision `0.428`, recall `0.389`, mAP50 `0.318`, mAP50-95 `0.106`
|
|
- Live API preflight passed for `geointel-building-yolov8n-expanded160e50-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Live 45-run multi-sample QA matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/expanded160e50-live/multi_sample_quality_summary.json`
|
|
- command compared `geointel-building-yolov8n-expanded160e50-pt`, `geointel-building-yolov8n-tile30-pt` and `yolov8s-building-segmentation-pt` over Geel, Mol, Turnhout, Retie and Kasterlee-bos with tile `640`, overlap `64`, thresholds `0.25`/`0.15`/`0.05`.
|
|
- best overall score and recall: Geel, `geointel-building-yolov8n-expanded160e50-pt`, tile `640`, threshold `0.05`, 300 detections, 91 matches, 209 false positives, 526 false negatives, precision `0.30333333333333334`, recall `0.14748784440842788`, F1 `0.1984732824427481`.
|
|
- dense-sample score winners: Geel, Mol, Turnhout and Retie all selected `geointel-building-yolov8n-expanded160e50-pt`.
|
|
- hard-negative/sparse-sample winner: Kasterlee-bos selected `yolov8s-building-segmentation-pt`, threshold `0.25`, F1 `0.16666666666666666`; the expanded local model produced too many false positives there.
|
|
|
|
Open:
|
|
- None for the sample-preparation and expanded-training runtime proof itself.
|
|
|
|
Limitations:
|
|
- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
|
|
- `geointel-building-yolov8n-expanded160e50-pt` is the best tested candidate on dense operator AOIs, but it is still experimental and should not become the V1 default until hard-negative false positives improve.
|
|
- The next model pass should add more sparse/background AOIs, tune confidence/NMS/max-detection settings and compare a stronger architecture or longer run against the same persisted QA matrix.
|
|
|
|
Next recommended pass:
|
|
- Build a hard-negative model-quality pass: expand sparse/background AOIs, export a balanced tile dataset, train a stronger candidate, and rerun the multi-sample QA matrix with dense and background samples scored separately.
|
|
|
|
## Sprint 130 Operator YOLO tile-level dataset tooling (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/export_operator_yolo_tile_dataset.py`.
|
|
- The exporter reads `operator_samples_manifest.json`, opens each raster/reference pair, creates overlapping tile windows, clips GRB building bounding boxes into tile-local YOLO labels, writes `dataset.yaml`, and reports `yolo_tile_dataset_summary.json`.
|
|
- Added deterministic negative tile retention through `negative_keep_ratio`.
|
|
- Added readiness compile coverage for the tile exporter.
|
|
- Added regression coverage in `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`.
|
|
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- RED: `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` failed while `scripts/export_operator_yolo_tile_dataset.py` did not exist.
|
|
- `python -m pytest backend\tests\test_sprint130_operator_yolo_tile_dataset.py -q` passed.
|
|
- `python scripts\export_operator_yolo_tile_dataset.py --help` passed without requiring local GIS dependencies.
|
|
- `python -m py_compile scripts\export_operator_yolo_tile_dataset.py` passed.
|
|
- Live Tower tile export passed:
|
|
- dataset: `/app/storage/operator-data/yolo-building-tile-dataset`
|
|
- samples: Geel, Mol and Turnhout
|
|
- tile size: `192`
|
|
- stride: `96`
|
|
- exported tiles: `75`
|
|
- positive tiles: `75`
|
|
- labels: `5321`
|
|
- validation split: Turnhout
|
|
- Live Tower 30-epoch CPU tile training passed:
|
|
- output model: `/app/models/geointel-building-yolov8n-tile30.pt`
|
|
- catalog asset: `geointel-building-yolov8n-tile30-pt`
|
|
- SHA256: `b9e228202500d7c85836d12a72e320f4f2f0cef24cbb1b5bf7fa78a6778390af`
|
|
- final validation: precision `0.208`, recall `0.271`, mAP50 `0.122`, mAP50-95 `0.0308`
|
|
- Live API preflight passed for `geointel-building-yolov8n-tile30-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Live 48-run multi-sample QA matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T190720Z/multi_sample_quality_summary.json`
|
|
- command compared `geointel-building-yolov8n-tile30-pt` with `yolov8s-building-segmentation-pt` over Geel, Mol and Turnhout, tile sizes `512`/`640`, overlap `64`, thresholds `0.50`/`0.25`/`0.15`/`0.05`.
|
|
- best overall score: Mol, `geointel-building-yolov8n-tile30-pt`, tile `640`, threshold `0.15`, 272 detections, 37 matches, 235 false positives, 337 false negatives, precision `0.13602941176470587`, recall `0.09893048128342247`, F1 `0.11455108359133127`.
|
|
- best overall recall: Mol, `geointel-building-yolov8n-tile30-pt`, tile `512`, threshold `0.05`, 544 detections, 44 matches, 500 false positives, 330 false negatives, precision `0.08088235294117647`, recall `0.11764705882352941`, F1 `0.09586056644880174`.
|
|
- best overall precision: Turnhout, `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.25`, precision `0.4`, recall `0.01034928848641656`, F1 `0.0201765447667087`.
|
|
- per-sample score winners: Geel `geointel-building-yolov8n-tile30-pt` F1 `0.09671179883945842`; Mol `geointel-building-yolov8n-tile30-pt` F1 `0.11455108359133127`; Turnhout `yolov8s-building-segmentation-pt` F1 `0.09971777986829727`.
|
|
|
|
Open:
|
|
- None for the tile exporter/training runtime proof itself.
|
|
|
|
Limitations:
|
|
- This remains operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
|
|
- `geointel-building-yolov8n-tile30-pt` is an experimental local candidate, not a V1 default. It improves the operator-trained baseline materially but still has low recall and many false positives on the current 3-sample corpus.
|
|
- The current corpus is too small and all exported tiles were positive; the next model pass needs more AOIs and deliberate negative/background tiles.
|
|
|
|
Next recommended pass:
|
|
- Expand the operator sample corpus beyond Geel/Mol/Turnhout, include negative/background AOIs, regenerate the tile dataset, then train a longer/larger local model candidate and rerun the same persisted QA matrix.
|
|
|
|
## Sprint 129 Operator YOLO training dataset tooling (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/export_operator_yolo_dataset.py` to export prepared operator samples into a local YOLO detection dataset:
|
|
- input manifest: `operator_samples_manifest.json`
|
|
- output: `dataset.yaml`, `images/train`, `labels/train`, `images/val`, `labels/val`, `yolo_dataset_summary.json`
|
|
- labels are derived from GRB building references with `source_name=grb` and `reference_layer_name=buildings`.
|
|
- Added `scripts/train_operator_yolo_detector.sh` as an explicit operator/runtime wrapper around a local Ultralytics training smoke:
|
|
- requires `OPERATOR_YOLO_DATASET_DIR`
|
|
- requires an existing `YOLO_BASE_MODEL_PATH`
|
|
- writes a local `TRAIN_MODEL_OUTPUT_PATH`
|
|
- uses `PYTHON_BIN=python3` by default for the all-in-one container
|
|
- writes `training_summary.json`.
|
|
- Added readiness coverage for exporter compile and train-wrapper shell syntax.
|
|
- Added regression coverage in `backend/tests/test_sprint129_operator_yolo_training_dataset.py`.
|
|
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- RED: `python -m pytest backend\tests\test_sprint129_operator_yolo_training_dataset.py -q` failed while the exporter and training wrapper contracts were incomplete.
|
|
- `python -m pytest backend\tests\test_sprint129_operator_yolo_training_dataset.py -q` passed.
|
|
- `python scripts\export_operator_yolo_dataset.py --help` passed without requiring local GIS dependencies.
|
|
- `python -m py_compile scripts\export_operator_yolo_dataset.py` passed.
|
|
- `bash -n scripts/train_operator_yolo_detector.sh` passed.
|
|
- Live Tower export passed: `/app/storage/operator-data/yolo-building-dataset` contains 3 images and 1427 labels from the current Geel/Mol/Turnhout samples.
|
|
- Live Tower training smoke passed with the existing `/app/models/yolov8n.pt` base model and wrote `/app/models/geointel-building-yolov8n-smoke.pt`.
|
|
- The first live smoke showed Ultralytics fetching an auxiliary plot font. The wrapper now sets `plots=False` and seeds `Arial.ttf` from an existing local system font before importing Ultralytics so the operator smoke path does not invoke plot/font network behavior.
|
|
- Full local readiness passed after the final wrapper hardening: `bash scripts/run_readiness_check.sh` ran 396 backend tests, frontend typecheck/build, Alembic head and smoke syntax checks.
|
|
- Pushed commits through `3936768` to Gitea and reset Tower `/mnt/user/appdata/geointel` to the same commit.
|
|
- Live Tower no-font smoke passed after deleting `/app/storage/ultralytics/Ultralytics/Arial.ttf`; the wrapper seeded the font locally and the 1-epoch run completed without the previous font fetch line.
|
|
- Live Tower final 8-epoch training smoke passed:
|
|
- output model: `/app/models/geointel-building-yolov8n-operator8.pt`
|
|
- catalog asset: `geointel-building-yolov8n-operator8-pt`
|
|
- SHA256: `071d64c89a8fd28f915f7a5a553e2d164942721292693a1612c725284c6e2e1e`
|
|
- training summary: `/app/storage/training/operator-yolo/geointel-building-yolov8n-operator8/training_summary.json`.
|
|
- Live API preflight passed for `geointel-building-yolov8n-operator8-pt` with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Live 36-run multi-sample QA matrix completed:
|
|
- command compared `geointel-building-yolov8n-operator8-pt` with `yolov8s-building-segmentation-pt` over Geel, Mol and Turnhout, tile sizes `512`/`640`, thresholds `0.50`/`0.25`/`0.15`.
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T033048Z/multi_sample_quality_summary.json`.
|
|
- `geointel-building-yolov8n-operator8-pt` produced zero detections for every sample/tile combination at thresholds `0.15` through `0.50`.
|
|
- best overall remained `yolov8s-building-segmentation-pt` on Mol, tile `640`, threshold `0.15`: 55 detections, 9 matches, 46 false positives, 365 false negatives, precision `0.16363636363636364`, recall `0.02406417112299465`, F1 `0.04195804195804196`.
|
|
- Live low-threshold operator8 matrix completed:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T033230Z/multi_sample_quality_summary.json`.
|
|
- at threshold `0.05`, operator8 still produced zero detections on all samples.
|
|
- at threshold `0.01`, operator8 produced many false positives and almost no matches; best case was Turnhout tile `512` with 280 detections, 2 matches, 278 false positives, precision `0.007142857142857143`, recall `0.00258732212160414`, F1 `0.003798670465337132`.
|
|
|
|
Open:
|
|
- None for the operator training/export tooling itself.
|
|
|
|
Limitations:
|
|
- This is operator tooling only. It does not add Training Studio, browser training controls, provider fetching, fake detections, model auto-provisioning or API contract changes.
|
|
- The 3-sample/8-epoch operator model is not production-useful and should not be activated as the default building detector.
|
|
|
|
Next recommended pass:
|
|
- Improve the training dataset before more model work: generate tile-level YOLO labels with enough positive/negative tiles, add more AOIs, then train a longer local model and rerun the same persisted QA matrix.
|
|
|
|
## Sprint 128 Stronger building model runtime benchmark (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `keremberke/yolov8s-building-segmentation` as an explicit runtime model asset on Tower:
|
|
- path: `/mnt/user/appdata/geointel/models/yolov8s-building-segmentation.pt`
|
|
- source: `https://huggingface.co/keremberke/yolov8s-building-segmentation/resolve/main/best.pt`
|
|
- SHA256: `a27af31654c6a4edbdc85581c33d93c13986b5919de7de410f8d85d801b3bb34`
|
|
- Did not commit model weights to Git and did not add app-side model download behavior.
|
|
- Ran a stronger-candidate multi-sample matrix comparing:
|
|
- `yolov8n-building-segmentation-pt`
|
|
- `yolov8s-building-segmentation-pt`
|
|
- samples: Geel, Mol, Turnhout
|
|
- tile sizes: `512`, `640`
|
|
- overlap: `64`
|
|
- thresholds: `0.50`, `0.25`, `0.15`
|
|
|
|
Tested:
|
|
- Hugging Face API metadata check showed `keremberke/yolov8s-building-segmentation` is public, uses the Ultralytics library, exposes `best.pt`, and reports validation mAP@0.5 values for the `keremberke/satellite-building-segmentation` dataset.
|
|
- Live model asset catalog reported `yolov8s-building-segmentation-pt`, `size_bytes=23814274`, `will_download_models=false`, SHA256 `a27af31654c6a4edbdc85581c33d93c13986b5919de7de410f8d85d801b3bb34`.
|
|
- Live preflight passed with `model_asset_id=yolov8s-building-segmentation-pt`, `check_model_load=true`, `model_file_exists=true`, `model_load_ok=true`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Tower 36-run matrix passed:
|
|
- command: `OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json QUALITY_MODEL_ASSET_IDS="yolov8n-building-segmentation-pt yolov8s-building-segmentation-pt" QUALITY_TILE_SIZES="512 640" QUALITY_TILE_OVERLAPS="64" QUALITY_THRESHOLDS="0.50 0.25 0.15" bash scripts/run_multi_sample_detection_quality_matrix.sh http://192.168.10.150:1202`
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/multi_sample_quality_summary.json`
|
|
- best overall score: Mol, `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.15`, 55 detections, 9 matches, 46 false positives, 365 false negatives, precision `0.16363636363636364`, recall `0.02406417112299465`, F1 `0.04195804195804196`.
|
|
- best overall recall: Mol, `yolov8s-building-segmentation-pt`, tile `512`, threshold `0.15`, 77 detections, 9 matches, 68 false positives, 365 false negatives, precision `0.11688311688311688`, recall `0.02406417112299465`, F1 `0.03991130820399113`.
|
|
- best overall precision: Turnhout, `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.25`, 20 detections, 8 matches, 12 false positives, 765 false negatives, precision `0.4`, recall `0.01034928848641656`, F1 `0.0201765447667087`.
|
|
- best Geel score: `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.15`, F1 `0.0187207488299532`.
|
|
- best Mol score: `yolov8s-building-segmentation-pt`, tile `640`, threshold `0.15`, F1 `0.04195804195804196`.
|
|
- best Turnhout score remained `yolov8n-building-segmentation-pt`, tile `512`, threshold `0.15`, F1 `0.03934426229508197`.
|
|
- Best-score evidence export passed for every sample in the new matrix:
|
|
- Geel evidence: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/geel/calibration_evidence_review.html`, 641 features, 611 false negatives, 18 false positives, 6 matched detections and 6 matched references.
|
|
- Mol evidence: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/mol/calibration_evidence_review.html`, 429 features, 365 false negatives, 46 false positives, 9 matched detections and 9 matched references.
|
|
- Turnhout evidence: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025932Z/turnhout/calibration_evidence_review.html`, 915 features, 755 false negatives, 124 false positives, 18 matched detections and 18 matched references.
|
|
|
|
Open:
|
|
- None for adding and benchmarking the `yolov8s` candidate.
|
|
|
|
Limitations:
|
|
- `yolov8s` improves precision and F1 on Geel/Mol but still misses most reference buildings. Best recall is about 2.4%, which is not sufficient for a usable V1 building extraction default.
|
|
- The model is a runtime artifact and remains outside Git.
|
|
|
|
Next recommended pass:
|
|
- Source or train a materially stronger aerial-building model. The GeoIntel pipeline is now good enough to benchmark candidates quickly, but the current public YOLO building-segmentation candidates are still too weak for the target Flemish orthophoto/GRB workflow.
|
|
|
|
## Sprint 127 Multi-sample detection quality calibration tooling (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/prepare_operator_real_data_samples.py` as an explicit operator/runtime helper for documented Geel, Mol and Turnhout real-data samples.
|
|
- The helper downloads small Digitaal Vlaanderen OMWRGBMRVL WMS `Ortho` GeoTIFFs and GRB OGC API Features `GBG` building GeoJSON references for the documented AOIs only.
|
|
- The helper writes `operator_samples_manifest.json`, sample metadata, source URLs and attribution under the runtime operator-data directory and reuses existing files by default.
|
|
- Added `scripts/run_multi_sample_detection_quality_matrix.sh` to run `scripts/run_detection_quality_matrix.sh` once per manifest sample.
|
|
- The multi-sample wrapper combines per-sample `quality_matrix_summary.json` files into `multi_sample_quality_summary.json` with `best_overall_by_score`, `best_overall_by_recall`, `best_overall_by_precision` and `best_by_sample`.
|
|
- Added readiness checks for Python compile and shell syntax.
|
|
- Added regression coverage in `backend/tests/test_sprint127_operator_sample_quality_matrix.py`.
|
|
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- RED: `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py -q` failed because the sample-preparation and multi-sample scripts did not exist.
|
|
- RED: `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py::test_prepare_operator_real_data_samples_help_does_not_require_gis_dependencies -q` failed because `--help` required missing GIS dependencies.
|
|
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py -q` passed.
|
|
- `python scripts\prepare_operator_real_data_samples.py --help` passed without requiring local GIS dependencies.
|
|
- `python -m py_compile scripts\prepare_operator_real_data_samples.py` passed.
|
|
- `bash -n scripts/run_multi_sample_detection_quality_matrix.sh` passed.
|
|
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint126_detection_quality_matrix.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py backend\tests\test_sprint124_detection_calibration_sweep.py -q` passed.
|
|
- `python scripts\smoke_docs.py` passed.
|
|
- `git diff --check` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 393 backend tests, frontend typecheck/build, Alembic head `202606120900`, live smoke syntax checks and the new sample/multi-sample checks.
|
|
- Pushed commit `06dfc5f` to Gitea and fast-forwarded Tower `/mnt/user/appdata/geointel` to the same commit.
|
|
- Tower sample preparation passed by running the new helper in the live `geointel` all-in-one container through stdin:
|
|
- Geel reused existing runtime files: 617 GRB GBG reference features.
|
|
- Mol was newly prepared: 374 GRB GBG reference features.
|
|
- Turnhout was newly prepared: 773 GRB GBG reference features.
|
|
- Manifest: `/mnt/user/appdata/geointel/storage/operator-data/operator_samples_manifest.json`.
|
|
- Tower multi-sample quality matrix passed:
|
|
- command: `OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json QUALITY_MODEL_ASSET_IDS="yolov8n-building-segmentation-pt yolov8n-pt" QUALITY_TILE_SIZES="512 640" QUALITY_TILE_OVERLAPS="64" QUALITY_THRESHOLDS="0.50 0.15" bash scripts/run_multi_sample_detection_quality_matrix.sh http://192.168.10.150:1202`
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025303Z/multi_sample_quality_summary.json`
|
|
- run count: 24 real persisted workflows across 3 samples.
|
|
- best overall score/recall: Turnhout, `yolov8n-building-segmentation-pt`, tile `512`, overlap `64`, threshold `0.15`, 142 detections, 18 matches, 124 false positives, 755 false negatives, precision `0.1267605633802817`, recall `0.02328589909443726`, F1 `0.03934426229508197`.
|
|
- best overall precision: Geel, `yolov8n-building-segmentation-pt`, tile `640`, overlap `64`, threshold `0.50`, 4 detections, 1 match, 3 false positives, 616 false negatives, precision `0.25`, recall `0.0016207455429497568`, F1 `0.0032206119162640897`.
|
|
- best Geel score: tile `512`, threshold `0.15`, 80 detections, 6 matches, 74 false positives, 611 false negatives, F1 `0.017216642754662843`.
|
|
- best Mol score: tile `640`, threshold `0.15`, 77 detections, 7 matches, 70 false positives, 367 false negatives, F1 `0.03104212860310421`.
|
|
- best Turnhout score: tile `512`, threshold `0.15`, 142 detections, 18 matches, 124 false positives, 755 false negatives, F1 `0.03934426229508197`.
|
|
- generic `yolov8n-pt` produced zero building detections across every sample, tile size and threshold.
|
|
- Tower best-score evidence export passed for every sample:
|
|
- Geel: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025303Z/geel/calibration_evidence_review.html`, 697 features, 611 false negatives, 74 false positives, 6 matched detections and 6 matched references.
|
|
- Mol: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025303Z/mol/calibration_evidence_review.html`, 451 features, 367 false negatives, 70 false positives, 7 matched detections and 7 matched references.
|
|
- Turnhout: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/20260707T025303Z/turnhout/calibration_evidence_review.html`, 915 features, 755 false negatives, 124 false positives, 18 matched detections and 18 matched references.
|
|
|
|
Open:
|
|
- None for multi-sample tooling.
|
|
|
|
Limitations:
|
|
- This is operator tooling only. It does not add a live GRB provider, live orthophoto provider, application endpoint, migration, frontend feature, model download or fixture inference path.
|
|
- The prepared sample files are runtime artifacts under appdata/storage and remain excluded from Git.
|
|
- The three-sample benchmark confirms the current building-segmentation evaluation model is not extraction-quality for V1: best recall is only about 2.3% and false negatives dominate every sample.
|
|
|
|
Next recommended pass:
|
|
- Replace or add a stronger aerial/building model candidate and rerun the same multi-sample matrix; optionally add IoU-threshold sweeps after a model produces materially better candidate detections.
|
|
|
|
## Sprint 126 Detection quality matrix tooling (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/run_detection_quality_matrix.sh` as an operator-facing model/tile/threshold matrix for the configured-YOLO real-data path.
|
|
- The matrix reuses `scripts/verify_real_data_detection_qa_workflow.sh` for each row so every result is backed by persisted Project, Dataset, AnalysisRun, Detection, QualityCheck, Metric and Export records.
|
|
- The script accepts `QUALITY_MODEL_ASSET_IDS`, `QUALITY_TILE_SIZES`, `QUALITY_TILE_OVERLAPS` and `QUALITY_THRESHOLDS`, writes per-run logs and produces `quality_matrix_summary.json`.
|
|
- The summary reports model asset, tile size, overlap, confidence threshold, detection count, QA score, precision, recall, F1, mean IoU, matches, false positives and false negatives.
|
|
- Added `best_by_score`, `best_by_recall` and `best_by_precision` rankings for operator model-quality decisions.
|
|
- Added readiness syntax coverage and regression coverage in `backend/tests/test_sprint126_detection_quality_matrix.py`.
|
|
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- RED: `python -m pytest backend\tests\test_sprint126_detection_quality_matrix.py -q` failed because `scripts/run_detection_quality_matrix.sh` did not exist.
|
|
- `python -m pytest backend\tests\test_sprint126_detection_quality_matrix.py -q` passed.
|
|
- `python -m pytest backend\tests\test_sprint126_detection_quality_matrix.py backend\tests\test_sprint124_detection_calibration_sweep.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py -q` passed.
|
|
- `bash -n scripts/run_detection_quality_matrix.sh` passed.
|
|
- `bash scripts/run_detection_quality_matrix.sh --help` passed.
|
|
- `python scripts\smoke_docs.py` passed.
|
|
- `git diff --check` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 390 backend tests, frontend typecheck/build, Alembic head `202606120900`, live smoke syntax checks and the new matrix syntax check.
|
|
- Pushed commit `e728f70` to Gitea and fast-forwarded Tower `/mnt/user/appdata/geointel` to the same commit.
|
|
- Tower live matrix passed against the Geel operator sample:
|
|
- command: `QUALITY_MODEL_ASSET_IDS="yolov8n-building-segmentation-pt yolov8n-pt" QUALITY_TILE_SIZES="512 640" QUALITY_TILE_OVERLAPS="64" QUALITY_THRESHOLDS="0.50 0.15" bash scripts/run_detection_quality_matrix.sh http://192.168.10.150:1202`
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/quality_matrix_summary.json`
|
|
- run count: 8
|
|
- best by score and recall: `yolov8n-building-segmentation-pt`, tile `512`, overlap `64`, threshold `0.15`, 80 detections, 6 matches, 74 false positives, 611 false negatives, precision `0.075`, recall `0.009724473257698542`, F1 `0.017216642754662843`.
|
|
- best by precision: `yolov8n-building-segmentation-pt`, tile `640`, overlap `64`, threshold `0.50`, 4 detections, 1 match, 3 false positives, 616 false negatives, precision `0.25`, recall `0.0016207455429497568`, F1 `0.0032206119162640897`.
|
|
- generic `yolov8n-pt` produced zero building detections for all tested tile/threshold combinations.
|
|
- Tower best-run evidence export passed from the matrix summary:
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/calibration_evidence.geojson`
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/calibration_evidence_summary.json`
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/20260707T023000Z/calibration_evidence_review.html`
|
|
- evidence features: 697 total, 611 false negatives, 74 false positives, 6 matched detections and 6 matched references.
|
|
|
|
Open:
|
|
- None for matrix tooling.
|
|
|
|
Limitations:
|
|
- This is operator benchmarking tooling only. It does not change inference behavior, add model downloads, seed fixture detections, fetch providers, change API contracts or change migrations.
|
|
- A single Geel sample is not enough to declare a production V1 building-extraction baseline; additional orthophoto/reference samples are still needed before picking defaults.
|
|
|
|
Next recommended pass:
|
|
- Add at least two more local orthophoto/reference samples and run the same quality matrix before choosing V1 defaults; the current Geel evidence says runtime plumbing works, but the active model is still not extraction-quality.
|
|
|
|
## Sprint 125 Detection calibration evidence bundle (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/export_detection_calibration_evidence.sh` to turn a persisted detection calibration summary into visual QA evidence artifacts.
|
|
- The script reads `calibration_summary.json`, fetches the existing project quality-check evidence GeoJSON endpoint for each `quality_check_id`, enriches features with threshold/score/provenance and writes:
|
|
- `calibration_evidence.geojson`
|
|
- `calibration_evidence_summary.json`
|
|
- `calibration_evidence_review.html`
|
|
- The HTML review artifact renders a simple SVG overview with distinct roles for `match_candidate`, `match_reference`, `false_positive` and `false_negative`.
|
|
- Registered the script in `scripts/run_readiness_check.sh` as a syntax check.
|
|
- Added regression coverage in `backend/tests/test_sprint125_detection_calibration_evidence_bundle.py`.
|
|
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend/tests/test_sprint125_detection_calibration_evidence_bundle.py -q` failed because `scripts/export_detection_calibration_evidence.sh` did not exist.
|
|
- `python -m pytest backend/tests/test_sprint125_detection_calibration_evidence_bundle.py -q` passed.
|
|
- `bash -n scripts/export_detection_calibration_evidence.sh` passed.
|
|
- `bash scripts/export_detection_calibration_evidence.sh --help` passed.
|
|
- Missing-input guard printed usage and did not fetch evidence.
|
|
- `bash scripts/run_readiness_check.sh` passed: 389 backend tests, Alembic head check, frontend typecheck/build and shell syntax checks.
|
|
- Tower pulled commit `ea8dcb2` with `git pull --ff-only origin main`.
|
|
- Tower evidence export passed against `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_summary.json`.
|
|
- Evidence artifacts written on Tower:
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_evidence.geojson`
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_evidence_summary.json`
|
|
- `/mnt/user/appdata/geointel/artifacts/detection-calibration/20260707T002103Z/calibration_evidence_review.html`
|
|
- Evidence counts from the Geel calibration export:
|
|
- total evidence features: 2555
|
|
- false negatives: 2460
|
|
- false positives: 79
|
|
- matched detections: 8
|
|
- matched references: 8
|
|
- Copied `calibration_evidence_review.html` and `calibration_evidence_summary.json` to the local ignored `artifacts/detection-calibration/20260707T002103Z/` directory for inspection.
|
|
- Internal browser validation passed through a temporary local static server: the review page loaded with title `GeoIntel Detection Calibration Evidence`, 1 SVG, 2555 SVG paths, 4 calibration table rows and 0 console errors.
|
|
|
|
Open:
|
|
- None for evidence export tooling.
|
|
|
|
Limitations:
|
|
- This is operator tooling only. It does not rerun inference, change API behavior, add UI behavior, fetch providers, seed demo data or download models.
|
|
- The SVG review is a lightweight geometry overview, not a replacement for full MapLibre evidence review in the workbench.
|
|
- The evidence distribution confirms the current active model misses most reference buildings on this sample. This points to model suitability and/or tiling strategy as the next bottleneck, not runtime plumbing.
|
|
|
|
Next recommended pass:
|
|
- Add a model-quality decision pass: compare the current evaluation model against another building/aerial model or adjusted tile/overlap settings on at least two additional local orthophoto/reference samples.
|
|
|
|
## Sprint 124 Detection calibration sweep tooling (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/run_detection_calibration_sweep.sh` as an operator-facing confidence-threshold sweep for the configured-YOLO real-data path.
|
|
- The sweep reuses `scripts/verify_real_data_detection_qa_workflow.sh` once per threshold, so each row is backed by persisted Project, Dataset, AnalysisRun, Detection, QualityCheck, Metric and export records.
|
|
- The sweep fetches project `quality-checks` after each run and writes per-threshold summaries plus `calibration_summary.json` with detection count, QA score, precision, recall, F1, mean IoU, matches, false positives and false negatives.
|
|
- Registered the sweep in `scripts/run_readiness_check.sh` as a syntax check.
|
|
- Added regression coverage in `backend/tests/test_sprint124_detection_calibration_sweep.py`.
|
|
- Updated `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend/tests/test_sprint124_detection_calibration_sweep.py -q` failed because `scripts/run_detection_calibration_sweep.sh` did not exist.
|
|
- `python -m pytest backend/tests/test_sprint124_detection_calibration_sweep.py -q` passed.
|
|
- `bash -n scripts/run_detection_calibration_sweep.sh` passed.
|
|
- `bash scripts/run_detection_calibration_sweep.sh --help` passed.
|
|
- Missing-input guard printed usage and did not start a live workflow.
|
|
- `bash scripts/run_readiness_check.sh` passed: 388 backend tests, Alembic head check, frontend typecheck/build and shell syntax checks.
|
|
- Tower pulled commit `590e597` with `git pull --ff-only origin main` and ran the default Geel calibration sweep.
|
|
- The first Tower sweep exposed that the summary script looked for `f1_score` while persisted metrics use `f1`; the script was patched with a `metrics.get("f1")` fallback and regression coverage.
|
|
- RED: `python -m pytest backend/tests/test_sprint124_detection_calibration_sweep.py -q` failed while the script lacked the `f1` fallback.
|
|
- `python -m pytest backend/tests/test_sprint124_detection_calibration_sweep.py -q` passed after the fallback.
|
|
- Tower pulled commit `e8cd463` and reran the Geel calibration sweep successfully:
|
|
- output: `artifacts/detection-calibration/20260707T002103Z/calibration_summary.json`
|
|
- threshold `0.50`: 4 detections, score/F1 `0.0032206119162640897`, precision `0.25`, recall `0.0016207455429497568`, 1 match, 3 false positives, 616 false negatives
|
|
- threshold `0.35`: 9 detections, score/F1 `0.003194888178913738`, precision `0.1111111111111111`, recall `0.0016207455429497568`, 1 match, 8 false positives, 616 false negatives
|
|
- threshold `0.25`: 20 detections, score/F1 `0.0031397174254317113`, precision `0.05`, recall `0.0016207455429497568`, 1 match, 19 false positives, 616 false negatives
|
|
- threshold `0.15`: 54 detections, score/F1 `0.014903129657228018`, precision `0.09259259259259259`, recall `0.008103727714748784`, 5 matches, 49 false positives, 612 false negatives
|
|
- best by score for this sample: threshold `0.15`
|
|
|
|
Open:
|
|
- None for calibration tooling.
|
|
|
|
Limitations:
|
|
- The sweep is intentionally mutating and creates one real workflow run per threshold.
|
|
- It is calibration tooling only; it does not change inference, add model downloads, fetch providers, seed demo detections or change API/UI behavior.
|
|
- The current active building model still performs poorly on the Geel validation sample. Threshold `0.15` is best among the tested values, but recall remains under 1%; this is model/data-quality evidence, not a production-ready extraction baseline.
|
|
|
|
Next recommended pass:
|
|
- Inspect false-positive/false-negative evidence for the Geel runs, then add at least two more local orthophoto/reference samples before choosing V1 default confidence/IoU guidance.
|
|
|
|
## Sprint 123 YOLO class and tile CRS normalization (2026-07-07)
|
|
|
|
Changed:
|
|
- Investigated the Geel real-data smoke that persisted zero detections despite the configured building model being available.
|
|
- Confirmed on Tower that `/app/models/yolov8n-building-segmentation.pt` reports model class `Building` and returns 4 raw detections at confidence `0.5` on the same real Geel tile manifest.
|
|
- Fixed configured-YOLO detection persistence so model class names are compared case-insensitively against `class_filter`, persisted as canonical lowercase domain classes, and preserve the original model class name in `properties_json.model_class_name`.
|
|
- Found a second live GIS correctness issue: generated tile manifests carried Lambert bounds/transforms but no CRS, so detection GeoJSON could expose EPSG:31370 coordinates as if they were EPSG:4326.
|
|
- Fixed raster tile manifest generation to include `crs`, `source_crs` and `dataset_crs` on the manifest and `crs` on each tile entry when the source raster CRS is known.
|
|
- Added regression coverage in `backend/tests/test_sprint8b_yolo_foundation.py` and `backend/tests/test_raster_operations_service.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend/tests/test_sprint8b_yolo_foundation.py::test_yolo_class_filter_is_case_insensitive_and_persists_canonical_class -q` failed with `detection_count=0` because `Building` did not match `building`.
|
|
- `python -m pytest backend/tests/test_sprint8b_yolo_foundation.py::test_yolo_class_filter_is_case_insensitive_and_persists_canonical_class -q` passed.
|
|
- `python -m pytest backend/tests/test_sprint8b_yolo_foundation.py backend/tests/test_model_asset_catalog.py backend/tests/test_sprint121_real_data_detection_qa_smoke.py backend/tests/test_sprint122_raster_upload_metadata_mapping.py -q` passed: 20 tests.
|
|
- RED: `python -m pytest backend/tests/test_raster_operations_service.py::test_raster_tile_returns_manifest_payload -q` failed because the tile manifest had no `crs`.
|
|
- `python -m pytest backend/tests/test_raster_operations_service.py::test_raster_tile_returns_manifest_payload -q` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 387 backend tests, Alembic head check, frontend typecheck/build and shell syntax checks.
|
|
- Tower deploy from commit `71c2cd9` passed with `GEOINTEL_INSTALL_AI=true`.
|
|
- Deploy-time live migration smoke passed; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`.
|
|
- Real-data smoke after the class-normalization deploy passed and persisted 4 detections:
|
|
- project: `cb80638d-dbef-48ac-b19c-cec7c3efc96e`
|
|
- raster dataset: `ae0ff76d-70c0-404f-b777-54d14517179a`
|
|
- reference dataset: `8ac01b4f-bd6a-4d6a-b625-a0950ae0f3eb`
|
|
- analysis run: `7ba34274-411d-45e3-8f54-c37baec598b1`
|
|
- quality check: `66907e6a-9ed7-4b0c-976f-5ad1ba9b8b7a`
|
|
- detection export: `516d37a3-2305-48a2-a3dd-b56f70eb055e`
|
|
- persisted detections used canonical `class_name=building` and preserved `model_class_name=Building`.
|
|
- Tower deploy from commit `bc87681` passed with `GEOINTEL_INSTALL_AI=true`.
|
|
- Deploy-time live migration smoke passed; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`.
|
|
- Real-data smoke after the tile-CRS deploy passed and persisted 4 detections:
|
|
- project: `6ced26f3-4486-451e-a05e-26a8f859e361`
|
|
- raster dataset: `d54c3420-03d4-45a2-9501-bf1cce165a89`
|
|
- reference dataset: `319dad51-15e9-4ae6-815b-8e39023ed962`
|
|
- manifest: `/app/storage/tiles/6ced26f3-4486-451e-a05e-26a8f859e361/d54c3420-03d4-45a2-9501-bf1cce165a89/721c9557-5910-4994-b0ac-ba3418d12246/manifest.json`
|
|
- analysis run: `e99bcd07-ba48-4ebe-9ba9-24a94a8c1dcb`
|
|
- quality check: `32c8252b-fe60-4062-9639-125c62ca677f`
|
|
- detection export: `9d3a7f60-07a2-415c-9f75-a3c1988d0904`
|
|
- Verified the new tile manifest now carries `crs=EPSG:31370`, `source_crs=EPSG:31370`, `dataset_crs=EPSG:31370` and per-tile `crs=EPSG:31370`.
|
|
- Verified Detection GeoJSON now returns WGS84 coordinates around Geel instead of raw Belgian Lambert coordinates.
|
|
- Live detection QA persisted honest metrics for the Geel sample: 1 match, 3 false positives, 616 false negatives and score `0.0032206119162640897`.
|
|
|
|
Open:
|
|
- None for availability, persistence and CRS propagation.
|
|
|
|
Limitations:
|
|
- This fixes class routing, persistence and future tile manifest CRS propagation. Existing tile manifests generated before this fix remain missing CRS and should be regenerated before AI runs.
|
|
- The active building model is operational but not calibrated for production-quality local Belgian/Kempen orthophoto extraction. The Geel sample proves end-to-end persistence and QA, while the 1/617 reference match result shows that confidence thresholds, tiling strategy, class mapping and IoU defaults still need model-quality calibration.
|
|
|
|
Next recommended pass:
|
|
- Run a detection calibration pass on several local orthophoto/reference samples: sweep confidence thresholds, inspect false positives/false negatives, tune tile size/overlap where needed and record a practical V1 baseline.
|
|
|
|
## Sprint 122 Real operator data availability and raster metadata fix (2026-07-07)
|
|
|
|
Changed:
|
|
- Created real operator validation files on Tower under `/mnt/user/appdata/geointel/storage/operator-data`.
|
|
- Generated `/mnt/user/appdata/geointel/storage/operator-data/geel_orthophoto_wms_512.tif` from the official Digitaal Vlaanderen OMWRGBMRVL WMS `Ortho` layer for a 500 m x 500 m AOI around Geel.
|
|
- Generated `/mnt/user/appdata/geointel/storage/operator-data/geel_grb_gbg_buildings.geojson` from the official Digitaal Vlaanderen GRB OGC API Features `GBG` building collection for the same AOI; it contained 617 features.
|
|
- Fixed `DatasetService` raster upload metadata mapping so extracted `bounds`, `resolution` and `dtype` populate persisted `bounds_json`, `resolution_json` and `bands_json`.
|
|
- Added `backend/tests/test_sprint122_raster_upload_metadata_mapping.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend/tests/test_sprint122_raster_upload_metadata_mapping.py -q` failed because uploaded raster `bounds_json` was `None`.
|
|
- `python -m pytest backend/tests/test_sprint122_raster_upload_metadata_mapping.py -q` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 386 backend tests, Alembic head check, frontend typecheck/build and shell syntax checks.
|
|
- Tower deploy from commit `2468945` passed with `GEOINTEL_INSTALL_AI=true`.
|
|
- Deploy-time live migration smoke passed; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`.
|
|
- Real-data smoke passed:
|
|
- command: `REAL_RASTER_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_orthophoto_wms_512.tif REAL_REFERENCE_VECTOR_PATH=/mnt/user/appdata/geointel/storage/operator-data/geel_grb_gbg_buildings.geojson bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202`
|
|
- project: `e1ec201b-bd83-4c95-be5a-f7225902d3c5`
|
|
- raster dataset: `fca14d23-bc0c-4b1f-8bfa-ba968f2e05d0`
|
|
- reference dataset: `5bb4a9b4-2a4e-4680-a729-777a75ecd51f`
|
|
- model asset: `yolov8n-building-segmentation-pt`
|
|
- manifest: `/app/storage/tiles/e1ec201b-bd83-4c95-be5a-f7225902d3c5/fca14d23-bc0c-4b1f-8bfa-ba968f2e05d0/0f29f110-951f-48eb-a2fb-636b587e31ee/manifest.json`
|
|
- analysis run: `c6798559-590c-42e3-a192-45a2409fa832`
|
|
- detections: `0`
|
|
- quality check: `cd0d13f9-f6ce-4acf-8859-7cb7aa666125`
|
|
- detection export: `0081c230-1766-4ff3-8df0-e47236f529d1`
|
|
|
|
Limitations:
|
|
- The workflow is operational against real operator data. A follow-up class-normalization pass found that the active evaluation model returned `Building` while the workflow filtered on `building`; see Sprint 123.
|
|
- The prepared files are runtime artifacts on Tower, not repository fixtures.
|
|
|
|
Next recommended pass:
|
|
- Redeploy the class-normalization fix, rerun the real-data smoke and calibrate confidence/IoU thresholds against persisted detection and QA metrics.
|
|
|
|
## Sprint 121 Real data detection and QA workflow smoke (2026-07-07)
|
|
|
|
Changed:
|
|
- Added `scripts/verify_real_data_detection_qa_workflow.sh` for live-runtime validation with operator-provided real GIS inputs.
|
|
- The smoke creates a project, uploads a real GeoTIFF-style raster as a source dataset, uploads a real reference-building GeoJSON/JSON as `dataset_role=reference`, validates CRS/bounds/features, tiles the raster, selects a mounted local model asset, runs read-only YOLO preflight, runs configured YOLO detection, runs detection QA against persisted `vector_features`, and exports the detection run GeoJSON.
|
|
- Registered the smoke in `scripts/run_readiness_check.sh` as a syntax check only, so normal readiness does not require real orthophotos, reference vectors, optional AI dependencies or model files.
|
|
- Documented Tower usage and limitations in `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend/tests/test_sprint121_real_data_detection_qa_smoke.py -q` failed while `scripts/verify_real_data_detection_qa_workflow.sh` did not exist.
|
|
- `python -m pytest backend/tests/test_sprint121_real_data_detection_qa_smoke.py -q` passed: 1 test.
|
|
- `bash -n scripts/verify_real_data_detection_qa_workflow.sh` passed.
|
|
- `bash scripts/verify_real_data_detection_qa_workflow.sh --help` passed and printed required `REAL_RASTER_PATH` and `REAL_REFERENCE_VECTOR_PATH` usage.
|
|
- `python -m compileall backend/app` passed.
|
|
- `python scripts/smoke_docs.py` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 385 backend tests, Alembic head check, frontend typecheck, frontend production build and shell syntax checks.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- Missing-input guard passed: `bash scripts/verify_real_data_detection_qa_workflow.sh http://localhost:1202` returned exit code 2 and printed usage.
|
|
- Local `docker compose config` could not run in this Windows Codex environment because the `docker` command is not installed.
|
|
|
|
Limitations:
|
|
- The full real-data smoke was not executed in this Codex workspace because no operator-provided real GeoTIFF and reference GeoJSON were found locally.
|
|
- The script enforces real inputs and never seeds demo data, enables fixture detections, fetches live GRB/OSM/Sentinel data or downloads model weights.
|
|
|
|
Next recommended pass:
|
|
- Place a target orthophoto/GeoTIFF and matching reference-building GeoJSON under the Tower appdata path and run `REAL_RASTER_PATH=... REAL_REFERENCE_VECTOR_PATH=... bash scripts/verify_real_data_detection_qa_workflow.sh http://192.168.10.150:1202`.
|
|
|
|
## Sprint 120 Model asset detection workflow smoke (2026-07-06)
|
|
|
|
Changed:
|
|
- Added `scripts/verify_model_asset_detection_workflow.sh` to validate the configured-YOLO runtime path against a live Docker/Tower deployment.
|
|
- The smoke seeds the explicit offline demo workflow, creates a raster tile manifest, selects the active local model asset from `GET /api/v1/detection/model-assets`, verifies read-only YOLO preflight, submits the existing detection run endpoint and checks persisted AnalysisRun, Detection list and Detection GeoJSON outputs.
|
|
- Registered the script in `scripts/run_readiness_check.sh` as a syntax check only, so ordinary readiness runs remain valid on machines without optional AI dependencies or mounted model files.
|
|
- Documented the smoke in `scripts/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend/tests/test_sprint120_model_asset_detection_workflow_smoke.py -q` failed because `scripts/verify_model_asset_detection_workflow.sh` did not exist yet.
|
|
- `python -m pytest backend/tests/test_sprint120_model_asset_detection_workflow_smoke.py -q` passed: 1 test.
|
|
- `bash -n scripts/verify_model_asset_detection_workflow.sh` passed.
|
|
- Live Tower smoke passed: `bash scripts/verify_model_asset_detection_workflow.sh http://192.168.10.150:1202`.
|
|
- Live smoke selected `model_asset_id=yolov8n-building-segmentation-pt`, generated manifest `/app/storage/tiles/c0b00f1f-80bf-4992-be94-f5e5e6f6bf63/f9160f51-ee78-43b3-9353-d5390576fa1d/e9acd488-c376-45ed-b259-0dd79886f21e/manifest.json`, persisted analysis run `7f9e7ecb-c43d-4ed3-9f98-424bc0317805` and returned `detection_count=0`.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 384 tests with the existing Pydantic `model_*` namespace warnings.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 384 backend tests, frontend typecheck/build, API contract audit, Alembic head and shell syntax checks.
|
|
- Live browser/API smoke passed: `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`.
|
|
- Live GIS capability smoke passed: `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202`.
|
|
- Live raster workflow smoke passed: `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202`.
|
|
- Live workbench default-state smoke passed: `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202`.
|
|
- Live workbench backing-state smoke passed: `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202`.
|
|
- Live demo/export workflow smoke passed: `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202`.
|
|
- `bash scripts/verify_ai_handoff_interactions.sh http://192.168.10.150:1202` could not run in this local Codex shell because Node cannot import Playwright; the script remains syntax-checked in readiness and the internal browser was used for live visual verification instead.
|
|
- Internal browser validation passed on `http://192.168.10.150:1202`: AI Labs rendered Detection Lab and Segmentation Lab, selecting `yolo-configured` showed the Local model assets selector with `yolov8n-building-segmentation (active)` and `yolov8n`, no-download copy was visible and no console errors were emitted.
|
|
- `git push origin main` pushed commit `b2fe7fa`.
|
|
- Tower deploy from commit `b2fe7fa` completed with `GEOINTEL_INSTALL_AI=true`; the all-in-one container is published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`, API proxy and icon.
|
|
- Post-deploy live model asset detection workflow smoke passed: `bash scripts/verify_model_asset_detection_workflow.sh http://192.168.10.150:1202` selected `yolov8n-building-segmentation-pt`, persisted analysis run `f758c992-4fb8-4dca-ab29-5c216cb14078` and returned `detection_count=0`.
|
|
|
|
Limitations:
|
|
- The smoke proves the configured-YOLO runtime path, provenance and persistence. It does not prove production model quality because it runs against the synthetic demo raster.
|
|
- Real operational validation still requires uploading a georeferenced Kempen orthophoto/GeoTIFF, running the configured building model on that raster and comparing persisted detections against reference building vectors through QA/QC.
|
|
|
|
Next recommended pass:
|
|
- Create the real-data validation path for orthophoto upload, tile generation, configured building-model run and reference-vector QA/QC.
|
|
|
|
## Sprint 118 Local model and reference catalog clarity (2026-07-06)
|
|
|
|
Changed:
|
|
- Added a read-only backend model asset catalog through `GET /api/v1/detection/model-assets`.
|
|
- Added `YOLO_MODELS_DIR` to backend settings, Compose, Unraid env examples and all-in-one runtime startup so `/app/models` is the explicit model catalog directory.
|
|
- Extended configured YOLO preflight and detection runs with optional `model_asset_id`, resolved server-side against the model asset catalog.
|
|
- Detection jobs and analysis runs now persist selected model asset ID, path and SHA-256 in parameters for reproducibility.
|
|
- Detection Lab now loads local model assets, selects the active model by default and lets operators choose a cataloged local model file for `yolo-configured`.
|
|
- Provider Capabilities now distinguishes GRB/OSM/manual/fixture reference-data sources from AI model choices.
|
|
- Updated `docs/API_CONTRACTS.md`, `docs/AI_PIPELINES.md`, `backend/README.md`, `frontend/README.md`, `deploy/unraid/README.md`, `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
- Added design/plan documents under `docs/superpowers/`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend/tests/test_model_asset_catalog.py -q` failed before implementation because `app.services.model_asset_catalog_service` did not exist.
|
|
- `python -m pytest backend/tests/test_model_asset_catalog.py -q` passed: 5 tests.
|
|
- RED: `python -m pytest backend/tests/test_sprint118_yolo_preflight_ui.py -q` failed before frontend wiring because the model asset types/API/hook/UI were absent.
|
|
- `python -m pytest backend/tests/test_sprint118_yolo_preflight_ui.py -q` passed: 2 tests.
|
|
- RED: runtime config tests failed before `YOLO_MODELS_DIR` was added to env examples, Unraid runtime and `scripts/configure_yolo_model.py`.
|
|
- `python -m pytest backend/tests/test_docker_runtime_config.py::test_env_example_uses_runtime_env_names_read_by_backend_and_frontend backend/tests/test_docker_runtime_config.py::test_unraid_deploy_passes_ai_build_arg_and_yolo_runtime_env backend/tests/test_sprint119_yolo_model_configuration.py -q` passed: 6 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 383 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python scripts/audit_api_contracts.py` passed: 81 implemented routes match docs; 2 explicit non-envelope endpoints tracked.
|
|
- `bash scripts/run_readiness_check.sh` passed: 383 backend tests plus frontend typecheck/build, API contract audit, Alembic head and shell syntax checks.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh; bash -n deploy/unraid/run-dockerman-container.sh; bash -n deploy/unraid/all-in-one-start.sh; bash -n scripts/deploy_tower.sh` passed.
|
|
- Local `docker compose config` could not run because Docker is not installed in this Windows Codex environment; Tower deploy validation remains required.
|
|
- `git push origin main` passed and pushed commits through `6e2a8cb`.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` with `GEOINTEL_INSTALL_AI=true` passed and redeployed Tower from commit `6e2a8cb`.
|
|
- Deploy-time live migration smoke passed on Tower; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`, API proxy and icon.
|
|
- Live `GET http://192.168.10.150:1202/api/v1/detection/model-assets` returned two cataloged local assets: `yolov8n-building-segmentation-pt` active with SHA-256 `152d6a9c5c76c9f2fd2fd5cc167efaed7c8c02e31002b15415899710f1d71f98`, and `yolov8n-pt`.
|
|
- Live `GET /api/v1/detection/yolo/preflight?model_asset_id=yolov8n-pt` resolved `/app/models/yolov8n.pt`, reported dependencies available, no downloads and no inference.
|
|
- Live `GET /api/v1/detection/yolo/preflight?model_asset_id=yolov8n-building-segmentation-pt&check_model_load=true` passed model load smoke with `model_load_ok=true`, `torch_version=2.12.1`, `ultralytics_version=8.4.89`, no downloads and no inference.
|
|
- Internal browser validation passed on `http://192.168.10.150:1202`: Detection Lab showed `Local model assets`, listed `yolov8n-building-segmentation (active)` and `yolov8n`, auto-selected the active asset, showed no-download copy, and emitted no console errors.
|
|
- Internal browser validation passed for System/Provider Capabilities: `Official reference sources`, GRB, OSM and `not AI model choices` copy were visible with no console errors.
|
|
|
|
Limitations:
|
|
- The catalog is intentionally filesystem-backed and read-only. It does not download, validate semantic class metadata, train models or manage model lifecycle records in the database.
|
|
- GRB/OSM remain provider capabilities only; no live external fetching was added.
|
|
|
|
Next recommended pass:
|
|
- Redeploy Tower, verify `/api/v1/detection/model-assets`, confirm Detection Lab shows the local model picker, then continue with real raster/model workflow validation on non-synthetic imagery.
|
|
|
|
## Sprint 117 Reusable GIS run and AI runtime opt-in (2026-07-05)
|
|
|
|
Changed:
|
|
- Added a Map workspace full-run mode selector with `Create new dataset/export` and `Reuse latest saved dataset for QA`.
|
|
- Reuse mode runs QA/QC against the latest saved derived map-selection dataset without creating another derived dataset/export pair.
|
|
- Added opt-in Docker and Unraid AI build support through `GEOINTEL_INSTALL_AI=true`; default builds still install only the GIS runtime.
|
|
- Passed YOLO runtime environment variables and a `/app/models` volume into the all-in-one Unraid container so local PyTorch/Ultralytics models can be mounted explicitly.
|
|
- Hardened the AI image path after Tower validation showed `torch` imported but `ultralytics` failed on a missing OpenCV native library. The Dockerfiles now include the required OpenCV runtime shared libraries and YOLO dependency detection performs real imports instead of `find_spec` checks.
|
|
- Added a writable `YOLO_CONFIG_DIR` default under application storage after Tower validation showed Ultralytics otherwise falls back to `/tmp` because root config is not writable in the container.
|
|
- Added YOLO preflight runtime diagnostics so operators can see dependency assumption state, model directory, `YOLO_CONFIG_DIR`, installed `torch`/`ultralytics` package versions and CUDA availability without loading a model, running inference or downloading weights.
|
|
- Added a read-only `GET /api/v1/detection/yolo/preflight` endpoint and Detection Lab YOLO runtime preflight panel so browser operators can inspect live AI runtime readiness without loading a model, running inference or downloading weights.
|
|
- Updated `.env.example`, `backend/README.md`, `frontend/README.md`, `scripts/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint116_operational_gis_map_workflow.py`, `backend/tests/test_sprint8b_yolo_foundation.py` and `backend/tests/test_docker_runtime_config.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py backend\tests\test_docker_runtime_config.py -q` failed before implementation because `fullWorkflowMode`, AI build args and YOLO runtime env wiring were absent.
|
|
- `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py backend\tests\test_docker_runtime_config.py -q` passed: 22 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `python -m py_compile scripts\yolo_preflight.py backend\scripts\yolo_preflight.py` passed.
|
|
- `python -m pytest backend\tests\test_sprint31_unraid_template.py backend\tests\test_docker_runtime_config.py -q` passed: 27 tests.
|
|
- RED: `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py backend\tests\test_docker_runtime_config.py -q` failed before the runtime hardening because YOLO dependency detection still used `find_spec` and the Dockerfiles lacked OpenCV native runtime libraries.
|
|
- `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py backend\tests\test_docker_runtime_config.py -q` passed: 30 tests.
|
|
- RED: `python -m pytest backend\tests\test_docker_runtime_config.py -q` failed before `YOLO_CONFIG_DIR` wiring because the Compose, Unraid and startup paths did not define a writable Ultralytics config directory.
|
|
- `python -m pytest backend\tests\test_docker_runtime_config.py -q` passed: 21 tests.
|
|
- `python -m compileall backend/app` passed after the AI runtime hardening.
|
|
- `cd backend && python -m pytest -q` passed: 367 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 367 backend tests plus frontend typecheck/build.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh; bash -n scripts/deploy_tower.sh; bash -n deploy/unraid/run-dockerman-container.sh` passed.
|
|
- Local Codex host could not run `docker compose config` because Docker is not installed in this Windows environment; Tower Docker validation is required after push/deploy.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` with `GEOINTEL_INSTALL_AI=true` passed after the AI runtime hardening and redeployed Tower from commit `9cad8d4`.
|
|
- Deploy-time live migration smoke passed on Tower; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`, API proxy and icon.
|
|
- Tower container check passed: `torch` imported as `2.12.1+cu130`, `torch.cuda.is_available()` returned `False`, `ultralytics` imported as `8.4.87`, and `scripts/yolo_preflight.py --enabled --json` returned `dependencies_available=true`, `status=not_configured`, `will_download_models=false`, `will_run_inference=false` because no local model path is configured yet.
|
|
- Tower runtime `YOLO_CONFIG_DIR` is `/app/storage/ultralytics`; the directory exists, is writable and Ultralytics writes settings there instead of root config.
|
|
- Internal browser validation passed against `http://192.168.10.150:1202`: the live shell and Map workspace rendered without console errors, with database layer selection, Operational GIS controls, and both full-run modes visible.
|
|
- RED: `python -m pytest backend\tests\test_sprint13_yolo_preflight.py -q` failed before runtime diagnostics were implemented because `runtime` was absent from preflight output.
|
|
- `python -m pytest backend\tests\test_sprint13_yolo_preflight.py -q` passed: 8 tests.
|
|
- `python -m pytest backend\tests\test_sprint13_yolo_preflight.py backend\tests\test_sprint8b_yolo_foundation.py backend\tests\test_docker_runtime_config.py -q` passed: 39 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 369 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 369 backend tests plus frontend typecheck/build and Alembic head.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh`, `bash -n scripts/deploy_tower.sh`, `bash -n deploy/unraid/run-dockerman-container.sh` and `bash -n deploy/unraid/all-in-one-start.sh` passed.
|
|
- Local Codex host still cannot run `docker compose config` because Docker is not installed in this Windows environment; Tower Docker validation is required after push/deploy.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` with `GEOINTEL_INSTALL_AI=true` passed and redeployed Tower from commit `7a29e78`.
|
|
- Deploy-time live migration smoke passed on Tower; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`, API proxy and icon.
|
|
- Tower container check passed: remote checkout is `7a29e78`, `geointel` is healthy on `0.0.0.0:1202->80/tcp`, and `scripts/yolo_preflight.py --enabled --json` reports `dependencies_available=true`, `torch_version=2.12.1`, `ultralytics_version=8.4.88`, `cuda_available=false`, `yolo_config_dir=/app/storage/ultralytics`, `status=not_configured`, `will_download_models=false` and `will_run_inference=false`.
|
|
- RED: `python -m pytest backend\tests\test_sprint13_yolo_preflight.py::test_yolo_preflight_api_returns_canonical_envelope backend\tests\test_sprint118_yolo_preflight_ui.py -q` failed before implementation because `/api/v1/detection/yolo/preflight` returned 404 and the Detection Lab did not surface a YOLO runtime preflight panel.
|
|
- `python -m pytest backend\tests\test_sprint13_yolo_preflight.py::test_yolo_preflight_api_returns_canonical_envelope backend\tests\test_sprint118_yolo_preflight_ui.py -q` passed: 2 tests.
|
|
- `cd frontend && npm run typecheck` passed after adding the preflight API client and Detection Lab panel.
|
|
- `python -m pytest backend\tests\test_sprint13_yolo_preflight.py backend\tests\test_sprint118_yolo_preflight_ui.py backend\tests\test_sprint48_api_contract_audit.py -q` passed: 13 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 371 tests.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 371 backend tests plus frontend typecheck/build and API contract audit for 80 documented routes.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` and `bash -n scripts/deploy_tower.sh` passed.
|
|
- `git push` passed after Tower/Gitea became reachable again and pushed commit `7aa9382`.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` with `GEOINTEL_INSTALL_AI=true` passed and redeployed Tower from commit `7aa9382`.
|
|
- Deploy-time live migration smoke passed on Tower; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for `http://192.168.10.150:1202`, API proxy and icon.
|
|
- Live API check passed for `GET http://192.168.10.150:1202/api/v1/detection/yolo/preflight`: it returned the canonical `data` envelope, `status=not_configured`, `YOLO_ENABLED=false`, `torch_version=2.12.1`, `ultralytics_version=8.4.89`, `yolo_config_dir=/app/storage/ultralytics`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Tower container check passed with explicit CLI `--enabled`: `dependencies_available=true`, `torch_version=2.12.1`, `ultralytics_version=8.4.89`, `cuda_available=false`, `status=not_configured`, `will_download_models=false` and `will_run_inference=false` because no local model path is configured yet.
|
|
|
|
Limitations:
|
|
- `GEOINTEL_INSTALL_AI=true` installs optional PyTorch/Ultralytics dependencies but still requires a user-provided local model file; GeoIntel does not download weights.
|
|
- Reuse mode intentionally reuses only the latest saved map-selection dataset for QA/QC. It does not delete or mutate older derived datasets/exports.
|
|
|
|
Next recommended pass:
|
|
- Run full readiness, deploy Tower, and browser-verify both Map run modes plus configured-YOLO preflight status in the live container.
|
|
|
|
## Sprint 116 Operational GIS map workflow (2026-07-04)
|
|
|
|
Changed:
|
|
- Switched the default MapLibre basemap to an OpenStreetMap road raster style with visible attribution while preserving `VITE_MAP_STYLE_URL` as the override for managed/production map styles.
|
|
- Added a persisted database layer selector to the Map workspace so ready vector datasets can be opened directly from stored project data.
|
|
- Added an Operational GIS run panel that reuses the selected AOI bbox or active layer bbox and calls the existing persisted `vector_features` bbox selection workflow.
|
|
- Added a visible basemap policy notice when the public OpenStreetMap fallback is active.
|
|
- Added a guided operational workflow that brings query, derived dataset save, GeoJSON export, reference selection, QA/QC run and evidence handoff into the Map workspace.
|
|
- Added a one-click full GIS workflow action that runs persisted selection, saves the derived dataset, saves a GeoJSON export and optionally runs QA/QC against the selected reference dataset.
|
|
- Updated `.env.example`, `CHANGELOG.md`, `docs/TODO.md`, `docs/ENVIRONMENT_SPEC.md` and `frontend/README.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint116_operational_gis_map_workflow.py`.
|
|
|
|
Validation:
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py backend\tests\test_sprint85_map_workspace_density.py backend\tests\test_sprint106_map_bbox_extract.py backend\tests\test_sprint107_map_selection_export.py backend\tests\test_sprint108_map_selection_derived_dataset.py -q` passed: 18 tests.
|
|
- `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py backend\tests\test_sprint109_map_selection_qa_shortcut.py backend\tests\test_sprint107_map_selection_export.py backend\tests\test_sprint108_map_selection_derived_dataset.py -q` passed: 11 tests.
|
|
- `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py -q` passed: 2 tests after aligning `.env.example` with the managed-style override policy.
|
|
- RED: `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py -q` failed before implementation because `Run full GIS workflow`, `runFullGisWorkflow` and `fullWorkflowStatus` were not present.
|
|
- `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py -q` passed: 2 tests after adding the full GIS workflow action.
|
|
- `python -m pytest backend\tests\test_sprint116_operational_gis_map_workflow.py backend\tests\test_sprint106_map_bbox_extract.py backend\tests\test_sprint107_map_selection_export.py backend\tests\test_sprint108_map_selection_derived_dataset.py backend\tests\test_sprint109_map_selection_qa_shortcut.py backend\tests\test_sprint110_map_qa_evidence_drilldown.py -q` passed: 18 tests.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 364 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 364 backend tests plus frontend typecheck/build.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202` from commit `350d50a`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Internal Codex browser validation passed against `http://192.168.10.150:1202`: the Map workspace rendered a nonblank road basemap, persisted vector layer selector, operational GIS run panel and bbox query action with no console errors.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed again after the guided workflow update; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202` from commit `14b66e6`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Internal Codex browser validation passed: the Map workspace showed the basemap policy notice, database layer selector, guided operational GIS steps/actions and nonblank map canvas with no console errors.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed after aligning `.env.example`; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202` from commit `fc88ceb`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Tower runtime check passed after the one-click GIS workflow update: remote checkout is commit `5331358`, the `geointel` all-in-one container is healthy and published on `0.0.0.0:1202->80/tcp`.
|
|
- `LIVE_SMOKE_CONTAINER=geointel bash scripts/live_migration_smoke.sh` passed on Tower; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202` passed on Tower for frontend, API proxy and icon.
|
|
- Internal Codex browser validation passed against `http://192.168.10.150:1202`: the Map workspace rendered without console warnings/errors or horizontal overflow, the `Run full GIS workflow` action was present, and a live run produced the persisted query/save/export status `Dataset/export complete. Select a reference dataset to add QA/QC.`
|
|
|
|
Limitations:
|
|
- The default OpenStreetMap tile service is appropriate for local V1 testing and demos, but production or heavier deployments should set `VITE_MAP_STYLE_URL` to a managed tile/style provider.
|
|
- This pass does not add live GRB/OSM fetching, new API routes, new migrations or fake data.
|
|
- The one-click workflow can still create a new derived dataset/export each time it is clicked; a future polish pass should add explicit reuse/replace behavior for repeated operator runs.
|
|
|
|
Next recommended pass:
|
|
- Add reuse/replace behavior for repeated map workflow runs and make reference selection for QA/QC more guided in the Map workspace.
|
|
|
|
## Sprint 115 QA/QC and Exports usability layout pass (2026-07-04)
|
|
|
|
Changed:
|
|
- Added a QA/Exports usability layer to the frontend shell CSS to reduce evidence/history density without changing behavior.
|
|
- Rebalanced QA/QC and Exports workspace columns for review-first usage.
|
|
- Made QA/QC summary, handoff, drilldown, feature evidence, metric history and raw provenance surfaces more compact.
|
|
- Reduced raw QA provenance height so JSON evidence remains available but no longer dominates the page.
|
|
- Made export handoff cards, latest-artifact cards, action cards and export history controls denser and easier to scan.
|
|
- Updated `CHANGELOG.md`, `docs/TODO.md` and `frontend/README.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint115_quality_export_usability_layout.py`.
|
|
|
|
Validation:
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m pytest backend\tests\test_sprint115_quality_export_usability_layout.py backend\tests\test_sprint86_quality_workspace_density.py backend\tests\test_sprint89_export_system_density.py backend\tests\test_sprint70_quality_handoff_polish.py backend\tests\test_sprint78_export_preview_readability.py -q` passed: 15 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 362 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 362 backend tests plus frontend typecheck/build.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202` from commit `915b34c`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
|
|
Limitations:
|
|
- This pass is visual/layout only; it does not change API contracts, persistence, export generation, QA metrics or map behavior.
|
|
- Live Tower redeploy has been run; the QA/QC and Exports usability layout pass is available at `http://192.168.10.150:1202`.
|
|
|
|
Next recommended pass:
|
|
- Run a live visual audit after redeploy, then refine AI Labs/System or address any remaining visual rough edges found in the browser.
|
|
|
|
## Sprint 114 Data and Map usability layout pass (2026-07-04)
|
|
|
|
Changed:
|
|
- Added a Data/Map usability layer to the frontend shell CSS to improve the core select-and-extract workflow without changing behavior.
|
|
- Rebalanced the Data workspace columns and made dataset upload, role summaries, catalog cards, metrics and actions more compact.
|
|
- Made the Map workspace more map-first by ordering the MapLibre frame before dense controls and increasing desktop map height.
|
|
- Compressed map context, provenance, layer controls, bbox selection, selected-feature extraction and inspector surfaces.
|
|
- Updated `CHANGELOG.md`, `docs/TODO.md` and `frontend/README.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint114_data_map_usability_layout.py`.
|
|
|
|
Validation:
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m pytest backend\tests\test_sprint113_calm_workbench_layout.py backend\tests\test_sprint114_data_map_usability_layout.py backend\tests\test_sprint74_data_map_mobile_polish.py backend\tests\test_sprint85_map_workspace_density.py -q` passed: 10 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 360 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 360 backend tests plus frontend typecheck/build.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202` from commit `9721937`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
|
|
Limitations:
|
|
- This pass is visual/layout only; it does not change API contracts, persistence, map query behavior, exports or QA logic.
|
|
- Live Tower redeploy has been run; the Data and Map usability layout pass is available at `http://192.168.10.150:1202`.
|
|
|
|
Next recommended pass:
|
|
- Run a live visual audit after redeploy and then refine QA/QC and Exports in the same quieter style.
|
|
|
|
## Sprint 113 calm workbench layout pass (2026-07-04)
|
|
|
|
Changed:
|
|
- Added a calm-density layer to the frontend shell CSS to reduce visual pressure without changing workflows.
|
|
- Softened the base palette, borders and shadows.
|
|
- Made the top context bar, left workspace navigation, main heading, readiness tiles and inspector surfaces more compact.
|
|
- Hid the duplicated workspace command bar because the sidebar remains the primary persistent navigation.
|
|
- Updated `CHANGELOG.md`, `docs/TODO.md` and `frontend/README.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint113_calm_workbench_layout.py`.
|
|
|
|
Validation:
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `python -m pytest backend\tests\test_sprint49_workbench_shell_refactor.py backend\tests\test_sprint62_frontend_visual_polish.py backend\tests\test_sprint82_shell_density_polish.py backend\tests\test_sprint83_workspace_panel_hierarchy.py -q` passed: 10 tests.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m pytest backend\tests\test_sprint113_calm_workbench_layout.py -q` passed: 2 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 358 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 358 backend tests plus frontend typecheck/build.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202` from commit `0355a3d`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
|
|
Limitations:
|
|
- This pass improves visual density and hierarchy only; no API, persistence, workflow or map behavior changed.
|
|
- Live Tower redeploy has been run; the calmer layout is available at `http://192.168.10.150:1202`.
|
|
|
|
Next recommended pass:
|
|
- Do a live visual audit after redeploy and then tune individual workspaces, starting with Data and Map, based on actual screenshots.
|
|
|
|
## Sprint 112 QA evidence map overlay (2026-06-25)
|
|
|
|
Changed:
|
|
- Added `QualityEvidenceService` to resolve persisted QA/QC evidence ids back to stored geometries.
|
|
- Added `GET /api/v1/projects/{project_id}/quality-checks/{quality_check_id}/evidence/geojson`.
|
|
- The endpoint returns a canonical envelope with `quality_check_id`, dataset/run provenance, warnings and a GeoJSON FeatureCollection.
|
|
- Evidence resolution supports candidate dataset `vector_features`, candidate persisted `detections`/`segmentations` for analysis-run QA, and reference `vector_features`.
|
|
- Added QA/QC panel actions to show selected or latest evidence on the Map workspace.
|
|
- Added a MapLibre QA evidence source/layers with distinct match candidate, match reference, false-positive and false-negative styling.
|
|
- Added map overlay loading/error/clear state and a compact legend.
|
|
- Updated `docs/API_CONTRACTS.md`, `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint112_qa_evidence_overlay.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint112_qa_evidence_overlay.py -q` failed before implementation because `app.services.quality_evidence_service` did not exist.
|
|
- RED: after backend implementation, the same test failed until frontend `qaEvidenceData`/API wiring existed.
|
|
- `python -m pytest backend\tests\test_sprint112_qa_evidence_overlay.py -q` passed: 4 tests.
|
|
- `cd frontend && npm run typecheck` passed after making the MapLibre expression type explicit.
|
|
- `python -m pytest backend\tests\test_sprint112_qa_evidence_overlay.py backend\tests\test_qa_service.py backend\tests\test_sprint8c_detection_visualization_qa.py backend\tests\test_sprint9_segmentation_foundation.py -q` passed: 25 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 356 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 356 backend tests plus frontend typecheck/build.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202` from commit `b9674a0`.
|
|
- Deploy-time live migration smoke passed against the container database after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live QA evidence API smoke passed: `GET /api/v1/projects/c0b00f1f-80bf-4992-be94-f5e5e6f6bf63/quality-checks/2a291f88-5ed1-4110-ba6b-d200ec722098/evidence/geojson` returned a GeoJSON FeatureCollection with 4 persisted evidence features and 0 warnings.
|
|
- Internal Codex browser validation passed against `http://192.168.10.150:1202`: the QA/QC evidence drilldown rendered, `Show evidence overlay` opened the Map workspace, MapLibre canvas was present, the QA/QC evidence overlay reported 4 rendered features, and the mobile viewport had no horizontal overflow.
|
|
|
|
Limitations:
|
|
- The overlay is generated read-only from existing persisted evidence and geometries; no new evidence table or migration was introduced.
|
|
- Missing evidence ids are reported as warnings and do not create fake geometries.
|
|
- No provider fetching, AI dependency, real model behavior or new product domain was added.
|
|
|
|
Next recommended pass:
|
|
- Consider a small export/handoff action for the evidence overlay GeoJSON or a reusable browser smoke script that verifies the QA evidence overlay after deploy.
|
|
|
|
## Sprint 111 QA feature evidence persistence (2026-06-25)
|
|
|
|
Changed:
|
|
- Added feature-level evidence extraction to the shared QA IoU matcher.
|
|
- Dataset QA now returns and persists `match_evidence`, `false_positive_evidence` and `false_negative_evidence`.
|
|
- Detection QA and segmentation QA now use the same evidence-aware matcher and persist the same evidence keys in `quality_checks.findings_json`.
|
|
- Extended the QA/QC drilldown with compact matched, false-positive and false-negative feature id lists before the raw findings JSON.
|
|
- Updated `docs/API_CONTRACTS.md`, `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added/extended regression coverage in `backend/tests/test_qa_service.py`, `backend/tests/test_sprint7a_persistence_foundation.py` and `backend/tests/test_sprint111_qa_feature_evidence.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_qa_service.py -q` failed before implementation because `QaProviderComparisonResult` had no `match_evidence`.
|
|
- RED: `python -m pytest backend\tests\test_sprint111_qa_feature_evidence.py -q` failed before docs were updated because `docs/API_CONTRACTS.md` did not document the evidence keys.
|
|
- `python -m pytest backend\tests\test_qa_service.py -q` passed: 3 tests.
|
|
- `python -m pytest backend\tests\test_sprint8c_detection_visualization_qa.py backend\tests\test_sprint9_segmentation_foundation.py -q` passed: 18 tests.
|
|
- `python -m pytest backend\tests\test_sprint111_qa_feature_evidence.py -q` passed: 2 tests.
|
|
- `python -m pytest backend\tests\test_qa_service.py backend\tests\test_sprint7a_persistence_foundation.py -q` passed: 10 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 352 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Deploy-time live migration smoke passed against the container database, including PostGIS/version/schema checks.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live QA API smoke passed: `POST /api/v1/qa/detections-vs-reference` returned `match_evidence`, `false_positive_evidence` and `false_negative_evidence`.
|
|
- Live persisted quality-check smoke passed: `GET /api/v1/projects/c0b00f1f-80bf-4992-be94-f5e5e6f6bf63/quality-checks` returned the evidence arrays in `findings_json`.
|
|
|
|
Limitations:
|
|
- Feature-level evidence is persisted as ids/IoU metadata in `quality_checks.findings_json`; no `quality_check_items` table or first-class evidence geometry table was introduced.
|
|
- Evidence map overlays can now be built from persisted ids, but overlay generation remains future work.
|
|
- No migration, provider fetching, AI dependency, real model behavior or new product domain was added.
|
|
|
|
Next recommended pass:
|
|
- Add QA evidence overlay generation by resolving persisted evidence ids back to candidate/reference geometries and rendering false positives/false negatives as MapLibre layers.
|
|
|
|
## Sprint 110 Map QA evidence drilldown (2026-06-25)
|
|
|
|
Changed:
|
|
- Extended `QaComparisonResult` frontend typing with optional `quality_check_id`, matching the existing backend QA result payload.
|
|
- Updated `useMapSelectionQa` to keep the latest persisted map-selection quality-check id after a successful QA/QC comparison.
|
|
- Extended the Map workspace QA/QC result state with inline evidence: quality-check id, matches, false positives, false negatives, mean IoU and QA warnings.
|
|
- Added `Open QA/QC evidence` handoff from Map workspace to the existing QA/QC workspace drilldown, avoiding a parallel QA detail system.
|
|
- Added compact styling for the Map QA evidence and warning surface.
|
|
- Updated `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint110_map_qa_evidence_drilldown.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint110_map_qa_evidence_drilldown.py -q` failed before implementation because the hook/evidence wiring was absent.
|
|
- `python -m pytest backend\tests\test_sprint110_map_qa_evidence_drilldown.py -q` passed: 2 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 349 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Deploy-time live migration smoke passed against the container database, including PostGIS/version/schema checks.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live API proxy smoke passed: `GET http://192.168.10.150:1202/api/v1/projects` returned the demo project envelope.
|
|
|
|
Limitations:
|
|
- The Map evidence handoff opens the existing QA/QC workspace; selecting a specific historical check inside that workspace remains governed by the QA/QC panel's own latest-check behavior.
|
|
- False-positive and false-negative geometries are summarized by persisted metrics/findings; dedicated map overlays for unmatched evidence remain future work.
|
|
- No backend API contract, migration, provider fetching, AI dependency, real model behavior or new product domain was added.
|
|
|
|
Next recommended pass:
|
|
- Add optional unmatched-evidence map overlays once the QA persistence model stores explicit matched/unmatched feature ids or geometries.
|
|
|
|
## Sprint 109 Map selection QA shortcut (2026-06-25)
|
|
|
|
Changed:
|
|
- Added `useMapSelectionQa` to keep Map workspace QA/QC orchestration out of `App.tsx`.
|
|
- Added a Map workspace QA/QC shortcut after `Save as dataset`, allowing the latest derived map selection dataset to be compared against a selected reference dataset.
|
|
- The shortcut reuses the existing `qaApi.runQa` flow and refreshes persisted quality checks/project data after completion.
|
|
- Added inline precision, recall, F1 and quality-check status feedback in the Map workspace.
|
|
- Added compact styling for the Map selection QA surface.
|
|
- Updated `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint109_map_selection_qa_shortcut.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint109_map_selection_qa_shortcut.py -q` failed before implementation because the hook and wiring were absent.
|
|
- `python -m pytest backend\tests\test_sprint109_map_selection_qa_shortcut.py -q` passed: 2 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 347 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Deploy-time live migration smoke passed against the container database, including PostGIS/version/schema checks.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live API QA smoke passed: `POST /api/v1/qa/detections-vs-reference` compared derived map selection dataset `266aa1de-908e-4bc4-987e-996374ad747b` against reference dataset `21dd968a-2ba4-4798-89d4-f3af91f9d24c`, returned `precision=0.5`, `recall=0.5`, `f1_score=0.5`, `mean_iou=0.8339768339652592` and persisted quality check `1730c270-6dee-4000-8813-12aa91df6b33`.
|
|
- Live quality-check list smoke passed: `GET /api/v1/projects/c0b00f1f-80bf-4992-be94-f5e5e6f6bf63/quality-checks` returned the persisted map-selection QA check with precision, recall, F1, mean IoU, false-positive and false-negative metrics.
|
|
|
|
Limitations:
|
|
- The Map shortcut currently uses the existing QA comparison defaults with IoU threshold `0.5`.
|
|
- QA results are summarized inline; detailed false-positive/false-negative evidence remains in the QA/QC workspace.
|
|
- No backend API contract, migration, provider fetching, AI dependency, real model behavior or new product domain was added.
|
|
|
|
Next recommended pass:
|
|
- Add a QA result drilldown/handoff from the Map workspace so operators can inspect persisted false-positive and false-negative evidence directly after running selection QA/QC.
|
|
|
|
## Sprint 108 Map selection derived datasets (2026-06-25)
|
|
|
|
Changed:
|
|
- Added `POST /api/v1/projects/{project_id}/datasets/{dataset_id}/vector/select/derive`.
|
|
- Added `VectorSelectionDeriveRequest` for bbox selection-to-derived-dataset requests.
|
|
- Added `VectorOperationsService.derive_selection_dataset`, which selects persisted PostGIS `vector_features`, writes a derived GeoJSON dataset artifact, stores source provenance and re-indexes the derived features into `vector_features`.
|
|
- Added `Save as dataset` to the Map workspace selection result state with loading/error/latest-dataset feedback.
|
|
- Updated frontend dataset API typing and wiring for selection-derived datasets.
|
|
- Updated `docs/API_CONTRACTS.md`, `backend/README.md`, `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint108_map_selection_derived_dataset.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint108_map_selection_derived_dataset.py -q` failed before implementation because the service, route and frontend contracts were absent.
|
|
- `python -m pytest backend\tests\test_sprint108_map_selection_derived_dataset.py -q` passed: 4 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 345 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Deploy-time live migration smoke passed against the container database, including PostGIS/version/schema checks.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live API smoke passed: `POST /api/v1/projects/c0b00f1f-80bf-4992-be94-f5e5e6f6bf63/datasets/6e5a3926-1a15-4136-b54f-ba5de9ec0e84/vector/select/derive` created derived dataset `266aa1de-908e-4bc4-987e-996374ad747b` with `dataset_role="derived"`, `source_name="map_selection"` and 2 features.
|
|
- Live API content smoke passed: `GET /api/v1/projects/c0b00f1f-80bf-4992-be94-f5e5e6f6bf63/datasets/266aa1de-908e-4bc4-987e-996374ad747b/content` returned a GeoJSON `FeatureCollection` with 2 features.
|
|
- Live PostGIS query smoke passed: `POST /api/v1/projects/c0b00f1f-80bf-4992-be94-f5e5e6f6bf63/datasets/266aa1de-908e-4bc4-987e-996374ad747b/vector/select` returned 2 features from the derived dataset's persisted `vector_features`.
|
|
|
|
Limitations:
|
|
- Selection-derived datasets are bbox-only and use EPSG:4326 coordinates.
|
|
- Empty selections are rejected with `VECTOR_OPERATION_EMPTY_RESULT`.
|
|
- No migrations, live provider fetching, AI dependency, real model behavior or new product domain were added.
|
|
|
|
Next recommended pass:
|
|
- Add an explicit QA/QC shortcut from a selected derived map dataset to compare it against a reference dataset without leaving the Map workspace.
|
|
|
|
## Sprint 107 Map selection export handoff (2026-06-25)
|
|
|
|
Changed:
|
|
- Added `vector_selection` to the GeoJSON export contract.
|
|
- Added `ExportService.export_vector_selection_geojson`, which queries persisted PostGIS `vector_features` through `VectorFeatureService.select_features_by_bbox`, writes the selected FeatureCollection and persists an `exports` row with `export_type="vector_selection_geojson"`.
|
|
- Extended `POST /api/v1/exports/geojson` to accept `export_kind="vector_selection"` with EPSG:4326 bbox and feature limit.
|
|
- Added frontend export API typing for bbox/limit and `useExportWorkflow.exportMapSelectionGeoJson`.
|
|
- Added `Save area export` to the Map workspace selection result state with loading/error/latest-path feedback.
|
|
- Updated `docs/API_CONTRACTS.md`, `backend/README.md`, `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint107_map_selection_export.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint107_map_selection_export.py -q` failed before implementation because the selection export service, route contract and frontend wiring were absent.
|
|
- `python -m pytest backend\tests\test_sprint107_map_selection_export.py -q` passed: 3 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 341 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed; rebuilt and redeployed the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Deploy-time live migration smoke passed against the container database, including PostGIS/version/schema checks.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live API smoke passed: `POST /api/v1/exports/geojson` with `export_kind="vector_selection"` created export `1d16c78b-7e8c-4081-8edc-5be38811b50e` with `export_type="vector_selection_geojson"`, `feature_count=2` and `source_table="vector_features"`.
|
|
- Live API content smoke passed: `GET /api/v1/exports/1d16c78b-7e8c-4081-8edc-5be38811b50e/content` returned a GeoJSON `FeatureCollection` with 2 persisted vector features.
|
|
|
|
Limitations:
|
|
- Selection exports are bbox-only and reuse the same EPSG:4326 constraints as the Map area selection endpoint.
|
|
- Saving a selection creates an export artifact, not a derived dataset.
|
|
- No migrations, live provider fetching, AI dependency, real model behavior or new product domain were added.
|
|
|
|
Next recommended pass:
|
|
- Add a live browser smoke that exercises `Save area export`, confirms the export appears in Export Center history and previews/downloads the persisted `vector_selection_geojson` artifact.
|
|
|
|
## Sprint 106 Map area selection extract (2026-06-25)
|
|
|
|
Changed:
|
|
- Added `POST /api/v1/projects/{project_id}/datasets/{dataset_id}/vector/select` for read-only bbox selection over persisted PostGIS `vector_features`.
|
|
- Added `VectorSelectionBBox`, `VectorSelectionRequest` and `VectorSelectionResponse` schemas and exported them through the backend schema module.
|
|
- Added `VectorFeatureService.select_features_by_bbox`, including EPSG:4326 bbox validation, feature limit capping, PostGIS `ST_Intersects` query and GeoJSON FeatureCollection conversion from persisted geometries.
|
|
- Added frontend `selectVectorFeatures` API client support and `useMapSelectionExtract`.
|
|
- Extended the Map workspace with an `Area selection` panel, two-click map bbox selection, manual bbox inputs, selected-feature/AOI/active-layer bbox shortcuts, area GeoJSON download/copy actions and a compact selected-feature table.
|
|
- Extended `GeoMap` with `selection-bbox` and `selection-result` MapLibre GeoJSON overlays.
|
|
- Updated `docs/API_CONTRACTS.md`, `frontend/README.md`, `backend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added regression coverage in `backend/tests/test_sprint106_map_bbox_extract.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint106_map_bbox_extract.py -q` failed before implementation because the vector selection service, route and frontend contracts were absent.
|
|
- `python -m pytest backend\tests\test_sprint106_map_bbox_extract.py -q` passed: 5 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 338 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed commit `851d722` to Tower on port 1202.
|
|
- Deploy-time live migration smoke passed after database readiness attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed with demo project, AOI, 3/3 ready datasets and seeded QA/QC.
|
|
- Live API smoke passed against `POST /api/v1/projects/{project_id}/datasets/{dataset_id}/vector/select`: demo vector dataset returned 2 persisted features as a GeoJSON FeatureCollection.
|
|
- Internal browser validation passed against `http://192.168.10.150:1202`: Map workspace rendered `Area selection`, `Start map bbox` and `Run area extract`; `Use layer bbox` filled EPSG:4326 bbox values; `Run area extract` returned 2 features, showed `vector_features` source plus `Download area GeoJSON` and `Copy area GeoJSON`; console warnings/errors were empty.
|
|
|
|
Limitations:
|
|
- Selection shape is bbox-only in EPSG:4326. Polygon/lasso selection remains future work.
|
|
- The endpoint is read-only and does not create exports or derived datasets; operators can download the selected GeoJSON client-side.
|
|
- No migrations, live provider fetching, AI dependency, real model behavior or new product domain were added.
|
|
|
|
Next recommended pass:
|
|
- Add a browser/live smoke around the area selection panel after deploy, then consider export-center handoff for persisted selection artifacts if V1 needs server-side audit retention.
|
|
|
|
## Sprint 105 Map feature extract (2026-06-25)
|
|
|
|
Changed:
|
|
- Added a `Selection & extract` surface to `frontend/src/components/map/MapWorkspace.tsx`.
|
|
- Clicking a visible map feature now gives operators a focused extraction panel with geometry type, coordinate count, EPSG:4326 bbox, property count and property table.
|
|
- Added client-side `Download selected GeoJSON`, `Copy selected properties` and `Clear selection` actions for the clicked feature.
|
|
- Added a dedicated `selected-feature` MapLibre source with fill/line/circle highlight layers in `frontend/src/components/GeoMap.tsx`.
|
|
- Wired the selected feature highlight through `frontend/src/App.tsx`.
|
|
- Added responsive selection/extract CSS in `frontend/src/styles/app.css`.
|
|
- Updated `frontend/README.md`, `CHANGELOG.md` and `docs/TODO.md`.
|
|
- Added `backend/tests/test_sprint105_map_feature_extract.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint105_map_feature_extract.py -q` failed before implementation because the extract panel, selected-feature highlight layer and CSS contracts were absent.
|
|
- `python -m pytest backend\tests\test_sprint105_map_feature_extract.py backend\tests\test_sprint19_map_workbench.py backend\tests\test_sprint85_map_workspace_density.py -q` passed: 9 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -q` passed: 333 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed commit `561304c` to Tower on port 1202.
|
|
- Deploy-time live migration smoke passed after database readiness attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Internal browser live check confirmed the Map workspace renders the `Selection & extract` panel and empty selection guidance at `http://192.168.10.150:1202`.
|
|
- Live frontend bundle check confirmed `Selection & extract`, `selected-feature` and `Download selected GeoJSON` markers in `/assets/index-DVYT-JPu.js`.
|
|
|
|
Limitations:
|
|
- This pass extracts the single currently clicked and loaded map feature only.
|
|
- Rectangle, lasso or polygon selection over persisted PostGIS `vector_features` still requires a backend spatial query endpoint and drawing workflow.
|
|
- Internal browser canvas clicks did not reliably hit the small demo polygon features during validation; static regression tests, TypeScript build and live bundle markers verify the MapLibre selection wiring, and the live UI panel is present.
|
|
- No backend API contract, migration, provider fetching, AI dependency or persistence behavior changed.
|
|
|
|
Next recommended pass:
|
|
- Add map area/rectangle selection backed by a PostGIS spatial-query endpoint for multi-feature extraction, then expose export handoff for the selected result set.
|
|
|
|
## Sprint 100 raster tile Segmentation Lab handoff (2026-06-24)
|
|
|
|
Changed:
|
|
- Added `segmentationTileManifestPath` state to `frontend/src/hooks/useSegmentationWorkflow.ts`.
|
|
- Sent `tile_manifest_path` in existing segmentation run requests when the field is populated.
|
|
- Added a `Tile manifest` input to `frontend/src/components/segmentation/SegmentationLab.tsx`.
|
|
- Added a `Use in Segmentation Lab` handoff beside the existing Detection Lab handoff in `frontend/src/components/datasets/RasterControls.tsx`.
|
|
- Wired the handoff through `frontend/src/components/datasets/DatasetDetailPanel.tsx` and `frontend/src/App.tsx`.
|
|
- Updated `frontend/README.md` and `CHANGELOG.md`.
|
|
- Added `backend/tests/test_sprint100_segmentation_manifest_handoff.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint100_segmentation_manifest_handoff.py -q` failed before implementation because segmentation manifest state and handoff wiring did not exist.
|
|
- `python -m pytest backend\tests\test_sprint100_segmentation_manifest_handoff.py -q` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `python -m pytest backend\tests\test_sprint100_segmentation_manifest_handoff.py backend\tests\test_sprint99_raster_ui_handoff.py backend\tests\test_sprint88_ai_lab_density.py backend\tests\test_sprint9_segmentation_foundation.py -q` passed: 17 tests.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `python -m pytest -q` from `backend/` passed: 323 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `140314a` to Tower on port 1202.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202` passed: inspect, preview, stats and `raster.tile`.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed with `3/3 ready` datasets.
|
|
- `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202` passed.
|
|
- Live frontend bundle check confirmed `Latest tile manifest`, `Use in Detection Lab`, `Use in Segmentation Lab` and `Raster tile manifest path` markers in `/assets/index-DygxAZpB.js`.
|
|
|
|
Limitations:
|
|
- Frontend handoff only; no backend route, migration, provider fetching, AI dependency or model behavior changes.
|
|
- Segmentation remains governed by the existing model registry and explicit-run controls.
|
|
|
|
Next recommended pass:
|
|
- Add a real browser interaction smoke for clicking both raster manifest handoff buttons into AI Labs, then continue with small usability polish around AI model readiness.
|
|
|
|
## Sprint 99 raster tile Detection Lab handoff (2026-06-23)
|
|
|
|
Changed:
|
|
- Added latest raster tile manifest tracking to `frontend/src/hooks/useDatasetWorkflow.ts` from persisted `raster.tile` job results and direct tile generation responses.
|
|
- Surfaced the latest manifest path in `frontend/src/components/datasets/RasterControls.tsx`.
|
|
- Added a `Use in Detection Lab` handoff that fills the selected raster dataset and tile manifest path in the existing Detection Lab state.
|
|
- Updated `frontend/src/components/datasets/DatasetDetailPanel.tsx`, `frontend/src/App.tsx`, `frontend/README.md` and `CHANGELOG.md`.
|
|
- Added `backend/tests/test_sprint99_raster_ui_handoff.py`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint99_raster_ui_handoff.py -q` failed before implementation because the raster tile manifest state and handoff wiring did not exist.
|
|
- `python -m pytest backend\tests\test_sprint99_raster_ui_handoff.py -q` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `python -m pytest backend\tests\test_sprint99_raster_ui_handoff.py backend\tests\test_sprint95_raster_pipeline_hardening.py backend\tests\test_sprint28_dataset_workflow_hook.py -q` passed: 6 tests.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `python -m pytest -q` from `backend/` passed: 322 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `aa41dfa` to Tower on port 1202.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202` passed: inspect, preview, stats and `raster.tile`.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed with `3/3 ready` datasets.
|
|
- `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202` passed.
|
|
- Live frontend bundle check confirmed `Latest tile manifest` and `Use in Detection Lab` markers in `/assets/index-gJWV1CAi.js`.
|
|
|
|
Limitations:
|
|
- Frontend handoff only; no API contract, migration, provider fetching, AI inference or model behavior changes.
|
|
- The Detection Lab still follows the existing model availability and explicit-run constraints.
|
|
|
|
Next recommended pass:
|
|
- Add a browser interaction smoke for clicking the raster manifest handoff into Detection Lab, or continue with segmentation tile-manifest prefill parity.
|
|
|
|
## Sprint 98 demo raster workflow smoke (2026-06-23)
|
|
|
|
Changed:
|
|
- Added `scripts/verify_demo_raster_workflow.sh`, a browser-facing runtime smoke for the seeded `demo_context_raster.tif` fixture.
|
|
- Added the new smoke to readiness syntax checks.
|
|
- Hardened `RasterOperationsService.tile` so tile manifest bounds accept both Rasterio tuple bounds and object bounds.
|
|
- Updated `backend/tests/test_raster_operations_service.py` to cover tuple-based window bounds.
|
|
- Added `backend/tests/test_sprint98_raster_workflow_smoke.py`.
|
|
- Updated `scripts/README.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint98_raster_workflow_smoke.py -q` failed before implementation because `scripts/verify_demo_raster_workflow.sh` did not exist.
|
|
- `python -m pytest backend\tests\test_sprint98_raster_workflow_smoke.py backend\tests\test_readiness_gate.py -q` passed: 14 tests.
|
|
- `bash -n scripts/verify_demo_raster_workflow.sh` passed.
|
|
- Live diagnostic run of `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202` showed inspect, preview and stats passed, then raster tile failed with HTTP 500.
|
|
- Tower backend logs identified the root cause: `rasterio.windows.bounds(...)` returned a tuple, while tile manifest generation expected `.left/.bottom/.right/.top` attributes.
|
|
- RED: `python -m pytest backend\tests\test_raster_operations_service.py::test_raster_tile_returns_manifest_payload -q` reproduced the live `AttributeError` after updating the fixture to tuple bounds.
|
|
- `python -m pytest backend\tests\test_raster_operations_service.py::test_raster_tile_returns_manifest_payload backend\tests\test_sprint98_raster_workflow_smoke.py -q` passed: 2 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `python -m pytest -q` from `backend/` passed: 321 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `7b093c0` to Tower on port 1202.
|
|
- `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202` initially proved the tile fix but exposed the existing job terminal status convention as `success` rather than `completed`; the smoke now accepts `success` and `completed`.
|
|
- `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202` passed after the smoke status fix: inspect, preview, stats and raster tile manifest all succeeded.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/run_readiness_check.sh` passed again after the smoke status update.
|
|
|
|
Limitations:
|
|
- The raster smoke creates a small tile set each run. It is appropriate for local/runtime validation, not high-volume benchmarking.
|
|
- The smoke intentionally does not run AI inference or external imagery/provider fetching.
|
|
|
|
Next recommended pass:
|
|
- Run full readiness, redeploy, then verify the raster smoke live against `http://192.168.10.150:1202`.
|
|
|
|
## Sprint 97 demo raster fixture workflow (2026-06-23)
|
|
|
|
Changed:
|
|
- Added `raster_dataset_id` to the demo workflow response contract.
|
|
- Extended `DemoWorkflowService` with a deterministic in-memory `demo_context_raster.tif` GeoTIFF fixture persisted through `StorageService` and the existing `datasets` table as a ready `raster`/`fixture` source dataset.
|
|
- Updated the frontend demo workflow hook so Detection and Segmentation Labs receive the seeded raster dataset while the candidate vector remains selected for Data, Map and Export review.
|
|
- Updated default-state and interaction smokes to require candidate vector, reference vector and raster fixture datasets as `3/3 ready`.
|
|
- Updated `scripts/README.md`, `frontend/README.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `python -m pytest backend\tests\test_sprint97_demo_raster_fixture.py -q` failed before implementation because the demo schema/service and frontend hook did not expose or select a raster fixture.
|
|
- `python -m pytest backend\tests\test_sprint97_demo_raster_fixture.py -q` passed: 2 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `python -m pytest backend\tests\test_sprint97_demo_raster_fixture.py backend\tests\test_sprint15_demo_workflow.py backend\tests\test_sprint21_demo_workflow_smoke.py backend\tests\test_readiness_gate.py -q` passed: 21 tests.
|
|
- `bash -n scripts/verify_workbench_default_state.sh` passed.
|
|
- `bash -n scripts/verify_workbench_interactions.sh` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `python -m pytest backend\tests\test_sprint96_useful_default_context.py backend\tests\test_sprint39_frontend_orchestration_hooks.py -q` passed: 11 tests.
|
|
- `python -m pytest -q` from `backend/` passed: 320 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `ca730ed` to Tower on port 1202.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed and reported `Datasets: 3/3 ready`.
|
|
- `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202` passed and verified candidate, reference and raster fixture datasets.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202` passed.
|
|
- Live API check confirmed `demo_context_raster.tif` as `raster`/`fixture`/`ready` with `EPSG:4326`.
|
|
- Chrome headless screenshot of `http://192.168.10.150:1202` showed the populated Overview workspace, selected demo context and no obvious desktop horizontal overflow.
|
|
|
|
Limitations:
|
|
- The raster fixture is a tiny generated local GeoTIFF for V1 workflow validation only. It does not represent external imagery and does not enable real AI inference.
|
|
- No migrations, provider fetching, real YOLO/SAM behavior or product scope beyond the existing offline demo workflow changed.
|
|
- Optional `scripts/capture_workbench_screenshots.sh` could not run in this Windows runner because Playwright is not installed; Chrome headless was used for the live visual check instead.
|
|
|
|
Next recommended pass:
|
|
- Continue with a focused raster operation happy-path smoke against the seeded raster fixture: inspect, preview, stats and tiling through the browser-facing API.
|
|
|
|
## Sprint 48 Backend API contract audit (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `scripts/audit_api_contracts.py` to import the FastAPI app, enumerate the implemented `GET`/`POST`/`PATCH`/`DELETE` route surface and compare it with active `### METHOD route` headings in `docs/API_CONTRACTS.md`.
|
|
- Added the API contract audit to `scripts/run_readiness_check.sh`.
|
|
- Corrected contract drift in `docs/API_CONTRACTS.md`:
|
|
- documented `GET/PATCH /api/v1/projects/{project_id}/areas/{area_id}`;
|
|
- corrected vector stats from `POST` to implemented `GET`;
|
|
- documented `GET /api/v1/projects/{project_id}/datasets/{dataset_id}/content`;
|
|
- changed non-implemented building-stats, legacy analysis object-detection/segmentation and YOLO export entries from active route headings to future-route notes.
|
|
- Added `backend/tests/test_sprint48_api_contract_audit.py`.
|
|
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `cd backend && python -m pytest tests/test_sprint48_api_contract_audit.py -q` failed before implementation because the audit script, readiness integration and route docs were missing.
|
|
- RED: `python scripts/audit_api_contracts.py` reported missing docs for 4 implemented routes and 5 stale documented routes.
|
|
- `python scripts/audit_api_contracts.py` passed: 76 implemented routes matched docs and 2 explicit non-envelope endpoints were tracked.
|
|
- `cd backend && python -m pytest tests/test_sprint48_api_contract_audit.py tests/test_readiness_gate.py -q` passed: 12 tests.
|
|
- `python -m py_compile scripts/audit_api_contracts.py` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 201 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed and included `API contract audit OK`.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed the all-in-one container to Tower on port 1202.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202` passed after redeploy.
|
|
|
|
Limitations:
|
|
- This pass audits route documentation presence, implemented/stale route drift and explicit non-envelope exceptions. It does not yet exercise every error path response body at runtime.
|
|
- No API behavior, migrations, provider fetching, AI behavior or product capabilities changed.
|
|
|
|
Next recommended pass:
|
|
- Add an error-envelope runtime audit for representative invalid/missing-resource paths across projects, datasets, providers, detection, segmentation, QA and exports.
|
|
|
|
## Sprint 47 Workbench interaction smoke (2026-06-17)
|
|
|
|
Changed:
|
|
- Added stable `data-testid` anchors to the existing project, area, map, dataset, QA/QC and export controls so browser checks can target real controls instead of brittle text/layout selectors.
|
|
- Added `scripts/verify_workbench_interactions.sh`, a dependency-light runtime smoke that verifies the backing state for project switching, AOI/map selection, dataset selection, QA refresh and export refresh through the browser-facing API proxy.
|
|
- Added the script syntax check to `scripts/run_readiness_check.sh`.
|
|
- Added `backend/tests/test_sprint47_workbench_interaction_smoke.py` to keep the UI anchors, readiness gate and interaction smoke contract in place.
|
|
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `cd backend && python -m pytest tests/test_sprint47_workbench_interaction_smoke.py -q` failed before implementation because the UI anchors, readiness script reference and interaction smoke script were missing.
|
|
- `cd backend && python -m pytest tests/test_sprint47_workbench_interaction_smoke.py -q` passed: 3 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `bash -n scripts/verify_workbench_interactions.sh` passed.
|
|
- `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202` passed against the pre-deploy runtime API surface.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 198 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed the all-in-one container to Tower on port 1202.
|
|
- `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- Live browser check against `http://192.168.10.150:1202` passed using the new anchors: project, map, dataset, QA/QC and export panels were present; QA refresh and export metadata actions worked; latest export updated; no horizontal overflow was detected.
|
|
|
|
Limitations:
|
|
- The shell smoke validates the state behind the controls but does not click rendered controls by itself. The added `data-testid` anchors are intended for Codex/browser click checks and future browser artifact automation.
|
|
- No API contracts, migrations, provider fetching, AI behavior or product capabilities changed.
|
|
|
|
Next recommended pass:
|
|
- Add persisted screenshot artifact automation for the anchored browser pass, or move to a backend service contract audit if UI stabilization is sufficient.
|
|
|
|
## Sprint 46 Workbench default-state smoke (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `scripts/verify_workbench_default_state.sh`, a dependency-light runtime smoke for the browser-facing workbench default demo state.
|
|
- The smoke calls the offline demo workflow, then verifies `GeoIntel Demo - Building QA`, `Demo AOI - Geel buildings`, `2/2 ready` datasets and a persisted QA/QC result through canonical `data.items` envelopes.
|
|
- Added the script syntax check to `scripts/run_readiness_check.sh`.
|
|
- Extended `backend/tests/test_readiness_gate.py` so the readiness gate and script keep covering the default-state smoke contract.
|
|
- Updated `scripts/README.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- RED: `cd backend && python -m pytest tests/test_readiness_gate.py -q` failed before implementation because `verify_workbench_default_state.sh` was missing and readiness did not reference it.
|
|
- `cd backend && python -m pytest tests/test_readiness_gate.py -q` passed: 9 tests.
|
|
- `bash -n scripts/verify_workbench_default_state.sh` passed.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 195 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed the all-in-one container to Tower on port 1202.
|
|
- `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed after redeploy.
|
|
- Live browser check against `http://192.168.10.150:1202` passed at 1280x720: selected project `GeoIntel Demo - Building QA`, status showed `1 area`, `2/2 ready` datasets and `2 checks`, map workspace rendered and no horizontal overflow was detected.
|
|
|
|
Limitations:
|
|
- The committed smoke validates the connected default workbench state through HTTP/API contracts, not pixel layout. Browser screenshot and overflow validation remain a Codex/browser verification step.
|
|
- No API contracts, migrations, provider fetching, AI behavior or product capabilities changed.
|
|
|
|
Next recommended pass:
|
|
- Add a deeper browser interaction smoke for core controls, starting with project switching, area selection and map layer opacity.
|
|
|
|
## Sprint 45 Default demo selection polish (2026-06-17)
|
|
|
|
Changed:
|
|
- Updated `frontend/src/hooks/useProjectWorkspace.ts` so cold-start project selection preserves an existing selected project, honors an explicit preferred project and otherwise prefers a populated demo/workbench project over an empty first project.
|
|
- New project creation now selects the newly created project immediately after creation.
|
|
- Updated `frontend/src/hooks/useDemoWorkflow.ts` so demo seed refresh passes the seeded project id to `loadProjects`.
|
|
- Extended orchestration tests to cover preferred demo selection and project creation selection behavior.
|
|
|
|
Validation:
|
|
- `cd backend && python -m pytest tests/test_sprint39_frontend_orchestration_hooks.py tests/test_sprint21_demo_workflow_smoke.py -q` passed: 11 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 194 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Browser check against local Vite preview with live backend passed after async settle: selected project `GeoIntel Demo - Building QA`, status showed `1 area`, `2/2 ready` datasets and `2 checks`.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed the all-in-one container to Tower on port 1202.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
- Live browser check against `http://192.168.10.150:1202` passed after async settle: selected project `GeoIntel Demo - Building QA`, status showed `1 area`, `2/2 ready` datasets and `2 checks`, with no horizontal overflow.
|
|
|
|
Limitations:
|
|
- No API contracts, migrations, provider fetching, AI behavior or product capabilities changed.
|
|
- The heuristic probes up to eight project candidates on cold start; a future backend list endpoint with area/dataset counts would make this cleaner.
|
|
|
|
Next recommended pass:
|
|
- Add a lightweight browser regression script for the demo start page once the UI state is stable enough to automate end to end.
|
|
|
|
## Sprint 44 Workbench UI polish pass (2026-06-17)
|
|
|
|
Changed:
|
|
- Reworked `frontend/src/styles/app.css` from a minimal browser-default stylesheet into a compact GIS workbench skin with modern controls, restrained neutral/green accents, scroll-contained long panels and responsive layout rules.
|
|
- Promoted `MapWorkspace` above the dense workflow grid in `frontend/src/App.tsx` so GIS context is visible before lower-detail provider, AI, QA and export panels.
|
|
- Moved `DatasetPanel` into the first workflow row beside project/area/provider setup.
|
|
- Added a static layout regression test in `backend/tests/test_sprint30_workbench_components.py` for map-first ordering and scroll-contained workflow panels.
|
|
- Updated frontend README and changelog.
|
|
|
|
Validation:
|
|
- Browser visual check against local Vite preview passed at desktop width: map-first layout visible, workflow panels scroll-contained, page height reduced from roughly 14.9k px to roughly 3.2k px.
|
|
- Browser responsive check at 390px width passed with no horizontal overflow.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 194 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `geointel-all-in-one:latest` to Tower on port 1202.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
- Live browser check against `http://192.168.10.150:1202` passed: map-first layout rendered, scroll height roughly 3.2k px and no desktop horizontal overflow detected.
|
|
|
|
Limitations:
|
|
- No API contracts, migrations, provider fetching, AI behavior or product capabilities changed.
|
|
- This is still a dense operational workbench rather than a designed product shell with navigation or tabs.
|
|
- The live default selected project can still be an empty project with the demo name; project selection polish remains a separate follow-up.
|
|
|
|
Next recommended pass:
|
|
- Add UI browser regression coverage for the map-first demo workflow and polish the default project/demo selection state.
|
|
|
|
## Sprint 43 Workbench bootstrap hook decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved frontend bootstrap, project-change reload/reset and detection/segmentation result reload effects from `frontend/src/App.tsx` into `frontend/src/hooks/useWorkbenchBootstrap.ts`.
|
|
- Kept `App.tsx` as a composition root that wires hook outputs into panels; it no longer imports `useEffect`.
|
|
- Extended orchestration tests so lifecycle side effects stay in the bootstrap hook.
|
|
- Updated frontend README, changelog and TODO status.
|
|
|
|
Validation:
|
|
- `cd backend && python -m pytest tests/test_sprint39_frontend_orchestration_hooks.py -q` passed: 9 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 193 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `geointel-all-in-one:latest` to Tower on port 1202.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
- `App.tsx` size audit after extraction: 621 lines; `useWorkbenchBootstrap.ts`: 81 lines.
|
|
|
|
Limitations:
|
|
- No UX behavior, API contracts, migrations, provider fetching or AI behavior changed.
|
|
- `App.tsx` line count remains high because it explicitly wires many panel props; the remaining size is primarily composition.
|
|
|
|
Next recommended pass:
|
|
- Pick the next V1 stabilization focus: UI browser regression coverage, backend service contract audit, or golden dataset expansion.
|
|
|
|
## Sprint 42 App entrypoint cleanup (2026-06-17)
|
|
|
|
Changed:
|
|
- Removed the stale `FormEvent`/`useState` React imports from `frontend/src/App.tsx`.
|
|
- Removed the UTF-8 BOM from `App.tsx` so patches and static checks use normal UTF-8 text.
|
|
- Added a regression test that verifies the clean entrypoint encoding and React import set.
|
|
- Recorded the current `App.tsx` size audit: 622 lines after the orchestration hook decomposition passes.
|
|
|
|
Validation:
|
|
- `cd backend && python -m pytest tests/test_sprint39_frontend_orchestration_hooks.py -q` passed: 8 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 192 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `geointel-all-in-one:latest` to Tower on port 1202.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
|
|
Limitations:
|
|
- No UX behavior, API contracts, migrations, provider fetching or AI behavior changed.
|
|
- `App.tsx` remains a large composition root; the remaining size is primarily panel wiring and hook outputs.
|
|
|
|
Next recommended pass:
|
|
- Optional bootstrap-effect extraction if another no-behavior size reduction is useful.
|
|
|
|
## Sprint 41 Demo workflow hook decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved offline demo workflow orchestration from `frontend/src/App.tsx` into `frontend/src/hooks/useDemoWorkflow.ts`.
|
|
- Preserved the existing backend fixture seed flow and all cross-module selection updates for project, candidate/reference datasets, map AOI, QA/QC, detection, segmentation and exports.
|
|
- Extended static frontend orchestration tests so `demoApi.seedWorkflow` is owned by the new hook and not `App.tsx`.
|
|
- Updated frontend README, changelog and TODO status.
|
|
|
|
Validation:
|
|
- `cd backend && python -m pytest tests/test_sprint21_demo_workflow_smoke.py tests/test_sprint39_frontend_orchestration_hooks.py -q` passed: 9 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 191 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `cd backend && python -m alembic heads` passed: `202606120900 (head)`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1` passed and redeployed `geointel-all-in-one:latest` to Tower on port 1202.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
|
|
Limitations:
|
|
- No UX behavior, API contracts, migrations, provider fetching or AI behavior changed.
|
|
- `App.tsx` still has a UTF-8 BOM and a stale React import cleanup opportunity; TypeScript build is unaffected.
|
|
|
|
Next recommended pass:
|
|
- Do a final `App.tsx` size/import cleanup pass.
|
|
|
|
## Sprint 40 Project workspace hook decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved project list/create state, area create state and project-scoped area/dataset loading into `frontend/src/hooks/useProjectWorkspace.ts`.
|
|
- Kept demo workflow orchestration in `App.tsx` because it coordinates project, dataset, QA/QC, detection, segmentation and export selections across multiple hooks.
|
|
- Moved default clip-area fallback selection into `frontend/src/hooks/useDatasetWorkflow.ts`.
|
|
- Moved default map-area fallback selection into `frontend/src/hooks/useMapWorkspaceState.ts`.
|
|
- Extended static frontend orchestration tests to lock these ownership boundaries.
|
|
- Updated frontend README, changelog and TODO status.
|
|
|
|
Validation:
|
|
- `cd backend && python -m pytest tests/test_sprint39_frontend_orchestration_hooks.py -q` passed: 6 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 190 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the all-in-one Tower container at `http://192.168.10.150:1202`.
|
|
- Tower live migration smoke passed and reported `Database collation version: ok`.
|
|
- Tower browser runtime verification passed.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- No UX behavior, API contracts, migrations, provider fetching or AI behavior changed.
|
|
- Demo workflow orchestration remains in `App.tsx` until it can be safely split without obscuring cross-module state updates.
|
|
|
|
Next recommended pass:
|
|
- Run full readiness and Tower deploy verification, then consider extracting the demo workflow coordinator into a dedicated hook.
|
|
|
|
## Sprint 39 Frontend orchestration decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved provider capability loading state and `externalApi.listProviders` calls into `frontend/src/hooks/useProviderCapabilities.ts`.
|
|
- Moved change-detection state, validation and `analysisApi.runChangeDetection` calls into `frontend/src/hooks/useChangeDetectionWorkflow.ts`.
|
|
- Moved map-layer derived state, area GeoJSON feature construction and selected-feature reset behavior into `frontend/src/hooks/useMapWorkspaceState.ts`.
|
|
- Kept `App.tsx` as the cross-module composition layer without changing panel props, API contracts, migrations or product behavior.
|
|
- Added static regression tests for the extracted orchestration hooks.
|
|
- Updated frontend README, changelog and TODO status.
|
|
|
|
Validation:
|
|
- `cd backend && python -m pytest tests/test_sprint39_frontend_orchestration_hooks.py -q` passed: 4 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 188 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the all-in-one Tower container at `http://192.168.10.150:1202`.
|
|
- Tower live migration smoke passed and reported `Database collation version: ok`.
|
|
- Tower browser runtime verification passed.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Project/area/dataset cross-load orchestration still lives in `App.tsx`; it is a good next low-risk decomposition target.
|
|
|
|
Next recommended pass:
|
|
- Run full readiness and Tower deploy verification, then extract project/area loading into a dedicated hook if behavior remains stable.
|
|
|
|
## Sprint 38 Export Center preview hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Hardened the export content preview path so HTML report artifacts return `EXPORT_CONTENT_UNSUPPORTED` instead of a generic JSON parse failure.
|
|
- Updated the frontend Export Center to offer JSON preview only for JSON/GeoJSON artifacts.
|
|
- HTML project report artifacts now display as download-only in the export list.
|
|
- Extracted export preview rendering from `frontend/src/App.tsx` into `frontend/src/components/exports/ExportPreview.tsx`.
|
|
- Updated API/frontend docs, changelog and TODO status.
|
|
|
|
Validation:
|
|
- `cd backend && python -m pytest tests/test_sprint17_export_foundation.py -q` passed: 10 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning` passed: 184 tests.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed.
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql` passed.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the all-in-one Tower container at `http://192.168.10.150:1202`.
|
|
- Tower live migration smoke passed and reported `Database collation version: ok`.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This pass does not add new export formats, PDF reports, provider fetching, AI inference or migrations.
|
|
|
|
Next recommended pass:
|
|
- Run the full readiness gate, frontend build and Tower deploy smoke; then continue with shared workbench orchestration decomposition or export cleanup/history filtering.
|
|
|
|
## Sprint 37 Tower PostgreSQL collation maintenance (2026-06-17)
|
|
|
|
Changed:
|
|
- Performed the manual PostgreSQL collation maintenance action reported by the live migration smoke on the Tower all-in-one runtime.
|
|
- Created a pre-maintenance custom-format database backup at `backups/geointel-before-collation-refresh-20260617-065707.dump`.
|
|
- Ran `REINDEX DATABASE geointel;` followed by `ALTER DATABASE "geointel" REFRESH COLLATION VERSION;`.
|
|
- Confirmed the database collation metadata now matches the runtime collation version: `geointel stored=2.36 actual=2.36`.
|
|
|
|
Validation:
|
|
- `LIVE_SMOKE_CONTAINER=geointel bash scripts/live_migration_smoke.sh` passed on Tower and reported `Database collation version: ok`.
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202` passed.
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202` passed.
|
|
|
|
Notes:
|
|
- An initial SSH script attempt was interrupted by PowerShell BOM/stdin handling before reindex or refresh executed; the subsequent base64-delivered script ran without `docker exec -i` stdin interference.
|
|
- Earlier backup attempts were left in `backups/` alongside the final validated backup.
|
|
- No code, API contracts, migrations, product features, provider fetching or AI behavior changed.
|
|
|
|
Next recommended pass:
|
|
- Continue with export/download polish or frontend export center hardening, using the now-clean Tower runtime as the validation target.
|
|
|
|
## Sprint 32 Unraid all-in-one runtime (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `docker-compose.unraid.yml` for a single editable `geointel` container on Unraid.
|
|
- Added `deploy/unraid/Dockerfile.all-in-one` to build one image containing embedded PostGIS, backend GIS runtime, nginx and frontend static assets.
|
|
- Added `deploy/unraid/all-in-one-start.sh` to start embedded PostGIS, apply Alembic migrations, start FastAPI on internal localhost and serve nginx on container port 80.
|
|
- Added `deploy/unraid/nginx-all-in-one.conf` so `/api` and `/health` proxy to `127.0.0.1:8000` inside the same container.
|
|
- Updated `deploy/unraid/geointel.env.example`, Unraid XML template and README for one-container operation and editable web/storage/database paths.
|
|
- Updated Tower deploy scripts to stop the old multi-container stack without deleting volumes and start `docker-compose.unraid.yml`.
|
|
- Updated `scripts/live_migration_smoke.sh` to support both all-in-one `geointel` and legacy `backend` Compose services.
|
|
- Added root `.dockerignore` for all-in-one builds.
|
|
- Updated Sprint 31 tests to cover the all-in-one Dockerfile, startscript, nginx config, deploy scripts and template metadata.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint31_unraid_template.py backend/tests/test_live_migration_smoke_script.py backend/tests/test_docker_runtime_config.py` passed: 26 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 179 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 179 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- `bash -n deploy/unraid/all-in-one-start.sh` passed.
|
|
|
|
Notes:
|
|
- No API contracts, migrations, product features, provider fetching or AI behavior changed.
|
|
- Local Windows environment does not have `docker` in PATH; Tower deployment should provide the live all-in-one Docker validation.
|
|
|
|
## Sprint 31 Unraid deployment template (2026-06-17)
|
|
|
|
Changed:
|
|
- Made `docker-compose.yml` configurable through `.env` defaults for frontend port, backend port, storage path, PostGIS database/user/password, CORS origins and upload limit.
|
|
- Added `deploy/unraid/geointel.env.example` for Unraid/Tower runtime configuration.
|
|
- Added `deploy/unraid/geointel-unraid-template.xml` as Unraid/DockerMan-style metadata for the editable Compose stack settings.
|
|
- Added `deploy/unraid/geointel-icon.svg` and `deploy/unraid/geointel-icon.png`, served through matching `frontend/public` assets.
|
|
- Added the frontend favicon link for the GeoIntel icon.
|
|
- Added Sprint 31 tests for Unraid template coverage, compose variable coverage, docs and icon availability.
|
|
- Updated root README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint31_unraid_template.py backend/tests/test_docker_runtime_config.py` passed: 22 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 177 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 177 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `3a8b82f`.
|
|
- Tower Docker Compose config/build/up passed with the env-default compose file.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML, `GET /api/v1/projects`, `/geointel-icon.svg` and `/geointel-icon.png`.
|
|
|
|
Notes:
|
|
- Local Windows environment does not have `docker` in PATH, so local `docker compose config` could not be run from this machine.
|
|
- Tower deployment validated Docker Compose config and live runtime after commit.
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching or AI behavior changed.
|
|
|
|
## Sprint 30 workbench component decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved persisted QA/QC result rendering from `frontend/src/App.tsx` into `frontend/src/components/quality/QualityResultsPanel.tsx`.
|
|
- Moved map layer controls, MapLibre composition and feature inspector rendering into `frontend/src/components/map/MapWorkspace.tsx`.
|
|
- Updated map/workbench and QA regression tests for the new component boundaries.
|
|
- Added Sprint 30 component wiring tests to keep QA and map markup out of `App.tsx`.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint30_workbench_components.py backend/tests/test_sprint27_frontend_workflow_hooks.py backend/tests/test_sprint19_map_workbench.py` passed: 9 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 173 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 173 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `acf9590`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Next maintainability pass should split export preview and remaining shared workbench orchestration into focused components/hooks.
|
|
|
|
## Sprint 29 dataset component decomposition (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved dataset upload/list rendering from `frontend/src/App.tsx` into `frontend/src/components/datasets/DatasetPanel.tsx`.
|
|
- Moved dataset detail and job-list rendering into `frontend/src/components/datasets/DatasetDetailPanel.tsx`.
|
|
- Split raster controls and vector controls into `frontend/src/components/datasets/RasterControls.tsx` and `frontend/src/components/datasets/VectorControls.tsx`.
|
|
- Updated Sprint 28 regression tests for the new component boundary and added Sprint 29 component wiring tests.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint29_dataset_components.py backend/tests/test_sprint28_dataset_workflow_hook.py backend/tests/test_sprint27_frontend_workflow_hooks.py` passed: 10 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 170 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 170 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `1cbb356`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Next maintainability pass should split change detection, QA/QC results and map workspace controls into focused presentational components.
|
|
|
|
## Sprint 28 dataset workflow hook hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved dataset selection, upload form state, dataset detail loading, dataset jobs and raster/vector operation orchestration from `frontend/src/App.tsx` into `frontend/src/hooks/useDatasetWorkflow.ts`.
|
|
- Kept project dataset listing in `App.tsx` so project/area loading remains the shared workbench boundary.
|
|
- Added regression tests to verify App uses the dataset workflow hook and still wires dataset, raster and vector callbacks.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint28_dataset_workflow_hook.py backend/tests/test_sprint27_frontend_workflow_hooks.py backend/tests/test_sprint26_frontend_workflow_hooks.py` passed: 11 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 167 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 167 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Initial Tower rebuild hit Docker btrfs storage exhaustion; safely reclaimed build cache only with `docker builder prune -af` without pruning volumes.
|
|
- Tower Docker storage recovered from `/var/lib/docker` 94% used to 58% used after cleanup and rebuild.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `361776c`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Next maintainability pass should split dataset detail, raster controls and vector controls into presentational components fed by the hook state.
|
|
|
|
## Sprint 27 export and QA workflow hook hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved Export Center orchestration state and API calls from `frontend/src/App.tsx` into `frontend/src/hooks/useExportWorkflow.ts`.
|
|
- Moved QA/QC comparison state and persisted quality-check listing from `frontend/src/App.tsx` into `frontend/src/hooks/useQualityWorkflow.ts`.
|
|
- Added regression tests to verify App uses export/quality hooks and still wires QA/QC results plus ExportCenter callbacks.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint27_frontend_workflow_hooks.py backend/tests/test_sprint26_frontend_workflow_hooks.py` passed: 8 tests.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m pytest` passed: 164 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 164 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `cd backend && python -m alembic heads` passed: single head `202606120900`.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` passed after commit `fd0000f`.
|
|
- Tower live migration smoke passed with PostGIS 3.4 and Alembic head `202606120900`.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Additional HTTP smoke passed for frontend HTML and `GET /api/v1/projects` through the frontend proxy.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
- Local Windows Docker CLI was unavailable (`docker` command not found); Tower deployment remains handled through `scripts/deploy_tower.ps1`.
|
|
- Next maintainability pass should extract dataset/raster/vector operation workflows from `App.tsx`.
|
|
|
|
## Sprint 26 frontend workflow hook hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Moved Detection Lab orchestration state and API calls from `frontend/src/App.tsx` into `frontend/src/hooks/useDetectionWorkflow.ts`.
|
|
- Moved Segmentation Lab orchestration state and API calls from `frontend/src/App.tsx` into `frontend/src/hooks/useSegmentationWorkflow.ts`.
|
|
- Added shared frontend `formatError` helper under `frontend/src/lib/formatError.ts`.
|
|
- Added regression tests to verify App uses workflow hooks and still wires DetectionLab/SegmentationLab callbacks.
|
|
- Updated frontend README, TODO and changelog docs.
|
|
|
|
Validation:
|
|
- `python -m pytest backend/tests/test_sprint26_frontend_workflow_hooks.py backend/tests/test_sprint22_workbench_status_strip.py` passed: 6 tests.
|
|
- `cd frontend && npm run typecheck` passed.
|
|
- `cd frontend && npm run build` passed.
|
|
- `bash scripts/run_readiness_check.sh` passed: 160 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy/live migration/browser runtime passed after commit `6c32f29`.
|
|
- In-app browser check passed on `http://192.168.10.150:1202`: workbench, status strip, Detection Lab, Segmentation Lab and Export Center visible with no console error logs.
|
|
|
|
Notes:
|
|
- No API contracts, backend behavior, migrations, product features, provider fetching, AI behavior or UI redesign changed.
|
|
## Sprint 25 YOLO compatibility smoke hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Added explicit `--check-model-load` support to `scripts/yolo_preflight.py`, `backend/scripts/yolo_preflight.py` and `YoloPreflightService`.
|
|
- The model-load smoke requires real optional AI dependencies, loads only an existing local model file, runs no inference and does not download weights.
|
|
- The CLI rejects `--check-model-load` with `--assume-dependencies` to avoid false-positive AI readiness.
|
|
- Added regression tests for mocked successful load, load failure reporting and CLI guard behavior.
|
|
- Added Python compile validation for both YOLO preflight entrypoints to the readiness gate.
|
|
- Updated AI pipeline, scripts, backend, TODO and changelog docs, including Docker runtime usage.
|
|
|
|
Validation:
|
|
- `python -m py_compile scripts/yolo_preflight.py` passed.
|
|
- `python -m pytest backend/tests/test_sprint13_yolo_preflight.py backend/tests/test_readiness_gate.py` passed: 14 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed: 156 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` passed.
|
|
- Tower deploy/live migration/browser runtime passed after commit `6c32f29`.
|
|
- In-app browser check passed on `http://192.168.10.150:1202`: workbench, status strip, Detection Lab, Segmentation Lab and Export Center visible with no console error logs.
|
|
- Tower deploy/live migration/browser runtime passed after commit `382dcad`.
|
|
- Backend-container YOLO preflight sanity passed: default status `not_configured`, `will_download_models=false`, `will_run_inference=false`.
|
|
- Backend-container CLI guard passed: `--check-model-load` with `--assume-dependencies` is rejected.
|
|
|
|
Notes:
|
|
- No base dependencies, API contracts, migrations, product features, provider fetching or detection persistence behavior changed.
|
|
## Sprint 24 demo/export artifact cleanup tooling (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `scripts/cleanup_demo_artifacts.py` for dry-run-first cleanup of old offline demo export artifacts.
|
|
- Added `backend/scripts/cleanup_demo_artifacts.py` so the same cleanup can run inside the backend Docker container.
|
|
- Cleanup is constrained to an exact demo project name by default, keeps the newest exports per project and refuses file deletion outside `STORAGE_ROOT`.
|
|
- Added regression tests for cleanup selection, path safety, dry-run candidate reporting and readiness gate coverage.
|
|
- Added Python compile validation for the cleanup script to `scripts/run_readiness_check.sh`.
|
|
- Documented cleanup usage in `scripts/README.md`, `docs/STORAGE_ARCHITECTURE.md`, `backend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Validation:
|
|
- `python -m py_compile scripts/cleanup_demo_artifacts.py` passed.
|
|
- `python -m pytest backend/tests/test_sprint24_cleanup_demo_artifacts.py backend/tests/test_readiness_gate.py` passed: 10 tests.
|
|
- `bash scripts/run_readiness_check.sh` passed twice after adding the backend container entrypoint: 152 backend tests, frontend typecheck/build, Alembic head check and script syntax checks.
|
|
- `python -m compileall backend/app` passed.
|
|
- `cd backend && python -m alembic upgrade head --sql` passed.
|
|
- `bash -n scripts/live_migration_smoke.sh` and `bash -n scripts/verify_demo_export_workflow.sh` passed.
|
|
- Local `docker compose config` could not run because the Windows Docker CLI is not installed in this Codex environment.
|
|
- Tower deploy/live migration/browser runtime passed after final commit `2d4e6bd`.
|
|
- Live backend-container cleanup dry-run exposed and then fixed confusing dry-run labeling so candidates are reported as `candidate_files`, not `deleted_files`; final dry-run reported 2 candidate files, 0 deleted files and 0 deleted export rows.
|
|
|
|
Notes:
|
|
- No API contracts, migrations, product features, provider fetching, AI inference or source dataset cleanup behavior changed.
|
|
## Sprint 23 V1 report handoff summary (2026-06-17)
|
|
|
|
Changed:
|
|
- Added V1 readiness summary data to project metadata exports.
|
|
- Added V1 Readiness Summary and Known Limitations sections to lightweight HTML project report exports.
|
|
- Included persisted AOI, dataset readiness, QA/QC and export-history counts in the handoff summary.
|
|
- Updated export tests, API contract docs, frontend README, TODO and changelog.
|
|
|
|
Tested:
|
|
- Passed: backend compile, export tests through full readiness, full backend pytest, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, live smoke syntax check, Tower deploy, live migration smoke and live project report smoke.
|
|
|
|
Known limitations:
|
|
- Report export remains a lightweight HTML artifact, not a PDF designer or custom report builder.
|
|
|
|
Next recommended pass:
|
|
- Add optional cleanup tooling for stale demo/export artifacts if repeated smoke runs keep accumulating local artifacts.
|
|
## Sprint 22 V1 workbench status strip (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `frontend/src/components/WorkbenchStatusStrip.tsx` to summarize existing V1 state for project, AOI, datasets, active map layer, QA/QC and exports.
|
|
- Wired the status strip into `frontend/src/App.tsx` using existing orchestration state only.
|
|
- Added compact status-strip styling and regression tests for the frontend wiring contract.`n- Hardened `frontend/src/components/GeoMap.tsx` so MapLibre source/layer updates wait for style readiness before adding sources.`n- Hardened demo project lookup so duplicate historical demo projects prefer complete fixture state before repairing incomplete state.
|
|
- Updated frontend README, TODO and changelog.
|
|
|
|
Tested:
|
|
- Passed: backend compile, focused pytest, full backend pytest with DeprecationWarning as error, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, readiness via Git Bash and live smoke syntax check via Git Bash.
|
|
|
|
Known limitations:
|
|
- The strip is a read-only operator summary; it intentionally does not add new backend status APIs or product workflows.
|
|
|
|
Next recommended pass:
|
|
- Add a compact project handoff summary in exports/report output if the browser-facing V1 workflow remains green.
|
|
|
|
## Sprint 21 V1 demo workflow smoke hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Hardened `scripts/verify_demo_export_workflow.sh` so the explicit offline demo smoke validates area GeoJSON, fixture datasets, vector FeatureCollection content, vector feature summaries, persisted QA/QC metrics and export downloads through the frontend proxy.
|
|
- Updated the frontend demo workflow action to open the candidate vector fixture dataset after seeding/loading the demo, so the Map Workbench is populated without a manual dataset click.
|
|
- Added regression tests for the strengthened smoke script and frontend demo loading contract.
|
|
- Updated scripts/frontend documentation, changelog and TODO status.
|
|
|
|
Tested:
|
|
- Passed: backend compile, backend pytest with DeprecationWarning as error, readiness, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, Tower deploy, live migration smoke, browser-runtime smoke and expanded demo/export workflow smoke.
|
|
|
|
Known limitations:
|
|
- The demo smoke intentionally seeds fixture demo data when run; it should be used as an explicit verification command, not as an implicit healthcheck.
|
|
|
|
Next recommended pass:
|
|
- Add a compact V1 dashboard/status strip for project, AOI, datasets, QA and exports so operators can see readiness at a glance after opening a project.
|
|
## Sprint 20 V1 selected area map overlay (2026-06-17)
|
|
|
|
Changed:
|
|
- Added GeoJSON geometry serialization for project areas so persisted AOIs can be displayed by the map workbench.
|
|
- Added a dedicated MapLibre area overlay layer with separate visibility and opacity controls.
|
|
- Added area list actions and map workspace controls to select the active AOI.
|
|
- Updated API/frontend docs, changelog and TODO status for selected area display.
|
|
|
|
Tested:
|
|
- Passed: backend compile, backend pytest with DeprecationWarning as error, readiness, frontend typecheck/build, Alembic heads, Alembic SQL upgrade, Tower deploy, live migration smoke, browser-runtime smoke and browser UI audit.
|
|
|
|
Known limitations:
|
|
- Area geometry is displayed as a simple filled/outlined GeoJSON overlay; no drawing/editing workflow is introduced in this pass.
|
|
|
|
Next recommended pass:
|
|
- Add a small V1 workflow polish pass for richer dataset/area empty states and a fixture-driven end-to-end browser smoke once the new build is deployed.
|
|
## Sprint 19 V1 map workbench controls (2026-06-17)
|
|
|
|
Changed:
|
|
- Added active MapLibre layer visibility and opacity controls.
|
|
- Added click-to-inspect feature property display for the active GeoJSON workbench layer.
|
|
- Added active layer label and feature count to the Map workspace panel.
|
|
- Updated the app header from the stale Sprint 9 label to the GeoIntel Kempen V1 Workbench identity.
|
|
- Added regression tests for the frontend map control and feature inspection wiring.
|
|
|
|
Limitations:
|
|
- The current workbench still shows one active GeoJSON overlay at a time; multi-layer stack ordering remains a later UI enhancement.
|
|
- Raster preview display still remains metadata/path-oriented unless the backend exposes a browser-safe raster image/tile URL.
|
|
- No API contracts, migrations, backend behavior, provider fetching, AI inference or new dependencies were introduced.
|
|
|
|
Validation planned:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` via Git Bash on Windows
|
|
- Tower redeploy through `scripts/deploy_tower.ps1`
|
|
## Sprint 18 vector change detection foundation (2026-06-16)
|
|
|
|
Changed:
|
|
- Added `POST /api/v1/analysis/change-detection` for comparing two vector datasets in the same project through the existing synchronous job envelope.
|
|
- Added `ChangeDetectionService` with persisted `vector_features` as the primary source of comparable geometries and explicit stored-GeoJSON fallback warnings for older datasets.
|
|
- Added frontend Change Detection controls, summary counts and MapLibre overlay styling for `added`, `removed` and `unchanged` feature properties.
|
|
- Added nginx no-cache headers for frontend HTML/assets after browser verification exposed stale cached modules on the LAN deployment.
|
|
- Added backend tests for persisted-vector comparison and canonical API envelope behavior.
|
|
|
|
Limitations:
|
|
- The foundation classifies `added`, `removed` and `unchanged` only. It does not emit fake `changed` objects without durable object ids/versioning.
|
|
- No migrations, live GRB/OSM/Sentinel fetching, AI inference, new dependencies, LiDAR, Copilot, Training Studio or separate Reports module were introduced.
|
|
|
|
Validation:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` via Git Bash on Windows
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
# Codex Execution Log
|
|
|
|
## Post-V1 national coverage completion: Wallonia (2026-07-22)
|
|
|
|
- Located and live-validated the stable official WALOUS 2018 GeoTIFF
|
|
distribution. The retained raster SHA-256 is
|
|
`a788cf4619363664d1e80def79b32176b4703317d123c8cd703f2eb10e8d56b2`;
|
|
the full source is EPSG:3812 at 1 m and the merged live provisioning report
|
|
now covers 2018, 2020 and 2023.
|
|
- Added the official 2018 stacked-class to view-class crosswalk. Greenhouse
|
|
code `62` enters the construction class and unmarked source value `0` is
|
|
normalized to internal nodata `255`; both behaviors have regression tests
|
|
so background pixels cannot inflate hectare metrics.
|
|
- Selected the official SPW MNT 2021-2022 1 m GeoTIFF as the governed Walloon
|
|
terrain source after a live storage audit showed sufficient capacity. Added
|
|
fail-closed operator provisioning, bounded `bbox intersect Area`
|
|
persistence, raster validation and terrain analysis with explicit
|
|
DNG/EPSG:5710 vertical-reference provenance.
|
|
- Replaced the frontend-only AI geography warning with a machine-readable
|
|
runtime contract. Detection capabilities now expose training/validation
|
|
scope, validated regions, national status and the review requirement; the
|
|
configured local weights remain bound to Mol/Kempen evidence and fail closed
|
|
as not nationally validated.
|
|
- Live Tower provisioning exposed an incorrect assumption that 11 WALOUS
|
|
classes implied numeric codes 1 through 11. The official SPW legend and the
|
|
downloaded 2020 raster confirm codes `1,2,3,4,5,6,7,8,9,80,90`. Corrected
|
|
validation, class semantics, colours and all hectare aggregations; added a
|
|
provisioner regression containing codes 80/90 and retained exact source
|
|
observation ranges.
|
|
- Validated the official WALOUS 2020 and 2023 archives, their EPSG:3812 1 m
|
|
raster contract, 11 classes, CC BY 4.0 attribution and published edition
|
|
accuracy. Added a fail-closed operator provisioner with archive-size,
|
|
extraction, CRS, resolution, class and checksum gates.
|
|
- Added bounded WALOUS persistence through `DatasetService`, semantic hectare
|
|
metrics, a governed PNG overlay and automatic 2020/2023 temporal-series
|
|
materialization. The existing temporal API now compares persisted WALOUS
|
|
raster metrics while keeping object-level change unavailable.
|
|
- Added the queryable legal SPW flood-hazard polygon product and class-aware
|
|
metrics through the existing official-vector engine.
|
|
- Wired both integrations through Compose, the all-in-one Unraid runner,
|
|
editable DockerMan template and readiness gate. Added canonical API,
|
|
persistence, rendering, temporal and runtime-parity regression tests.
|
|
- Revalidated both official Walloon DTM distributions. The 1 m archive is
|
|
operator-provisioned once on persistent storage and selections read bounded
|
|
windows from it; the approximately 213 GB 0.5 m distribution remains
|
|
deliberately unprovisioned because it is not required for the V1 analysis
|
|
contract.
|
|
- Reprobed the documented MDK WCS endpoints. Strict TLS still fails hostname
|
|
validation; acquisition remains disabled and no insecure fallback was added.
|
|
- A live Tower deploy exposed PostGIS crash recovery exceeding the former
|
|
startup/deploy waits. Raised both bounded waits to 15/16 minutes and added a
|
|
release regression so recovery can finish without a premature rollback.
|
|
- The same live restart exposed an unconditional recursive `chown` over the
|
|
persistent database. Replaced it with root-directory ownership only to avoid
|
|
rewriting relation metadata and delaying every all-in-one startup.
|
|
|
|
This file must be updated by Codex after each implementation pass.
|
|
|
|
## Format
|
|
|
|
```md
|
|
## Pass X — Title
|
|
Date:
|
|
|
|
### Completed
|
|
- ...
|
|
|
|
### Files changed
|
|
- ...
|
|
|
|
### Tests run
|
|
- ...
|
|
|
|
### Known limitations
|
|
- ...
|
|
|
|
### Next recommended pass
|
|
- ...
|
|
```
|
|
|
|
## Initial status
|
|
Specification phase completed through M0. No code foundation has been implemented yet.
|
|
|
|
## M2 Engineering Package
|
|
|
|
- [x] Add ADR decision records.
|
|
- [x] Add RFC placeholders for future modules.
|
|
- [x] Add API/database/event contracts.
|
|
- [x] Add model registry and class catalogs.
|
|
- [x] Add queue architecture.
|
|
- [x] Add acceptance matrix and test catalog.
|
|
- [x] Add Codex M2 build prompts.
|
|
- [ ] Start Codex Pass 01 backend foundation.
|
|
|
|
## M8 preparation
|
|
|
|
Added the tomorrow execution layer for Codex autonomy: Day 1 master prompt, pass prompts, failure recovery, quality gates, operator checklist and smoke script scaffold.
|
|
|
|
## Sprint 1 readiness hardening (2026-06-11)
|
|
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Hardened backend dependency declarations for Sprint 1 runtime modules in `backend/pyproject.toml`.
|
|
- Normalized helper scripts for backend/frontend install/test/dev/readiness across `python`/`python3` interpreter availability.
|
|
- Fixed PostGIS/geospatial stack compatibility issues (`geojson_service` import path, package config).
|
|
- Updated backend/frontend/docs/readme commands for import smoke, setup, and readiness verification.
|
|
- Fixed frontend shell script line-ending parsing failures and added `npm` availability checks.
|
|
- Added missing frontend type path fixes and TypeScript typing corrections.
|
|
- Added/verified minimal Sprint 1 tests for health and GeoJSON parsing/rejection behaviors.
|
|
|
|
### Files changed
|
|
- `backend/pyproject.toml`
|
|
- `backend/app/core/config.py`
|
|
- `backend/app/services/geojson_service.py`
|
|
- `backend/tests/test_health.py`
|
|
- `backend/tests/test_geojson_dataset_service.py`
|
|
- `backend/README.md`
|
|
- `frontend/package.json`
|
|
- `frontend/tsconfig.json`
|
|
- `frontend/src/components/GeoMap.tsx`
|
|
- `frontend/src/services/api/areas.ts`
|
|
- `frontend/src/services/api/projects.ts`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/README.md`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `scripts/backend_install.sh`
|
|
- `scripts/backend_test.sh`
|
|
- `scripts/backend_dev.sh`
|
|
- `scripts/codex_pass_end_check.sh`
|
|
- `scripts/smoke_backend_import.sh`
|
|
- `scripts/frontend_install.sh`
|
|
- `scripts/frontend_typecheck.sh`
|
|
- `scripts/frontend_build.sh`
|
|
- `scripts/frontend_dev.sh`
|
|
- `README.md`
|
|
- `.env.example`
|
|
- `CHANGELOG.md`
|
|
- `Makefile`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend; python -m pytest` (pass, 5 tests)
|
|
- `cd backend; python -c "from app.main import app; print(app.title)"` (pass)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `bash scripts/frontend_install.sh` (pass)
|
|
- `bash scripts/frontend_typecheck.sh` (pass)
|
|
- `bash scripts/frontend_build.sh` (pass)
|
|
- `make`-based targets not runnable in this environment (`make` command missing)
|
|
- `docker compose config` not runnable in this environment (`docker` command missing)
|
|
- `python3` commands fail in this environment because `python3` maps to Microsoft Store stub; use `python` instead.
|
|
|
|
### Known limitations
|
|
- `docker` and `make` are not installed in the current local shell environment.
|
|
- `python3` is not a usable interpreter in this environment; `python` must be used for all backend runtime/tests.
|
|
|
|
### Next recommended pass
|
|
- Run `docker compose` validation and DB-backed migration checks in an environment with Docker + PostGIS service available.
|
|
|
|
## Pass 12 — Sprint 2 Foundation
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Added vector/raster dataset typing and lifecycle states (`uploaded`, `validating`, `ready`, `failed`) in dataset service.
|
|
- Added vector metadata extraction details (feature counts, geometry types, bounds, area, CRS assumptions).
|
|
- Added raster metadata service with dependency-aware unavailable behavior and explicit `RASTER_PROCESSING_UNAVAILABLE` handling.
|
|
- Added dataset vector inspect/summary and raster metadata endpoints for project-scoped datasets.
|
|
- Persisted deterministic storage metadata for uploads (original/stored filename, MIME, size, checksum).
|
|
- Extended frontend dataset manager details panel with type/status/file metadata/feature counts and raster summary readiness.
|
|
- Added minimal Sprint 2 tests for vector metadata, legacy `geojson` compatibility, storage metadata persistence, and raster dependency fallback.
|
|
|
|
### Files changed
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/services/raster_service.py`
|
|
- `backend/app/services/geojson_service.py`
|
|
- `backend/app/schemas/dataset.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/app/services/storage_service.py`
|
|
- `backend/tests/test_geojson_dataset_service.py`
|
|
- `backend/tests/test_raster_service.py`
|
|
- `backend/tests/test_storage_service.py`
|
|
- `backend/README.md`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/types.ts`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `.env.example`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `README.md`
|
|
|
|
### Tests run
|
|
- Sprint 2 tests to be executed via readiness commands (see below).
|
|
|
|
### Known limitations
|
|
- `rasterio` is not required by default; raster metadata extraction is unavailable until installed in the environment.
|
|
- Docker and PostGIS validation still depends on local availability of Docker/DB runtime.
|
|
|
|
### Next recommended pass
|
|
- Keep Sprint 1/2 architecture; implement raster clipping/tiling APIs and status jobs before Detection/Segmentation modules.
|
|
|
|
## Pass 13 — Sprint 2 validation and legacy compatibility hardening
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Verified Sprint 2 readiness commands in current environment (where tools are available).
|
|
- Revalidated backend tests and frontend typecheck/build after a Sprint 2 compatibility patch.
|
|
- Added frontend vector-detail compatibility for legacy `geojson` dataset rows in Sprint 1 records.
|
|
- Confirmed backend import smoke and dataset/raster/vector service behavior remain intact.
|
|
|
|
### Files changed
|
|
- `frontend/src/App.tsx`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 11 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
- `cd backend; python3 -m compileall backend/app` (fails: python3 shim unavailable in this shell)
|
|
- `cd backend; python3 -c "from app.main import app; print(app.title)"` (fails: python3 shim unavailable in this shell)
|
|
- `docker compose config` (fails: docker command unavailable in this shell)
|
|
|
|
### Known limitations
|
|
- `python3` is not available in the current environment; use `python` commands for verification.
|
|
- `docker` is not available in the current environment.
|
|
|
|
### Next recommended pass
|
|
- Run the same sprint verification commands in an environment with `python3` and `docker` installed.
|
|
|
|
## Pass 14 — Sprint 3 operations + job foundation
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Added lightweight job model/migration and service layer with statuses `queued`, `running`, `success`, `failed`.
|
|
- Added job API endpoints for create/list/read/status under project scope.
|
|
- Added vector operation service foundation:
|
|
- inspect
|
|
- bbox
|
|
- stats
|
|
- clip by area
|
|
- buffer
|
|
- intersect
|
|
- Added raster operation foundation:
|
|
- inspect
|
|
- metadata
|
|
- preview readiness
|
|
- clip placeholder (dependency-aware)
|
|
- tile placeholder (dependency-aware)
|
|
- Added job-based execution wrappers for vector/raster operation endpoints.
|
|
- Updated dataset detail UI with available operations, job list/status, and derived output navigation.
|
|
- Added/updated Sprint 3 API contract and backend/frontend documentation updates.
|
|
|
|
### Files changed
|
|
- `backend/app/schemas/operations.py`
|
|
- `backend/app/schemas/__init__.py`
|
|
- `backend/app/services/vector_operations_service.py`
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/app/services/job_service.py`
|
|
- `backend/app/api/routes/jobs.py`
|
|
- `backend/app/models/entities.py`
|
|
- `backend/app/models/__init__.py`
|
|
- `backend/alembic/versions/20260611212435_add_jobs_table.py`
|
|
- `backend/tests/test_vector_operations_service.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `backend/tests/test_storage_service.py`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/services/api/jobs.ts`
|
|
- `frontend/src/types.ts`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
|
|
### Known limitations
|
|
- Raster processing beyond readiness/metadata is intentionally dependency-aware placeholder in Sprint 3.
|
|
- `docker` remains unavailable in this environment.
|
|
- `python3` still resolves to Windows Store stub; use `python` for verification.
|
|
|
|
### Next recommended pass
|
|
- Install rasterio/GDAL for real raster clipping/tile implementations.
|
|
- Add queue-backed worker (Redis/RQ) behind existing synchronous job facade.
|
|
- Add deeper operation acceptance tests for output dataset geometry correctness and persisted metadata.
|
|
|
|
## Pass 15 — Sprint 4 raster foundation
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Finalized dependency-aware raster processing behavior for clip/tile/preview and improved metadata enrichment.
|
|
- Ensured raster metadata and preview responses include deterministic file metadata (`size_bytes`, `checksum_sha256`, `path`) where available.
|
|
- Added raster tile manifest generation with deterministic storage paths and returned manifest payload.
|
|
- Updated API contracts and execution documentation from placeholder wording to actual raster-op foundations.
|
|
- Added targeted Sprint 4 tests for missing-area clip behavior and tile manifest payload shape.
|
|
|
|
### Files changed
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/app/api/routes/health.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `README.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `bash scripts/smoke_backend_import.sh`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
### Known limitations
|
|
- Raster dependency packages remain optional and will report `RASTER_PROCESSING_UNAVAILABLE` if absent.
|
|
- Raster clip/tile remain synchronous under the current in-process job wrapper.
|
|
|
|
### Next recommended pass
|
|
- Run full raster end-to-end tests with real GeoTIFF fixtures and validate output dataset metadata persistence.
|
|
## Pass 16 � Sprint 4 raster operations foundation hardening
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Realized the Sprint 4 raster foundation readiness in implementation and docs alignment:
|
|
- metadata extraction now returns real raster profile fields when rasterio is available
|
|
- preview generation writes deterministic PNG artifacts and reuses cached previews
|
|
- clip and tile operations persist deterministic outputs and manifest structure
|
|
- all raster processing paths now use explicit dependency-aware errors when rasterio/numpy/pillow are missing
|
|
- Fixed remaining frontend render/type issues introduced during raster path handling.
|
|
- Strengthened raster tests for tile manifest minimum size and dependency-aware behavior.
|
|
- Updated sprint milestone docs to reflect Sprint 4 status:
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
|
|
### Files changed
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/app/schemas/operations.py`
|
|
- `frontend/src/App.tsx`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 26 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
|
|
### Known limitations
|
|
- `docker` command is unavailable in this environment, so `docker compose config` could not be executed.
|
|
- `python3` command is unavailable as an alias in this environment; use `python` instead.
|
|
- Full raster dependency stack may be unavailable in some dev environments; those cases intentionally return structured `RASTER_PROCESSING_UNAVAILABLE` responses.
|
|
|
|
### Next recommended pass
|
|
- Sprint 5: add end-to-end RasterOps coverage for CRS-preserving re-projection, export-quality metadata policy, and tile set artifact cleanup lifecycle.
|
|
## Pass 17 — Sprint 5 raster analytics hardening
|
|
|
|
Date: 2026-06-11
|
|
|
|
### Completed
|
|
- Implemented Sprint 5 raster analytics foundation:
|
|
- band statistics endpoint and service with dependency-aware unavailable mode.
|
|
- reproject operation with CRS validation and reprojection metadata persistence.
|
|
- clip/tile hardening for missing/invalid geometry and empty clip output handling.
|
|
- enriched tile manifest fields (`tile_set_id`, `tile_size`, `overlap`, `source_dataset_id`, `source_raster_id`, `bounds`, `count`, `tile_paths`, `ai_inference`, `tile_server`, `created_at`, parameters).
|
|
- Updated raster stats/job error tests:
|
|
- dependency-aware stats failure coverage.
|
|
- invalid CRS request validation for reproject.
|
|
- persisted derived dataset assertions for reproject and clip.
|
|
- failure persistence test for failed raster jobs (`_run_job_sync`).
|
|
- Strengthened raster frontend detail panel rendering:
|
|
- job result JSON is visible for raster/vector operations.
|
|
- clearer raster metadata/status visibility retained for CRS/bounds/resolution display.
|
|
- Updated docs:
|
|
- backend/README Sprint 5 section.
|
|
- frontend/README Sprint 5 section.
|
|
- CHANGELOG entry for Sprint 5.
|
|
|
|
### Files changed
|
|
- `backend/app/services/raster_operations_service.py` (final reproject and manifest hardening alignment)
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `frontend/src/App.tsx`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/RASTER_OPERATIONS_SPEC.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `bash scripts/smoke_backend_import.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass)
|
|
|
|
### Known limitations
|
|
- Raster statistics and reproject operations still depend on environment availability of `rasterio`/`numpy`.
|
|
- Raster dependency checks and operation errors remain dependency-aware when libraries are unavailable.
|
|
|
|
### Next recommended pass
|
|
- Sprint 6: add raster index/mask workflows (NDVI/NDWI/NDBI), tile serving or export packaging, and AI-ready dataset linking.
|
|
## Pass 18 - Sprint 6 spectral indices
|
|
|
|
### Completed
|
|
- Added local raster index operations (`ndvi`, `ndwi`, `ndbi`) under the existing raster operation architecture.
|
|
- Added typed band payload validation and explicit `INVALID_PARAMETERS` handling for missing/invalid band indexes.
|
|
- Implemented dependency-aware failure behavior for index execution (`RASTER_PROCESSING_UNAVAILABLE`) when `rasterio` or `numpy` are unavailable.
|
|
- Implemented local index output generation with float32 raster derivation and persisted provenance metadata on derived datasets.
|
|
- Extended dataset detail UI with spectral index controls (NDVI/NDWI/NDBI), run actions, and output dataset navigation from jobs.
|
|
- Updated API contracts, raster operation spec, and project documentation for Sprint 6 behavior.
|
|
|
|
### Files changed
|
|
- `backend/app/schemas/operations.py`
|
|
- `backend/app/services/raster_operations_service.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/tests/test_raster_operations_service.py`
|
|
- `frontend/src/services/api/datasets.ts`
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/types.ts`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/RASTER_OPERATIONS_SPEC.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `bash scripts/smoke_backend_import.sh`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
### Known limitations
|
|
- Raster dependency checks remain optional; missing raster packages return explicit `RASTER_PROCESSING_UNAVAILABLE` results.
|
|
- Job execution remains synchronous under current Sprint 3 job facade.
|
|
- `docker` and `python3` availability still depend on developer environment.
|
|
|
|
### Next recommended pass
|
|
- Keep operation architecture stable, then add threshold/mask workflows and output export packaging in a follow-up pass.
|
|
|
|
|
|
## Pass 19 - RC-2 stabilization
|
|
Date: 2026-06-12
|
|
|
|
### Completed
|
|
- Fixed backend compile/import blockers identified in RC-1:
|
|
- corrected dataset upload parameter ordering.
|
|
- corrected schema package re-exports for area schemas.
|
|
- Fixed frontend typecheck/build blockers in `App.tsx`.
|
|
- Added Alembic migration for dataset reference/provenance metadata columns required by current ORM models.
|
|
- Fixed QA comparison runtime crash and added focused QA service coverage.
|
|
- Corrected envelope response-model mismatches for vector inspect and raster stats endpoints.
|
|
- Strengthened `scripts/run_readiness_check.sh` so readiness runs backend compile, backend tests, frontend typecheck and frontend build.
|
|
- Improved readiness Python interpreter selection so it chooses an interpreter capable of running pytest.
|
|
|
|
### Files changed
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/schemas/__init__.py`
|
|
- `backend/app/api/routes/datasets.py`
|
|
- `backend/app/services/qa_service.py`
|
|
- `backend/alembic/versions/202606120001_add_dataset_reference_metadata.py`
|
|
- `backend/tests/test_qa_service.py`
|
|
- `frontend/src/App.tsx`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend\\app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 40 tests)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass; bundle-size warning only)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `cd backend && python -m alembic heads` (pass, single head `202606120001`)
|
|
- `cd backend && python -m alembic upgrade head --sql` (pass, generated SQL includes required dataset metadata columns)
|
|
- `docker compose config` not runnable in this environment because `docker` is not installed.
|
|
|
|
### Known limitations
|
|
- Fresh online database migration was validated only as generated Alembic SQL in this environment; run `cd backend && python -m alembic upgrade head` against a live PostgreSQL/PostGIS database on a machine with Docker or Postgres available.
|
|
- Existing non-RC architecture limitations from RC-1 remain intentionally unfixed: synchronous job facade, file-first vector outputs, monolithic frontend component, and incomplete future AI/storage architecture.
|
|
|
|
### Next recommended pass
|
|
- Do not start Sprint 7 until RC-2 verification is repeated against a live Docker/PostGIS environment.
|
|
|
|
## Pass 20 - Sprint 7A persistence and QA foundation
|
|
Date: 2026-06-12
|
|
|
|
### Completed
|
|
- Added first-class `vector_features` ORM model and Alembic migration with dataset and GiST geometry indexes.
|
|
- Persisted uploaded vector GeoJSON features into PostGIS-backed `vector_features` while keeping original file storage intact.
|
|
- Added first-class `quality_checks` and `metrics` ORM models and Alembic migration indexes.
|
|
- Added `QualityService` for persisted QA/QC domain records and metric rows.
|
|
- Updated QA candidate-vs-reference route so successful QA jobs also persist a `QualityCheck` and metrics, and return `quality_check_id` in `result_json`.
|
|
- Hardened provider capability contracts for GRB and OSM as `not_configured` stubs with supported layers, geometry types and query modes.
|
|
- Added Sprint 7A tests for vector feature persistence, quality check persistence, metrics persistence, dataset role validation, provider contracts, migration integrity and QA route persistence.
|
|
- Updated database/API documentation for Vector Features Architecture, Quality Check Architecture, Metrics Architecture and Provider Architecture.
|
|
|
|
### Files changed
|
|
- `backend/app/models/entities.py`
|
|
- `backend/app/models/__init__.py`
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/services/vector_feature_service.py`
|
|
- `backend/app/services/quality_service.py`
|
|
- `backend/app/api/routes/qa.py`
|
|
- `backend/app/providers/base.py`
|
|
- `backend/app/providers/grb.py`
|
|
- `backend/app/providers/osm.py`
|
|
- `backend/app/schemas/health.py`
|
|
- `backend/alembic/versions/202606120700_sprint7a_persistence_foundation.py`
|
|
- `backend/tests/test_sprint7a_persistence_foundation.py`
|
|
- `docs/DATABASE_IMPLEMENTATION_PLAN.md`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
- `CHANGELOG.md`
|
|
|
|
### Tests run
|
|
- `python -m compileall backend/app` (pass)
|
|
- `python -m pytest backend -q` (pass, 47 tests)
|
|
|
|
### Known limitations
|
|
- Sprint 7A intentionally does not implement GRB downloads, OSM downloads, Detection Lab, Segmentation Lab, LiDAR, AI Copilot, Training Studio or Reports.
|
|
- Job execution remains the existing synchronous facade.
|
|
- Live database migration still needs validation against a running PostgreSQL/PostGIS service in an environment with Docker or Postgres available.
|
|
|
|
### Next recommended pass
|
|
- Complete full release validation commands, including Alembic heads/SQL generation, readiness script, frontend typecheck/build and Docker config if Docker is available.
|
|
|
|
### Validation addendum
|
|
Date: 2026-06-12
|
|
|
|
Additional Sprint 7A validation completed after migration index cleanup:
|
|
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 47 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass; Vite chunk-size warning only)
|
|
- `cd backend && python -m alembic heads` (pass, single head `202606120700`)
|
|
- `cd backend && python -m alembic upgrade head --sql` (pass; generated SQL includes `vector_features`, `quality_checks`, `metrics` and the named GiST index `ix_vector_features_geometry`)
|
|
- `docker compose config` could not run because Docker is not installed in this shell.
|
|
|
|
## Pass 21 - Sprint 7B provider integration skeleton (2026-06-12)
|
|
|
|
- Implemented central provider registry for `grb`, `osm`, `manual` and `fixture`.
|
|
- Added provider capability, layer, status and future import-contract endpoints using the existing response envelope style.
|
|
- Preserved GRB and OSM as explicit `not_configured` providers; no live WFS, Overpass, download or fake provider data was introduced.
|
|
- Documented and tested provider-to-dataset mapping rules; future provider output must flow through `DatasetService` / `VectorFeatureService` rather than direct `vector_features` writes.
|
|
- Added frontend Provider Capabilities panel without live import buttons for GRB/OSM.
|
|
- Added opt-in `scripts/live_migration_smoke.sh` for real PostGIS migration smoke checks.
|
|
- Added Sprint 7B tests for provider registry, API envelopes, invalid provider handling, import contract and smoke script presence.
|
|
|
|
## Pass 22 - Sprint 8 Detection Lab foundation (2026-06-12)
|
|
|
|
### Completed
|
|
- Added first-class `detections` ORM model and Alembic migration with project, dataset, analysis run, class and GiST geometry indexes.
|
|
- Hardened `analysis_runs` with dataset, job, model, result and created-at fields while keeping jobs conceptually separate from analysis lifecycle.
|
|
- Added model registry capability service for `yolo-placeholder` (`not_configured`) and `manual-fixture-detector` (explicit fixture/demo only).
|
|
- Added `DetectionService` boundary for model listing, request validation, analysis run creation, job creation, unavailable model responses and explicit fixture persistence.
|
|
- Added Detection Lab API endpoints under `/api/v1/detection` using the existing response envelope style.
|
|
- Added minimal frontend Detection Lab panel for model capability status, raster dataset selection, confidence threshold and run result/error display.
|
|
- Updated database, API, AI pipeline, backend/frontend README, TODO and changelog docs.
|
|
|
|
### Known limitations
|
|
- Real YOLO/PyTorch inference is not enabled and no model downloads are performed.
|
|
- Fixture detector requires `fixture_mode=true` and explicit fixture detections; it is not production inference.
|
|
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
|
|
|
|
### Next recommended pass
|
|
- Verify Sprint 8 with full backend/frontend/readiness/Alembic gates, then perform a Sprint 8 verification audit before Sprint 8B real YOLO integration.
|
|
|
|
## Pass 23 - Sprint 8B configured YOLO foundation (2026-06-12)
|
|
|
|
### Completed
|
|
- Added optional backend `ai` dependency group for `ultralytics` and `torch`; normal backend startup remains import-safe without those packages.
|
|
- Added YOLO configuration settings:
|
|
- `YOLO_ENABLED`
|
|
- `YOLO_MODEL_PATH`
|
|
- `YOLO_MODEL_ID`
|
|
- `YOLO_MODEL_DISPLAY_NAME`
|
|
- `YOLO_MODEL_VERSION`
|
|
- `YOLO_DEVICE`
|
|
- `YOLO_IMAGE_SIZE`
|
|
- `YOLO_MAX_TILES`
|
|
- `YOLO_BATCH_SIZE`
|
|
- Added `yolo-configured` model registry capability with honest `not_configured`, `dependency_unavailable` and `configured` states.
|
|
- Added `YoloDetectionAdapter` that imports Ultralytics only in the load path and refuses missing local model files before model construction.
|
|
- Added raster tile manifest validation and configured tile-limit enforcement for real YOLO runs.
|
|
- Added pixel bbox to EPSG:4326 polygon georeferencing from tile transform or bounds metadata.
|
|
- Routed configured YOLO outputs through existing `DetectionService`, `Job`, `AnalysisRun` and first-class `Detection` persistence.
|
|
- Added Detection Lab tile manifest path input for the configured YOLO model.
|
|
- Added mocked Sprint 8B tests for model registry status, dependency-unavailable behavior, tile manifest validation, georeferencing and persisted detections.
|
|
- Updated API, AI pipeline, backend/frontend README and changelog documentation.
|
|
|
|
### Known limitations
|
|
- Sprint 8B does not add workers/queues; configured YOLO runs remain synchronous behind the existing job abstraction.
|
|
- Real model loading is validated at execution time. The registry reports configured when dependencies and local model path are present.
|
|
- No model weights are downloaded by GeoIntel.
|
|
- Detection visualization/map overlays are deferred.
|
|
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
|
|
|
|
### Next recommended pass
|
|
- Run full Sprint 8B validation and then perform a Sprint 8B verification audit before advancing to detection visualization/QA or segmentation planning.
|
|
|
|
## Pass 24 - Sprint 8C detection visualization and QA integration (2026-06-12)
|
|
|
|
### Completed
|
|
- Added detection result review endpoints for listing runs, listing detections by run/dataset, retrieving detection detail and returning persisted detections as GeoJSON FeatureCollections.
|
|
- Added lightweight detection filters for class name and minimum confidence.
|
|
- Added detection QA against persisted reference `vector_features` using the existing `QualityService`, `quality_checks` and `metrics` persistence path.
|
|
- Added frontend Detection Lab run selection, detection table, class/confidence filters and MapLibre detection GeoJSON overlay via the existing map component.
|
|
- Added frontend detection QA controls and metric summary display.
|
|
- Added Sprint 8C tests for GeoJSON output, list/filter behavior, detection detail, API envelope shape, QA persistence and no-match QA behavior.
|
|
- Added direct Sprint 8B tests for missing and invalid tile manifest files.
|
|
- Updated API, AI pipeline, backend/frontend README and changelog documentation.
|
|
|
|
### Known limitations
|
|
- Detection QA requires reference datasets to have persisted `vector_features`; unsupported references return a clear error instead of fake metrics.
|
|
- Detection overlays reuse the existing single GeoJSON map layer styling; complex class-based map styling is deferred.
|
|
- Segmentation, LiDAR, AI Copilot, Training Studio and Reports remain out of scope.
|
|
|
|
### Next recommended pass
|
|
- Run full Sprint 8C validation and perform a Sprint 8C verification audit before starting Sprint 9 Segmentation Lab.
|
|
|
|
## Pass 25 - Sprint 9 Segmentation Lab foundation (2026-06-12)
|
|
|
|
### Completed
|
|
- Added first-class `segmentations` ORM model and Alembic migration with project, dataset, job, analysis run, class and GiST geometry indexes.
|
|
- Added segmentation model registry capabilities for `segmentation-placeholder`, `fixture-segmenter`, `yolo-seg-configured` and `sam-configured`.
|
|
- Added `SegmentationService` boundary for raster validation, job/analysis-run lifecycle, unavailable model responses and explicit fixture-only persistence.
|
|
- Added segmentation adapter placeholder module with no SAM, YOLO-seg, torch or ultralytics imports.
|
|
- Added persisted segmentation GeoJSON output generated from PostGIS geometry and provenance properties.
|
|
- Added segmentation QA against persisted reference `vector_features` using existing `quality_checks` and `metrics`.
|
|
- Added minimal frontend Segmentation Lab panel for model states, raster selection, runs/results, GeoJSON map overlay and QA metric display.
|
|
- Updated API, AI pipeline, storage, database, backend/frontend README and changelog documentation.
|
|
|
|
### Files changed
|
|
- `backend/app/models/entities.py`
|
|
- `backend/app/models/__init__.py`
|
|
- `backend/alembic/versions/202606120900_sprint9_segmentation_foundation.py`
|
|
- `backend/app/schemas/segmentation.py`
|
|
- `backend/app/schemas/__init__.py`
|
|
- `backend/app/services/model_registry_service.py`
|
|
- `backend/app/services/segmentation_adapter.py`
|
|
- `backend/app/services/segmentation_service.py`
|
|
- `backend/app/api/routes/segmentation.py`
|
|
- `backend/app/main.py`
|
|
- `backend/tests/test_sprint9_segmentation_foundation.py`
|
|
- `frontend/src/types.ts`
|
|
- `frontend/src/services/api/segmentation.ts`
|
|
- `frontend/src/services/api/index.ts`
|
|
- `frontend/src/App.tsx`
|
|
- `docs/API_CONTRACTS.md`
|
|
- `docs/AI_PIPELINES.md`
|
|
- `docs/STORAGE_ARCHITECTURE.md`
|
|
- `docs/DATABASE_IMPLEMENTATION_PLAN.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Tests run
|
|
- `python -m pytest backend/tests/test_sprint9_segmentation_foundation.py -q` (red first: missing `Segmentation` import, then pass)
|
|
- `cd backend && python -m pytest` (pass, 88 tests)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
|
|
### Known limitations
|
|
- Sprint 9 intentionally does not implement real SAM, real YOLO-seg, model downloads, new AI dependencies or production-scale async inference.
|
|
- Fixture segmenter requires explicit `fixture_mode=true` and explicit fixture segmentations; it is not production inference.
|
|
- Metric area is only persisted when provided by the fixture/output payload; Sprint 9 does not compute authoritative area from masks.
|
|
|
|
### Next recommended pass
|
|
- Run full Sprint 9 validation and then perform Sprint 9 Verification Audit before considering future real SAM/YOLO-seg integration.
|
|
|
|
### Validation addendum
|
|
- `python -m compileall backend/app` (pass)
|
|
- `cd backend && python -m pytest` (pass, 88 tests)
|
|
- `bash scripts/run_readiness_check.sh` (pass)
|
|
- `cd frontend && npm run typecheck` (pass)
|
|
- `cd frontend && npm run build` (pass; existing Vite chunk-size warning only)
|
|
- `cd backend && python -m alembic heads` (pass, single head `202606120900`)
|
|
- `cd backend && python -m alembic upgrade head --sql` (pass; generated SQL includes `segmentations` and GiST index)
|
|
- `bash -n scripts/live_migration_smoke.sh` (pass)
|
|
- `docker compose config` could not run because Docker is not installed in this shell.
|
|
|
|
## Pass 26 - Sprint 10 release hardening and frontend modularization (2026-06-13)
|
|
|
|
### Completed
|
|
- Extracted Provider Capabilities, Detection Lab and Segmentation Lab sections from `frontend/src/App.tsx` into focused frontend components.
|
|
- Preserved existing workbench state ownership, API client calls, map overlay behavior and UI copy.
|
|
- Hardened `scripts/run_readiness_check.sh` with Alembic head verification and live migration smoke script syntax validation.
|
|
- Updated frontend README and changelog documentation for Sprint 10 maintainability work.
|
|
|
|
### Files changed
|
|
- `frontend/src/App.tsx`
|
|
- `frontend/src/components/providers/ProviderPanel.tsx`
|
|
- `frontend/src/components/detection/DetectionLab.tsx`
|
|
- `frontend/src/components/segmentation/SegmentationLab.tsx`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Sprint 10 intentionally does not add new backend capabilities, migrations, product features, AI dependencies or live provider fetching.
|
|
- `App.tsx` still owns shared workbench state orchestration; further extraction can be considered in a later maintainability pass if needed.
|
|
|
|
### Sprint 10 addendum - additional frontend extraction
|
|
- Extracted `frontend/src/components/project/ProjectPanel.tsx` and `frontend/src/components/project/AreaPanel.tsx` from `frontend/src/App.tsx`.
|
|
- Kept project and area form state owned by `App.tsx`; extracted components receive state and callbacks only.
|
|
- `cd frontend && npm run typecheck` passed after the additional extraction.
|
|
|
|
## Pass 27 - Sprint 11 Live Docker/PostGIS Runtime Validation (2026-06-13)
|
|
|
|
### Completed
|
|
- Hardened `scripts/live_migration_smoke.sh` so it runs `SELECT 1`, applies `alembic upgrade head`, then checks `PostGIS_Version()`.
|
|
- Added migrated schema-object checks for core tables and geometry indexes after the live migration step.
|
|
- Added `backend/tests/test_live_migration_smoke_script.py` to lock the smoke-script ordering and schema-check contract.
|
|
- Documented the Docker/PostGIS validation command sequence, expected local `DATABASE_URL` and cleanup commands in `backend/README.md`.
|
|
|
|
### Files changed
|
|
- `scripts/live_migration_smoke.sh`
|
|
- `backend/tests/test_live_migration_smoke_script.py`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Runtime status
|
|
- Docker is not installed or not available in this shell, so `docker compose config`, `docker compose up -d db` and the live container-backed smoke could not be completed here.
|
|
- On a Docker-enabled machine, run:
|
|
- `docker compose config`
|
|
- `docker compose up -d db`
|
|
- `DATABASE_URL=postgresql+psycopg://geointel:geointel@localhost:5432/geointel bash scripts/live_migration_smoke.sh`
|
|
|
|
### Known limitations
|
|
- Sprint 11 did not add product behavior, API contracts, migrations, AI dependencies or provider fetching.
|
|
- Live runtime validation is partially blocked until Docker/PostGIS is available in the execution environment.
|
|
|
|
## Pass 28 - Sprint 12 QA/QC golden dataset and benchmarking (2026-06-15)
|
|
|
|
### Completed
|
|
- Added deterministic golden QA/QC fixtures for reference and predicted building polygons.
|
|
- Added `fixtures/golden/expected_qa_metrics.json` with the expected partial-match baseline.
|
|
- Added `scripts/run_golden_qa_benchmark.py` to run existing `QaService` logic and verify `QualityService` persistence output.
|
|
- Added backend tests for expected golden metrics, benchmark command output and persisted metric keys.
|
|
- Updated QA/QC specification, backend README and changelog documentation.
|
|
|
|
### Files changed
|
|
- `fixtures/golden/reference_buildings.geojson`
|
|
- `fixtures/golden/predicted_buildings.geojson`
|
|
- `fixtures/golden/expected_qa_metrics.json`
|
|
- `scripts/run_golden_qa_benchmark.py`
|
|
- `backend/tests/test_sprint12_golden_qa_benchmark.py`
|
|
- `docs/QA_QC_SPECIFICATION.md`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Expected benchmark metrics
|
|
- precision: `0.5`
|
|
- recall: `0.5`
|
|
- F1: `0.5`
|
|
- mean IoU: `0.8339768339761133`
|
|
- false positives: `1`
|
|
- false negatives: `1`
|
|
|
|
### Known limitations
|
|
- The benchmark uses explicit fixture/demo data and an in-memory persistence session; it does not replace the pending Docker/PostGIS live smoke.
|
|
- Sprint 12 does not add product features, API contracts, migrations, live providers, AI model execution or new dependencies.
|
|
|
|
## Pass 29 - Sprint 13 Real YOLO operational hardening (2026-06-15)
|
|
|
|
### Completed
|
|
- Added `YoloPreflightService` for local configured-YOLO readiness checks without model loading, inference or downloads.
|
|
- Added `scripts/yolo_preflight.py` for CLI checks of enabled state, dependencies, local model file, tile manifest validity, tile limit and tile paths.
|
|
- Added Sprint 13 backend tests for disabled, dependency-unavailable and ready preflight states plus CLI JSON output.
|
|
- Updated AI pipeline, backend README and changelog documentation.
|
|
|
|
### Files changed
|
|
- `backend/app/services/yolo_preflight_service.py`
|
|
- `scripts/yolo_preflight.py`
|
|
- `backend/tests/test_sprint13_yolo_preflight.py`
|
|
- `docs/AI_PIPELINES.md`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Preflight does not prove model compatibility or inference correctness; it intentionally avoids loading YOLO models.
|
|
- Optional AI dependencies are still not installed by default.
|
|
- Docker/PostGIS live validation remains pending until Docker is available.
|
|
|
|
## Pass 30 - Release hardening audit pass (2026-06-15)
|
|
|
|
### Completed
|
|
- Audited release-readiness signals after Sprint 13, including timestamp warnings, frontend bundle output, migration SQL rendering and readiness coverage.
|
|
- Replaced backend `datetime.utcnow()` calls with timezone-aware UTC timestamps in service paths.
|
|
- Verified the affected backend tests with `DeprecationWarning` promoted to errors.
|
|
- Split frontend production output into app, React vendor and MapLibre vendor chunks, with an explicit chunk warning threshold for the known MapLibre GIS runtime.
|
|
- Updated backend/frontend README, TODO and changelog documentation.
|
|
|
|
### Files changed
|
|
- `backend/app/services/dataset_service.py`
|
|
- `backend/app/services/geojson_service.py`
|
|
- `backend/app/services/job_service.py`
|
|
- `backend/app/services/qa_service.py`
|
|
- `backend/app/services/quality_service.py`
|
|
- `frontend/vite.config.ts`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/TODO.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- This pass does not add product features, migrations, API contracts, AI dependencies, provider fetching or model execution.
|
|
- Docker/PostGIS live validation still requires a Docker-enabled machine.
|
|
- Larger frontend architectural decomposition remains a separate low-risk planning item; this pass only hardened build output.
|
|
|
|
## Pass 31 - Extended release hardening sweep (2026-06-15)
|
|
|
|
### Completed
|
|
- Promoted the backend readiness gate to run `pytest` with `-W error::DeprecationWarning`.
|
|
- Added backend tests that verify readiness and pass-end scripts keep the stricter release checks in place.
|
|
- Hardened `scripts/codex_pass_end_check.sh` so placeholder scans skip `node_modules`, `dist` and `__pycache__` folders.
|
|
- Updated `docs/TODO.md` with a current implementation status layer while preserving older planning context.
|
|
- Re-ran pass-end checks and strict backend warning checks.
|
|
|
|
### Files changed
|
|
- `scripts/run_readiness_check.sh`
|
|
- `scripts/codex_pass_end_check.sh`
|
|
- `backend/tests/test_readiness_gate.py`
|
|
- `backend/README.md`
|
|
- `docs/TODO.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- This pass still does not add API contracts, migrations, product features, provider fetching, AI dependencies or model execution.
|
|
- Docker/PostGIS live validation remains blocked in this local environment because Docker is unavailable.
|
|
|
|
## Pass 32 - Readiness contract gate hardening (2026-06-15)
|
|
|
|
### Completed
|
|
- Added API contract smoke validation to `scripts/run_readiness_check.sh`.
|
|
- Added a regression test that requires the readiness gate to keep running `scripts/smoke_contracts.py`.
|
|
- Re-ran the full readiness gate after the change.
|
|
|
|
### Files changed
|
|
- `scripts/run_readiness_check.sh`
|
|
- `backend/tests/test_readiness_gate.py`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker/PostGIS live validation remains blocked in this local environment because Docker is unavailable.
|
|
|
|
## Pass 33 - Docker runtime build hardening (2026-06-15)
|
|
|
|
### Completed
|
|
- Investigated Unraid/Tower Docker build failure from pasted server output.
|
|
- Fixed backend Docker build ordering so `README.md` and `app/` exist before `pip install .`.
|
|
- Removed mandatory root `.env` references from Compose; default local runtime now uses checked-in environment values.
|
|
- Added PostGIS healthcheck and backend `depends_on` health condition.
|
|
- Made the backend container run `python -m alembic upgrade head` before starting Uvicorn.
|
|
- Added backend and frontend `.dockerignore` files to reduce Docker build context and exclude dependency/build/cache outputs.
|
|
- Added regression tests for Dockerfile package-source ordering, Compose env behavior, DB health/migration startup and Docker ignore coverage.
|
|
|
|
### Files changed
|
|
- `backend/Dockerfile`
|
|
- `docker-compose.yml`
|
|
- `backend/.dockerignore`
|
|
- `frontend/.dockerignore`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `backend/README.md`
|
|
- `docs/TODO.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker commands still cannot be executed in this local Codex environment because Docker is unavailable here.
|
|
- The server should rerun `docker compose build --no-cache && docker compose up -d` to verify the real Docker runtime.
|
|
|
|
## Pass 34 - Docker browser port 1202 (2026-06-15)
|
|
|
|
### Completed
|
|
- Changed Docker Compose frontend host publishing from `5173:5173` to `1202:5173`.
|
|
- Added backend Docker CORS defaults for `http://localhost:1202` and `http://127.0.0.1:1202`.
|
|
- Updated `.env.example` and local/Docker documentation to point browser users to `http://localhost:1202` for Docker Compose.
|
|
- Added a regression assertion to Docker runtime config tests.
|
|
|
|
### Files changed
|
|
- `docker-compose.yml`
|
|
- `.env.example`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `backend/README.md`
|
|
- `frontend/README.md`
|
|
- `docs/LOCAL_DEVELOPMENT_RUNBOOK.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker commands still cannot be executed in this local Codex environment because Docker is unavailable here.
|
|
|
|
## Pass 35 - Docker backend database startup retry (2026-06-16)
|
|
|
|
### Completed
|
|
- Investigated Tower runtime logs showing backend Alembic startup failed with `connection refused` even after the db container reported healthy.
|
|
- Added `backend/docker_start.sh` to retry a real SQLAlchemy `SELECT 1` connection before migrations.
|
|
- Updated Compose to run `sh /app/docker_start.sh` for backend startup.
|
|
- Added regression tests that require the Docker start script and SQL readiness retry before migrations.
|
|
|
|
### Files changed
|
|
- `backend/docker_start.sh`
|
|
- `docker-compose.yml`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `backend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker commands still cannot be executed in this local Codex environment because Docker is unavailable here.
|
|
|
|
## Pass 36 - Alembic Docker log formatting fix (2026-06-16)
|
|
|
|
### Completed
|
|
- Investigated backend Docker logs showing repeated literal `%(levelname)-5.5s [%(name)s] %(message)s` lines during migrations.
|
|
- Fixed `backend/alembic.ini` logging formatter from escaped `%%(...)` tokens to runtime interpolation `%(...)` tokens.
|
|
- Added a regression test for Alembic logging formatter correctness.
|
|
- Verified Alembic SQL rendering no longer emits literal formatter spam.
|
|
|
|
### Files changed
|
|
- `backend/alembic.ini`
|
|
- `backend/tests/test_alembic_logging_config.py`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
## Pass 37 - Frontend same-origin API proxy for Docker LAN access (2026-06-16)
|
|
|
|
### Completed
|
|
- Audited the running app at `http://192.168.10.150:1202` from Codex: frontend HTML and backend `/health` were reachable, but `/api/v1/projects` on the frontend origin returned the frontend HTML fallback.
|
|
- Changed the frontend API client default from `http://localhost:8000` to same-origin requests.
|
|
- Added Vite proxy routes for `/api` and `/health`.
|
|
- Added Docker Compose `VITE_API_PROXY_TARGET=http://backend:8000` so LAN browsers use `http://192.168.10.150:1202` only and the frontend container proxies API calls internally.
|
|
- Added regression tests for same-origin API/proxy behavior.
|
|
|
|
### Files changed
|
|
- `frontend/src/services/api/client.ts`
|
|
- `frontend/vite.config.ts`
|
|
- `docker-compose.yml`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `frontend/README.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- The running Tower deployment needs a rebuild/restart before this fix is active.
|
|
|
|
## Pass 38 - Browser runtime proxy guard (2026-06-16)
|
|
|
|
### Completed
|
|
- Added `scripts/verify_browser_runtime.sh` to verify the browser-facing frontend URL, `/api/v1/projects` proxy behavior and optional backend health endpoint.
|
|
- Added readiness syntax validation for the browser runtime verification script.
|
|
- Updated environment and local development documentation to prefer same-origin frontend API calls with Vite proxying in Docker/LAN deployments.
|
|
|
|
### Files changed
|
|
- `scripts/verify_browser_runtime.sh`
|
|
- `scripts/run_readiness_check.sh`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `docs/ENVIRONMENT_SPEC.md`
|
|
- `docs/LOCAL_DEVELOPMENT_RUNBOOK.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- The currently running Tower deployment at `http://192.168.10.150:1202` still returns frontend HTML for `/api/v1/projects` until the frontend image is rebuilt and restarted.
|
|
|
|
## Pass 39 - Environment contract cleanup (2026-06-16)
|
|
|
|
### Completed
|
|
- Corrected `.env.example` and `docs/ENVIRONMENT_SPEC.md` to use the YOLO environment variable names read by backend settings: `YOLO_ENABLED`, `YOLO_MODEL_PATH`, `YOLO_MODEL_VERSION` and `YOLO_MAX_TILES`.
|
|
- Updated frontend example settings to keep `VITE_API_BASE_URL` empty by default and expose `VITE_API_PROXY_TARGET` for Vite proxy routing.
|
|
- Added regression coverage for example environment names and browser runtime proxy verification.
|
|
|
|
### Files changed
|
|
- `.env.example`
|
|
- `docs/ENVIRONMENT_SPEC.md`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker cannot be executed in this local Codex environment; Tower still needs an image rebuild/restart to activate the frontend proxy fix.
|
|
|
|
## Pass 40 - Frontend Docker nginx reverse proxy (2026-06-16)
|
|
|
|
### Completed
|
|
- Replaced the Docker frontend runtime with a production Vite build served by nginx.
|
|
- Added `frontend/nginx.conf` with explicit reverse proxy rules for `/api/` and `/health` to `backend:8000`.
|
|
- Changed Docker Compose frontend publishing from `1202:5173` to `1202:80`.
|
|
- Updated Docker runtime regression tests and documentation so the browser-facing API path is no longer dependent on Vite dev-server proxy behavior.
|
|
|
|
### Files changed
|
|
- `frontend/Dockerfile`
|
|
- `frontend/nginx.conf`
|
|
- `docker-compose.yml`
|
|
- `backend/tests/test_docker_runtime_config.py`
|
|
- `README.md`
|
|
- `frontend/README.md`
|
|
- `docs/ENVIRONMENT_SPEC.md`
|
|
- `docs/LOCAL_DEVELOPMENT_RUNBOOK.md`
|
|
- `CHANGELOG.md`
|
|
- `docs/CODEX_EXECUTION_LOG.md`
|
|
|
|
### Known limitations
|
|
- Docker still cannot be executed in this local Codex environment. Tower must rebuild the frontend image to activate the nginx runtime.
|
|
|
|
## Sprint 14 - Docker GIS runtime enablement (2026-06-16)
|
|
|
|
- Added backend `gis` optional dependency group for the approved Rasterio/GeoPandas runtime stack.
|
|
- Updated backend Docker image to install `.[gis]` and GDAL/GEOS/PROJ system packages.
|
|
- Added `scripts/verify_gis_runtime.sh` to verify PostGIS, Rasterio and GeoPandas capabilities through the browser-facing frontend proxy.
|
|
- Added readiness syntax coverage for the GIS runtime verification script.
|
|
- Added regression tests for Docker GIS dependency installation and capability verification coverage.
|
|
- Updated backend, environment, root README and changelog documentation with local/LAN verification commands.
|
|
- No API contracts, migrations, AI dependencies, provider fetching or product features were changed.
|
|
- Added `scripts/gis_import_smoke.py` and wired it into the backend Docker build so broken Rasterio/GeoPandas/pyogrio imports fail during image build.
|
|
- Added Docker Compose healthchecks for backend and frontend; frontend now waits for backend service health before starting.
|
|
- Corrected the GIS import smoke placement so the backend Docker build can access it inside the `./backend` build context; the root script now wraps the backend script.
|
|
|
|
## Sprint 15 - Explicit demo workflow seed (2026-06-16)
|
|
|
|
- Added `POST /api/v1/demo/workflow` for an explicit offline demo workflow seed.
|
|
- Added `DemoWorkflowService` to create or return a demo project, AOI, fixture reference dataset, fixture candidate dataset and persisted QA/QC metrics.
|
|
- Added `scripts/seed_demo_workflow.py` for terminal-based demo seeding.
|
|
- Added frontend `Load demo workflow` action in the Projects panel.
|
|
- Added endpoint/fixture-contract tests and readiness compile coverage for the demo seed script.
|
|
- No live GRB/OSM fetching, AI inference, migrations or new dependencies were introduced.
|
|
|
|
## Sprint 16 - QA/QC result visibility (2026-06-16)
|
|
|
|
- Added `GET /api/v1/projects/{project_id}/quality-checks` for read-only project QA/QC result listing.
|
|
- Added `QualityCheckService` to return persisted `quality_checks` with metric rows.
|
|
- Added frontend QA/QC Results panel and API client support.
|
|
- Demo workflow loading and QA actions now refresh persisted QA/QC results in the UI.
|
|
- Added backend tests for quality check listing and canonical envelopes.
|
|
- No migrations, live provider fetching, AI inference or new dependencies were introduced.
|
|
|
|
## Sprint 17 export foundation (2026-06-16)
|
|
|
|
Changed:
|
|
- Hardened GeoJSON exports so vector dataset, detection run and segmentation run exports persist `exports` rows and write JSON artifacts.
|
|
- Added project metadata JSON export plus export list/read/content endpoints.
|
|
- Added a frontend Export Center panel for creating exports, listing export records and previewing JSON content.
|
|
- Added backend tests for export persistence, artifact writing, raster rejection and canonical envelope behavior.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh`
|
|
|
|
Open:
|
|
- Docker/live browser validation must be rerun on the deployment host after rebuild.
|
|
- YOLO-format export and report export remain documented future work; this pass only implements JSON/GeoJSON export foundation.
|
|
|
|
Limitations:
|
|
- Export artifacts are returned through API JSON content preview; browser file-download UX is not implemented yet.
|
|
- Detection and segmentation exports require existing persisted runs; no inference or fake output generation is introduced.
|
|
|
|
Next recommended pass:
|
|
- Rebuild/redeploy the Docker stack and verify `/api/v1/exports/*` through the LAN frontend proxy, then consider a lightweight file-download endpoint or report artifact pass.
|
|
|
|
## Sprint 17 export download hardening (2026-06-16)
|
|
|
|
Changed:
|
|
- Added `GET /api/v1/exports/{export_id}/download` as a raw file response for stored JSON/GeoJSON export artifacts.
|
|
- Reused the same export artifact existence validation for content preview and downloads.
|
|
- Added frontend Export Center download buttons using the configured/same-origin API base URL.
|
|
- Added backend tests for missing artifacts and file download response headers/content.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh`
|
|
|
|
Open:
|
|
- Docker/LAN validation should be rerun after redeploy on the Tower host.
|
|
|
|
Limitations:
|
|
- Download endpoint is intentionally a raw file response, not a canonical JSON envelope, because it is a browser/file artifact path.
|
|
|
|
Next recommended pass:
|
|
- Rebuild Docker and verify Export Center create/preview/download against `http://192.168.10.150:1202`.
|
|
|
|
## Sprint 17 lightweight report artifact export (2026-06-16)
|
|
|
|
Changed:
|
|
- Added `POST /api/v1/exports/report` to create a lightweight HTML project report artifact from persisted project, dataset and QA/QC summary state.
|
|
- Added HTML escaping for report-rendered project and dataset values.
|
|
- Updated export downloads to return `text/html` for HTML report artifacts and `application/json` for JSON/GeoJSON artifacts.
|
|
- Added a frontend Export Center action for project report HTML export.
|
|
- Added backend tests for HTML report artifact creation and HTML download response behavior.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh`
|
|
|
|
Open:
|
|
- Docker/LAN verification should be rerun after deployment rebuild.
|
|
|
|
Limitations:
|
|
- Report export is intentionally a simple HTML artifact, not a PDF designer or standalone Reports module.
|
|
- Report content is summary-only and uses existing persisted project, dataset and QA/QC rows.
|
|
|
|
Next recommended pass:
|
|
- Rebuild Docker and verify Export Center metadata, GeoJSON, report HTML and download flows through the LAN URL.
|
|
|
|
## Sprint 17 export audit trail and LAN demo/export smoke (2026-06-16)
|
|
|
|
Changed:
|
|
- Added export history to project metadata JSON and lightweight HTML report artifacts.
|
|
- Added `scripts/verify_demo_export_workflow.sh` to verify the browser-facing demo workflow, persisted QA/QC listing, metadata export, report export, vector GeoJSON export, export listing and artifact downloads.
|
|
- Included the demo/export workflow script in the main readiness syntax gate.
|
|
- Added backend tests to lock export-history content and demo/export script coverage.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh && bash -n scripts/verify_browser_runtime.sh && bash -n scripts/verify_gis_runtime.sh && bash -n scripts/verify_demo_export_workflow.sh`
|
|
|
|
Open:
|
|
- `scripts/verify_demo_export_workflow.sh` still needs to be run against the rebuilt Tower deployment URL.
|
|
|
|
Limitations:
|
|
- The smoke script intentionally uses explicit fixture demo data and does not fetch live GRB/OSM or run AI inference.
|
|
|
|
Next recommended pass:
|
|
- Rebuild Docker on the Tower host and run `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202`.
|
|
|
|
## Sprint 32 Unraid all-in-one runtime verification (2026-06-17)
|
|
|
|
Changed:
|
|
- Converted the Unraid/Tower runtime to a single editable `geointel` Docker container via `docker-compose.unraid.yml`.
|
|
- Built the all-in-one image from `postgres:16-bookworm` with PostgreSQL 16/PostGIS packages, the FastAPI backend, nginx and the React frontend.
|
|
- Removed the default nginx site from the image so `/api/v1/*` is proxied to the embedded backend instead of returning nginx 404s.
|
|
- Hardened live migration smoke and browser runtime verification with startup retries.
|
|
- Browser runtime verification now checks the frontend, `/api/v1/projects` canonical JSON envelope and `/geointel-icon.png`.
|
|
|
|
Tested:
|
|
- `python -m pytest backend\tests\test_sprint31_unraid_template.py`
|
|
- `bash -n deploy/unraid/all-in-one-start.sh`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- `bash -n scripts/verify_browser_runtime.sh`
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\deploy_tower.ps1`
|
|
- `docker compose -f docker-compose.unraid.yml ps` on Tower
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`
|
|
|
|
Verified runtime:
|
|
- Tower runs one GeoIntel app container: `geointel`.
|
|
- Published browser port is `0.0.0.0:1202->80/tcp`.
|
|
- Container health is `healthy`.
|
|
- Live migration smoke passed with PostGIS available and one Alembic head.
|
|
- Frontend, API proxy and icon are reachable at `http://192.168.10.150:1202`.
|
|
|
|
Open:
|
|
- Existing reused PostgreSQL volume logs a collation version mismatch because the old database was created on Debian glibc 2.31 and the all-in-one runtime uses glibc 2.36. Runtime and migrations pass; a future maintenance pass can rebuild/refresh collation metadata if needed.
|
|
|
|
## Sprint 32 DockerMan template/icon follow-up (2026-06-17)
|
|
|
|
Changed:
|
|
- Added a PNG icon for Unraid/DockerMan because DockerMan may not reliably render app-served SVG icons.
|
|
- Changed the Unraid template `<Name>` to `geointel` so it matches the running all-in-one container name.
|
|
- Changed the template icon URL to the absolute PNG URL `http://192.168.10.150:1202/geointel-icon.png`.
|
|
- Added DockerMan labels to `docker-compose.unraid.yml` so the running Compose container exposes Unraid managed/webui/icon metadata.
|
|
- Updated Tower deploy scripts to copy the editable template to `/boot/config/plugins/dockerMan/templates-user/my-geointel.xml`.
|
|
- Updated Tower deploy scripts to copy the PNG icon to `/boot/config/plugins/dockerMan/images/geointel-icon.png`.
|
|
|
|
Expected Unraid behavior:
|
|
- Refresh the Unraid Docker page after deploy so DockerMan reloads the user template metadata.
|
|
- The running `geointel` container should have template-backed editable fields and a PNG icon.
|
|
|
|
## Sprint 32 automatic DockerMan-native deploy follow-up (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `deploy/unraid/run-dockerman-container.sh` as the single source of truth for the final Unraid container launch.
|
|
- Updated Tower deploy scripts so repository deploys validate the Compose reference, build the image with plain `docker build` and start the final container with `docker run` plus DockerMan labels.
|
|
- The launch script installs the DockerMan template/icon, removes old `geointel` containers, preserves/migrates the old Compose PostGIS volume when needed and starts the final `geointel` container as DockerMan-managed.
|
|
- Updated live migration smoke to support direct container execution through `LIVE_SMOKE_CONTAINER=geointel`.
|
|
|
|
Expected Unraid behavior:
|
|
- A deploy from the repo should no longer leave the final app as a plain Compose-owned container.
|
|
- The final image/container should avoid Compose metadata labels that can confuse Unraid's Docker page.
|
|
- The running `geointel` container should expose `net.unraid.docker.managed=dockerman`, web UI metadata and icon metadata immediately after deploy.
|
|
|
|
## Sprint 33 QA/QC benchmark readiness hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Added `scripts/verify_golden_qa_benchmark.sh` as a shell wrapper for the deterministic QA/QC golden benchmark.
|
|
- Made `scripts/run_readiness_check.sh` run `scripts/run_golden_qa_benchmark.py --json` so QA/QC metric drift fails the main release gate.
|
|
- Added a readiness syntax check for the golden benchmark wrapper.
|
|
- Hardened `scripts/validate_fixtures.py` so `fixtures/golden` GeoJSON files and expected fixture paths are validated alongside the general GeoJSON fixtures.
|
|
- Added backend regression tests that keep the golden benchmark wired into readiness.
|
|
- Updated script/backend docs, TODO and changelog.
|
|
|
|
Tested:
|
|
- `python scripts/validate_fixtures.py`
|
|
- `python scripts/run_golden_qa_benchmark.py --json`
|
|
- `bash -n scripts/verify_golden_qa_benchmark.sh`
|
|
- `bash scripts/verify_golden_qa_benchmark.sh`
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- The benchmark intentionally uses explicit local fixtures only. It does not fetch live GRB/OSM data and does not run AI inference.
|
|
|
|
Next recommended pass:
|
|
- Continue with broader QA/QC golden demo coverage or frontend export preview decomposition.
|
|
|
|
## Sprint 34 browser-facing golden QA demo hardening (2026-06-17)
|
|
|
|
Changed:
|
|
- Hardened `scripts/verify_demo_export_workflow.sh` so the browser-facing demo/export smoke loads `fixtures/golden/expected_qa_metrics.json`.
|
|
- The smoke now verifies persisted QA/QC status, F1 score, precision, recall, mean IoU, false positives, false negatives and match counts against the golden baseline.
|
|
- Corrected the offline demo AOI to cover the golden fixture geometries instead of an older broad Kempen placeholder outside the fixture coordinates.
|
|
- Made existing demo workflows self-heal stale/unsupported QA checks by syncing the demo AOI and persisting a fresh golden QA/QC result.
|
|
- Added regression checks in backend tests so the demo/export smoke cannot regress back to key-existence-only QA/QC validation.
|
|
- Updated script documentation, TODO and changelog.
|
|
|
|
Tested:
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest tests/test_sprint15_demo_workflow.py tests/test_sprint21_demo_workflow_smoke.py tests/test_readiness_gate.py -q`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202`
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202`
|
|
|
|
Open:
|
|
- Docker build still emits a warning about `GEOINTEL_POSTGRES_PASSWORD` being present as an image `ENV`; runtime behavior is green, but a future secret-hygiene pass should move that default out of the Dockerfile.
|
|
|
|
Limitations:
|
|
- The demo/export smoke remains intentionally fixture-based and idempotent. It does not fetch live providers and does not run AI inference.
|
|
|
|
Next recommended pass:
|
|
- Run the full release-readiness gate and then rebuild/deploy to Tower for browser-facing verification.
|
|
|
|
## Sprint 35 Docker runtime secret hygiene (2026-06-17)
|
|
|
|
Changed:
|
|
- Removed embedded PostGIS database name/user/password defaults from `deploy/unraid/Dockerfile.all-in-one` image metadata.
|
|
- Kept database credentials as runtime configuration supplied by `.env`, the Unraid template, Compose or `docker run -e`.
|
|
- Added regression coverage so `GEOINTEL_POSTGRES_PASSWORD` is not baked into the all-in-one Dockerfile again.
|
|
- Updated Unraid runtime documentation and changelog.
|
|
|
|
Tested:
|
|
- `cd backend && python -m pytest tests/test_sprint31_unraid_template.py tests/test_docker_runtime_config.py -q`
|
|
- `bash -n deploy/unraid/all-in-one-start.sh`
|
|
- `bash -n deploy/unraid/run-dockerman-container.sh`
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `cd backend && python -m alembic heads && python -m alembic upgrade head --sql`
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202`
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202`
|
|
|
|
Open:
|
|
- Local Windows shell has no `docker` command in PATH, so Docker build/config verification must run on Tower during deploy.
|
|
- Tower rebuild completed and the previous Docker BuildKit `SecretsUsedInArgOrEnv` warning no longer appears.
|
|
|
|
Limitations:
|
|
- The start script still has safe local defaults for standalone/dev startup. Shared deployments should set `GEOINTEL_POSTGRES_PASSWORD` via runtime configuration.
|
|
|
|
Next recommended pass:
|
|
- Run release readiness, rebuild/deploy Tower and verify browser/GIS/demo smoke again.
|
|
|
|
## Sprint 36 PostgreSQL collation maintenance visibility (2026-06-17)
|
|
|
|
Changed:
|
|
- Added PostgreSQL database collation version reporting to `scripts/live_migration_smoke.sh`.
|
|
- The live smoke now prints `COLLATION_VERSION_MISMATCH` with stored and actual versions when a reused PostGIS volume was created under an older libc/collation runtime.
|
|
- The smoke also prints the exact `ALTER DATABASE "... " REFRESH COLLATION VERSION;` acknowledgement command, but does not run it automatically.
|
|
- Documented the Unraid maintenance procedure and backup/index review guidance.
|
|
- Added regression coverage for the collation reporting path.
|
|
|
|
Tested:
|
|
- `cd backend && python -m pytest tests/test_live_migration_smoke_script.py -q`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -W error::DeprecationWarning`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`
|
|
- `bash scripts/verify_gis_runtime.sh http://192.168.10.150:1202`
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202`
|
|
|
|
Open:
|
|
- Tower live smoke reports `COLLATION_VERSION_MISMATCH: database=geointel stored=2.31 actual=2.36`.
|
|
- Runtime remains green. The next maintenance action is optional/manual acknowledgement after backup/index review: `ALTER DATABASE "geointel" REFRESH COLLATION VERSION;`.
|
|
|
|
Limitations:
|
|
- The smoke reports and documents the maintenance action. It intentionally does not mutate collation metadata automatically.
|
|
|
|
Next recommended pass:
|
|
- Run release readiness, deploy Tower and decide whether to manually acknowledge the current collation version.
|
|
|
|
## Sprint 49 Workbench shell UI refactor (2026-06-17)
|
|
|
|
Changed:
|
|
- Audited the live workbench UI and confirmed the main usability issue was information architecture: all V1 workflows were mounted as one long vertical panel stack.
|
|
- Refactored the frontend into a task-based workbench shell with Overview, Data, Map, QA/QC, AI Labs, Exports and System workspaces.
|
|
- Added a persistent top context bar for active project, AOI, dataset and layer state.
|
|
- Moved selected dataset details into a persistent right-side inspector while keeping the same dataset/raster/vector operation callbacks.
|
|
- Added stable primary navigation test anchors.
|
|
- Documented the new frontend shell structure and updated the changelog.
|
|
|
|
Tested:
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local Vite visual audit on `http://127.0.0.1:5175`
|
|
|
|
Open:
|
|
- Local Vite visual audit shows `Request failed (404)` when no local backend/proxy target is available. Docker/nginx same-origin proxy behavior remains the production path.
|
|
- The next pass should run full release readiness and deploy to Tower for browser-facing verification on `http://192.168.10.150:1202`.
|
|
|
|
Limitations:
|
|
- This pass intentionally changes UI structure only. It does not add product capabilities, alter API contracts, change migrations, fetch live providers or enable new AI models.
|
|
|
|
Next recommended pass:
|
|
- Run full readiness, rebuild/deploy Tower, then perform a live browser smoke through the new workbench navigation.
|
|
|
|
## Sprint 50 Workspace usability polish (2026-06-17)
|
|
|
|
Changed:
|
|
- Refined the task-based shell workspaces after the first UI refactor.
|
|
- Converted Project, AOI and Dataset panels into compact forms and card-based lists for faster scanning.
|
|
- Converted the Map workspace controls into a toolbar with dedicated AOI/layer controls and status.
|
|
- Converted Detection Lab and Segmentation Lab into model, run, result and QA blocks.
|
|
- Added CSS utilities for entity cards, dataset cards, model cards, lab blocks, primary/secondary actions and responsive nested forms.
|
|
- Added regression coverage for the polished workspace structure.
|
|
|
|
Tested:
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `cd backend && python -m pytest tests/test_sprint47_workbench_interaction_smoke.py tests/test_sprint49_workbench_shell_refactor.py -q`
|
|
|
|
Open:
|
|
- Run full release readiness and deploy the polish to Tower.
|
|
|
|
Limitations:
|
|
- This pass remains frontend-only. It does not add features, change API contracts, alter migrations, fetch live providers or enable new AI models.
|
|
|
|
Next recommended pass:
|
|
- Deploy to Tower and verify the Data, Map and AI Labs workspaces on `http://192.168.10.150:1202`.
|
|
|
|
## Sprint 51 QA/QC and export workspace polish (2026-06-17)
|
|
|
|
Changed:
|
|
- Refined the QA/QC workspace so persisted checks are shown as summary tiles, quality-check cards and metric chips instead of a raw nested list.
|
|
- Refined the Exports workspace with grouped export actions, latest-export status, artifact history cards and a framed JSON/GeoJSON preview panel.
|
|
- Kept the existing `useQualityWorkflow` and `useExportWorkflow` dataflow intact; no API client calls or backend contracts changed.
|
|
- Added regression coverage for the polished QA/QC and Exports workspace structure.
|
|
|
|
Tested:
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd backend && python -m pytest tests/test_sprint27_frontend_workflow_hooks.py tests/test_sprint30_workbench_components.py tests/test_sprint47_workbench_interaction_smoke.py -q`
|
|
- `cd backend && python -m pytest tests/test_sprint51_quality_export_polish.py -q`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`207 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- Tower live migration smoke passed.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
|
|
Open:
|
|
- Manual visual review of populated QA/QC and Exports states with real project artifacts remains useful after the next demo workflow run.
|
|
|
|
Limitations:
|
|
- This pass remains UI polish only. It does not add product capabilities, change API contracts, alter migrations, fetch live providers or enable new AI models.
|
|
|
|
Next recommended pass:
|
|
- Verify QA/QC and Exports on the deployed Tower workbench, then continue with selected-object inspector detail tabs.
|
|
|
|
## Sprint 52 selected context inspector tabs (2026-06-17)
|
|
|
|
Changed:
|
|
- Replaced the dataset-only right inspector with a tabbed `WorkbenchInspector`.
|
|
- Added Context, Dataset, QA/Exports and AI Runs tabs using existing project, AOI, dataset, QA, export and AI-run state.
|
|
- Kept the existing `DatasetDetailPanel` as the Dataset tab so raster/vector operations and callbacks remain behavior-compatible.
|
|
- Added inspector cards for selected map feature properties, latest QA/QC, latest export and selected detection/segmentation run summaries.
|
|
- Added regression coverage for inspector wiring and tab structure.
|
|
|
|
Tested:
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd backend && python -m pytest tests/test_sprint49_workbench_shell_refactor.py tests/test_sprint50_workspace_usability_polish.py tests/test_sprint51_quality_export_polish.py -q`
|
|
- `cd backend && python -m pytest tests/test_sprint28_dataset_workflow_hook.py tests/test_sprint29_dataset_components.py tests/test_sprint52_workbench_inspector_tabs.py -q`
|
|
- `cd backend && python -m pytest tests/test_sprint52_workbench_inspector_tabs.py -q`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`210 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- Tower live migration smoke passed.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
|
|
Open:
|
|
- Manual click-through of each inspector tab with populated demo data remains useful for visual fine-tuning.
|
|
|
|
Limitations:
|
|
- This pass remains UI orchestration only. It does not add features, change API contracts, alter migrations, fetch live providers or enable new AI models.
|
|
|
|
Next recommended pass:
|
|
- Verify the tabbed inspector live on Tower, then continue with map/dataset selection ergonomics.
|
|
|
|
## Sprint 53 map/dataset selection ergonomics (2026-06-17)
|
|
|
|
Changed:
|
|
- Added active-state styling to dataset cards so the selected dataset is visibly anchored in the catalog.
|
|
- Added dataset quick actions to open a dataset directly in the Map workspace or Exports workspace after loading details.
|
|
- Added inspector navigation actions to jump to Data, Map, QA/QC, Exports and AI Labs without hunting through the left navigation.
|
|
- Kept the existing dataset loading, map layer state, export flow and API clients unchanged.
|
|
- Added regression coverage for selection quick actions and inspector navigation wiring.
|
|
|
|
Tested:
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd backend && python -m pytest tests/test_sprint28_dataset_workflow_hook.py tests/test_sprint29_dataset_components.py tests/test_sprint52_workbench_inspector_tabs.py -q`
|
|
- `cd backend && python -m pytest tests/test_sprint53_selection_ergonomics.py -q`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`213 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- Tower live migration smoke passed.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
|
|
Open:
|
|
- Manual demo-data click-through can still tune copy/density after real populated cards are visible.
|
|
|
|
Limitations:
|
|
- This pass remains UI orchestration only. It does not add features, change API contracts, alter migrations, fetch live providers or enable new AI models.
|
|
|
|
Next recommended pass:
|
|
- Verify dataset quick actions with demo data on Tower, then improve populated map/detail readability if needed.
|
|
|
|
## Sprint 54 populated-state UI polish (2026-06-17)
|
|
|
|
Changed:
|
|
- Ran the live demo/export workflow against the Tower deployment to inspect real populated workbench states.
|
|
- Adjusted the Data workspace so Project and AOI remain side by side while the Dataset catalog spans the full row for readable populated dataset cards.
|
|
- Made dataset-card action rows responsive so Map/Exports/detail actions do not crowd or clip on populated cards.
|
|
- Limited the Exports artifact history to the latest 10 entries by default with an explicit show-all toggle.
|
|
- Kept the Export Preview panel visible even before an artifact is selected, avoiding a blank middle column in the Exports workspace.
|
|
- Shortened displayed export storage paths while keeping the full path available in the title attribute.
|
|
- Added regression coverage for the populated Data layout and Exports populated-state behavior.
|
|
|
|
Tested:
|
|
- `bash scripts/verify_demo_export_workflow.sh http://192.168.10.150:1202`
|
|
- Local Vite visual audit using the live Tower API proxy on `http://127.0.0.1:5176`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd backend && python -m pytest tests/test_sprint51_quality_export_polish.py tests/test_sprint53_selection_ergonomics.py -q`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`214 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- Tower live migration smoke passed.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Direct HTTP check passed for `http://192.168.10.150:1202` and `http://192.168.10.150:1202/api/v1/projects`.
|
|
|
|
Open:
|
|
- Continue monitoring export history growth during repeated demo workflow runs.
|
|
|
|
Limitations:
|
|
- This pass remains UI polish only. It does not add features, change API contracts, alter migrations, fetch live providers or enable new AI models.
|
|
|
|
Next recommended pass:
|
|
- Add export history filtering or retention controls if artifact history continues to grow during demo runs.
|
|
|
|
## Sprint 55 live visual shell polish (2026-06-17)
|
|
|
|
Changed:
|
|
- Performed a Browser-based visual audit of the live Tower workbench at `http://192.168.10.150:1202`.
|
|
- Confirmed the main visual defect was the cramped three-column desktop shell: left navigation, central workspace and inspector competed for width at 1280px.
|
|
- Compacted the sticky top context bar and left navigation.
|
|
- Changed the standard desktop breakpoint so the inspector moves below the workspace up to 1360px, while remaining a side panel on wider displays.
|
|
- Made the Map toolbar wrap responsively instead of forcing four controls into a narrow row.
|
|
- Added a workspace-change scroll reset so switching pages starts at the workspace heading instead of inheriting stale scroll position.
|
|
- Added regression coverage for the standard desktop shell width and scroll-reset behavior.
|
|
|
|
Tested:
|
|
- Local Browser visual audit against `http://127.0.0.1:5177` using the live Tower API proxy.
|
|
- Verified Overview, Data, Map and Exports workspaces visually after the shell changes.
|
|
- Browser console error/warning check returned no entries.
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd backend && python -m pytest tests/test_sprint53_selection_ergonomics.py tests/test_sprint50_workspace_usability_polish.py -q`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`215 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- Tower live migration smoke passed.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Live Browser visual verification passed for Overview and Map on `http://192.168.10.150:1202`.
|
|
|
|
Open:
|
|
- Export history still grows during repeated demo runs and should get filtering or retention controls.
|
|
|
|
Limitations:
|
|
- This pass remains UI shell polish only. It does not add features, change API contracts, alter migrations, fetch live providers or enable new AI models.
|
|
|
|
Next recommended pass:
|
|
- Continue with export history filtering or retention controls if repeated demo runs keep growing artifact history.
|
|
|
|
## Sprint 56 export history controls (2026-06-17)
|
|
|
|
Changed:
|
|
- Added frontend-only search for export type, status, id and storage path in the Exports workspace.
|
|
- Added export type and status filters generated from the loaded export records.
|
|
- Made the latest-10 limiter apply after filtering so large histories stay manageable without hiding matching records unexpectedly.
|
|
- Added a clear no-match empty state and reset view action.
|
|
- Slightly compacted export action buttons so the history controls are visible earlier on standard desktop viewports.
|
|
- Added regression coverage for the export history controls, filtered list limiting and no-match state.
|
|
|
|
Tested:
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd backend && python -m pytest tests/test_sprint51_quality_export_polish.py -q`
|
|
- `cd frontend && npm run build`
|
|
- Local Browser visual audit against `http://127.0.0.1:5178` using the live Tower API proxy.
|
|
- Verified Exports history controls render with 50 live artifacts and a `report` search narrows the list to 17 matching artifacts.
|
|
- Browser console error/warning check returned no entries.
|
|
|
|
Open:
|
|
- Run full readiness, deploy Tower and verify the live Exports filters on `http://192.168.10.150:1202`.
|
|
|
|
Limitations:
|
|
- This pass remains frontend UI hardening only. It does not add backend filtering, retention deletion, API changes, migrations, live provider fetching or AI model behavior.
|
|
|
|
Next recommended pass:
|
|
- Consider a safe export retention/cleanup command if the artifact table keeps growing beyond demo needs.
|
|
|
|
## Sprint 57 safer demo export cleanup (2026-06-17)
|
|
|
|
Changed:
|
|
- Hardened the existing dry-run-first demo export cleanup command instead of creating a parallel cleanup path.
|
|
- Added `--max-delete` with a default cap of 25 so large `--apply` runs are blocked until explicitly reviewed and raised.
|
|
- Added repeatable `--export-type` filters for targeted cleanup, e.g. reports only.
|
|
- Extended the cleanup summary with `keep_latest`, `max_delete`, `export_types`, `type_filtered_export_count`, `candidate_exports` and `blocked_reason`.
|
|
- Updated the root wrapper to expose the new filter helper.
|
|
- Updated `scripts/README.md`, `docs/STORAGE_ARCHITECTURE.md`, `backend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
- Added regression coverage for export-type filtering, parser defaults and max-delete options.
|
|
|
|
Tested:
|
|
- `python -m py_compile scripts/cleanup_demo_artifacts.py backend/scripts/cleanup_demo_artifacts.py`
|
|
- `cd backend && python -m pytest tests/test_sprint24_cleanup_demo_artifacts.py tests/test_readiness_gate.py -q` (`16 passed`)
|
|
- Tower dry-run command passed without deleting data:
|
|
`docker exec geointel /opt/geointel/venv/bin/python /app/scripts/cleanup_demo_artifacts.py --keep-latest 10 --max-delete 100 --export-type project_report_html`
|
|
- Tower dry-run reported `matched_export_count=51`, `type_filtered_export_count=17`, `selected_export_count=7`, `deleted_export_count=0`.
|
|
|
|
Open:
|
|
- Run full readiness and redeploy Tower with the clearer `candidate_exports` dry-run output.
|
|
|
|
Limitations:
|
|
- Cleanup still targets demo export records/files only. It does not delete source uploads, vector features, projects, AOIs, QA/QC records, rasters, tiles, masks or production data.
|
|
|
|
Next recommended pass:
|
|
- Add a small live maintenance smoke that runs cleanup in dry-run mode through the deployed all-in-one container.
|
|
|
|
## Sprint 58 demo cleanup dry-run smoke (2026-06-18)
|
|
|
|
Changed:
|
|
- Added `scripts/verify_demo_cleanup_dry_run.sh` as a live maintenance smoke for the demo export cleanup path.
|
|
- The smoke auto-detects a running compose backend or all-in-one `geointel` container, with explicit `CLEANUP_MODE=local|compose|container` overrides.
|
|
- The smoke runs cleanup without `--apply`, then verifies `dry_run=true`, `deleted_export_count=0`, `deleted_files=[]`, expected filter settings and candidate dry-run fields.
|
|
- Added the smoke syntax check to `scripts/run_readiness_check.sh`.
|
|
- Added regression coverage that the readiness gate checks the smoke and that the smoke contract remains non-mutating.
|
|
- Updated `scripts/README.md`, `docs/STORAGE_ARCHITECTURE.md`, `backend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- `bash -n scripts/verify_demo_cleanup_dry_run.sh`
|
|
- `python -m py_compile scripts/cleanup_demo_artifacts.py backend/scripts/cleanup_demo_artifacts.py`
|
|
- `cd backend && python -m pytest tests/test_readiness_gate.py tests/test_sprint24_cleanup_demo_artifacts.py -q` (`18 passed`)
|
|
- `bash scripts/run_readiness_check.sh` (`219 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- Tower live migration smoke passed.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Tower cleanup dry-run smoke passed with `matched=51`, `type_filtered=17`, `selected=7`, `deleted=0`.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- The readiness gate checks this live smoke's syntax only. The actual cleanup dry-run still requires a running backend/PostGIS runtime.
|
|
|
|
Next recommended pass:
|
|
- Add browser screenshot artifact automation for visual regression handoff.
|
|
|
|
## Sprint 59 workbench screenshot artifacts (2026-06-18)
|
|
|
|
Changed:
|
|
- Added `scripts/capture_workbench_screenshots.sh` for optional visual regression handoff screenshots.
|
|
- The script seeds the explicit offline demo workflow, opens Overview, Data, Map, QA/QC, AI Labs, Exports and System, then writes viewport screenshots plus `manifest.json`.
|
|
- Desktop capture is always enabled; mobile capture is enabled by default and can be disabled with `CAPTURE_MOBILE=0`.
|
|
- Added `/artifacts/` to `.gitignore` so screenshot outputs stay local.
|
|
- Added readiness syntax coverage and regression checks for the screenshot capture contract.
|
|
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- `bash -n scripts/capture_workbench_screenshots.sh`
|
|
- `cd backend && python -m pytest tests/test_readiness_gate.py -q` (`13 passed`)
|
|
- `bash scripts/run_readiness_check.sh` (`221 passed`)
|
|
- Local script execution without Playwright fails cleanly with setup instructions instead of producing partial artifacts.
|
|
- Browser-plugin visual capture against `http://192.168.10.150:1202` created viewport screenshots for all seven workspaces with no console errors or warnings.
|
|
|
|
Open:
|
|
- Commit, deploy Tower and keep the visual artifact script available for future Playwright-enabled runners.
|
|
|
|
Limitations:
|
|
- Playwright/Chromium is intentionally optional and not added to the frontend dependency set. The readiness gate checks script syntax only.
|
|
|
|
Next recommended pass:
|
|
- Run a backend error-envelope audit for expected user-error paths.
|
|
|
|
## Sprint 60 API error-envelope contract hardening (2026-06-18)
|
|
|
|
Changed:
|
|
- Audited the backend error payload shape against `docs/API_CONTRACTS.md`.
|
|
- Changed central FastAPI error serialization to return top-level `error`, `message`, `details` and `request_id` fields.
|
|
- Preserved HTTPException detail dict support for future explicit error codes.
|
|
- Updated the frontend API client to parse the canonical top-level error contract while remaining tolerant of the older nested `error.code/message/details` shape.
|
|
- Added regression tests for AppError, HTTPException and validation-error envelopes.
|
|
- Updated provider-registry error assertions to the canonical top-level schema.
|
|
- Added static frontend parser coverage for canonical and legacy error payloads.
|
|
- Updated `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- `cd backend && python -m pytest tests/test_error_envelope_contract.py tests/test_sprint7b_provider_registry.py::test_provider_api_envelopes_and_invalid_provider -q` (`4 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd backend && python -m pytest -q` (`224 passed`)
|
|
- `cd frontend && npm run build`
|
|
- `cd backend && python -m pytest tests/test_error_envelope_contract.py tests/test_frontend_api_client_error_parser.py tests/test_sprint7b_provider_registry.py::test_provider_api_envelopes_and_invalid_provider -q` (`6 passed`)
|
|
- `bash scripts/run_readiness_check.sh` (`226 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1`
|
|
- Tower live migration smoke passed.
|
|
- Tower browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- Live `GET /api/v1/external/providers/unknown` returned `{"error":"PROVIDER_NOT_FOUND","message":"Provider not found","details":{},"request_id":null}`.
|
|
- Live `GET /api/v1/projects/not-a-uuid` returned top-level `VALIDATION_ERROR`.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This pass changes only the centralized response envelope and frontend parser. It does not rename route-level error codes or change product behavior.
|
|
|
|
Next recommended pass:
|
|
- Expand golden datasets beyond the current building QA fixtures.
|
|
|
|
## Sprint 61 Golden QA scenario expansion (2026-06-18)
|
|
|
|
Changed:
|
|
- Added `fixtures/golden/golden_qa_benchmarks.json` as the explicit scenario manifest for deterministic QA/QC regression coverage.
|
|
- Added local golden fixture pairs for perfect-match, no-overlap and MultiPolygon building comparisons.
|
|
- Updated `scripts/run_golden_qa_benchmark.py` to execute all manifest scenarios, assert expected metric drift, persist simulated `QualityCheck`/`Metric` rows for each scenario and report aggregate persistence totals.
|
|
- Preserved backward-compatible top-level benchmark fields for scripts that still read the original single-scenario output shape.
|
|
- Updated `scripts/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `cd backend && python -m pytest tests/test_sprint12_golden_qa_benchmark.py -q` failed on missing multi-scenario manifest/output.
|
|
- `cd backend && python -m pytest tests/test_sprint12_golden_qa_benchmark.py -q` (`4 passed`)
|
|
- `python scripts/run_golden_qa_benchmark.py --json`
|
|
|
|
Open:
|
|
- Run full readiness and deploy Tower after the expanded golden benchmark passes the release gate.
|
|
|
|
Limitations:
|
|
- These are deterministic local QA/QC fixtures only. They do not introduce new production QA metrics or provider data.
|
|
|
|
Next recommended pass:
|
|
- Continue with frontend visual polish backlog or add a live golden benchmark smoke only if a running PostGIS environment needs that extra release signal.
|
|
|
|
## Sprint 62 Workbench visual polish (2026-06-18)
|
|
|
|
Changed:
|
|
- Added a compact workspace command bar under the active workspace heading for quick movement between the main workbench surfaces.
|
|
- Polished the shared frontend visual system with raised/sunken surface tokens, softer shadows, tighter topbar spacing and consistent workspace panel styling.
|
|
- Replaced raw project/dataset empty-state text with structured empty-state blocks.
|
|
- Wrapped Detection Lab and Segmentation Lab result summaries in scan-friendly result cards and long result tables in scroll-safe containers.
|
|
- Improved mobile navigation density by making the primary nav and command chips horizontal rails on narrow screens.
|
|
- Added `backend/tests/test_sprint62_frontend_visual_polish.py` to guard the visual polish contracts.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `cd backend && python -m pytest tests/test_sprint62_frontend_visual_polish.py -q` failed on missing command bar, panel polish and empty/result wrappers.
|
|
- `cd backend && python -m pytest tests/test_sprint62_frontend_visual_polish.py tests/test_sprint49_workbench_shell_refactor.py tests/test_sprint50_workspace_usability_polish.py -q` (`6 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local Chrome/Playwright visual check against `http://127.0.0.1:5175` with live API proxy: desktop and mobile screenshots captured, no console warnings/errors, no horizontal page overflow.
|
|
|
|
Open:
|
|
- Run full readiness and deploy Tower after this visual polish pass.
|
|
|
|
Limitations:
|
|
- This is a frontend usability polish pass only. It does not add new API behavior, data processing, provider fetching or AI model functionality.
|
|
|
|
Next recommended pass:
|
|
- Continue with map/result overlay ergonomics, especially making selected dataset/result provenance easier to see beside the map.
|
|
|
|
## Sprint 63 Map overlay ergonomics (2026-06-18)
|
|
|
|
Changed:
|
|
- Added active map layer source/provenance/draw-state context in the Map workspace using existing selected dataset, detection, segmentation and change-detection frontend state.
|
|
- Added a clear map empty-state when no vector/result layer is active.
|
|
- Added selected-feature property chips before the raw JSON feature inspector so common properties are scan-friendly.
|
|
- Tightened panel-title alignment with a scoped CSS override after the broader visual polish exposed a specificity issue in the existing `section > div:not(...)` rule.
|
|
- Added `backend/tests/test_sprint63_map_overlay_ergonomics.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `cd backend && python -m pytest tests/test_sprint63_map_overlay_ergonomics.py -q` failed on missing map provenance/feature-summary UI.
|
|
- `cd backend && python -m pytest tests/test_sprint63_map_overlay_ergonomics.py tests/test_sprint30_workbench_components.py tests/test_sprint53_selection_ergonomics.py -q` (`11 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local Chrome/Playwright check against `http://127.0.0.1:5175` with live API proxy: Map workspace opened, provenance rail present, no console warnings/errors, no horizontal page overflow.
|
|
|
|
Open:
|
|
- Run full readiness and deploy Tower after this pass.
|
|
|
|
Limitations:
|
|
- This is a frontend ergonomics pass only. It does not add new map layers, backend endpoints, provider fetching or geospatial processing.
|
|
|
|
Next recommended pass:
|
|
- Continue with export/report handoff polish or add a live browser smoke that explicitly visits every workspace after deployment.
|
|
|
|
## Sprint 64 Export/report handoff polish (2026-06-18)
|
|
|
|
Changed:
|
|
- Added a handoff readiness card to the Export Center using existing project, selected dataset, selected detection run, selected segmentation run and latest export state.
|
|
- Reworked existing export actions into artifact cards for refresh, project report, project metadata, selected vector GeoJSON, detection GeoJSON and segmentation GeoJSON.
|
|
- Added formatted export-type badges and extra export-card provenance for analysis-run ids and created timestamps when those fields are available.
|
|
- Added responsive styling for handoff summary/action cards.
|
|
- Added `backend/tests/test_sprint64_export_handoff_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint64_export_handoff_polish.py -q` failed on missing handoff summary/action structure and styles.
|
|
- `python -m pytest backend/tests/test_sprint64_export_handoff_polish.py backend/tests/test_sprint51_quality_export_polish.py -q` (`4 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local Chrome/Playwright check against `http://127.0.0.1:5175` with live API proxy: Exports workspace opened, handoff summary/actions present, no console warnings/errors, no horizontal page overflow on desktop or mobile.
|
|
- `bash scripts/run_readiness_check.sh` (`233 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the all-in-one container on Tower, with live migration smoke and browser runtime verification passing.
|
|
- Live Chrome/Playwright check against `http://192.168.10.150:1202`: Exports workspace opened, 6 handoff action cards present, no console warnings/errors, no horizontal page overflow on desktop or mobile.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is a frontend handoff usability pass only. It does not add export endpoints, change export persistence, introduce provider fetching or add AI/model behavior.
|
|
|
|
Next recommended pass:
|
|
- Run a live browser smoke across Exports and Overview after deployment, then continue with report artifact readability if the exported HTML itself needs visual polish.
|
|
|
|
## Sprint 65 Project report readability polish (2026-06-18)
|
|
|
|
Changed:
|
|
- Reworked the lightweight `project_report_html` renderer into a self-contained handoff layout with hero, readiness pill, scorecards and sectioned report content.
|
|
- Added print-friendly CSS and scroll-safe table wrappers to the HTML artifact.
|
|
- Added source and CRS columns to the dataset inventory section.
|
|
- Preserved existing export type, endpoint behavior, download behavior and storage flow.
|
|
- Added `backend/tests/test_sprint65_project_report_polish.py`.
|
|
- Updated `backend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint65_project_report_polish.py -q` failed on missing report shell, scorecards, print styles and readiness pill classes.
|
|
- `python -m pytest backend/tests/test_sprint65_project_report_polish.py backend/tests/test_sprint17_export_foundation.py -q` (`12 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`235 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live report smoke through `http://192.168.10.150:1202`: seeded explicit demo workflow, created `project_report_html`, downloaded the HTML artifact and verified it in Chrome with 5 scorecards, 5 report sections, no console warnings/errors and no horizontal overflow on desktop or mobile.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This remains a lightweight HTML handoff artifact. It does not add PDF generation, a report designer, new endpoints, live provider fetching or AI/model behavior.
|
|
|
|
Next recommended pass:
|
|
- Continue with a full-workspace browser smoke and then refine any remaining dense panels found during populated-state review.
|
|
|
|
## Sprint 66 Live workspace smoke polish (2026-06-19)
|
|
|
|
Changed:
|
|
- Ran a live browser smoke against `http://192.168.10.150:1202` across Overview, Data, Map, QA/QC, AI Labs, Exports and System.
|
|
- Captured desktop and mobile screenshots under `artifacts/sprint66-live-workspace-smoke/`.
|
|
- Confirmed all primary workspaces load with no console warnings/errors and no page-level horizontal overflow.
|
|
- Tightened Export Center handoff card CSS so action cards and readiness cells wrap by available width instead of forcing cramped three/four-column layouts in the populated Exports workspace.
|
|
- Added `backend/tests/test_sprint66_live_workspace_smoke_polish.py`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint66_live_workspace_smoke_polish.py -q` failed on missing width-aware export grid CSS and Sprint 66 log entry.
|
|
- Live desktop browser smoke: all seven primary workspaces opened, screenshots captured, no console warnings/errors, no horizontal page overflow.
|
|
- Live mobile browser smoke: all seven primary workspaces opened, screenshots captured, no console warnings/errors, no page-level horizontal overflow. The mobile primary nav intentionally remains a horizontal rail.
|
|
- Local post-fix browser check against `http://127.0.0.1:5175` with live API proxy: Exports handoff action cards wrap to 2 desktop columns and 1 mobile column, no console warnings/errors and no horizontal page overflow.
|
|
- `bash scripts/run_readiness_check.sh` (`237 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy Exports check on `http://192.168.10.150:1202`: handoff action cards render as 2 usable columns at desktop width, no console warnings/errors and no horizontal overflow.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is a visual ergonomics hardening pass only. It does not add endpoints, migrations, provider fetching, AI/model behavior or new product capabilities.
|
|
|
|
Next recommended pass:
|
|
- Continue with populated Map/Data interaction polish, especially making it easier to activate the demo vector layer from the map empty state.
|
|
|
|
## Sprint 67 Map empty-state quick actions (2026-06-19)
|
|
|
|
Changed:
|
|
- Added ready vector/GeoJSON dataset quick actions to the Map workspace empty state.
|
|
- Reused the existing `openDatasetInMap` frontend flow so quick actions load the same persisted dataset layer as the Data workspace button.
|
|
- Added width-aware `.map-empty-action-grid` styling for desktop and mobile.
|
|
- Added `backend/tests/test_sprint67_map_empty_state_quick_actions.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint67_map_empty_state_quick_actions.py -q` failed on missing Map quick-action props, App wiring and CSS.
|
|
- `python -m pytest backend/tests/test_sprint67_map_empty_state_quick_actions.py backend/tests/test_sprint63_map_overlay_ergonomics.py backend/tests/test_sprint30_workbench_components.py -q` (`9 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local Chrome/Playwright check against `http://127.0.0.1:5175` with live API proxy: Map empty state showed 2 ready dataset actions, clicking the first loaded `demo_predicted_buildings.geojson` as a 2-feature active layer, no console warnings/errors and no horizontal overflow.
|
|
- `bash scripts/run_readiness_check.sh` (`240 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy Map quick-action check on `http://192.168.10.150:1202`: Map empty state showed 2 ready dataset actions, clicking the first loaded `demo_predicted_buildings.geojson` as a 2-feature active layer, no console warnings/errors and no horizontal overflow.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is a frontend interaction polish pass only. It does not add map layers, backend endpoints, migrations, provider fetching or geospatial processing.
|
|
|
|
Next recommended pass:
|
|
- Continue with Data catalog density polish, especially making selected/reference/candidate dataset roles easier to scan in populated demo projects.
|
|
|
|
## Sprint 68 Data catalog density polish (2026-06-19)
|
|
|
|
Changed:
|
|
- Added a compact role summary to the Data catalog for Selected, Reference, Candidate and Source datasets.
|
|
- Added scan-friendly dataset card badges plus source, reference layer and CRS context.
|
|
- Treated non-reference vector/GeoJSON datasets as QA Candidate display roles in the frontend only, preserving persisted `dataset_role` values and API contracts.
|
|
- Added responsive role-summary and role-badge CSS.
|
|
- Added `backend/tests/test_sprint68_dataset_catalog_density.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint68_dataset_catalog_density.py -q` failed on missing role summary, role badges and responsive CSS.
|
|
- `python -m pytest backend/tests/test_sprint68_dataset_catalog_density.py backend/tests/test_sprint67_map_empty_state_quick_actions.py backend/tests/test_sprint30_workbench_components.py -q` (`10 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5175` with live API proxy: Data workspace showed Selected, Reference, Candidate and Source summary cards, with 1 Reference, 1 Candidate and 0 Source in the demo catalog; dataset badges rendered correctly on desktop and mobile, no console warnings/errors and no horizontal overflow.
|
|
- Screenshots captured under `artifacts/sprint68-dataset-catalog-density/`.
|
|
- `bash scripts/run_readiness_check.sh` (`243 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy Data catalog check on `http://192.168.10.150:1202`: Data workspace showed Selected, Reference, Candidate and Source summary cards, candidate/reference badges rendered, no console warnings/errors and no horizontal overflow on desktop or mobile.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is a frontend density/readability pass only. It does not change dataset persistence, backend role validation, API contracts, migrations, provider fetching or AI/model behavior.
|
|
|
|
Next recommended pass:
|
|
- Continue with Data workspace action polish, especially making QA/export affordances clearer once candidate and reference layers are present.
|
|
|
|
## Sprint 69 Data catalog action polish (2026-06-19)
|
|
|
|
Changed:
|
|
- Added role-aware recommended action hints to dataset cards.
|
|
- Reworked dataset card actions into compact two-line buttons for Inspect, Map, Export / QA and Metadata.
|
|
- Kept all existing Data workspace handlers and API calls unchanged.
|
|
- Added disabled-action explanation copy for unsupported export/QA and raster metadata refresh cases.
|
|
- Added responsive `.dataset-action-grid` and `.dataset-action-button` CSS.
|
|
- Added `backend/tests/test_sprint69_dataset_action_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint69_dataset_action_polish.py -q` failed on missing action hints, action grid and responsive styles.
|
|
- `python -m pytest backend/tests/test_sprint69_dataset_action_polish.py backend/tests/test_sprint68_dataset_catalog_density.py backend/tests/test_sprint29_dataset_components.py -q` (`9 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5175` with live API proxy: Data workspace rendered 2 role-aware hints and 8 compact dataset actions across the demo candidate/reference datasets, no console warnings/errors and no horizontal overflow on desktop or mobile.
|
|
- Screenshots captured under `artifacts/sprint69-dataset-action-polish/`.
|
|
- `bash scripts/run_readiness_check.sh` (`246 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy Data action check on `http://192.168.10.150:1202`: Data workspace rendered 2 role-aware hints and 8 compact dataset actions across the demo candidate/reference datasets, no console warnings/errors and no horizontal overflow on desktop or mobile.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is a frontend affordance/readability pass only. It does not add new workflows, change dataset persistence, change API contracts, add migrations, fetch providers or alter AI/model behavior.
|
|
|
|
Next recommended pass:
|
|
- Continue with QA/QC workspace handoff polish, especially linking the existing candidate/reference dataset context more clearly to persisted QA results.
|
|
|
|
## Sprint 70 QA/QC handoff polish (2026-06-19)
|
|
|
|
Changed:
|
|
- Added candidate/reference handoff cards to the QA/QC results panel.
|
|
- Resolved persisted quality-check candidate/reference dataset IDs to loaded dataset names when available.
|
|
- Passed `candidateDatasets` and `referenceDatasets` from `App.tsx` into `QualityResultsPanel`.
|
|
- Tightened QA candidate context to non-reference vector/GeoJSON datasets while preserving persisted `dataset_role` values.
|
|
- Added responsive `.quality-handoff-grid`, `.quality-dataset-name` and `.quality-check-dataset-link` CSS.
|
|
- Added `backend/tests/test_sprint70_quality_handoff_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint70_quality_handoff_polish.py -q` failed on missing dataset context props, handoff markup, App wiring and styles.
|
|
- `python -m pytest backend/tests/test_sprint70_quality_handoff_polish.py backend/tests/test_sprint39_frontend_orchestration_hooks.py backend/tests/test_sprint51_quality_export_polish.py -q` (`15 passed`)
|
|
- `python -m pytest backend/tests/test_sprint70_quality_handoff_polish.py backend/tests/test_sprint51_quality_export_polish.py backend/tests/test_sprint30_workbench_components.py backend/tests/test_sprint27_frontend_workflow_hooks.py -q` (`14 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5175` with live API proxy: QA/QC workspace showed 1 candidate layer, 1 reference layer, latest comparison names and named dataset links inside persisted quality-check cards, with no console warnings/errors and no horizontal overflow.
|
|
- Screenshots captured under `artifacts/sprint70-quality-handoff-polish/`.
|
|
- `bash scripts/run_readiness_check.sh` (`250 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy QA/QC handoff check on `http://192.168.10.150:1202`: QA/QC workspace showed 1 candidate layer, 1 reference layer, latest comparison names and named dataset links inside persisted quality-check cards, with no console warnings/errors and no horizontal overflow on desktop or mobile.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- This is a frontend handoff/readability pass only. It does not change QA persistence, metric calculations, API contracts, migrations, provider fetching or AI/model behavior.
|
|
|
|
Next recommended pass:
|
|
- Continue with QA/QC result card polish, especially making metric groups easier to scan in long-lived demo projects.
|
|
|
|
## Sprint 71 QA/QC metric card polish (2026-06-19)
|
|
|
|
Changed:
|
|
- Added core metric evidence cards to QA/QC result cards for precision, recall, F1, mean IoU, false positives and false negatives.
|
|
- Kept the raw persisted metric list visible below the promoted evidence cards.
|
|
- Added compact metric label/value formatting in `QualityResultsPanel`.
|
|
- Added responsive `.quality-metric-grid`, `.quality-metric-card` and `.quality-metric-card-critical` CSS.
|
|
- Added `backend/tests/test_sprint71_quality_metric_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint71_quality_metric_polish.py -q` failed on missing metric promotion helpers, markup and styles.
|
|
- `python -m pytest backend/tests/test_sprint71_quality_metric_polish.py backend/tests/test_sprint70_quality_handoff_polish.py backend/tests/test_sprint51_quality_export_polish.py backend/tests/test_sprint30_workbench_components.py -q` (`13 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5175` with live API proxy: QA/QC workspace rendered 12 promoted metric cards across 2 persisted quality checks, retained 2 raw metric sections, no console warnings/errors and no horizontal overflow on desktop or mobile.
|
|
- Screenshots captured under `artifacts/sprint71-quality-metric-polish/`.
|
|
- `bash scripts/run_readiness_check.sh` (`253 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy QA/QC metric check on `http://192.168.10.150:1202`: QA/QC workspace rendered 12 promoted metric cards across 2 persisted quality checks, retained 2 raw metric sections, no console warnings/errors and no body-level horizontal overflow on desktop or mobile.
|
|
- Live mobile screenshot captured at `artifacts/sprint71-quality-metric-polish/live-qa-metrics-mobile.png`.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend readability pass only; no QA calculation, persistence, API, migrations, provider or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with QA/QC metric/result filtering or result-card density for long-lived demo projects.
|
|
|
|
## Sprint 72 Mobile overflow hardening (2026-06-20)
|
|
|
|
Changed:
|
|
- Clamped page-level horizontal overflow for `html`, `body` and the workbench shell.
|
|
- Kept mobile sidebar navigation and workspace shortcut chips as contained horizontal scroll regions with overscroll containment.
|
|
- Added min-width/max-width containment for the workbench layout, main area, inspector panel and card surfaces.
|
|
- Allowed long QA check IDs, dataset links and inspector values to wrap instead of widening result cards.
|
|
- Switched inspector tabs to a two-column layout on narrow screens.
|
|
- Added `backend/tests/test_sprint72_mobile_overflow_hardening.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint72_mobile_overflow_hardening.py -q` failed on missing mobile overflow and identifier wrapping contracts.
|
|
- `python -m pytest backend/tests/test_sprint72_mobile_overflow_hardening.py backend/tests/test_sprint71_quality_metric_polish.py backend/tests/test_sprint62_frontend_visual_polish.py backend/tests/test_sprint30_workbench_components.py -q` (`11 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5176` with live API proxy at a 390px viewport: QA/QC workspace rendered 2 quality cards and 12 metric cards; body, shell, inspector, inspector panel and inspector tabs stayed within viewport width; sidebar/workspace nav retained contained horizontal scrolling; no console warnings/errors.
|
|
- Screenshot captured under `artifacts/sprint72-mobile-overflow-hardening/`.
|
|
- `bash scripts/run_readiness_check.sh` (`255 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy mobile check on `http://192.168.10.150:1202` at a 390px viewport: QA/QC workspace rendered 2 quality cards and 12 metric cards; body, shell, inspector, inspector panel and inspector tabs stayed within viewport width; sidebar/workspace nav retained contained horizontal scrolling; no console warnings/errors.
|
|
- Live screenshot captured at `artifacts/sprint72-mobile-overflow-hardening/live-mobile-qa.png`.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend CSS hardening only; no UI flow, API, persistence, migration, provider or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with QA/QC result filtering/density for long-lived demo projects, or a broader mobile visual pass across Data and Map once the overflow baseline is stable.
|
|
|
|
## Sprint 73 QA/QC result filtering (2026-06-20)
|
|
|
|
Changed:
|
|
- Added client-side QA/QC result search across check id, type, status, candidate/reference dataset ids, analysis/job ids and resolved dataset names.
|
|
- Added status and check-type filters derived from the loaded quality-check list.
|
|
- Added latest-eight result density control with a show-all toggle.
|
|
- Added a no-match empty state and reset action for filtered result views.
|
|
- Reused the dense history control styling pattern while allowing the QA/QC controls to wrap inside narrower workspace columns.
|
|
- Added `backend/tests/test_sprint73_quality_result_filtering.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint73_quality_result_filtering.py -q` failed on missing filter state, filtered list logic and styles.
|
|
- `python -m pytest backend/tests/test_sprint73_quality_result_filtering.py backend/tests/test_sprint72_mobile_overflow_hardening.py backend/tests/test_sprint71_quality_metric_polish.py backend/tests/test_sprint70_quality_handoff_polish.py backend/tests/test_sprint51_quality_export_polish.py -q` (`13 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5177` with live API proxy: QA/QC filters rendered against 2 persisted demo checks; search/status conflict showed the no-match state; reset restored 2 checks and 12 metric cards; no console warnings/errors and no body-level overflow.
|
|
- Local mobile browser check at a 390px viewport: filter controls stayed within viewport width, with 2 checks and 12 metric cards visible.
|
|
- Screenshots captured under `artifacts/sprint73-quality-result-filtering/`.
|
|
- `bash scripts/run_readiness_check.sh` (`257 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy QA/QC filtering check on `http://192.168.10.150:1202`: search/status conflict showed the no-match state; reset restored 2 checks and 12 metric cards; no console warnings/errors and no body-level overflow.
|
|
- Live mobile check at a 390px viewport: QA/QC filter controls stayed within viewport width, with 2 checks and 12 metric cards visible.
|
|
- Live screenshots captured at `artifacts/sprint73-quality-result-filtering/live-qa-filtering.png` and `artifacts/sprint73-quality-result-filtering/live-qa-filtering-mobile.png`.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend filtering/density pass only; filtering is client-side over already-loaded persisted checks and does not change API pagination, persistence, QA calculations, migrations, provider behavior or AI/model behavior.
|
|
|
|
Next recommended pass:
|
|
- Continue with Data/Map mobile visual polish, especially dataset upload/action forms and map toolbar density on narrow screens.
|
|
|
|
## Sprint 74 Data/Map mobile visual polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added mobile-density CSS for Data workspace upload forms, file input sizing and dataset action grids.
|
|
- Kept the desktop dataset action grid width contract intact while adding compact mobile tracks.
|
|
- Added touch/width containment for map toolbar controls, range sliders and empty-map quick actions.
|
|
- Reduced mobile map container height to keep controls and map visible together on narrow screens.
|
|
- Added `backend/tests/test_sprint74_data_map_mobile_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint74_data_map_mobile_polish.py -q` failed on missing mobile Data/Map CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint74_data_map_mobile_polish.py backend/tests/test_sprint72_mobile_overflow_hardening.py backend/tests/test_sprint69_dataset_action_polish.py backend/tests/test_sprint68_dataset_catalog_density.py backend/tests/test_sprint67_map_empty_state_quick_actions.py backend/tests/test_sprint63_map_overlay_ergonomics.py backend/tests/test_sprint50_workspace_usability_polish.py backend/tests/test_sprint30_workbench_components.py -q` (`21 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5178` with live API proxy at a 390px viewport: Data upload form, file input and 8 dataset action buttons stayed within the viewport with no body-level horizontal overflow; Map toolbar, 2 layer controls, range sliders and empty-map quick actions also stayed within the viewport.
|
|
- Screenshots captured under `artifacts/sprint74-data-map-mobile-polish/`.
|
|
- `bash scripts/run_readiness_check.sh` (`259 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy browser check on `http://192.168.10.150:1202` at a 390px viewport: Data upload form, file input and 8 dataset action buttons stayed within the viewport; Map toolbar, 2 layer controls, range sliders and empty-map quick actions stayed within the viewport; no body-level horizontal overflow was detected.
|
|
- Live screenshots captured at `artifacts/sprint74-data-map-mobile-polish/live-data-mobile.png` and `artifacts/sprint74-data-map-mobile-polish/live-map-mobile.png`.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend CSS polish only; no Data/Map workflow, API, persistence, migration, provider or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with AI Labs mobile/density polish, especially detection and segmentation forms/tables.
|
|
|
|
## Sprint 75 AI Labs mobile visual polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added mobile-density CSS for Detection and Segmentation Lab model cards, run forms and result summaries.
|
|
- Added overflow wrapping for long model ids, source tile paths, mask paths and QA summary values.
|
|
- Kept AI result tables scroll-contained while reducing their mobile minimum width.
|
|
- Added `backend/tests/test_sprint75_ai_labs_mobile_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint75_ai_labs_mobile_polish.py -q` failed on missing AI Labs mobile CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint75_ai_labs_mobile_polish.py backend/tests/test_sprint74_data_map_mobile_polish.py backend/tests/test_sprint72_mobile_overflow_hardening.py backend/tests/test_sprint62_frontend_visual_polish.py backend/tests/test_sprint30_workbench_components.py backend/tests/test_sprint8c_detection_visualization_qa.py backend/tests/test_sprint9_segmentation_foundation.py -q` (`30 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser check against `http://127.0.0.1:5179` with live API proxy at a 390px viewport: AI Labs rendered 2 panels, 7 model cards, 6 lab blocks and 4 lab forms with no body-level horizontal overflow.
|
|
- Screenshot captured under `artifacts/sprint75-ai-labs-mobile-polish/`.
|
|
- `bash scripts/run_readiness_check.sh` (`261 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, with live migration smoke and browser runtime verification passing.
|
|
- Live post-deploy browser check on `http://192.168.10.150:1202` at a 390px viewport: AI Labs rendered 2 panels, 7 model cards, 6 lab blocks and 4 lab forms with no body-level horizontal overflow.
|
|
- Live screenshot captured at `artifacts/sprint75-ai-labs-mobile-polish/live-ai-labs-mobile.png`.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend CSS polish only; no Detection/Segmentation workflow, API, persistence, migration, provider or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with Export/System mobile-density polish, especially export action cards and provider capability lists.
|
|
|
|
## Sprint 76 Export/System mobile visual polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added scan-friendly Provider Capabilities cards with structured provider header, status badge, authority/configuration/geometry/query mode fields and layer chips.
|
|
- Added mobile-density CSS for provider cards, export action cards, export history controls and export card headers.
|
|
- Added overflow wrapping for long provider limitations, attribution/license notes, export ids and artifact paths.
|
|
- Added `backend/tests/test_sprint76_export_system_mobile_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint76_export_system_mobile_polish.py -q` failed on missing Export/System mobile CSS and provider markup contracts.
|
|
- `python -m pytest backend/tests/test_sprint76_export_system_mobile_polish.py backend/tests/test_sprint75_ai_labs_mobile_polish.py backend/tests/test_sprint74_data_map_mobile_polish.py backend/tests/test_sprint31_unraid_template.py backend/tests/test_sprint64_export_handoff_polish.py backend/tests/test_sprint7b_provider_registry.py backend/tests/test_sprint30_workbench_components.py -q` (`27 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`263 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, with live migration smoke and browser runtime verification passing.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend presentation polish only; no export workflow, provider API, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with right-side inspector mobile polish, especially long dataset/vector/raster metadata and action groups.
|
|
|
|
## Sprint 77 Inspector mobile visual polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added compact inspector action button grids and mobile-safe wrapping for long inspector values.
|
|
- Added scroll/wrap containment for dataset detail text, job JSON and loaded feature metadata.
|
|
- Added `dataset-tool-panel` and `dataset-tool-group` classes to raster/vector operation controls.
|
|
- Added CSS containment for raster/vector tool labels, inputs, selects and buttons inside the inspector.
|
|
- Added `backend/tests/test_sprint77_inspector_mobile_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint77_inspector_mobile_polish.py -q` failed on missing inspector mobile CSS and dataset tool markup contracts.
|
|
- `python -m pytest backend/tests/test_sprint77_inspector_mobile_polish.py backend/tests/test_sprint76_export_system_mobile_polish.py backend/tests/test_sprint75_ai_labs_mobile_polish.py backend/tests/test_sprint72_mobile_overflow_hardening.py backend/tests/test_sprint52_workbench_inspector_tabs.py backend/tests/test_sprint30_workbench_components.py backend/tests/test_sprint29_dataset_components.py -q` (`18 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`265 passed`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, with live migration smoke and browser runtime verification passing.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend presentation polish only; no dataset workflow, raster/vector operation behavior, API, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with report/export preview readability polish, especially large JSON/HTML handoff artifacts.
|
|
|
|
## Sprint 78 Export preview readability polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added preview summary cards for JSON/GeoJSON export payload root type, root key count and rendered preview size.
|
|
- Added a scroll-contained export preview JSON shell and toolbar around the existing stored payload preview.
|
|
- Added wrapping for long JSON keys/values inside the preview body.
|
|
- Added `backend/tests/test_sprint78_export_preview_readability.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint78_export_preview_readability.py -q` failed on missing preview summary/shell CSS and markup contracts.
|
|
- `python -m pytest backend/tests/test_sprint78_export_preview_readability.py backend/tests/test_sprint77_inspector_mobile_polish.py backend/tests/test_sprint76_export_system_mobile_polish.py backend/tests/test_sprint51_quality_export_polish.py backend/tests/test_sprint64_export_handoff_polish.py backend/tests/test_sprint17_export_foundation.py -q` (`20 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`267 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend presentation polish only; no export content, export API, artifact storage, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with accessibility/keyboard focus polish across primary workbench controls.
|
|
|
|
## Sprint 79 Accessibility focus polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added a shared visible keyboard focus contract for primary buttons, workspace navigation, command chips, inspector tabs and dataset action buttons.
|
|
- Added explicit ARIA labels for workspace sidebar navigation, workspace command chips and overview quick actions.
|
|
- Bound inspector tabs to active tab panels with `aria-controls`, tab ids, `tabpanel` role and `aria-labelledby`.
|
|
- Added `backend/tests/test_sprint79_accessibility_focus_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint79_accessibility_focus_polish.py -q` failed on missing focus-visible CSS, navigation labels and inspector tab/panel bindings.
|
|
- `python -m pytest backend/tests/test_sprint79_accessibility_focus_polish.py backend/tests/test_sprint77_inspector_mobile_polish.py backend/tests/test_sprint53_selection_ergonomics.py backend/tests/test_sprint49_workbench_shell_refactor.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`15 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`270 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend accessibility/presentation polish only; no workflow behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with form-level validation/readability polish for dense raster/vector operation panels.
|
|
|
|
## Sprint 80 Operation form readability polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added structured headings, helper text, field wrappers, responsive field grids and action rows to raster operation controls.
|
|
- Added equivalent readability structure to vector clip, buffer and intersect controls.
|
|
- Added inline dataset tool error styling for raster clip/tile validation messages.
|
|
- Added `backend/tests/test_sprint80_operation_form_readability.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint80_operation_form_readability.py -q` failed on missing form readability CSS and markup contracts.
|
|
- `python -m pytest backend/tests/test_sprint80_operation_form_readability.py -q` (`3 passed`)
|
|
- `python -m pytest backend/tests/test_sprint80_operation_form_readability.py backend/tests/test_sprint77_inspector_mobile_polish.py backend/tests/test_sprint29_dataset_components.py backend/tests/test_sprint28_dataset_workflow_hook.py backend/tests/test_vector_operations_service.py backend/tests/test_raster_operations_service.py -q` (`37 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`273 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend readability/presentation polish only; no raster/vector operation behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with compact empty/error-state polish across QA, exports and AI lab result panels.
|
|
|
|
## Sprint 81 Result state consistency polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added shared `result-state` CSS variants for loading, error, empty and ready states.
|
|
- Applied result-state blocks to QA/QC error/empty/filter-empty states.
|
|
- Applied result-state blocks to export loading, error and empty/filter-empty states.
|
|
- Applied result-state blocks to Detection and Segmentation model loading/errors, empty registries, result counts and QA/run errors.
|
|
- Added `backend/tests/test_sprint81_result_state_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint81_result_state_polish.py -q` failed on missing result-state CSS and panel markup contracts.
|
|
- `python -m pytest backend/tests/test_sprint81_result_state_polish.py -q` (`3 passed`)
|
|
- `python -m pytest backend/tests/test_sprint81_result_state_polish.py backend/tests/test_sprint80_operation_form_readability.py backend/tests/test_sprint75_ai_labs_mobile_polish.py backend/tests/test_sprint76_export_system_mobile_polish.py backend/tests/test_sprint73_quality_result_filtering.py backend/tests/test_sprint70_quality_handoff_polish.py backend/tests/test_sprint8c_detection_visualization_qa.py backend/tests/test_sprint9_segmentation_foundation.py -q` (`34 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`276 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend presentation/state polish only; no workflow behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with visual density review for topbar/sidebar/responsive shell after another live browser pass.
|
|
|
|
## Sprint 82 Shell density polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added a keyboard skip link that targets the active workspace main region.
|
|
- Added an explicit `Primary workspaces` label to the sidebar navigation and a focusable `workspace-main` target.
|
|
- Tightened narrow-screen topbar, context chip, sidebar nav and workspace shortcut density while keeping intentional horizontal rails scroll-safe.
|
|
- Added `backend/tests/test_sprint82_shell_density_polish.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live browser pre-check against `http://192.168.10.150:1202` showed no console errors, no horizontal document overflow and a too-tall narrow viewport topbar/context stack.
|
|
- Red step: `python -m pytest backend/tests/test_sprint82_shell_density_polish.py -q` failed on missing skip-link, main focus target and compact mobile shell CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint82_shell_density_polish.py -q` (`3 passed`)
|
|
- `python -m pytest backend/tests/test_sprint82_shell_density_polish.py backend/tests/test_sprint79_accessibility_focus_polish.py backend/tests/test_sprint72_mobile_overflow_hardening.py backend/tests/test_sprint49_workbench_shell_refactor.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`13 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`279 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- Live post-deploy browser check found a narrow-screen cascade override that made `.context-bar` one-column again; tightened the regression test and kept the context rail scrollable through the smallest breakpoint.
|
|
- Re-ran `python -m pytest backend/tests/test_sprint82_shell_density_polish.py -q`, `cd frontend && npm run typecheck`, `cd frontend && npm run build` and `bash scripts/run_readiness_check.sh` (`279 passed`) after that fix.
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
- Final live browser check against `http://192.168.10.150:1202` passed on desktop and mobile viewports: no console warnings/errors, no horizontal document overflow, skip link/main target present, `Primary workspaces` nav label present and mobile context bar stayed compact at 64px tall.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend shell presentation/accessibility polish only; no workflow behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with a live visual review of dense workspace panel hierarchy after the shell density changes are deployed.
|
|
|
|
## Sprint 83 Workspace panel hierarchy polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Made the Overview readiness strip visually calmer with a compact section surface and tighter status tiles.
|
|
- Added explicit `overview-action-copy`, `overview-quick-actions` and `quick-action-button` regions to the recommended-action block.
|
|
- Restyled the recommended flow block as a lighter accent callout so it no longer competes visually with the readiness strip.
|
|
- Added `backend/tests/test_sprint83_workspace_panel_hierarchy.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live browser pre-check against `http://192.168.10.150:1202` showed no console errors or horizontal overflow, but the first workspace viewport still had several equally weighted white cards.
|
|
- Red step: `python -m pytest backend/tests/test_sprint83_workspace_panel_hierarchy.py -q` failed on missing Overview hierarchy regions and compact section-surface CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint83_workspace_panel_hierarchy.py -q` (`3 passed`)
|
|
- `python -m pytest backend/tests/test_sprint83_workspace_panel_hierarchy.py backend/tests/test_sprint82_shell_density_polish.py backend/tests/test_sprint62_frontend_visual_polish.py backend/tests/test_sprint50_workspace_usability_polish.py backend/tests/test_sprint22_workbench_status_strip.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`15 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`282 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
- Final live browser check against `http://192.168.10.150:1202` passed on desktop and mobile viewports: no console warnings/errors, no horizontal document overflow, Overview hierarchy regions present and four recommended-action buttons rendered.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend Overview presentation hierarchy only; no workflow behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with Data workspace panel hierarchy and selected-item density after this pass is deployed and visually checked.
|
|
|
|
## Sprint 84 Data workspace density polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added selected-summary regions to Project, AOI and Dataset panels so the active data context appears before creation/upload forms.
|
|
- Split Project and AOI panels into `data-panel-form-block` and `data-panel-list-block` regions.
|
|
- Split Dataset catalog into selected dataset, upload source data and available dataset regions.
|
|
- Restyled dataset upload as an embedded source-data block while preserving the existing upload flow.
|
|
- Added `backend/tests/test_sprint84_data_workspace_density.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live browser pre-check against `http://192.168.10.150:1202` showed Data workspace had no console errors or horizontal overflow, but the first Data viewport was still form-first and visually heavy.
|
|
- Red step: `python -m pytest backend/tests/test_sprint84_data_workspace_density.py -q` failed on missing selected-summary and named form/list/catalog regions.
|
|
- `python -m pytest backend/tests/test_sprint84_data_workspace_density.py -q` (`3 passed`)
|
|
- `python -m pytest backend/tests/test_sprint84_data_workspace_density.py backend/tests/test_sprint83_workspace_panel_hierarchy.py backend/tests/test_sprint50_workspace_usability_polish.py backend/tests/test_sprint29_dataset_components.py backend/tests/test_sprint68_dataset_catalog_density.py backend/tests/test_sprint69_dataset_action_polish.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`20 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`285 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
- Final live browser DOM/console check against `http://192.168.10.150:1202` passed on desktop and mobile viewports: Data workspace active, no console warnings/errors, no horizontal document overflow, three selected-summary regions, three form blocks and three list/catalog blocks rendered.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend Data workspace presentation hierarchy only; no upload behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with Map workspace panel hierarchy and layer-control density after this pass is deployed and visually checked.
|
|
|
|
## Sprint 85 Map workspace density polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added a compact Map workspace context summary for selected AOI, active layer and rendered feature state.
|
|
- Wrapped existing layer controls and provenance in a dedicated `map-control-surface`.
|
|
- Wrapped the existing MapLibre component in a `map-frame-surface` and the selected-feature inspector in a `map-inspection-surface`.
|
|
- Added `backend/tests/test_sprint85_map_workspace_density.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live browser pre-check against `http://192.168.10.150:1202` showed Map workspace had no console errors or horizontal overflow, but controls, provenance, map frame and inspector still read as a loose vertical stack.
|
|
- Red step: `python -m pytest backend/tests/test_sprint85_map_workspace_density.py -q` failed on missing Map workspace surface and context-summary contracts.
|
|
- `python -m pytest backend/tests/test_sprint85_map_workspace_density.py -q` (`3 passed`)
|
|
- `python -m pytest backend/tests/test_sprint85_map_workspace_density.py backend/tests/test_sprint63_map_overlay_ergonomics.py backend/tests/test_sprint67_map_empty_state_quick_actions.py backend/tests/test_sprint74_data_map_mobile_polish.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`13 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Live browser post-deploy check showed desktop Map workspace was structured and overflow-free, but the mobile breakpoint still stacked controls too tall above the map.
|
|
- Red step: `python -m pytest backend/tests/test_sprint85_map_workspace_density.py -q` then failed on missing compact mobile Map workspace breakpoint contracts.
|
|
- `python -m pytest backend/tests/test_sprint85_map_workspace_density.py -q` (`4 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`289 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
- Final live browser check against `http://192.168.10.150:1202` passed on desktop and mobile viewports: no console warnings/errors, no horizontal document overflow, three Map context summary cards, two layer control cards, separated control/map/inspector surfaces and mobile control preamble reduced after the compact breakpoint correction.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend Map workspace presentation hierarchy only; no overlay behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with QA/QC workspace result hierarchy and filter density after this pass is deployed and visually checked.
|
|
|
|
## Sprint 86 QA/QC workspace density polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Wrapped QA/QC summary metrics in `quality-summary-surface`.
|
|
- Wrapped candidate/reference/latest comparison context in `quality-evidence-surface`.
|
|
- Wrapped refresh, result states, filters and list limit controls in `quality-control-surface`.
|
|
- Wrapped persisted quality check cards in `quality-history-surface`.
|
|
- Added compact mobile CSS contracts for QA/QC summary, handoff, filter, score, metric and raw metric grids.
|
|
- Added `backend/tests/test_sprint86_quality_workspace_density.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live browser pre-check against `http://192.168.10.150:1202` showed QA/QC workspace had no console errors or horizontal overflow, but summary, handoff, filters and history were still visually stacked at equal weight. Mobile measured summary at 210px, handoff at 295px and filters at 278px before the result history.
|
|
- Red step: `python -m pytest backend/tests/test_sprint86_quality_workspace_density.py -q` failed on missing QA/QC surface and density CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint86_quality_workspace_density.py -q` (`3 passed`)
|
|
- `python -m pytest backend/tests/test_sprint86_quality_workspace_density.py backend/tests/test_sprint70_quality_handoff_polish.py backend/tests/test_sprint71_quality_metric_polish.py backend/tests/test_sprint73_quality_result_filtering.py backend/tests/test_sprint81_result_state_polish.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`18 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- First full readiness run caught a compatibility regression in `backend/tests/test_sprint51_quality_export_polish.py`: the root `quality-results-panel` class was no longer exact. Kept the old root class and moved the new shell into a wrapper.
|
|
- `python -m pytest backend/tests/test_sprint51_quality_export_polish.py backend/tests/test_sprint86_quality_workspace_density.py backend/tests/test_sprint70_quality_handoff_polish.py backend/tests/test_sprint73_quality_result_filtering.py -q` (`11 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`292 passed`; frontend typecheck/build passed; Alembic single head `202606120900`)
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/deploy_tower.ps1` rebuilt and redeployed the Tower all-in-one container on port 1202, then passed live migration smoke and browser runtime verification.
|
|
- Final live browser check against `http://192.168.10.150:1202` passed on desktop and mobile viewports: no console warnings/errors, no horizontal document overflow, QA/QC shell/surfaces present and two persisted quality check cards rendered.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend QA/QC workspace presentation hierarchy only; no quality-check behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with Change Detection panel hierarchy and analysis workspace balance after this pass is deployed and visually checked.
|
|
|
|
## Sprint 87 Change Detection density polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Wrapped Change Detection input controls in `change-detection-input-surface`.
|
|
- Replaced loose error/empty text with shared `result-state` cards inside `change-detection-state-stack`.
|
|
- Wrapped result metrics in `change-detection-result-surface`.
|
|
- Wrapped warning output in `change-detection-warning-surface`.
|
|
- Added compact responsive CSS contracts for Change Detection form and summary grids.
|
|
- Added `backend/tests/test_sprint87_change_detection_density.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live browser pre-check attempt against `http://192.168.10.150:1202` hit a transient browser automation click timeout on the QA/QC workspace nav; source review showed Change Detection was still the older header/form/summary stack.
|
|
- Red step: `python -m pytest backend/tests/test_sprint87_change_detection_density.py -q` failed on missing Change Detection surface and density CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint87_change_detection_density.py backend/tests/test_sprint18_change_detection.py backend/tests/test_sprint39_frontend_orchestration_hooks.py backend/tests/test_sprint86_quality_workspace_density.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`20 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`295 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy via `scripts/deploy_tower.ps1` rebuilt and restarted the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Tower deploy live migration smoke passed with PostGIS `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and required runtime schema objects present.
|
|
- Browser runtime verification passed for frontend, proxied API and icon.
|
|
- Direct Chrome live UI check passed on desktop `1440x1000` and mobile `390x844`: Change Detection shell/input surface rendered, no horizontal overflow and no console warnings/errors.
|
|
|
|
Open:
|
|
- None known after full readiness, Tower deploy and live browser validation.
|
|
|
|
Limitations:
|
|
- Frontend Change Detection presentation hierarchy only; no change-detection behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with AI Labs run-form hierarchy and detection/segmentation result density after this pass is deployed and visually checked.
|
|
|
|
## Sprint 88 AI Labs density polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Wrapped Detection Lab and Segmentation Lab in shared `ai-lab-shell` workspace shells.
|
|
- Grouped model registry states into `ai-lab-model-surface` with a shared `ai-lab-state-stack`.
|
|
- Grouped run controls into `ai-lab-run-surface`.
|
|
- Grouped persisted detection/segmentation result loading and tables into `ai-lab-results-surface`.
|
|
- Grouped QA controls and metric summaries into `ai-lab-qa-surface`.
|
|
- Added shared AI Lab CSS contracts for compact model grids, form grids and summary grids.
|
|
- Added `backend/tests/test_sprint88_ai_lab_density.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint88_ai_lab_density.py -q` failed on missing AI Lab shells/surfaces and CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint88_ai_lab_density.py backend/tests/test_sprint8c_detection_visualization_qa.py backend/tests/test_sprint9_segmentation_foundation.py backend/tests/test_sprint39_frontend_orchestration_hooks.py backend/tests/test_sprint47_workbench_interaction_smoke.py -q` (`34 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser UI check against `http://127.0.0.1:5174` passed on desktop and mobile: Detection/Segmentation shells and model/run/result/QA surfaces rendered, no horizontal overflow and no console warnings/errors. The local-only screenshot showed expected API 500 states because only the frontend Vite server was running.
|
|
- First full readiness run caught legacy AI Lab static contract expectations for exact `lab-block` and `result-summary-card` class names. Kept those compatibility anchors while preserving the new AI Lab surfaces.
|
|
- `python -m pytest backend/tests/test_sprint50_workspace_usability_polish.py backend/tests/test_sprint75_ai_labs_mobile_polish.py backend/tests/test_sprint88_ai_lab_density.py -q` (`8 passed`)
|
|
- `bash scripts/run_readiness_check.sh` (`299 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy via `scripts/deploy_tower.ps1` rebuilt and restarted the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Tower deploy live migration smoke passed with PostGIS `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and required runtime schema objects present.
|
|
- Browser runtime verification passed for frontend, proxied API and icon.
|
|
- Live browser AI Labs check against `http://192.168.10.150:1202` passed on desktop and mobile: Detection/Segmentation shells and model/run/result/QA surfaces rendered, no horizontal overflow and no console warnings/errors.
|
|
|
|
Open:
|
|
- None known after full readiness, Tower deploy and live browser validation.
|
|
|
|
Limitations:
|
|
- Frontend AI Labs presentation hierarchy only; no detection/segmentation behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with export/system final visual consistency or live workflow guidance once this pass is deployed and visually checked.
|
|
|
|
## Sprint 89 Export/System density polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Wrapped Export Center content in `export-center-shell`.
|
|
- Grouped export summary, handoff readiness, artifact actions, current state cards and history into focused `export-*` surfaces.
|
|
- Wrapped Provider Capabilities content in `system-provider-shell`.
|
|
- Replaced loose provider loading/error/empty text with shared `result-state` cards.
|
|
- Grouped provider registry content into `system-provider-capability-surface`.
|
|
- Split provider limitation/reason text and attribution/license data into `provider-detail-stack` and provenance cards.
|
|
- Added shared Export/System density CSS for compact summary, handoff, action, history and provider provenance grids.
|
|
- Added `backend/tests/test_sprint89_export_system_density.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint89_export_system_density.py -q` failed on missing Export/System shells, surfaces and CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint89_export_system_density.py backend/tests/test_sprint51_quality_export_polish.py backend/tests/test_sprint64_export_handoff_polish.py backend/tests/test_sprint76_export_system_mobile_polish.py backend/tests/test_sprint78_export_preview_readability.py -q` (`12 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- In-app browser fallback note: the Browser click path timed out on the local workspace navigation CDP command, so rendered validation used local Chrome via Playwright.
|
|
- Local Chrome UI check against `http://127.0.0.1:5174` passed on desktop and mobile: Export/System shells and surfaces rendered with no horizontal overflow. Local console showed expected Vite proxy 500s because only the frontend server was running.
|
|
- `bash scripts/run_readiness_check.sh` (`303 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy via `scripts/deploy_tower.ps1` rebuilt and restarted the all-in-one container on `http://192.168.10.150:1202`.
|
|
- Tower deploy live migration smoke passed with PostGIS `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and required runtime schema objects present.
|
|
- Browser runtime verification passed for frontend, proxied API and icon.
|
|
- Live Chrome UI check against `http://192.168.10.150:1202` passed on desktop and mobile: Export/System shells and surfaces rendered, four provider cards and eight provider provenance cards were visible, no horizontal overflow and no console warnings/errors.
|
|
|
|
Open:
|
|
- None known after full readiness, Tower deploy and live browser validation.
|
|
|
|
Limitations:
|
|
- Frontend Export/System presentation hierarchy only; no export behavior, API contract, persistence, migration, provider fetching or AI/model changes.
|
|
|
|
Next recommended pass:
|
|
- Continue with end-to-end workflow guidance after this pass is deployed and visually checked.
|
|
|
|
## Sprint 90 workflow guidance polish (2026-06-20)
|
|
|
|
Changed:
|
|
- Added an Overview `workflow-guidance-panel` that shows the V1 flow from Project & AOI through Data, Map, QA / AI and Export.
|
|
- Added ready/waiting/next state labels from existing loaded project, dataset, map, QA/AI and export state.
|
|
- Routed the guidance cards through the existing `setActiveWorkspace` navigation only.
|
|
- Added compact responsive CSS for the guidance rail.
|
|
- Added `backend/tests/test_sprint90_workflow_guidance.py`.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend/tests/test_sprint90_workflow_guidance.py -q` failed on missing workflow guidance App and CSS contracts.
|
|
- `python -m pytest backend/tests/test_sprint90_workflow_guidance.py backend/tests/test_sprint83_workspace_panel_hierarchy.py backend/tests/test_sprint49_workbench_shell_refactor.py backend/tests/test_sprint62_frontend_visual_polish.py backend/tests/test_sprint82_shell_density_polish.py -q` (`13 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`306 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Local browser UI check against `http://127.0.0.1:5174` passed on desktop and mobile: workflow guidance panel rendered with five steps and no horizontal overflow. Local console showed expected Vite proxy 500s because only the frontend server was running.
|
|
- Tower deploy via `scripts/deploy_tower.ps1` rebuilt and restarted the all-in-one container on `http://192.168.10.150:1202` from commit `2a0e7e7`.
|
|
- Tower deploy live migration smoke passed with PostGIS `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and required runtime schema objects present.
|
|
- Browser runtime verification passed for frontend, proxied API and icon.
|
|
- Live browser UI check against `http://192.168.10.150:1202` passed on desktop and mobile: workflow guidance panel rendered with five steps, no horizontal overflow and no console warnings/errors.
|
|
|
|
Open:
|
|
- None known after full readiness, Tower deploy and live browser validation.
|
|
|
|
Limitations:
|
|
- Frontend Overview workflow guidance only; no API contract, persistence, migration, provider fetching or AI/model behavior changes.
|
|
|
|
Next recommended pass:
|
|
- After deploy and visual validation, continue with any remaining end-to-end workflow handoff polish surfaced by the live audit.
|
|
|
|
## Sprint 91 populated workflow audit polish (2026-06-21)
|
|
|
|
Changed:
|
|
- Audited the live populated demo state on `http://192.168.10.150:1202` across Overview, Data, Map, QA/QC, AI Labs and Exports.
|
|
- Changed the Overview workflow guidance badge to show `Ready for handoff` when project, dataset, map, QA/AI and export state are all present.
|
|
- Clarified the Map workflow step so it reports layer feature count separately from AOI context and shows `AOI loaded` for AOI-only map context.
|
|
- Extended `backend/tests/test_sprint90_workflow_guidance.py` with complete-state and Map-copy regression coverage.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live pre-change browser audit passed for the populated demo workflow: Data had 2 ready datasets, Map rendered `demo_predicted_buildings.geojson` with 2 layer features, QA/QC showed 2 checks, AI Labs showed model/result/QA surfaces, Exports showed 50 artifacts, and no checked workspace had console warnings/errors or horizontal overflow.
|
|
- Red step: `python -m pytest backend/tests/test_sprint90_workflow_guidance.py -q` failed on missing `workflowGuidanceComplete` / `Ready for handoff` / precise Map copy contracts.
|
|
- Post-deploy live check found the AOI-only reload state rendered `0 layer features + AOI`; a second red step failed on missing `AOI loaded` copy before the fix.
|
|
- `python -m pytest backend/tests/test_sprint90_workflow_guidance.py -q` (`4 passed`)
|
|
- `python -m pytest backend/tests/test_sprint90_workflow_guidance.py backend/tests/test_sprint83_workspace_panel_hierarchy.py backend/tests/test_sprint82_shell_density_polish.py -q` (`10 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser UI check against `http://127.0.0.1:5174` passed for the Overview guidance panel with no horizontal overflow. Local console showed expected Vite proxy 500s because only the frontend server was running.
|
|
- `bash scripts/run_readiness_check.sh` (`307 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy via `scripts/deploy_tower.ps1` rebuilt and restarted the all-in-one container on `http://192.168.10.150:1202` from commit `97b943d`.
|
|
- Tower deploy live migration smoke passed with PostGIS `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1` and required runtime schema objects present.
|
|
- Browser runtime verification passed for frontend, proxied API and icon.
|
|
- Live post-change browser check passed: AOI-only Overview state showed `Map AOI loaded`; after opening `demo_predicted_buildings.geojson` on the map, Overview showed `Map 2 layer features + AOI`; both states showed `Ready for handoff`, no horizontal overflow and no console warnings/errors.
|
|
|
|
Open:
|
|
- None known after full readiness, Tower deploy and live browser validation.
|
|
|
|
Limitations:
|
|
- Frontend Overview copy/state polish only; no API contract, persistence, migration, provider fetching or AI/model behavior changes.
|
|
|
|
Next recommended pass:
|
|
- After deploy and live validation, continue with any remaining populated-state visual issues found in the next audit.
|
|
|
|
## Sprint 92 workflow rail interaction polish (2026-06-21)
|
|
|
|
Changed:
|
|
- Audited the live Overview workflow rail click path on `http://192.168.10.150:1202`.
|
|
- Added `openWorkflowGuidanceStep` so Map and Export rail clicks can reuse existing dataset context flows.
|
|
- Map rail click now opens the first ready vector/GeoJSON dataset through `openDatasetInMap` when no layer is active.
|
|
- Export rail click now opens the first ready vector/GeoJSON dataset through `openDatasetExport` when no dataset is selected.
|
|
- Extended `backend/tests/test_sprint90_workflow_guidance.py` with regression coverage for the context-aware rail handler.
|
|
- Updated `frontend/README.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Live pre-change browser audit showed the rail navigated correctly but Map landed with no active layer and Exports landed with `Selected dataset: none`.
|
|
- Red step: `python -m pytest backend/tests/test_sprint90_workflow_guidance.py -q` failed on missing `openWorkflowGuidanceStep` / dataset-context handler contracts.
|
|
- `python -m pytest backend/tests/test_sprint90_workflow_guidance.py backend/tests/test_sprint53_selection_ergonomics.py backend/tests/test_sprint67_map_empty_state_quick_actions.py backend/tests/test_sprint64_export_handoff_polish.py -q` (`15 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- Local browser UI check against `http://127.0.0.1:5174` passed for rail rendering with no horizontal overflow. Local console showed expected Vite proxy 500s because only the frontend server was running.
|
|
- `bash scripts/run_readiness_check.sh` (`308 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy from commit `1872c60` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live targeted browser validation against `http://192.168.10.150:1202` passed for `Open Map step`: `demo_predicted_buildings.geojson` became active, `2 features` rendered, no horizontal overflow and no console/page errors.
|
|
- Live targeted browser validation against `http://192.168.10.150:1202` passed for `Open Export step`: `demo_predicted_buildings.geojson` became the selected export dataset, no horizontal overflow and no console/page errors.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend workflow navigation polish only; no API contract, persistence, migration, provider fetching or AI/model behavior changes.
|
|
|
|
Next recommended pass:
|
|
- After deploy and live validation, continue with export artifact action ergonomics or QA result drill-down, depending on the next live friction point.
|
|
|
|
## Sprint 93 export handoff artifact polish (2026-06-21)
|
|
|
|
Changed:
|
|
- Added a `Latest handoff artifacts` section to `frontend/src/components/exports/ExportCenter.tsx`.
|
|
- Grouped newest persisted artifacts for project report, project metadata, dataset GeoJSON, detection GeoJSON and segmentation GeoJSON.
|
|
- Reused existing JSON preview and artifact download callbacks from latest artifact cards.
|
|
- Added responsive latest artifact card styling in `frontend/src/styles/app.css`.
|
|
- Added `backend/tests/test_sprint93_export_handoff_completion.py` to lock the grouped handoff surface, actions and responsive CSS.
|
|
- Updated `CHANGELOG.md`, `docs/TODO.md` and `frontend/README.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint93_export_handoff_completion.py -q` failed on missing latest artifact grouping/actions/styles.
|
|
- `python -m pytest backend\tests\test_sprint93_export_handoff_completion.py -q` (`3 passed`)
|
|
- `python -m pytest backend\tests\test_sprint93_export_handoff_completion.py backend\tests\test_sprint64_export_handoff_polish.py backend\tests\test_sprint89_export_system_density.py backend\tests\test_sprint51_quality_export_polish.py backend\tests\test_sprint78_export_preview_readability.py -q` (`13 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`311 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy from commit `0a9054c` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live browser validation against `http://192.168.10.150:1202` passed for `Open Export step`: selected dataset stayed `demo_predicted_buildings.geojson`, `Latest handoff artifacts` rendered, project report/project metadata/dataset GeoJSON cards were present, detection/segmentation cards showed `No artifact yet`, preview opened from the latest artifact card, no horizontal overflow and no console/page errors.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Export Center frontend handoff polish only; no API contract, persistence, migration, provider fetching or AI/model behavior changes.
|
|
|
|
Next recommended pass:
|
|
- After deploy and live validation, continue with QA/QC drilldown and map evidence layers.
|
|
|
|
## Sprint 94 QA/QC evidence drilldown (2026-06-22)
|
|
|
|
Changed:
|
|
- Added selected-check drilldown state to `frontend/src/components/quality/QualityResultsPanel.tsx`.
|
|
- Added a `QA/QC evidence drilldown` surface with selected check, candidate/reference layer, analysis run, job, status, score and timestamp provenance.
|
|
- Added false-positive evidence, false-negative evidence and map evidence handoff cards using persisted metric rows and dataset names.
|
|
- Added parameter and findings JSON panes for persisted QA/QC provenance.
|
|
- Added `Inspect latest check` and per-result `Inspect check` controls.
|
|
- Added responsive QA drilldown styles to `frontend/src/styles/app.css`.
|
|
- Added `backend/tests/test_sprint94_quality_drilldown.py`.
|
|
- Updated `CHANGELOG.md`, `docs/TODO.md` and `frontend/README.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint94_quality_drilldown.py -q` failed on missing drilldown state/surfaces/styles.
|
|
- `python -m pytest backend\tests\test_sprint94_quality_drilldown.py backend\tests\test_sprint70_quality_handoff_polish.py backend\tests\test_sprint71_quality_metric_polish.py backend\tests\test_sprint73_quality_result_filtering.py backend\tests\test_sprint86_quality_workspace_density.py -q` (`15 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`314 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- Tower deploy from commit `f3ad9c3` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live browser validation against `http://192.168.10.150:1202` passed for `Open QA / AI step`: QA/QC evidence drilldown rendered, candidate/reference names resolved, false-positive/negative evidence rendered, parameters/findings JSON rendered, per-result `Inspect check` controls were available, no horizontal overflow and no console/page errors.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- QA/QC frontend drilldown only; no API contract, persistence, migration, provider fetching or AI/model behavior changes.
|
|
- Map evidence handoff points reviewers to candidate/reference datasets already persisted in the workbench; it does not create a new matched/unmatched geometry export.
|
|
|
|
Next recommended pass:
|
|
- After deploy and live validation, continue with QA map overlay affordances or raster pipeline hardening.
|
|
|
|
## Sprint 95 raster pipeline hardening (2026-06-22)
|
|
|
|
Changed:
|
|
- Added a `Raster pipeline readiness` surface to `frontend/src/components/datasets/RasterControls.tsx`.
|
|
- Surfaced metadata profile, CRS readiness, preview artifact, tile manifest handoff and clip AOI state before raster operations.
|
|
- Added `Processing guardrails` for missing selected dataset, unavailable raster processing, missing metadata, missing CRS, missing preview, invalid tile parameters and missing clip areas.
|
|
- Added responsive raster readiness, guardrail and manifest-handoff styles to `frontend/src/styles/app.css`.
|
|
- Added `backend/tests/test_sprint95_raster_pipeline_hardening.py`.
|
|
- Updated `CHANGELOG.md`, `docs/TODO.md` and `frontend/README.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint95_raster_pipeline_hardening.py -q` failed on missing readiness/handoff component structure and CSS.
|
|
- `python -m pytest backend\tests\test_sprint95_raster_pipeline_hardening.py -q` (`2 passed`)
|
|
- `python -m pytest backend\tests\test_sprint80_operation_form_readability.py backend\tests\test_sprint77_inspector_mobile_polish.py -q` (`5 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -q` (`316 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`316 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- `cd backend && python -m alembic heads` (`202606120900 (head)`)
|
|
- `cd backend && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- Tower deploy from commit `330b677` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live browser validation against `http://192.168.10.150:1202` passed for the workbench shell: Overview/Data/Map/QA/QC/AI Labs/Exports markers were present, no horizontal overflow and no console/page errors.
|
|
- Live dataset API validation confirmed the default demo currently contains vector fixtures only, so the raster-specific panel is not visible in the default live state without a raster upload.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend raster inspector hardening only; no API contract, persistence, migration, provider fetching or AI/model behavior changes.
|
|
- The panel explains the existing tile manifest handoff but does not create a new manifest picker or new detection/segmentation behavior.
|
|
- Raster-specific live visual validation requires a raster dataset in the live project; the current offline demo seed contains vector fixtures only.
|
|
|
|
Next recommended pass:
|
|
- After deploy and live validation, add a raster fixture/demo upload smoke or continue with map evidence overlay affordances for QA/QC.
|
|
|
|
## Sprint 96 useful default context (2026-06-22)
|
|
|
|
Changed:
|
|
- Added a guarded default dataset effect in `frontend/src/hooks/useDatasetWorkflow.ts`.
|
|
- When project data loads and no dataset is selected, the workbench now auto-opens the first ready vector dataset, falling back to any ready dataset and then the first dataset.
|
|
- This gives Data, Map and Exports an immediately useful selected dataset/map layer context for the populated demo flow.
|
|
- Added explicit no-raster guidance cards to Detection Lab and Segmentation Lab run controls.
|
|
- Added `backend/tests/test_sprint96_useful_default_context.py`.
|
|
- Updated `CHANGELOG.md`, `docs/TODO.md` and `frontend/README.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint96_useful_default_context.py -q` failed on missing default dataset selection and no-raster AI Lab guidance.
|
|
- `python -m pytest backend\tests\test_sprint96_useful_default_context.py -q` (`2 passed`)
|
|
- `python -m pytest backend\tests\test_sprint39_frontend_orchestration_hooks.py backend\tests\test_sprint29_dataset_components.py backend\tests\test_sprint88_ai_lab_density.py -q` (`16 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -q` (`318 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`318 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- `cd backend && python -m alembic heads` (`202606120900 (head)`)
|
|
- `cd backend && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- Tower deploy from commit `aaf1299` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live browser validation against `http://192.168.10.150:1202` passed: Data auto-selected `demo_predicted_buildings.geojson`, Map rendered `2 features`, Exports showed the same selected dataset, AI Labs showed no-raster guidance for detection and segmentation, desktop/mobile had no horizontal overflow and no console/page errors.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend usability/default-context hardening only; no API contract, persistence, migration, provider fetching or AI/model behavior changes.
|
|
- Detection and segmentation remain correctly blocked without a raster dataset.
|
|
|
|
Next recommended pass:
|
|
- After deploy and live validation, add a raster fixture/demo upload smoke so AI Labs and raster controls can be validated with a connected raster state.
|
|
|
|
## Sprint 101 AI Lab handoff browser smoke (2026-06-24)
|
|
|
|
Changed:
|
|
- Added `scripts/verify_ai_handoff_interactions.sh`.
|
|
- The script seeds the explicit offline demo workflow through the frontend-facing API, creates a small raster tile manifest, opens the browser workbench with Playwright/Chromium, clicks `Use in Detection Lab` and `Use in Segmentation Lab`, and verifies both AI Lab forms receive the selected raster dataset plus `Raster tile manifest path`.
|
|
- Added syntax coverage for the new script to `scripts/run_readiness_check.sh`.
|
|
- Added `backend/tests/test_sprint101_ai_handoff_interaction_smoke.py`.
|
|
- Updated `scripts/README.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint101_ai_handoff_interaction_smoke.py -q` failed while `scripts/verify_ai_handoff_interactions.sh` was absent.
|
|
- `python -m pytest backend\tests\test_sprint101_ai_handoff_interaction_smoke.py -q` (`1 passed`)
|
|
- `bash -n scripts/verify_ai_handoff_interactions.sh`
|
|
- Local Node Playwright availability check failed with `ERR_MODULE_NOT_FOUND`, so live browser-click execution requires Playwright to be installed or exposed in the runner environment.
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -q` (`324 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`324 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- `cd backend && python -m alembic heads` (`202606120900 (head)`)
|
|
- `cd backend && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- Tower deploy from commit `aab51ed` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live smoke passed: `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`.
|
|
- Live browser-click validation with Codex Playwright plus installed Chrome passed against `http://192.168.10.150:1202`: the smoke seeded the demo workflow, generated a raster tile manifest, selected the raster in Data, opened the Dataset inspector tab, clicked `Use in Detection Lab`, selected `yolo-configured`, verified Detection Lab dataset and manifest values, clicked `Use in Segmentation Lab`, verified Segmentation Lab dataset and manifest values, and observed no console/page errors.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Test/smoke tooling only; no product behavior, API contract, migration, AI dependency, provider fetching or model behavior changes.
|
|
- `scripts/verify_ai_handoff_interactions.sh` requires a runner with Playwright plus a browser. This Codex environment had Playwright available through the Node REPL and used installed Chrome; plain local `node` still reports `ERR_MODULE_NOT_FOUND` for `playwright`.
|
|
|
|
Next recommended pass:
|
|
- After validation and deploy, keep using the AI handoff smoke as the browser-level regression guard for raster-to-AI workspace wiring.
|
|
|
|
## Sprint 102 Detection Lab handoff polish (2026-06-24)
|
|
|
|
Changed:
|
|
- Updated `useRasterTileManifestForDetection` in `frontend/src/App.tsx` to set `selectedDetectionModelId` to `yolo-configured` when a raster tile manifest is handed off from the dataset inspector.
|
|
- Tightened `scripts/verify_ai_handoff_interactions.sh` so it verifies that Detection Lab is already on `yolo-configured` after the handoff instead of selecting that model inside the smoke.
|
|
- Updated `backend/tests/test_sprint99_raster_ui_handoff.py` and `backend/tests/test_sprint101_ai_handoff_interaction_smoke.py`.
|
|
- Updated `CHANGELOG.md` and `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint99_raster_ui_handoff.py -q` failed while the detection handoff did not set `yolo-configured`.
|
|
- `python -m pytest backend\tests\test_sprint99_raster_ui_handoff.py backend\tests\test_sprint101_ai_handoff_interaction_smoke.py -q` (`2 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash -n scripts/verify_ai_handoff_interactions.sh`
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -q` (`324 passed`)
|
|
- `bash scripts/run_readiness_check.sh` (`324 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- `cd backend && python -m alembic heads` (`202606120900 (head)`)
|
|
- `cd backend && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- First Tower redeploy attempt failed with Docker btrfs `no space left on device` while writing build context files.
|
|
- Remote Docker diagnosis showed `/var/lib/docker` had ordinary free space but btrfs metadata was 94.80% used with the 300GB docker image fully allocated.
|
|
- Removed dangling Docker images and cleared orphaned hung Docker CLI processes from prior inspection/prune attempts.
|
|
- Ran a limited Docker btrfs balance: `btrfs balance start -dusage=40 -musage=80 /var/lib/docker`; metadata improved to 75.63% used and 94.95GiB became unallocated.
|
|
- Tower deploy from commit `4367400` then completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live smoke passed: `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`.
|
|
- Internal Codex browser validation passed against `http://192.168.10.150:1202`: Data workspace selected the raster fixture, Dataset inspector handoff clicked `Use in Detection Lab`, Detection Lab auto-selected `yolo-configured`, the raster dataset and tile manifest matched the generated manifest, the manifest input was visible, and no console/page errors were reported.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend handoff polish only; no backend API, persistence, migration, provider fetching, AI dependency or model execution behavior changed.
|
|
- The Tower Docker image required btrfs metadata balancing before redeploy. If this recurs, inspect `btrfs filesystem usage /var/lib/docker`; metadata near full can fail builds even when `df` reports free GB.
|
|
|
|
Next recommended pass:
|
|
- After live validation, continue with the next V1 usability gap from the workbench flow rather than adding new model/provider scope.
|
|
|
|
## Sprint 103 AI Lab run readiness (2026-06-24)
|
|
|
|
Changed:
|
|
- Added compact run-readiness panels to Detection Lab and Segmentation Lab.
|
|
- Detection readiness now checks selected raster dataset, selected model availability and the configured-YOLO tile manifest requirement before a run is submitted.
|
|
- Segmentation readiness now checks selected raster dataset, configured segmentation model state and whether a tile manifest is present for provenance.
|
|
- Added shared AI Lab readiness styling and regression coverage in `backend/tests/test_sprint103_ai_lab_run_readiness.py`.
|
|
- Updated `CHANGELOG.md` and `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint103_ai_lab_run_readiness.py -q` failed while the readiness panels and CSS contracts were absent.
|
|
- `python -m pytest backend\tests\test_sprint103_ai_lab_run_readiness.py -q` (`3 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -q` (`327 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`327 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- `cd backend && python -m alembic heads` (`202606120900 (head)`)
|
|
- `cd backend && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- Tower deploy from commit `98fff63` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live smoke passed: `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`.
|
|
- Internal Codex browser validation passed against `http://192.168.10.150:1202`: AI Labs rendered Detection and Segmentation run-readiness panels; Detection stayed blocked when `Configured YOLO detector` had no local YOLO configuration but marked dataset and tile manifest as provided; Segmentation became ready with the explicit fixture segmenter and manifest; desktop and mobile viewports had no horizontal overflow and no console warnings/errors.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend readiness guidance only; no backend API, persistence, migration, provider fetching, AI dependency or model execution behavior changed.
|
|
|
|
Next recommended pass:
|
|
- Continue with V1 usability work that reduces operator confusion without expanding frozen product scope.
|
|
|
|
## Sprint 122 Detection model asset activation guardrails (2026-07-08)
|
|
|
|
Changed:
|
|
- Hardened Detection Lab local model handling so reported runtime model assets are read-only choices and are not auto-selected by the frontend hook.
|
|
- Configured YOLO run readiness now blocks submission when local model assets exist but no explicit `model_asset_id` has been selected.
|
|
- Added an explicit model asset section with active runtime env status, `will_download_models`, SHA-256 preview, file size and mounted model path.
|
|
- Surfaced the current benchmark candidate `geointel-building-yolov8s-hardneg160r4e50-pt` with recommended starting threshold `0.25`.
|
|
- Added compact UI guidance styling for the benchmark/threshold warning.
|
|
- Added regression coverage in `backend/tests/test_sprint122_model_asset_activation_guardrails.py`.
|
|
- Updated `CHANGELOG.md` and `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint122_model_asset_activation_guardrails.py -q` failed on the previous auto-selection behavior and missing guardrail copy.
|
|
- `python -m pytest backend\tests\test_sprint122_model_asset_activation_guardrails.py -q` (`3 passed`)
|
|
- `python -m pytest backend\tests\test_sprint118_yolo_preflight_ui.py backend\tests\test_sprint103_ai_lab_run_readiness.py backend\tests\test_sprint104_ai_lab_action_guardrails.py backend\tests\test_model_asset_catalog.py backend\tests\test_sprint122_model_asset_activation_guardrails.py -q` (`16 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `python -m pytest backend\tests -q` (`408 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
Open:
|
|
- Full threshold calibration comparison UI is still pending; this pass adds safe single-threshold guidance and explicit asset choice only.
|
|
|
|
Limitations:
|
|
- Frontend guardrail only; no backend API contracts, migrations, provider fetching, model downloads or model weight mutation behavior changed.
|
|
- The active runtime env model can still be configured by operators through existing deployment/env tooling, but the Detection Lab no longer silently chooses a local asset from the catalog for a run.
|
|
|
|
Next recommended pass:
|
|
- Add threshold calibration comparison UX over existing persisted runs so candidate models can be promoted with visible precision/recall/F1 and hard-negative counts.
|
|
|
|
## Operator YOLOv8s hard-negative model benchmark (2026-07-08)
|
|
|
|
Changed:
|
|
- No repository code, API contract, migration, UI or application behavior was changed in this pass.
|
|
- Downloaded the official Ultralytics YOLOv8s base model manually as an operator/runtime asset on Tower at `/mnt/user/appdata/geointel/models/yolov8s.pt`.
|
|
- Trained a local hard-negative building detector on Tower from the existing exported operator tile dataset `yolo-building-tile-hardneg160r4`.
|
|
- Produced the trained runtime model artifact `/mnt/user/appdata/geointel/models/geointel-building-yolov8s-hardneg160r4e50.pt`, mounted in the container as `/app/models/geointel-building-yolov8s-hardneg160r4e50.pt`.
|
|
- Verified the live model catalog exposes the trained model as `geointel-building-yolov8s-hardneg160r4e50-pt` with SHA256 `9bf71ad4742048ac77f07060b677bacd9757b8d310497fcada334d543e320d19`, size `22473194`, `will_download_models=false` and `active=false`.
|
|
- Reused existing persisted dense QA and hard-negative benchmark runs through the live API; no external provider fetching and no fake data were introduced.
|
|
|
|
Training evidence:
|
|
- Base model: `/app/models/yolov8s.pt`.
|
|
- Base model SHA256: `1f47a78bf100391c2a140b7ac73a1caae18c32779be7d310658112f7ac9aa78a`.
|
|
- Training run: `/app/storage/training/operator-yolo/geointel-building-yolov8s-hardneg160r4e50`.
|
|
- Best checkpoint copied to: `/app/models/geointel-building-yolov8s-hardneg160r4e50.pt`.
|
|
- Final validation from Ultralytics: precision `0.449`, recall `0.404`, mAP50 `0.322`, mAP50-95 `0.110`.
|
|
|
|
Live benchmark evidence:
|
|
- Created 24 persisted rescored detection runs with `model_asset_id=geointel-building-yolov8s-hardneg160r4e50-pt`.
|
|
- Dense persisted QA matrix results at IoU `0.1`:
|
|
- Four dense benchmark scenes saturated at 300 detections across thresholds `0.05`, `0.15` and `0.25`.
|
|
- Dense F1 scores observed: `0.6380`, `0.5627`, `0.5247`, `0.4678`.
|
|
- Dense precision ranged from `0.7167` to `0.8600`; recall ranged from `0.3247` to `0.5749`.
|
|
- Mean IoU ranged from `0.3962` to `0.4465`.
|
|
- The sparse/forest scene improved as the threshold increased: `49` detections and F1 `0.1429` at `0.05`; `18` detections and F1 `0.3200` at `0.15`; `10` detections and F1 `0.4706` at `0.25`.
|
|
- Hard-negative detection counts:
|
|
- `kasterlee_bos`: `49` at threshold `0.05`, `18` at `0.15`, `10` at `0.25`.
|
|
- `lommel_heide`: `0` at thresholds `0.05`, `0.15` and `0.25`.
|
|
- `postel_bos`: `1` at threshold `0.05`, `0` at `0.15` and `0.25`.
|
|
|
|
Assessment:
|
|
- The YOLOv8s hard-negative model is a materially better evaluation candidate than the previous tiny smoke models and is useful in dense building scenes.
|
|
- The model should not be made the silent default yet because `kasterlee_bos` still produces 10 hard-negative detections at threshold `0.25`.
|
|
- Threshold `0.25` is the safest observed operating point for the current candidate.
|
|
- The next product step should be model-selection and threshold workflow hardening before any operator-facing default activation.
|
|
|
|
Operational note:
|
|
- Tower root SSH works with `widefrog_unraid_deploy` on the default SSH port `22`.
|
|
- Gitea SSH works separately on port `222` through the `gitea-widefrog` host alias and `widefrog_gitea` key.
|
|
- Do not test Unraid/root deploy access against port `222`; that port belongs to Gitea and correctly rejects `root`.
|
|
- Live HTTP API verification on `http://192.168.10.150:1202` remained reachable.
|
|
|
|
Open:
|
|
- Add an operator-facing model selection/activation flow or documented command that can promote a chosen local model deliberately, with visible SHA256 and threshold guidance.
|
|
- Add more negative/background AOIs and a threshold calibration benchmark before activating this model by default.
|
|
|
|
Next recommended pass:
|
|
- Implement V1 model-catalog hardening: show available local model assets, make active model/threshold explicit, and prevent accidental silent default activation.
|
|
|
|
## Sprint 123 Raster detection manifest handoff (2026-07-08)
|
|
|
|
Changed:
|
|
- Added a typed `RasterTileHandoff` frontend contract so raster tile jobs can expose manifest path, tile count, tile size, overlap, tile-set id and source dataset provenance.
|
|
- Updated the dataset workflow hook to retain the latest structured raster tile manifest alongside the existing manifest path.
|
|
- Updated Raster Controls to show manifest details and explicit Detection/Segmentation Lab handoff actions.
|
|
- Updated the Detection Lab handoff so a raster manifest selects `yolo-configured`, selects the active raster dataset, applies threshold `0.25`, refreshes YOLO preflight and does not auto-select a local model asset.
|
|
- Added linked tile manifest, manifest validation, tile count and `will_run_inference` visibility to Detection Lab.
|
|
- Added regression coverage in `backend/tests/test_sprint123_raster_detection_handoff_operational.py`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint123_raster_detection_handoff_operational.py -q` failed while the typed handoff, Raster Controls details and Detection Lab preflight indicators were absent.
|
|
- `python -m pytest backend\tests\test_sprint123_raster_detection_handoff_operational.py -q` (`3 passed`)
|
|
- `python -m pytest backend\tests\test_sprint99_raster_ui_handoff.py backend\tests\test_sprint122_model_asset_activation_guardrails.py backend\tests\test_sprint120_model_asset_detection_workflow_smoke.py backend\tests\test_sprint123_raster_detection_handoff_operational.py -q` (`8 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
Limitations:
|
|
- This pass improves the operator handoff and preflight visibility only. It does not change backend inference contracts, migrations, model files, provider fetching or automatic model promotion.
|
|
|
|
Next recommended pass:
|
|
- Add threshold calibration comparison UX so an operator can compare candidate thresholds before promoting a local model.
|
|
|
|
## Sprint 133 Detection threshold calibration UX (2026-07-08)
|
|
|
|
Changed:
|
|
- Added a Detection Lab calibration comparison panel that combines existing persisted `DetectionRunRead` rows with existing persisted `QualityCheckRead`/metric rows.
|
|
- The panel shows confidence threshold, model, local model asset id, detection count, precision, recall, F1, false positives, false negatives and linked quality-check id.
|
|
- Added summary cards for best F1 candidate, best precision candidate and lowest false-positive pressure.
|
|
- Added a promotion guardrail that keeps model/threshold acceptance tied to QA evidence across AOIs instead of a single run.
|
|
- Passed project-level `qualityChecks` into Detection Lab without adding API routes, migrations or new AI execution behavior.
|
|
- Added regression coverage in `backend/tests/test_sprint133_detection_threshold_calibration_ux.py`.
|
|
- Marked the threshold calibration UX item complete in `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint133_detection_threshold_calibration_ux.py -q` failed while the persisted calibration comparison UI was absent.
|
|
- `python -m pytest backend\tests\test_sprint133_detection_threshold_calibration_ux.py -q` (`1 passed`)
|
|
- `python -m pytest backend\tests\test_sprint122_model_asset_activation_guardrails.py backend\tests\test_sprint123_raster_detection_handoff_operational.py backend\tests\test_sprint133_detection_threshold_calibration_ux.py -q` (`7 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
Limitations:
|
|
- This is a persisted-run comparison surface only. It does not launch batch calibration sweeps from the browser and does not auto-promote model assets or thresholds.
|
|
|
|
Next recommended pass:
|
|
- Add a guided in-app calibration runner that can queue a small explicit threshold set for one selected raster/reference pair, reusing the existing detection and QA APIs.
|
|
|
|
## Sprint 134 Guided detection calibration runner (2026-07-08)
|
|
|
|
Changed:
|
|
- Added an explicit guided calibration runner to Detection Lab for operator-selected confidence threshold sweeps.
|
|
- Added frontend detection workflow state for calibration thresholds, progress rows, running state and errors.
|
|
- The runner parses a space/comma/semicolon-separated threshold set, validates project/raster/reference/model/manifest/model-asset readiness and runs one existing `detectionApi.run` plus one existing detection QA comparison per threshold.
|
|
- Successful threshold rows report persisted analysis run/job/quality check ids, detection count, precision, recall, F1, false positives and false negatives.
|
|
- Added UI copy that this runs real configured YOLO jobs and QA comparisons and does not promote or mutate model files.
|
|
- Added regression coverage in `backend/tests/test_sprint134_guided_detection_calibration_runner.py`.
|
|
- Marked the guided calibration runner task complete in `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint134_guided_detection_calibration_runner.py -q` failed while the runner contract was absent.
|
|
- `python -m pytest backend\tests\test_sprint134_guided_detection_calibration_runner.py -q` (`1 passed`)
|
|
- `python -m pytest backend\tests\test_sprint133_detection_threshold_calibration_ux.py backend\tests\test_sprint134_guided_detection_calibration_runner.py backend\tests\test_sprint122_model_asset_activation_guardrails.py backend\tests\test_sprint123_raster_detection_handoff_operational.py -q` (`8 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
|
|
Limitations:
|
|
- The runner is intentionally sequential and explicit. It does not schedule background batches, compare multiple AOIs at once or promote model assets/thresholds automatically.
|
|
|
|
Next recommended pass:
|
|
- Add an evidence shortcut from each calibration row to the persisted QA evidence map/review flow so false positives and false negatives can be inspected faster.
|
|
|
|
## Sprint 117 Safe local YOLO model activation (2026-07-06)
|
|
|
|
Changed:
|
|
- Added `scripts/configure_yolo_model.py` to configure an existing local YOLO model into the deployment `.env` file without downloading model weights, loading a model or running inference.
|
|
- The helper scans a mounted model directory for `.pt`, `.onnx` and `.engine` files, refuses no-model and ambiguous multi-model states, and writes env updates only when `--apply` is provided.
|
|
- Added regression coverage in `backend/tests/test_sprint119_yolo_model_configuration.py` for no local model, ambiguous model selection, dry-run single model selection and env-file apply behavior.
|
|
- Downloaded the official Ultralytics `yolov8n.pt` smoke model to Tower under `/mnt/user/appdata/geointel/models/yolov8n.pt`, recorded checksum `f59b3d833e2ff32e194b5bb8e08d211dc7c5bdf144b90d2c8412c47ccfc83b36`, and applied the env configuration with the local helper.
|
|
- Hardened `YoloDetectionAdapter.predict_tile` so non-RGB raster tile artifacts are converted to a temporary RGB image before YOLO inference while georeferencing remains driven by the tile manifest.
|
|
- Wrapped YOLO prediction runtime errors as typed `DETECTION_INFERENCE_FAILED` `AppError`s instead of leaking raw runtime exceptions through FastAPI.
|
|
- Updated `scripts/README.md`, `deploy/unraid/README.md`, `backend/README.md`, `docs/AI_PIPELINES.md`, `docs/TODO.md` and `CHANGELOG.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint119_yolo_model_configuration.py -q` failed while `scripts/configure_yolo_model.py` was absent.
|
|
- Red step: `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py -q` failed because single-band TIFF tiles were passed through as mode `L` and prediction runtime errors leaked as raw `RuntimeError`.
|
|
- `python -m pytest backend\tests\test_sprint119_yolo_model_configuration.py -q` (`4 passed`)
|
|
- `python -m pytest backend\tests\test_sprint8b_yolo_foundation.py -q` (`12 passed`)
|
|
- `python -m py_compile scripts\configure_yolo_model.py`
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -q` (`377 passed`, existing Pydantic protected-namespace warnings remain)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`Run readiness check passed`)
|
|
- `cd backend && python -m alembic heads` (`202606120900 (head)`)
|
|
- `cd backend && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- `bash -n scripts/deploy_tower.sh`
|
|
- Tower deploy from commit `72ee623` completed with `GEOINTEL_INSTALL_AI=true`; the all-in-one container is published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live API check passed: `GET /api/v1/detection/yolo/preflight` returned canonical `data` with `status=not_configured`, `YOLO_ENABLED=false`, `torch_version=2.12.1`, `ultralytics_version=8.4.89`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Tower helper dry-run passed: `python scripts/configure_yolo_model.py --models-dir /mnt/user/appdata/geointel/models --env-file .env --json` returned `status=no_model_found`, empty candidates and no env updates.
|
|
- Tower model apply passed: `python scripts/configure_yolo_model.py --models-dir /mnt/user/appdata/geointel/models --env-file .env --model-file /mnt/user/appdata/geointel/models/yolov8n.pt --apply --json` returned `status=applied`, `YOLO_ENABLED=true` and `YOLO_MODEL_PATH=/app/models/yolov8n.pt`.
|
|
- Live YOLO preflight with generated demo raster tile manifest passed with `status=ready`, `model_load_ok=true`, `manifest_valid=true`, `tile_paths_exist=true`, `tile_count=1`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Live inference smoke before the RGB adapter fix reproduced the runtime bug: YOLOv8n expected 3 channels but the demo tile was single-band (`input[1, 1, 480, 640]`).
|
|
- Tower deploy from commit `add4768` completed with `GEOINTEL_INSTALL_AI=true`; the all-in-one container is published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 3; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live YOLO preflight after the RGB adapter fix passed with generated demo raster tile manifest and `status=ready`.
|
|
- Live real YOLO inference smoke after the RGB adapter fix passed: `POST /api/v1/detection/run` returned `status=success`, analysis run `bbd80690-baf3-4e83-8ffb-fdcb07a70c0c`, job `9636ed10-bbde-463f-a47a-6d7706c81e19`, `detection_count=0` and no error code. Zero detections is expected for the generic COCO smoke model on the synthetic demo raster.
|
|
- Downloaded `keremberke/yolov8n-building-segmentation` from Hugging Face as an explicit Tower runtime artifact at `/mnt/user/appdata/geointel/models/yolov8n-building-segmentation.pt`.
|
|
- Recorded building-model checksum `152d6a9c5c76c9f2fd2fd5cc167efaed7c8c02e31002b15415899710f1d71f98`, matching the Hugging Face file metadata for `best.pt`.
|
|
- Applied `.env` with `YOLO_MODEL_PATH=/app/models/yolov8n-building-segmentation.pt` through `scripts/configure_yolo_model.py --model-file ... --apply`.
|
|
- Restarted the all-in-one container without rebuild; it remained healthy and published on `0.0.0.0:1202->80/tcp`.
|
|
- Live building-model preflight passed with `model_load_ok=true`, `dependencies_available=true`, `torch_version=2.12.1`, `ultralytics_version=8.4.89`, `will_download_models=false` and `will_run_inference=false`.
|
|
- Live building-model preflight with generated demo raster tile manifest passed with `status=ready`.
|
|
- Live real building-model YOLO inference smoke passed: `POST /api/v1/detection/run` returned `status=success`, analysis run `be609d4a-3e01-42df-bca8-0d5a2289120d`, job `9c8bc26d-7753-4537-ab98-dc31b9029450`, `detection_count=0` and no error code. Zero detections remains expected on the synthetic demo raster.
|
|
|
|
Open:
|
|
- A local YOLO smoke model is now present and configured on Tower, but it is the generic COCO `yolov8n.pt` model. It proves the runtime path, not production-quality aerial building detection.
|
|
|
|
Limitations:
|
|
- The bundled Tower model file was placed as an operator/runtime artifact under appdata, not committed to Git.
|
|
- The configured model is a generic COCO model and should be replaced by a suitable aerial/building detector for meaningful GIS output.
|
|
- The active configured model is now a building segmentation YOLO model trained for satellite-building segmentation, but it is still a third-party Hugging Face PyTorch `.pt` artifact and should be treated as an evaluation model until validated on local Belgian/Kempen orthophotos.
|
|
- No model download behavior was added to the application; the manual operator placement remains explicit.
|
|
- If multiple local model files are present, the operator must choose one with `--model-file` so GeoIntel does not silently activate the wrong model.
|
|
|
|
Next recommended pass:
|
|
- Load a real georeferenced orthophoto/GeoTIFF for the target area, generate raster tiles, run the active building YOLO model and compare persisted detections against reference vector buildings through QA/QC.
|
|
|
|
## Sprint 104 AI Lab action guardrails (2026-06-24)
|
|
|
|
Changed:
|
|
- Added explicit action guardrails below the Detection Lab and Segmentation Lab run-readiness panels.
|
|
- Detection now distinguishes `configured` model registry state from UI-runnable action state, blocking the explicit test/demo-only `manual-fixture-detector` in the normal workbench run form.
|
|
- Segmentation now distinguishes `configured` model registry state from UI-runnable action state, blocking the explicit test/demo-only `fixture-segmenter` in the normal workbench run form.
|
|
- Updated run button disabled conditions to use the new readiness/action state.
|
|
- Added compact guardrail styling and regression coverage in `backend/tests/test_sprint104_ai_lab_action_guardrails.py`.
|
|
- Updated `CHANGELOG.md` and `docs/TODO.md`.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint104_ai_lab_action_guardrails.py -q` failed while the action guardrails and CSS contracts were absent.
|
|
- `python -m pytest backend\tests\test_sprint104_ai_lab_action_guardrails.py -q` (`3 passed`)
|
|
- `python -m pytest backend\tests\test_sprint103_ai_lab_run_readiness.py backend\tests\test_sprint104_ai_lab_action_guardrails.py -q` (`6 passed`)
|
|
- `python -m compileall backend/app`
|
|
- `cd backend && python -m pytest -q` (`330 passed`)
|
|
- `cd frontend && npm run typecheck`
|
|
- `cd frontend && npm run build`
|
|
- `bash scripts/run_readiness_check.sh` (`330 passed`; frontend typecheck/build passed; Alembic head `202606120900`; live smoke syntax passed)
|
|
- `cd backend && python -m alembic heads` (`202606120900 (head)`)
|
|
- `cd backend && python -m alembic upgrade head --sql`
|
|
- `bash -n scripts/live_migration_smoke.sh`
|
|
- Tower deploy from commit `0bc3b2f` completed; all-in-one container published on `0.0.0.0:1202->80/tcp`.
|
|
- Deploy-time live migration smoke passed after the database became ready on attempt 2; PostGIS reported `3.6 USE_GEOS=1 USE_PROJ=1 USE_STATS=1`, required runtime schema objects were present and Alembic head was `202606120900`.
|
|
- Deploy-time browser runtime verification passed for frontend, API proxy and icon.
|
|
- Live smoke passed: `bash scripts/verify_demo_raster_workflow.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_default_state.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_workbench_interactions.sh http://192.168.10.150:1202`.
|
|
- Live smoke passed: `bash scripts/verify_browser_runtime.sh http://192.168.10.150:1202`.
|
|
- Internal Codex browser validation passed against `http://192.168.10.150:1202`: AI Labs opened cleanly; run action guardrails rendered; Detection and Segmentation run buttons were disabled without a raster dataset; selecting explicit fixture models kept both run buttons disabled with `Fixture model is explicit test/demo-only` and `Fixture segmenter is explicit test/demo-only`; desktop and mobile viewports had no horizontal overflow and no console warnings/errors.
|
|
|
|
Open:
|
|
- None for this pass.
|
|
|
|
Limitations:
|
|
- Frontend action-guardrail guidance only; no backend API, persistence, migration, provider fetching, AI dependency or model execution behavior changed.
|
|
|
|
Next recommended pass:
|
|
- Continue with V1 usability work that reduces operator confusion without expanding frozen product scope.
|
|
|
|
## Sprint 142 Calibration evidence response uniqueness (2026-07-08)
|
|
|
|
Changed:
|
|
- Hardened `scripts/export_detection_calibration_evidence.sh` so calibration evidence response files include both confidence threshold and quality-check id.
|
|
- Prevented same-threshold runs from different model assets or calibration jobs from overwriting each other before portfolio assembly.
|
|
- Extended the multi-AOI calibration evidence portfolio regression test with two Geel runs at the same threshold and distinct quality checks.
|
|
|
|
Tested:
|
|
- Red step: `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py -q` failed because only two evidence features were retained when three same-threshold responses were expected.
|
|
- `python -m pytest backend\tests\test_sprint139_multi_aoi_calibration_evidence_portfolio.py backend\tests\test_sprint138_calibration_evidence_bundle_smoke.py backend\tests\test_sprint137_browser_calibration_summary_evidence_script.py backend\tests\test_sprint125_detection_calibration_evidence_bundle.py -q` (`4 passed`)
|
|
- `bash -n scripts/export_detection_calibration_evidence.sh`
|
|
- `bash -n scripts/assemble_detection_calibration_evidence_portfolio.sh`
|
|
- `python scripts\smoke_docs.py`
|
|
- `python -m compileall backend/app`
|
|
- `bash scripts/run_readiness_check.sh` (`418 passed`, frontend typecheck/build passed, Alembic head `202606120900`)
|
|
- Tower pulled commit `5314f16` and regenerated `/mnt/user/appdata/geointel/artifacts/detection-calibration-portfolio/positive-aoi-expanded-20260708/output`.
|
|
- The regenerated portfolio contains 7 samples, 38,007 evidence features and no missing model metadata.
|
|
- Best runs are now preserved across same-threshold model comparisons: Geel, Mol, Turnhout, Retie, Balen, Herentals and Westerlo all select `geointel-building-yolov8n-expanded160e50-pt` as their best scored run.
|
|
- Tower container remained healthy on `0.0.0.0:1202->80/tcp`.
|
|
|
|
Open:
|
|
- The expanded160e50 model is consistently best on the current positive AOI portfolio, but hard-negative/background AOI evidence still prevents blind default promotion.
|
|
# Sprint 146 - Operator YOLO dataset quality audit
|
|
|
|
## What changed
|
|
|
|
- Added `scripts/audit_operator_yolo_dataset_quality.py` to inspect generated operator YOLO tile datasets before further model training.
|
|
- Added a focused pytest that creates a synthetic tile summary and YOLO label files, then verifies JSON/Markdown audit output and warning gates.
|
|
- Added the audit script to the readiness syntax gate.
|
|
- Documented the operator audit command in `scripts/README.md`.
|
|
|
|
## What was tested
|
|
|
|
- `python -m pytest backend\tests\test_sprint146_operator_yolo_dataset_quality_audit.py -q`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- Tower live audit after pulling commit `5898e54`:
|
|
- `yolo-building-tile-dataset`: `needs_attention`; only 3 positive samples and no background negatives.
|
|
- `yolo-building-tile-expanded160`: `ok`; 10 samples, 8 positive samples, 360 tiles, 11,213 labels, no missing/invalid label rows.
|
|
- `yolo-building-tile-hardneg160r4`: `needs_attention`; repeated background negatives are 91.1% of negative tiles.
|
|
- `yolo-building-tile-hardneg160r8`: `needs_attention`; repeated background negatives are 95.4% of negative tiles.
|
|
|
|
## Known limitations
|
|
|
|
- The audit is evidence tooling only. It does not modify datasets, train models, fetch external data or change active YOLO configuration.
|
|
- The report flags likely dataset risks, but final promotion decisions must still come from persisted detection QA/QC matrices and hard-negative benchmarks.
|
|
|
|
## Next recommended pass
|
|
|
|
- Add more unique hard-negative/background AOIs before another hard-negative training run. The current label files are clean, so the bottleneck is dataset diversity and balance rather than label-file corruption.
|
|
|
|
# Sprint 147 - Unique hard-negative AOI expansion
|
|
|
|
## What changed
|
|
|
|
- Expanded `scripts/prepare_operator_real_data_samples.py` with six additional explicit background-candidate AOIs: Dessel-heide, Ravels-bos, Meerhout-bos, Geel-Bel, Arendonk-heide and Herenthout-bos.
|
|
- Background candidates remain operator/runtime samples only. They are not product providers, not fixtures and not automatic app fetches.
|
|
- Added sample-registry test coverage for minimum unique background count, unique centers and regional spread.
|
|
- Updated operator documentation with the expanded default corpus and the next required Tower regeneration step.
|
|
- Fixed the all-in-one Dockerfile so the documented operator scripts are copied into `/app/scripts/` during normal rebuilds.
|
|
- Fixed the all-in-one Dockerfile training-runtime gap so `/app/scripts/train_operator_yolo_detector.sh` is also copied into the rebuilt container and marked executable.
|
|
- Fixed `scripts/yolo_preflight.py` so it respects `YOLO_ENABLED`, `YOLO_MODEL_PATH` and `YOLO_MAX_TILES` from the runtime environment unless explicit CLI overrides are supplied.
|
|
- Prepared the expanded Tower operator manifest and exported `yolo-building-tile-uniquehardneg160`.
|
|
|
|
## What was tested
|
|
|
|
- `python -m pytest backend\tests\test_sprint131_operator_sample_expansion.py -q`
|
|
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint131_operator_sample_expansion.py -q`
|
|
- `python -m pytest backend\tests\test_sprint13_yolo_preflight.py -q`
|
|
- Red/green Docker runtime guard for the training wrapper: `python -m pytest backend\tests\test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_for_runtime_use -q`
|
|
- `python -m pytest backend\tests\test_docker_runtime_config.py -q`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- Tower live operator sample prep:
|
|
- manifest samples: 16 total, 7 reference and 9 background candidates.
|
|
- background candidate GRB feature counts: Postel-bos 0, Lommel-heide 0, Kasterlee-bos 7, Dessel-heide 30, Ravels-bos 3, Meerhout-bos 20, Geel-Bel 17, Arendonk-heide 0, Herenthout-bos 90.
|
|
- Tower live tile export and audit for `yolo-building-tile-uniquehardneg160`:
|
|
- status `ok`
|
|
- 576 tiles, 346 positive, 230 negative
|
|
- 16 samples, 13 positive samples, 9 background samples
|
|
- 11,757 labels, 0 missing label files, 0 invalid label rows
|
|
- repeated background negative share 0.0
|
|
- Tower all-in-one rebuild from commit `f949347` with `GEOINTEL_INSTALL_AI=true`:
|
|
- live migration smoke passed.
|
|
- browser runtime verification passed on `http://192.168.10.150:1202`.
|
|
- `/app/scripts/prepare_operator_real_data_samples.py`, `/app/scripts/export_operator_yolo_tile_dataset.py` and `/app/scripts/audit_operator_yolo_dataset_quality.py` are present in the rebuilt container.
|
|
- YOLO preflight with an existing raster tile manifest returned `status: ready`, `dependencies_available: true`, `model_file_exists: true`, `tile_paths_exist: true`, `will_download_models: false` and `will_run_inference: false`.
|
|
|
|
## Known limitations
|
|
|
|
- Some background candidates contain real GRB buildings. They are still useful as mixed rural/background samples, but the pure negative pressure currently comes mostly from Postel-bos, Lommel-heide and Arendonk-heide plus empty tiles inside sparse candidates.
|
|
- The rebuilt Tower image now contains the operator scripts automatically. The generated `yolo-building-tile-uniquehardneg160` dataset is ready for a controlled training candidate, but no model has been promoted from it yet.
|
|
|
|
## Next recommended pass
|
|
|
|
- Use `yolo-building-tile-uniquehardneg160` as the next safer hard-negative training dataset candidate. Benchmark after training before changing defaults.
|
|
|
|
# Sprint 152 - GRB reference paging for operator samples
|
|
|
|
## What changed
|
|
|
|
- Fixed `scripts/prepare_operator_real_data_samples.py` so GRB GBG reference exports follow OGC API `rel=next` pagination links instead of silently trusting only the first `limit=1000` page.
|
|
- Added operator controls:
|
|
- `--reference-page-limit` / `OPERATOR_GRB_PAGE_LIMIT`, default `1000`.
|
|
- `--reference-max-features` / `OPERATOR_GRB_MAX_FEATURES`, default `100000`.
|
|
- Generated reference GeoJSON now records `source_urls`, `reference_pages_fetched`, `reference_truncated`, `reference_page_limit` and `reference_max_features`.
|
|
- Kept the change operator-only: no GeoIntel API route calls this helper, no product provider endpoint changed, no live GRB/OSM import was added, no migration changed and no YOLO model was activated.
|
|
|
|
## What was tested
|
|
|
|
- RED: `python -m pytest backend\tests\test_sprint131_operator_sample_expansion.py::test_fetch_reference_follows_grb_next_links_until_complete -q` failed because only the first GRB page was persisted.
|
|
- GREEN: `python -m pytest backend\tests\test_sprint131_operator_sample_expansion.py::test_fetch_reference_follows_grb_next_links_until_complete -q`
|
|
- `python -m pytest backend\tests\test_sprint127_operator_sample_quality_matrix.py backend\tests\test_sprint131_operator_sample_expansion.py -q`
|
|
- `python -m py_compile scripts\prepare_operator_real_data_samples.py`
|
|
- `bash scripts/run_readiness_check.sh`
|
|
- Local live GRB paging smoke with `page_limit=2` and `max_features=5` fetched 3 source pages, returned 5 features and correctly marked `reference_truncated=true`.
|
|
- Tower deploy from commit `a63d4ea` passed live migration smoke and browser runtime verification on `http://192.168.10.150:1202`.
|
|
- Tower one-sample Geel AOI1024 paging smoke passed with `/opt/geointel/venv/bin/python` and produced 2,268 GRB reference features instead of the old 1,000-feature cap.
|
|
- Tower full AOI1024 operator sample regeneration passed:
|
|
- Geel: 2,268 features, 3 GRB pages, not truncated.
|
|
- Mol: 1,993 features, 2 GRB pages, not truncated.
|
|
- Turnhout: 3,278 features, 4 GRB pages, not truncated.
|
|
- Herentals: 2,478 features, 3 GRB pages, not truncated.
|
|
- Balen: 1,343 features, 2 GRB pages, not truncated.
|
|
- Retie: 1,734 features, 2 GRB pages, not truncated.
|
|
- Westerlo: 1,133 features, 2 GRB pages, not truncated.
|
|
- Re-exported `/app/storage/operator-data/yolo-building-aoi1024-visible025` from the regenerated paged references:
|
|
- 144 tiles, 117 positive tiles, 27 negative tiles, 29,170 labels.
|
|
- `min_label_visible_ratio=0.25`, `negative_keep_ratio=1.0`.
|
|
- Re-ran the Tower dataset audit:
|
|
- output: `/app/storage/operator-data/yolo-building-aoi1024-visible025-audit/operator_yolo_dataset_quality_audit.json`
|
|
- status remains `needs_attention`
|
|
- missing label files: 0
|
|
- invalid labels: 0
|
|
- median normalized box area: `0.000762939453125`
|
|
- small-box share: `0.34744600617072335`
|
|
- repeated background negative share: 0.0
|
|
|
|
## Known limitations
|
|
|
|
- The Tower AOI1024 operator samples and tile dataset have now been regenerated from paged GRB references, but the tile audit still reports `needs_attention` because small clipped boxes remain common.
|
|
- This does not make the Sprint 7B GRB provider a live product importer; it only fixes explicit operator sample preparation.
|
|
|
|
## Next recommended pass
|
|
|
|
- Before another training run, improve label quality: either raise `--min-label-visible-ratio`, increase tile size, reduce dense clipped-edge tiles or add cleaner AOIs. Do not activate or retrain from this dataset as-is without acknowledging the small-box warning.
|
|
|
|
# Sprint 151 - Runtime GIS upload and AOI1024 YOLO candidate
|
|
|
|
## What changed
|
|
|
|
- Fixed `scripts/train_operator_yolo_detector.sh` so the all-in-one image uses `/opt/geointel/venv/bin/python` by default when that AI venv exists. Explicit `PYTHON_BIN` still wins, and local shells still fall back to `python3`.
|
|
- Raised the Nginx request body limit to `250m` in both `frontend/nginx.conf` and `deploy/unraid/nginx-all-in-one.conf` after the live 1024px GeoTIFF upload path returned `413 Request Entity Too Large`.
|
|
- Raised Nginx proxy read/send timeouts to `600s` after the live low-threshold persisted YOLO/QA path reached `504 Gateway Timeout`.
|
|
- Kept the change runtime-only: no API contract, persistence model, migration, model-download behavior or default model selection changed.
|
|
|
|
## What was tested
|
|
|
|
- Red/green TDD guard for the training wrapper fallback:
|
|
- `python -m pytest backend\tests\test_sprint129_operator_yolo_training_dataset.py -q`
|
|
- Red/green TDD guard for real GIS upload payload support:
|
|
- `python -m pytest backend\tests\test_docker_runtime_config.py::test_nginx_runtime_allows_real_gis_upload_payloads -q`
|
|
- Red/green TDD guard for long AI/QA proxy requests:
|
|
- `python -m pytest backend\tests\test_docker_runtime_config.py::test_nginx_runtime_allows_long_ai_and_qa_requests -q`
|
|
- `bash -n scripts/train_operator_yolo_detector.sh`
|
|
- Tower live model training on `/app/storage/operator-data/yolo-building-aoi1024-visible025`:
|
|
- output model: `/app/models/geointel-building-yolov8s-aoi1024visible025e50.pt`
|
|
- model asset id: `geointel-building-yolov8s-aoi1024visible025e50-pt`
|
|
- final Ultralytics validation: precision approximately `0.275`, recall `0.331`, mAP50 `0.188`, mAP50-95 `0.0716`
|
|
- Tower live runtime validation after deploy:
|
|
- browser/runtime verification passed on `http://192.168.10.150:1202`
|
|
- the previous Geel 1024px upload `413` no longer occurs
|
|
- the previous Geel low-threshold persisted YOLO/QA `504` no longer occurs
|
|
- Geel `threshold=0.05` completed with 2,612 detections, 5,091 raw candidates, 2,479 suppressed duplicates, precision `0.06316998468606431`, recall `0.165` and F1 `0.09136212624584718`
|
|
- Full AOI1024 positive persisted QA matrix:
|
|
- Geel best F1: `0.09136212624584718` at threshold `0.05`
|
|
- Turnhout best F1: `0.058721074894252295` at threshold `0.05`
|
|
- Retie best F1: `0.15621436716077539` at threshold `0.15`
|
|
- Westerlo best F1: `0.28703703703703703` at threshold `0.15`
|
|
- Full AOI1024 hard-negative/background matrix:
|
|
- pure empty Postel/Lommel/Arendonk samples stayed at or near zero detections
|
|
- mixed background candidates still produced false-positive pressure: max detections were `59` at threshold `0.25`, `107` at `0.15` and `226` at `0.05`
|
|
- Fixed `scripts/build_detection_model_promotion_report.py` after discovering it accepted `multi_sample_quality_summary.json` but counted positive samples as `0`. The report now supports both `samples[].runs[]` portfolios and `items[]` multi-sample summaries.
|
|
- AOI1024 promotion report path: `artifacts/detection-model-promotion/aoi1024visible025e50-full/detection_model_promotion_report.md`; recommendation remains `none`.
|
|
- Corrected AOI1024 promotion decision after parser fix:
|
|
- threshold `0.05`: 4 positive samples, 9 background samples, mean F1 `0.13307746028311157`, max background detections `226`
|
|
- threshold `0.15`: 4 positive samples, 9 background samples, mean F1 `0.13900227809255514`, max background detections `107`
|
|
- threshold `0.25`: 4 positive samples, 9 background samples, mean F1 `0.09694707724016788`, max background detections `59`
|
|
|
|
## Known limitations
|
|
|
|
- The AOI1024 tile audit is still `needs_attention`: median normalized box area is below gate and small-box share remains high.
|
|
- Several dense 1024 GRB reference exports reached the current 1000-feature source cap. Treat those samples as useful but potentially reference-capped until the provider query path supports paging or smaller dense AOIs are chosen.
|
|
- The trained model is intentionally inactive. It needs persisted detection QA/QC matrix evidence and background/hard-negative evidence before default promotion.
|
|
- The first completed persisted Geel QA run confirms the candidate is not promotion-ready: recall improves at low threshold, but false-positive pressure is too high.
|
|
- The complete positive/background evidence confirms the candidate is not promotion-ready. The bottleneck is label/source quality and sample design, not runtime configuration.
|
|
|
|
## Next recommended pass
|
|
|
|
- Fix dense GRB reference completeness first: add provider-side paging or split dense AOIs so reference exports do not cap at 1000 features, then regenerate AOI1024 labels before another training run.
|
|
|
|
# Sprint 153 - AOI1024 clean-label YOLOv8s candidate gate
|
|
|
|
## What changed
|
|
|
|
- Built and audited AOI1024 label-quality candidate exports on Tower after the paged GRB reference regeneration:
|
|
- baseline `yolo-building-aoi1024-visible025`: 144 tiles, 117 positive, 27 negative, 29,170 labels, audit `needs_attention`, median normalized box area `0.000762939453125`, small-box share `0.34744600617072335`.
|
|
- `yolo-building-aoi1024-visible050`: 144 tiles, 117 positive, 27 negative, 28,552 labels, audit `needs_attention`, small-box share `0.34463435135892406`.
|
|
- `yolo-building-aoi1024-visible025-tile640`: 45,529 labels, audit `needs_attention`, small-box share `0.4952667530584902`.
|
|
- `yolo-building-aoi1024-visible025-tile384`: 400 tiles, 311 positive, 89 negative, 46,423 labels, audit `ok`, small-box share `0.215281218361588`.
|
|
- `yolo-building-aoi1024-visible050-minpx8`: 144 tiles, 117 positive, 27 negative, 21,530 labels, audit `ok`, median normalized box area `0.0010299684375`, small-box share `0.14681839294008361`.
|
|
- Selected `/app/storage/operator-data/yolo-building-aoi1024-visible050-minpx8` for the next controlled candidate because it passes the label audit while preserving the 512px runtime scale.
|
|
- Trained Tower-local inactive model asset:
|
|
- model path: `/app/models/geointel-building-yolov8s-aoi1024clean512e50.pt`
|
|
- asset id: `geointel-building-yolov8s-aoi1024clean512e50-pt`
|
|
- SHA256: `7cfadb684dd56623d2e35ebd65593211d3051908c87121bef438b231d3e47cce`
|
|
- training summary: `/app/storage/training/operator-yolo/geointel-building-yolov8s-aoi1024clean512e50/training_summary.json`
|
|
- base model: `/app/models/yolov8s.pt`
|
|
- epochs: 50
|
|
- image size: 512
|
|
- batch size: 2
|
|
- device: CPU
|
|
- final Ultralytics validation on `best.pt`: precision `0.441`, recall `0.380`, mAP50 `0.265`, mAP50-95 `0.096`.
|
|
- Verified the live model catalog reports the candidate as `status=available`, `active=false` and `will_download_models=false`.
|
|
|
|
## What was tested
|
|
|
|
- Tower training command:
|
|
- `docker exec -e OPERATOR_YOLO_DATASET_DIR=/app/storage/operator-data/yolo-building-aoi1024-visible050-minpx8 -e YOLO_BASE_MODEL_PATH=/app/models/yolov8s.pt -e TRAIN_OUTPUT_DIR=/app/storage/training/operator-yolo -e TRAIN_RUN_NAME=geointel-building-yolov8s-aoi1024clean512e50 -e TRAIN_MODEL_OUTPUT_PATH=/app/models/geointel-building-yolov8s-aoi1024clean512e50.pt -e TRAIN_EPOCHS=50 -e TRAIN_IMGSZ=512 -e TRAIN_BATCH=2 -e TRAIN_WORKERS=0 -e TRAIN_DEVICE=cpu geointel bash /app/scripts/train_operator_yolo_detector.sh`
|
|
- Full seven-reference AOI1024 persisted positive QA matrix:
|
|
- command used `scripts/run_multi_sample_detection_quality_matrix.sh` with manifest `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/operator_samples_manifest.json`, samples `geel mol turnhout herentals balen retie westerlo`, model `geointel-building-yolov8s-aoi1024clean512e50-pt`, tile `512`, overlap `64`, thresholds `0.25 0.15 0.05` and IoU threshold `0.25`.
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/aoi1024clean512e50-full/multi_sample_quality_summary.json`
|
|
- threshold `0.05`: 7 positive samples, mean F1 `0.4723513253430784`, mean precision `0.44861169484563357`, mean recall `0.5039340859731523`, min F1 `0.4442082890541977`.
|
|
- threshold `0.15`: 7 positive samples, mean F1 `0.4753322215541376`, mean precision `0.6673270868402211`, mean recall `0.37207511021815726`, min F1 `0.4055555555555556`.
|
|
- threshold `0.25`: 7 positive samples, mean F1 `0.31021575247724875`, mean precision `0.8163316419273472`, mean recall `0.19608909809008596`, min F1 `0.16430903155603915`.
|
|
- Full nine-sample hard-negative/background matrix:
|
|
- command used `scripts/run_operator_hard_negative_detection_matrix.sh` with the same AOI1024 manifest, model `geointel-building-yolov8s-aoi1024clean512e50-pt`, tile `512`, overlap `64` and thresholds `0.25 0.15 0.05`.
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/aoi1024clean512e50-full/hard_negative_matrix_summary.json`
|
|
- threshold `0.05`: total background detections `488`, max sample detections `137`.
|
|
- threshold `0.15`: total background detections `269`, max sample detections `75`.
|
|
- threshold `0.25`: total background detections `205`, max sample detections `55`.
|
|
- Promotion report:
|
|
- command used `scripts/build_detection_model_promotion_report.py` with `--min-positive-samples 7`, `--min-background-samples 9`, `--min-mean-f1 0.25` and `--max-background-detections-per-sample 0`.
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/aoi1024clean512e50-full/detection_model_promotion_report.md`
|
|
- recommendation: `none`.
|
|
|
|
## Known limitations
|
|
|
|
- The clean-label model is materially stronger on positive reference AOIs than the previous AOI1024-visible025 candidate, but it is still not safe as a V1 default because hard-negative/background false positives remain high in mixed wooded AOIs.
|
|
- The likely bottleneck has shifted from label-file integrity to discriminating buildings from visually similar background structures and wooded-edge artifacts.
|
|
- The model remains inactive. No default model, API contract, migration, provider fetching or download behavior changed.
|
|
|
|
## Next recommended pass
|
|
|
|
- Build a background-aware training pass instead of another positive-only clean-label pass: combine the `visible050-minpx8` positives with stronger unique hard-negative/background retention, then re-run the same seven-positive/nine-background promotion gate before any activation.
|
|
|
|
# Sprint 154 - Background-aware AOI1024 YOLOv8s candidate gate
|
|
|
|
## What changed
|
|
|
|
- Built a background-aware AOI1024 tile dataset on Tower from the clean positive labels plus explicit background retention:
|
|
- dataset path: `/app/storage/operator-data/yolo-building-aoi1024-bgaware512r3`
|
|
- audit path: `/mnt/user/appdata/geointel/storage/operator-data/yolo-building-aoi1024-bgaware512r3-audit/operator_yolo_dataset_quality_audit.json`
|
|
- tile count: `162`
|
|
- positive tiles: `117`
|
|
- negative tiles: `45`
|
|
- label count: `21530`
|
|
- background negative repeat: `3`
|
|
- audit status: `ok`
|
|
- warnings: `[]`
|
|
- Trained Tower-local inactive model asset:
|
|
- model path: `/app/models/geointel-building-yolov8s-aoi1024bg512r3e50.pt`
|
|
- asset id: `geointel-building-yolov8s-aoi1024bg512r3e50-pt`
|
|
- SHA256: `e0980572aac90e7efc514608eb16d7de5bfbf27a4bbec04e7bc1bc8c02f9601f`
|
|
- training summary: `/app/storage/training/operator-yolo/geointel-building-yolov8s-aoi1024bg512r3e50/training_summary.json`
|
|
- base model: `/app/models/yolov8s.pt`
|
|
- epochs: `50`
|
|
- image size: `512`
|
|
- batch size: `2`
|
|
- device: CPU
|
|
- final Ultralytics validation on `best.pt`: precision `0.440`, recall `0.362`, mAP50 `0.251`, mAP50-95 `0.0878`.
|
|
- Verified the live model catalog reports the candidate as `status=available`, `active=false` and `will_download_models=false`.
|
|
- No API contract, migration, provider fetching, fake detection data, model download behavior or active model default changed.
|
|
|
|
## What was tested
|
|
|
|
- Tower training command:
|
|
- `docker exec -e OPERATOR_YOLO_DATASET_DIR=/app/storage/operator-data/yolo-building-aoi1024-bgaware512r3 -e YOLO_BASE_MODEL_PATH=/app/models/yolov8s.pt -e TRAIN_OUTPUT_DIR=/app/storage/training/operator-yolo -e TRAIN_RUN_NAME=geointel-building-yolov8s-aoi1024bg512r3e50 -e TRAIN_MODEL_OUTPUT_PATH=/app/models/geointel-building-yolov8s-aoi1024bg512r3e50.pt -e TRAIN_EPOCHS=50 -e TRAIN_IMGSZ=512 -e TRAIN_BATCH=2 -e TRAIN_WORKERS=0 -e TRAIN_DEVICE=cpu geointel bash /app/scripts/train_operator_yolo_detector.sh`
|
|
- Full seven-reference AOI1024 persisted positive QA matrix:
|
|
- command used `scripts/run_multi_sample_detection_quality_matrix.sh` with manifest `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/operator_samples_manifest.json`, samples `geel mol turnhout herentals balen retie westerlo`, model `geointel-building-yolov8s-aoi1024bg512r3e50-pt`, tile `512`, overlap `64`, thresholds `0.25 0.15 0.05` and IoU threshold `0.25`.
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-full/multi_sample_quality_summary.json`
|
|
- threshold `0.05`: 7 positive samples, mean F1 `0.4908049127242224`, mean precision `0.468604`, mean recall `0.520016`, total detections `15277`.
|
|
- threshold `0.15`: 7 positive samples, mean F1 `0.5074022485589402`, mean precision `0.636639`, mean recall `0.424258`, total detections `9186`.
|
|
- threshold `0.25`: 7 positive samples, mean F1 `0.44378879337957716`, mean precision `0.762683`, mean recall `0.316314`, total detections `5584`.
|
|
- Conservative seven-reference AOI1024 persisted positive QA matrix:
|
|
- command used the same manifest/model/tile/overlap with thresholds `0.35 0.45 0.60`.
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-high-threshold/multi_sample_quality_summary.json`
|
|
- threshold `0.35`: 7 positive samples, mean F1 `0.32086574003576274`, mean precision `0.840006`, mean recall `0.202135`, total detections `3106`.
|
|
- threshold `0.45`: 7 positive samples, mean F1 `0.20093433626447627`, mean precision `0.885043`, mean recall `0.116348`, total detections `1592`.
|
|
- threshold `0.60`: 7 positive samples, mean F1 `0.06640476239173489`, mean precision `0.871611`, mean recall `0.035083`, total detections `418`.
|
|
- Full nine-sample hard-negative/background matrix:
|
|
- command used `scripts/run_operator_hard_negative_detection_matrix.sh` with the AOI1024 manifest, model `geointel-building-yolov8s-aoi1024bg512r3e50-pt`, tile `512`, overlap `64` and thresholds `0.25 0.15 0.05`.
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/aoi1024bg512r3e50-full/hard_negative_matrix_summary.json`
|
|
- threshold `0.05`: total background detections `638`, max sample detections `184`.
|
|
- threshold `0.15`: total background detections `390`, max sample detections `103`.
|
|
- threshold `0.25`: total background detections `278`, max sample detections `75`.
|
|
- Conservative nine-sample hard-negative/background matrix:
|
|
- output: `/mnt/user/appdata/geointel/artifacts/detection-hard-negatives/aoi1024bg512r3e50-high-threshold/hard_negative_matrix_summary.json`
|
|
- threshold `0.35`: total background detections `198`, max sample detections `55`.
|
|
- threshold `0.45`: total background detections `148`, max sample detections `38`.
|
|
- threshold `0.60`: total background detections `76`, max sample detections `18`.
|
|
- Promotion reports:
|
|
- balanced thresholds report: `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/aoi1024bg512r3e50-full/detection_model_promotion_report.md`
|
|
- conservative thresholds report: `/mnt/user/appdata/geointel/artifacts/detection-model-promotion/aoi1024bg512r3e50-high-threshold/detection_model_promotion_report.md`
|
|
- recommendation: `none`.
|
|
|
|
## Known limitations
|
|
|
|
- The background-aware model is the strongest positive-AOI candidate so far and is usable as an explicit review/demo candidate, especially at threshold `0.15` for balance or `0.35` for conservative high-precision review.
|
|
- It is still not safe as a V1 default because the full background-candidate gate fails `background_false_positive_pressure`.
|
|
- Some `background_candidate` AOIs contain GRB reference buildings and should be split into pure-empty negatives versus sparse-positive contextual samples before the next gate; otherwise the hard-negative score remains intentionally conservative but not perfectly diagnostic.
|
|
- The model remains inactive. Operators can select it explicitly from the local model asset catalog.
|
|
|
|
## Next recommended pass
|
|
|
|
- Build a V1 operator detection profile layer in the UI/docs: expose `balanced` (`threshold=0.15`) and `conservative review` (`threshold=0.35`) as explicit choices for local model assets, with clear warning that the model is not a default-approved detector.
|
|
- Clean the background corpus classification: separate pure-empty AOIs from sparse-building contextual AOIs, then retrain or recalibrate against that cleaner gate.
|
|
|
|
# Sprint 155 - Detection operator profiles
|
|
|
|
## What changed
|
|
|
|
- Added `frontend/src/components/detection/detectionProfiles.ts` with explicit operator profiles for the inactive `geointel-building-yolov8s-aoi1024bg512r3e50-pt` local model asset.
|
|
- Exposed two deliberate Detection Lab actions:
|
|
- `balanced-review`: confidence threshold `0.15`, positive-AOI F1 `0.5074022485589402`, precision `0.636639`, recall `0.424258`, max background detections `103`.
|
|
- `conservative-review`: confidence threshold `0.35`, positive-AOI F1 `0.32086574003576274`, precision `0.840006`, recall `0.202135`, max background detections `55`.
|
|
- Applying a profile selects `yolo-configured`, the local model asset id and the profile threshold. It does not auto-select assets on model catalog load and does not promote the candidate as a default detector.
|
|
- Detection Lab now marks both profiles as `Candidate only - not default-approved` because the promotion recommendation remains `none`.
|
|
- Updated frontend, AI pipeline and TODO documentation.
|
|
|
|
## What was tested
|
|
|
|
- Added regression coverage in `backend/tests/test_sprint155_detection_operator_profiles.py`.
|
|
- Ran `python -m pytest tests/test_sprint155_detection_operator_profiles.py tests/test_sprint122_model_asset_activation_guardrails.py -q`.
|
|
- Ran `python -m pytest` in `backend`: 435 passed.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `cd frontend && npm run typecheck`.
|
|
- Ran `cd frontend && npm run build`.
|
|
- Ran `bash scripts/run_readiness_check.sh`.
|
|
- Ran `cd backend && python -m alembic heads` and `cd backend && python -m alembic upgrade head --sql`.
|
|
- Ran `bash -n scripts/live_migration_smoke.sh`.
|
|
|
|
## Known limitations
|
|
|
|
- The profiles are review/demo aids only. The background corpus still needs to be split into pure-empty negatives and sparse-building contextual AOIs before retraining or recalibrating for a default detector decision.
|
|
- No backend API contract, migration, provider fetching, fake detection output, model download behavior or active runtime default changed.
|
|
|
|
# Sprint 156 - Background corpus classification
|
|
|
|
## What changed
|
|
|
|
- Added explicit background category classification to operator sample preparation:
|
|
- `pure_empty_negative` when a background candidate has zero GRB reference buildings.
|
|
- `sparse_building_context` when a background candidate has one or more GRB reference buildings.
|
|
- `reference_aoi` for normal positive reference samples.
|
|
- Persisted `background_category` into generated operator sample manifests and reference GeoJSON metadata.
|
|
- Added `OPERATOR_BACKGROUND_CATEGORIES` to `scripts/run_operator_hard_negative_detection_matrix.sh` so the strict default-promotion hard-negative gate can run only on `pure_empty_negative` samples, while `sparse_building_context` samples can be reviewed separately.
|
|
- Preserved `background_category` in YOLO tile export metadata so negative-tile provenance survives training dataset audits.
|
|
- Updated operator pipeline docs, TODO and changelog.
|
|
|
|
## What was tested
|
|
|
|
- Added regression coverage in `backend/tests/test_sprint156_background_corpus_classification.py`.
|
|
- Ran `python -m pytest tests/test_sprint156_background_corpus_classification.py -q`.
|
|
- Ran `python -m pytest tests/test_sprint156_background_corpus_classification.py tests/test_sprint131_operator_sample_expansion.py tests/test_sprint132_operator_hard_negative_matrix.py tests/test_sprint130_operator_yolo_tile_dataset.py -q`: 17 passed.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 439 passed.
|
|
- Ran `cd frontend && npm run typecheck`.
|
|
- Ran `cd frontend && npm run build`.
|
|
- Ran `bash scripts/run_readiness_check.sh`.
|
|
- Ran `cd backend && python -m alembic heads` and `cd backend && python -m alembic upgrade head --sql`.
|
|
- Ran `bash -n scripts/live_migration_smoke.sh` and `bash -n scripts/run_operator_hard_negative_detection_matrix.sh`.
|
|
|
|
## Known limitations
|
|
|
|
- This pass adds the cleaner corpus/gate contract only. It does not regenerate Tower manifests, retrain YOLO, rerun the live hard-negative matrices or change any model default.
|
|
- No backend API contract, database migration, provider fetching, fake detection output or model download behavior changed.
|
|
|
|
## Next recommended pass
|
|
|
|
- Redeploy/rebuild the runtime scripts, regenerate the operator sample manifest, then run:
|
|
- `OPERATOR_BACKGROUND_CATEGORIES="pure_empty_negative"` for the strict default-promotion false-positive gate.
|
|
- `OPERATOR_BACKGROUND_CATEGORIES="sparse_building_context"` for contextual review evidence.
|
|
- Retrain or recalibrate the inactive AOI1024 local model candidate only after those two matrices are available.
|
|
|
|
# Sprint 157 - Background split matrix runner
|
|
|
|
## What changed
|
|
|
|
- Added `scripts/run_background_corpus_split_matrix.sh` as the operator wrapper for the next Tower run.
|
|
- The wrapper runs `scripts/run_operator_hard_negative_detection_matrix.sh` twice:
|
|
- `OPERATOR_BACKGROUND_CATEGORIES="pure_empty_negative"` for the strict default-promotion false-positive gate.
|
|
- `OPERATOR_BACKGROUND_CATEGORIES="sparse_building_context"` for review-only contextual evidence.
|
|
- Added `scripts/build_background_corpus_split_report.py` to combine both summaries into:
|
|
- `background_corpus_split_summary.json`
|
|
- `background_corpus_split_summary.md`
|
|
- The combined report records `strict_default_gate`, `context_review`, `passes_zero_detection_gate`, max detection counts and the recommended next step.
|
|
- Added readiness coverage for the new Python and Bash scripts.
|
|
- Updated operator pipeline docs, TODO and changelog.
|
|
|
|
## What was tested
|
|
|
|
- Added regression coverage in `backend/tests/test_sprint157_background_split_matrix_runner.py`.
|
|
- Ran `python -m pytest tests/test_sprint157_background_split_matrix_runner.py -q`.
|
|
- Ran `python -m pytest tests/test_sprint157_background_split_matrix_runner.py tests/test_sprint156_background_corpus_classification.py tests/test_sprint132_operator_hard_negative_matrix.py -q`: 9 passed.
|
|
- Ran `python -m py_compile scripts/build_background_corpus_split_report.py`.
|
|
- Ran `bash -n scripts/run_background_corpus_split_matrix.sh`.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 443 passed.
|
|
- Ran `cd frontend && npm run typecheck`.
|
|
- Ran `cd frontend && npm run build`.
|
|
- Ran `bash scripts/run_readiness_check.sh`.
|
|
- Ran `cd backend && python -m alembic heads` and `cd backend && python -m alembic upgrade head --sql`.
|
|
- Ran `bash -n scripts/live_migration_smoke.sh` and `bash -n scripts/run_background_corpus_split_matrix.sh`.
|
|
|
|
## Known limitations
|
|
|
|
- This pass adds orchestration/report tooling only. It does not run live inference on Tower, retrain YOLO, rerun the split matrices or change any model default.
|
|
- No backend API contract, database migration, provider fetching, fake detection output or model download behavior changed.
|
|
|
|
## Next recommended pass
|
|
|
|
- Rebuild/redeploy the runtime, regenerate the operator manifest if needed, then run `scripts/run_background_corpus_split_matrix.sh` against `http://192.168.10.150:1202`.
|
|
- Use the emitted split report to decide whether to retrain, recalibrate thresholds or keep the AOI1024 candidate operator-only.
|
|
|
|
# Sprint 158 - Split-aware promotion report
|
|
|
|
## What changed
|
|
|
|
- Added `--background-split-summary` support to `scripts/build_detection_model_promotion_report.py`.
|
|
- The promotion report now resolves a split report's `pure_empty_negative` source summary as the strict default-promotion background gate.
|
|
- The same report records `sparse_building_context` as review-only evidence, including source path and detection-pressure context, without counting it as a default-promotion blocker.
|
|
- Kept direct `--hard-negative-summary` support unchanged for older operator workflows.
|
|
- Updated operator pipeline docs, TODO and changelog.
|
|
|
|
## What was tested
|
|
|
|
- Red step: `python -m pytest tests/test_sprint158_promotion_report_split_background.py -q` failed because the promotion report required `--hard-negative-summary` and did not yet accept `--background-split-summary`.
|
|
- Ran `python -m pytest tests/test_sprint158_promotion_report_split_background.py -q`: 1 passed.
|
|
- Ran `python -m pytest tests/test_sprint143_detection_model_promotion_report.py tests/test_sprint157_background_split_matrix_runner.py -q`: 7 passed.
|
|
- Ran `python -m py_compile scripts/build_detection_model_promotion_report.py scripts/build_background_corpus_split_report.py`.
|
|
- Ran `python -m pytest tests/test_sprint158_promotion_report_split_background.py tests/test_sprint143_detection_model_promotion_report.py tests/test_sprint157_background_split_matrix_runner.py -q`: 8 passed.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 444 passed, 17 existing Pydantic namespace warnings.
|
|
- Ran `cd frontend && npm run typecheck`.
|
|
- Ran `cd frontend && npm run build`.
|
|
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
|
- Ran `cd backend && python -m alembic heads`: `202606120900 (head)`.
|
|
- Ran `cd backend && python -m alembic upgrade head --sql`.
|
|
- Ran `bash -n scripts/live_migration_smoke.sh`.
|
|
|
|
## Known limitations
|
|
|
|
- This pass is report/tooling only. It does not run live split matrices on Tower, retrain YOLO, change a model default, change API contracts, change migrations, fetch providers, download model weights or create fake detections.
|
|
- The AOI1024 background-aware local model remains explicit operator-review only until the split matrices plus positive-AOI QA/QC evidence pass the documented gates.
|
|
|
|
## Next recommended pass
|
|
|
|
- After redeploy, run `scripts/run_background_corpus_split_matrix.sh` on Tower for `geointel-building-yolov8s-aoi1024bg512r3e50-pt`.
|
|
- Feed the generated `background_corpus_split_summary.json` into `scripts/build_detection_model_promotion_report.py --background-split-summary` together with the seven-AOI positive summary.
|
|
- If pure-empty false-positive pressure still fails, retrain or recalibrate before any default activation. If pure-empty passes, inspect sparse-context review evidence before deciding whether to keep the model operator-only or prepare a guarded default-candidate decision.
|
|
|
|
# Sprint 159 - Split-background promotion workflow wrapper
|
|
|
|
## What changed
|
|
|
|
- Added `scripts/run_split_background_promotion_workflow.sh`.
|
|
- The wrapper runs `scripts/run_background_corpus_split_matrix.sh`, verifies the emitted `background_corpus_split_summary.json`, then calls `scripts/build_detection_model_promotion_report.py --background-split-summary`.
|
|
- The wrapper requires `PROMOTION_POSITIVE_PORTFOLIO_PATH` and exposes promotion gate overrides through environment variables.
|
|
- Added readiness syntax coverage for the new wrapper.
|
|
- Updated operator pipeline docs, TODO and changelog.
|
|
|
|
## What was tested
|
|
|
|
- Red step: `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q` failed because the wrapper script and readiness syntax check did not exist.
|
|
- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q`: 2 passed.
|
|
- Ran `bash -n scripts/run_split_background_promotion_workflow.sh`.
|
|
- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py tests/test_sprint158_promotion_report_split_background.py tests/test_sprint157_background_split_matrix_runner.py -q`: 7 passed.
|
|
- Ran `bash -n scripts/run_readiness_check.sh`.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 446 passed, 17 existing Pydantic namespace warnings.
|
|
- Ran `cd frontend && npm run typecheck`.
|
|
- Ran `cd frontend && npm run build`.
|
|
- Ran `cd backend && python -m alembic heads`: `202606120900 (head)`.
|
|
- Ran `cd backend && python -m alembic upgrade head --sql`.
|
|
- Ran `bash -n scripts/live_migration_smoke.sh`.
|
|
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
|
|
|
## Known limitations
|
|
|
|
- This pass adds operator orchestration only. It does not run live inference locally, retrain YOLO, activate a model default, change backend API contracts, change migrations, fetch providers, fetch weights or create fake detections.
|
|
- `http://192.168.10.150:1202` is reachable from this workstation, but `ssh -p 222 root@192.168.10.150` returned `Permission denied (publickey)`. Port `222` appears to be the Gitea SSH endpoint rather than an Unraid shell endpoint, so Docker rebuild/restart still needs the existing Unraid deployment path or a separate shell-access route.
|
|
|
|
## Next recommended pass
|
|
|
|
- Redeploy/pull commit on Tower through the existing Gitea/Unraid flow.
|
|
- Run:
|
|
- `PROMOTION_POSITIVE_PORTFOLIO_PATH=... bash scripts/run_split_background_promotion_workflow.sh http://192.168.10.150:1202`
|
|
- Review the generated split-aware promotion report before any retraining, recalibration or default-model decision.
|
|
|
|
# Sprint 160 - Split-background promotion preflight
|
|
|
|
## What changed
|
|
|
|
- Added `--preflight-only` to `scripts/run_split_background_promotion_workflow.sh`.
|
|
- Preflight now checks:
|
|
- `PROMOTION_POSITIVE_PORTFOLIO_PATH` exists and is readable.
|
|
- `OPERATOR_SAMPLE_MANIFEST_PATH` exists and is readable.
|
|
- the manifest contains both `pure_empty_negative` and `sparse_building_context` background categories.
|
|
- Python and `curl` are available.
|
|
- the runtime frontend API proxy returns the canonical `data` envelope from `/api/v1/projects`.
|
|
- Updated operator docs, TODO and changelog with the quick post-redeploy preflight command.
|
|
|
|
## What was tested
|
|
|
|
- Red step: `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q` failed because `--preflight-only` was not yet present.
|
|
- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q`: 3 passed.
|
|
- Ran `bash -n scripts/run_split_background_promotion_workflow.sh`.
|
|
- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py tests/test_sprint158_promotion_report_split_background.py -q`: 4 passed.
|
|
- Ran `curl.exe -fsS http://192.168.10.150:1202/api/v1/projects`: runtime API proxy returned a canonical `data` envelope.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 447 passed, 17 existing Pydantic namespace warnings.
|
|
- Ran `cd frontend && npm run typecheck`.
|
|
- Ran `cd frontend && npm run build`.
|
|
- Ran `cd backend && python -m alembic heads`: `202606120900 (head)`.
|
|
- Ran `cd backend && python -m alembic upgrade head --sql`.
|
|
- Ran `bash -n scripts/live_migration_smoke.sh`.
|
|
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
|
|
|
## Known limitations
|
|
|
|
- Preflight is intentionally non-mutating and does not run inference, create datasets, retrain models, activate defaults, fetch providers, fetch weights or change API/database behavior.
|
|
- Full split-background matrix execution still requires redeploying this commit to the Tower/Unraid runtime and running the wrapper in the runtime checkout where the positive portfolio and operator manifest artifacts exist.
|
|
|
|
## Next recommended pass
|
|
|
|
- Redeploy/pull on Tower, then run:
|
|
- `PROMOTION_POSITIVE_PORTFOLIO_PATH=... OPERATOR_SAMPLE_MANIFEST_PATH=... bash scripts/run_split_background_promotion_workflow.sh --preflight-only http://192.168.10.150:1202`
|
|
- If preflight passes, rerun without `--preflight-only` to generate the split summary and split-aware promotion report.
|
|
|
|
# Sprint 161 - Widescreen workbench support
|
|
|
|
## What changed
|
|
|
|
- Added dedicated frontend CSS breakpoints for `1800px` and `2200px` workbench widths.
|
|
- Wide screens now expand the workbench shell columns, inspector padding, Data/Analysis/AI/Export grids and Map workspace controls without changing workflows.
|
|
- Ultrawide screens prioritize the MapLibre review frame with a taller map and wider toolbar/inspection layout.
|
|
- Added static regression coverage for widescreen/ultrawide layout contracts.
|
|
- Updated frontend README, TODO and changelog.
|
|
|
|
## What was tested
|
|
|
|
- Red step: `python -m pytest tests/test_sprint161_widescreen_workbench.py -q` failed before the widescreen CSS contracts existed.
|
|
- Ran `python -m pytest tests/test_sprint161_widescreen_workbench.py tests/test_sprint53_selection_ergonomics.py tests/test_sprint85_map_workspace_density.py tests/test_sprint113_calm_workbench_layout.py -q`: 13 passed.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 449 passed, 17 existing Pydantic protected-namespace warnings.
|
|
- Ran `npm run typecheck` in `frontend`.
|
|
- Ran `npm run build` in `frontend`.
|
|
- Ran `python -m alembic heads` in `backend`: `202606120900 (head)`.
|
|
- Ran `python -m alembic upgrade head --sql` in `backend`.
|
|
- Ran `bash -n scripts/live_migration_smoke.sh`.
|
|
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
|
- Ran a local Vite preview visual/layout check at `2560x1080` through the in-app browser. The workbench measured no horizontal overflow, shell columns `240px 1889px 416px`, and the Map workspace rendered an `832px` high MapLibre frame with the road basemap visible.
|
|
|
|
## Known limitations
|
|
|
|
- Local `docker compose config` could not be run on this Windows workstation because the Docker CLI is not installed or not on `PATH`.
|
|
- The live Tower/Unraid runtime at `http://192.168.10.150:1202` will show the widescreen changes only after redeploying this commit.
|
|
- The local preview showed an expected API error because it served the built frontend without the Docker backend proxy; the layout and MapLibre basemap rendered correctly.
|
|
|
|
## Next recommended pass
|
|
|
|
- Redeploy/pull this commit on Tower/Unraid and visually recheck `http://192.168.10.150:1202` on the actual widescreen monitor.
|
|
- Continue with the split-background promotion preflight/matrix flow once the runtime has been rebuilt.
|
|
|
|
# Sprint 161 - Tower redeploy and live widescreen runtime verification
|
|
|
|
## What changed
|
|
|
|
- Redeployed Tower/Unraid from commit `07ea8db` with the existing all-in-one deploy path.
|
|
- Preserved the remote AI runtime setting by deploying with `GEOINTEL_INSTALL_AI=true`.
|
|
- The deployed `geointel` container is healthy and published on `0.0.0.0:1202->80/tcp`.
|
|
- No application code, API contract, migration, provider fetching, model default, model download behavior or fake data path changed in this pass.
|
|
|
|
## What was tested
|
|
|
|
- Verified Tower SSH access through `root@192.168.10.150` with the `widefrog_unraid_deploy` key.
|
|
- Verified the remote checkout reset to `07ea8db`.
|
|
- Ran the Tower deploy script with AI dependencies enabled.
|
|
- Deploy build produced frontend asset `assets/index-CxHabIo7.css`; the live CSS contains the Sprint 161 wide/ultrawide layout rules including `15rem minmax(0,1fr) 26rem` and `min-height:42rem`.
|
|
- Live migration smoke passed inside the `geointel` container:
|
|
- database connection ok
|
|
- PostGIS `3.6`
|
|
- database collation ok
|
|
- required runtime schema objects ok
|
|
- Alembic head `202606120900`
|
|
- Browser runtime smoke passed against `http://192.168.10.150:1202`.
|
|
- GIS runtime smoke passed against `http://192.168.10.150:1202`.
|
|
- Health endpoint returned `status=ok`, `database=ok`.
|
|
- YOLO preflight returned:
|
|
- `dependencies_available=true`
|
|
- `model_file_exists=true`
|
|
- `torch_version=2.13.0`
|
|
- `ultralytics_version=8.4.92`
|
|
- `will_download_models=false`
|
|
- `will_run_inference=false` without a tile manifest
|
|
|
|
## Known limitations
|
|
|
|
- The optional local screenshot script could not run in this Windows runner because Node cannot import Playwright here. The script remains optional and reports the documented install commands.
|
|
- The in-app browser could not attach a fresh browser-use tab during this pass, so the final visual confirmation is based on deploy smoke, live CSS verification and runtime API/GIS/AI checks rather than a new inline screenshot.
|
|
|
|
## Next recommended pass
|
|
|
|
- Manually refresh the already-open `http://192.168.10.150:1202` browser tab on the actual widescreen monitor to observe the new layout.
|
|
- Continue with the split-background promotion preflight/matrix workflow now that Tower is rebuilt with the latest frontend and AI runtime.
|
|
|
|
# Sprint 162 - Split-background promotion runtime pass
|
|
|
|
## What changed
|
|
|
|
- Hardened `scripts/run_split_background_promotion_workflow.sh` preflight so legacy operator manifests without explicit `background_category` are handled consistently with `run_operator_hard_negative_detection_matrix.sh`.
|
|
- Missing background categories are now derived from `reference_feature_count`: `0` becomes `pure_empty_negative`, and background samples with references become `sparse_building_context`.
|
|
- Added a regression test for the derived-category preflight path.
|
|
- Updated operator docs and changelog. No API contract, migration, provider fetching, model default, fake output or model download behavior changed.
|
|
|
|
## What was tested locally
|
|
|
|
- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q`: 4 passed.
|
|
- Ran `python -m pytest tests/test_sprint157_background_split_matrix_runner.py tests/test_sprint158_promotion_report_split_background.py tests/test_sprint159_split_promotion_workflow.py -q`: 9 passed.
|
|
- Ran `bash -n scripts/run_split_background_promotion_workflow.sh`.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 450 passed, 17 existing Pydantic protected-namespace warnings.
|
|
- Ran `npm run typecheck` in `frontend`.
|
|
- Ran `npm run build` in `frontend`.
|
|
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
|
|
|
## Tower runtime evidence
|
|
|
|
- Pushed commit `a499c5f` and fast-forwarded the Tower checkout.
|
|
- Ran split-background preflight against `http://192.168.10.150:1202` with:
|
|
- `PROMOTION_POSITIVE_PORTFOLIO_PATH=artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-full/multi_sample_quality_summary.json`
|
|
- `OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator-samples-1024/operator_samples_manifest.json`
|
|
- result: passed.
|
|
- Ran the full low/mid-threshold split workflow into:
|
|
- background split: `artifacts/detection-hard-negatives/background-split/aoi1024bg512r3e50-split-20260710T222641Z`
|
|
- promotion report: `artifacts/detection-model-promotion/split-aware/aoi1024bg512r3e50-split-20260710T222641Z/detection_model_promotion_report.json`
|
|
- result: no recommended candidate because `0.15` produced 46 pure-empty detections on `postel_bos`, while `0.35` had no matching positive evidence in that portfolio.
|
|
- Ran high-threshold preflight and full split workflow using:
|
|
- `PROMOTION_POSITIVE_PORTFOLIO_PATH=artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-high-threshold/multi_sample_quality_summary.json`
|
|
- `QUALITY_THRESHOLDS="0.35 0.45 0.6"`
|
|
- background split: `artifacts/detection-hard-negatives/background-split/aoi1024bg512r3e50-high-threshold-split-20260710T222934Z`
|
|
- promotion report: `artifacts/detection-model-promotion/split-aware/aoi1024bg512r3e50-high-threshold-split-20260710T222934Z/detection_model_promotion_report.json`
|
|
- High-threshold strict pure-empty gate passed:
|
|
- 3 pure-empty samples: `arendonk_heide`, `lommel_heide`, `postel_bos`
|
|
- 9 pure-empty runs
|
|
- total detections: `0`
|
|
- max detections per sample/threshold run: `0`
|
|
- Recommended candidate:
|
|
- `geointel-building-yolov8s-aoi1024bg512r3e50-pt|512|64|0.35`
|
|
- positive samples: `7`
|
|
- pure-empty background samples: `3`
|
|
- mean F1: `0.32086574003576274`
|
|
- mean precision: `0.8400057773951873`
|
|
- mean recall: `0.20213514285308795`
|
|
- max pure-empty detections: `0`
|
|
- promotion status: `promote_candidate`
|
|
- Sparse-context review remained review-only:
|
|
- 6 sparse-context samples
|
|
- 18 sparse-context runs
|
|
- total detections: `422`
|
|
- max sparse-context detections: `55`
|
|
|
|
## Known limitations
|
|
|
|
- The recommended candidate has conservative recall (`0.2021`) at threshold `0.35`; it is suitable as an operator-review candidate, not as an automatically activated production default.
|
|
- The model default was not changed. Activation should remain a separate explicit operator decision after reviewing the high-threshold report.
|
|
|
|
## Next recommended pass
|
|
|
|
- Add a guarded model activation/operator-selection workflow that can mark a promoted candidate as active only after the report artifact and candidate key are explicitly supplied.
|
|
|
|
# Sprint 163 - Guarded promoted YOLO activation workflow
|
|
|
|
## What changed
|
|
|
|
- Added `scripts/activate_promoted_yolo_candidate.py`.
|
|
- The helper validates:
|
|
- the promotion report file exists and is valid JSON;
|
|
- the exact supplied `candidate_key` matches the report recommended candidate;
|
|
- the candidate has `promotion_status=promote_candidate`;
|
|
- positive sample count, background sample count, mean F1 and max background detections still satisfy report gates;
|
|
- the candidate `model_asset_id` resolves to an existing local model file under the mounted models directory.
|
|
- The helper emits `.env` updates in dry-run mode by default and writes them only when `--apply` is supplied.
|
|
- Updated Detection Lab operator profiles:
|
|
- `balanced-review` at threshold `0.15` remains candidate-only because pure-empty false-positive pressure failed.
|
|
- `conservative-review` at threshold `0.35` is marked as promoted/default-approved based on the split-background pure-empty gate.
|
|
- Added docs for the guarded activation command in `docs/AI_PIPELINES.md`, `scripts/README.md`, `backend/README.md` and `frontend/README.md`.
|
|
- Added readiness coverage for compiling the new helper.
|
|
- No API contract, database migration, provider fetching, fake detection path, model file mutation, model download or automatic runtime activation was introduced in code.
|
|
|
|
## What was tested locally
|
|
|
|
- RED: `python -m pytest tests/test_sprint162_promoted_model_activation.py -q` failed while `scripts/activate_promoted_yolo_candidate.py` was absent.
|
|
- RED: `python -m pytest tests/test_sprint155_detection_operator_profiles.py -q` failed before `conservative-review` was marked promoted.
|
|
- RED: `python -m pytest tests/test_sprint162_promoted_model_activation.py::test_readiness_gate_compiles_promoted_activation_script -q` failed before readiness compiled the helper.
|
|
- Ran `python -m pytest tests/test_sprint162_promoted_model_activation.py tests/test_sprint155_detection_operator_profiles.py -q`: 7 passed.
|
|
- Ran `python -m py_compile scripts/activate_promoted_yolo_candidate.py`.
|
|
- Ran `python -m compileall backend/app`.
|
|
- Ran `python -m pytest` in `backend`: 454 passed, 17 existing Pydantic protected-namespace warnings.
|
|
- Ran `npm run typecheck` in `frontend`.
|
|
- Ran `npm run build` in `frontend`.
|
|
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
|
|
|
## Known limitations
|
|
|
|
- The helper updates runtime environment only; a container restart or rebuild is still required for `YOLO_MODEL_PATH` changes to take effect.
|
|
- The promoted threshold is represented in the operator profile and promotion report. The backend detection endpoint still requires clients to submit the intended confidence threshold explicitly.
|
|
|
|
## Next recommended pass
|
|
|
|
- Push this helper to Tower, run it first as dry-run against the high-threshold promotion report, then apply and redeploy/restart only if the emitted `YOLO_MODEL_PATH` matches the promoted local asset.
|
|
|
|
# Sprint 164 - Live operational YOLO detection and QA smoke
|
|
|
|
## What changed
|
|
|
|
- No product code, API contract, migration, model artifact, fake-data path or provider-fetching behavior changed.
|
|
- Ran the deployed all-in-one Tower runtime at `http://192.168.10.150:1202` through a real raster/reference detection and QA workflow using existing operator data and the local promoted YOLO model asset.
|
|
|
|
## Tower runtime evidence
|
|
|
|
- Runtime URL: `http://192.168.10.150:1202`
|
|
- Raster input: `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/geel_orthophoto_wms_1024.tif`
|
|
- Reference input: `/mnt/user/appdata/geointel/storage/operator-data/operator-samples-1024/geel_grb_gbg_buildings.geojson`
|
|
- Model asset: `geointel-building-yolov8s-aoi1024bg512r3e50-pt`
|
|
- Project: `865746ee-10ce-40a1-a3da-98b2182200e5`
|
|
- Raster dataset: `c87ed009-0bf0-4a34-adc5-51e6747d847b`
|
|
- Reference dataset: `d9bca687-20ca-4609-8c15-d24d240cfae6`
|
|
- Tile manifest: `/app/storage/tiles/865746ee-10ce-40a1-a3da-98b2182200e5/c87ed009-0bf0-4a34-adc5-51e6747d847b/b1335951-4ead-4e08-9c5f-08c67e026a8f/manifest.json`
|
|
- Analysis run: `45159e0b-36be-4300-8132-fef3a1e6b667`
|
|
- Persisted detections: `333`
|
|
- Quality check: `2e696dca-dea6-42d1-af2a-4894b182d427`
|
|
- Detection export: `339344f7-38e3-4558-b66b-459726051bac`
|
|
|
|
## Validation
|
|
|
|
- Ran `scripts/verify_real_data_detection_qa_workflow.sh` against the deployed Tower runtime: passed.
|
|
- Confirmed detection run status: `success`.
|
|
- Confirmed detection list endpoint returned `333` persisted detections with real source tile provenance.
|
|
- Confirmed detection GeoJSON endpoint returned a `FeatureCollection` with `333` persisted geometry features.
|
|
- Confirmed detection export content returned a `detection_geojson` `FeatureCollection` with `333` features.
|
|
- Confirmed QA/QC persisted metrics:
|
|
- precision: `0.21621621621621623`
|
|
- recall: `0.031746031746031744`
|
|
- F1: `0.05536332179930796`
|
|
- mean IoU: `0.5697275247203281`
|
|
- false positives: `261`
|
|
- false negatives: `2196`
|
|
- Confirmed QA evidence overlay endpoint returned a `FeatureCollection` with `2601` features and no warnings.
|
|
|
|
## Known limitations
|
|
|
|
- The model is operational and demonstrable, but the Geel smoke confirms low recall at the current conservative threshold. It should remain an operator-review detector, not an automated decision engine.
|
|
- Further training data quality work remains required before treating the detector as production-grade.
|
|
|
|
## Next recommended pass
|
|
|
|
- Add/curate more high-quality positive AOIs and cleaner building labels, then rerun the multi-AOI calibration and promotion gate before changing default operator thresholds.
|
|
|
|
# Sprint 165 - Clean AOI1024 YOLO dataset and promotion rejection
|
|
|
|
## What changed
|
|
|
|
- Hardened `scripts/export_operator_yolo_tile_dataset.py` so legacy operator manifests without explicit `background_category` now derive background categories consistently with the split-background evaluator:
|
|
- `reference_feature_count == 0` -> `pure_empty_negative`
|
|
- `reference_feature_count > 0` -> `sparse_building_context`
|
|
- normal reference samples -> `reference_aoi`
|
|
- Added regression coverage in:
|
|
- `backend/tests/test_sprint130_operator_yolo_tile_dataset.py`
|
|
- `backend/tests/test_sprint156_background_corpus_classification.py`
|
|
- Documented the clean AOI1024 export/audit profile in `scripts/README.md`.
|
|
- Updated `docs/TODO.md` with the clean dataset and rejected candidate state.
|
|
- No API contract, migration, default model activation, provider fetch path, model download behavior or fake detection path changed.
|
|
|
|
## Local validation
|
|
|
|
- RED: `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py::test_background_category_is_derived_for_legacy_operator_manifests -q` failed before the exporter helper existed.
|
|
- GREEN: same targeted test passed after adding `background_category_for_sample`.
|
|
- Ran `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py backend/tests/test_sprint157_background_split_matrix_runner.py -q`: 12 passed.
|
|
- Ran `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint156_background_corpus_classification.py -q`: 11 passed.
|
|
- Ran `bash scripts/run_readiness_check.sh`: 457 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
|
|
|
|
## Tower runtime evidence
|
|
|
|
- Pushed commit `b3bd34c` and redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`.
|
|
- Deploy validation passed:
|
|
- live migration smoke passed;
|
|
- browser runtime verification passed;
|
|
- container exposed `0.0.0.0:1202->80/tcp`.
|
|
- Regenerated clean AOI1024 tile dataset:
|
|
- dataset: `/app/storage/operator-data/yolo-building-aoi1024-cleanpx12vis035`
|
|
- tile count: `144`
|
|
- positive tiles: `114`
|
|
- negative tiles: `30`
|
|
- labels: `14632`
|
|
- `min_label_px=12`
|
|
- `min_label_visible_ratio=0.35`
|
|
- background categories in tile summary:
|
|
- `pure_empty_negative=27`
|
|
- `sparse_building_context=54`
|
|
- `reference_aoi=63`
|
|
- Dataset audit:
|
|
- report: `/app/artifacts/operator-yolo-dataset-audit/aoi1024-cleanpx12vis035/operator_yolo_dataset_quality_audit.json`
|
|
- status: `ok`
|
|
- positive sample count: `13`
|
|
- background sample count: `9`
|
|
- invalid labels: `0`
|
|
- missing label files: `0`
|
|
- median normalized box area: `0.001373291015625`
|
|
- small-box share: `0.0`
|
|
|
|
## Training and evaluation
|
|
|
|
- Trained a new inactive local model:
|
|
- model asset id: `geointel-building-yolov8s-aoi1024cleanpx12vis035e50-pt`
|
|
- model file: `/app/models/geointel-building-yolov8s-aoi1024cleanpx12vis035e50.pt`
|
|
- SHA256: `4863b27717cb7fd3126ccf86a988b3c1863b5fb9271cf35370ca2473b9ed10f4`
|
|
- base model: `/app/models/yolov8s.pt`
|
|
- dataset: `/app/storage/operator-data/yolo-building-aoi1024-cleanpx12vis035/dataset.yaml`
|
|
- epochs: `50`
|
|
- image size: `512`
|
|
- batch: `4`
|
|
- device: `cpu`
|
|
- active runtime model: unchanged
|
|
- Ran positive multi-AOI matrix:
|
|
- output: `artifacts/detection-quality-matrix/multi-sample/aoi1024cleanpx12vis035e50-positive/multi_sample_quality_summary.json`
|
|
- samples: `geel`, `mol`, `turnhout`, `herentals`, `balen`, `retie`, `westerlo`
|
|
- thresholds: `0.35`, `0.25`, `0.15`
|
|
- Ran split-background promotion workflow:
|
|
- background split: `artifacts/detection-hard-negatives/background-split/aoi1024cleanpx12vis035e50-split/background_corpus_split_summary.json`
|
|
- promotion report: `artifacts/detection-model-promotion/split-aware/aoi1024cleanpx12vis035e50-split/detection_model_promotion_report.json`
|
|
- recommended candidate: `none`
|
|
|
|
## Promotion result
|
|
|
|
- `geointel-building-yolov8s-aoi1024cleanpx12vis035e50-pt|512|64|0.15`
|
|
- rejected: `positive_mean_f1_below_gate`, `background_false_positive_pressure`
|
|
- positive samples: `7`
|
|
- background samples: `3`
|
|
- mean F1: `0.1542209371995884`
|
|
- mean precision: `0.18801639524786692`
|
|
- mean recall: `0.13146097412721683`
|
|
- max pure-empty detections: `2`
|
|
- `geointel-building-yolov8s-aoi1024cleanpx12vis035e50-pt|512|64|0.25`
|
|
- rejected: `positive_mean_f1_below_gate`
|
|
- mean F1: `0.14443976458069788`
|
|
- mean precision: `0.24254741138809843`
|
|
- mean recall: `0.10383903038875668`
|
|
- max pure-empty detections: `0`
|
|
- `geointel-building-yolov8s-aoi1024cleanpx12vis035e50-pt|512|64|0.35`
|
|
- rejected: `positive_mean_f1_below_gate`
|
|
- mean F1: `0.11851668674424971`
|
|
- mean precision: `0.30588130541017644`
|
|
- mean recall: `0.07481388046665544`
|
|
- max pure-empty detections: `0`
|
|
|
|
## Known limitations
|
|
|
|
- The clean label filter fixed the small-box audit problem but did not improve promotion-quality detection metrics.
|
|
- The new model remains useful evidence only and must stay inactive.
|
|
- The current limiting factor is not script/runtime wiring; it is training data representativeness and label geometry quality for small/dense Belgian building footprints.
|
|
|
|
## Next recommended pass
|
|
|
|
- Add building-size/visibility diagnostics per AOI and use them to choose or generate better positive samples before another training run. Do not spend more CPU on the current cleanpx dataset without changing the sample/label strategy.
|
|
|
|
# Sprint 166 - Per-sample YOLO dataset audit diagnostics
|
|
|
|
## What changed
|
|
|
|
- Extended `scripts/audit_operator_yolo_dataset_quality.py` with per-sample YOLO label diagnostics in `sample_summaries`:
|
|
- parsed label count;
|
|
- invalid label count;
|
|
- missing label file count;
|
|
- median/mean normalized box area;
|
|
- median normalized width/height;
|
|
- small-box count/share;
|
|
- sample-specific quality warning codes.
|
|
- Kept the existing global audit gates unchanged.
|
|
- Updated the Markdown audit output so each AOI/sample line includes the new label-quality signals.
|
|
- Added regression assertions in `backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py`.
|
|
- Updated `scripts/README.md` to document per-sample diagnostics.
|
|
|
|
## Local validation
|
|
|
|
- RED: `python -m pytest backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py::test_operator_yolo_dataset_quality_audit_reports_dataset_risks -q` failed with `KeyError: 'parsed_label_count'` before sample diagnostics existed.
|
|
- GREEN: same targeted test passed after adding sample label stats.
|
|
- Ran `python -m pytest backend/tests/test_sprint146_operator_yolo_dataset_quality_audit.py backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint156_background_corpus_classification.py -q`: 12 passed.
|
|
- Ran `bash scripts/run_readiness_check.sh`: 457 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
|
|
|
|
## Tower runtime evidence
|
|
|
|
- Pushed commit `7ccabf5` and redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`.
|
|
- Deploy validation passed:
|
|
- live migration smoke passed;
|
|
- browser runtime verification passed;
|
|
- container exposed `0.0.0.0:1202->80/tcp`.
|
|
- Re-ran the clean AOI1024 audit with sample diagnostics:
|
|
- report: `/app/artifacts/operator-yolo-dataset-audit/aoi1024-cleanpx12vis035-sample-diagnostics/operator_yolo_dataset_quality_audit.json`
|
|
- status: `ok`
|
|
- tile count: `144`
|
|
- positive tiles: `114`
|
|
- negative tiles: `30`
|
|
- labels: `14632`
|
|
- parsed labels: `14632`
|
|
- median normalized box area: `0.001373291016`
|
|
- small-box share: `0.0`
|
|
- sample count: `16`
|
|
- All samples reported `quality_warnings=[]` under the current gates.
|
|
- Lowest positive/context label counts:
|
|
- `ravels_bos`: 22 parsed labels, median box area `0.001140594385`
|
|
- `geel_bel`: 56 parsed labels, median box area `0.001411437891`
|
|
- `kasterlee_bos`: 73 parsed labels, median box area `0.001522064229`
|
|
- `dessel_heide`: 77 parsed labels, median box area `0.001495361328`
|
|
- `meerhout_bos`: 99 parsed labels, median box area `0.001232147197`
|
|
|
|
## Known limitations
|
|
|
|
- Numeric label diagnostics now pass, but the rejected `aoi1024cleanpx12vis035e50` model proves that numeric gates alone are insufficient.
|
|
- The next unknown is visual alignment and semantic fit: whether GRB building boxes line up well enough with the orthophoto tiles and whether sparse/context AOIs represent the false-positive modes seen during promotion.
|
|
|
|
## Next recommended pass
|
|
|
|
- Design and add a visual YOLO label QA artifact generator: deterministic contact sheets of selected train/val tiles with YOLO boxes overlaid on imagery, grouped by AOI/sample and label density. Use it before another training run.
|
|
|
|
# Sprint 167 - Operator YOLO visual label QA contact sheets
|
|
|
|
## What changed
|
|
|
|
- Added `scripts/render_operator_yolo_label_qa_contact_sheets.py` to render deterministic operator-only contact sheets from existing YOLO tile datasets.
|
|
- The script reads the existing `yolo_tile_dataset_summary.json`, image tiles and label files, then writes:
|
|
- `operator_yolo_label_qa_summary.json`;
|
|
- `operator_yolo_label_qa_contact_sheet.md`;
|
|
- one or more PNG contact sheets with YOLO labels drawn over the tile imagery.
|
|
- Added explicit checks for missing images, missing label files, invalid YOLO rows and blank-looking/low-variance tiles.
|
|
- Added regression coverage in `backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py`.
|
|
- Updated the all-in-one Dockerfile so operator QA scripts are copied after the expensive dependency layer, keeping future script-only rebuilds cache-friendlier.
|
|
- Updated Docker runtime tests so the all-in-one image keeps packaging the operator scripts needed on Tower.
|
|
- No inference, training, provider fetch, database mutation, model activation or fake detection path was introduced.
|
|
|
|
## Local validation
|
|
|
|
- RED: `python -m pytest backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py -q` failed before the contact-sheet script existed.
|
|
- GREEN: same targeted test passed after adding the renderer.
|
|
- RED: the low-variance regression failed before `low_visual_variance_tile_count` existed.
|
|
- GREEN: same targeted test passed after adding low-variance tile reporting.
|
|
- Ran `python -m pytest backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_for_runtime_use backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_after_dependency_install backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py -q`: 3 passed.
|
|
- Ran `bash scripts/run_readiness_check.sh`: 459 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
|
|
|
|
## Tower runtime evidence
|
|
|
|
- Pushed commits `fbccf83` and `5688fee`, then redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`.
|
|
- Docker storage had filled during the first build attempt. Cleaned Docker build cache and dangling images only; application volumes and appdata were not pruned. Docker reclaimed `98.38GB`.
|
|
- Deploy validation for commit `5688fee` passed:
|
|
- live migration smoke passed;
|
|
- browser runtime verification passed;
|
|
- container exposed `0.0.0.0:1202->80/tcp`;
|
|
- Unraid icon check remained OK.
|
|
- Rendered the clean AOI1024 visual label QA artifact inside the live container:
|
|
- summary: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/operator_yolo_label_qa_summary.json`
|
|
- Markdown: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/operator_yolo_label_qa_contact_sheet.md`
|
|
- contact sheet: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035/contact_sheet_001.png`
|
|
- Runtime summary:
|
|
- status: `ok`
|
|
- selected tiles: `32`
|
|
- rendered tiles: `32`
|
|
- valid labels: `7670`
|
|
- invalid labels: `0`
|
|
- missing images: `0`
|
|
- missing label files: `0`
|
|
- low-variance tiles: `6`
|
|
- Visual inspection confirmed:
|
|
- dense positive tiles show yellow YOLO boxes over real orthophoto imagery;
|
|
- the six low-variance tiles are `arendonk_heide` pure-empty negative validation tiles with no labels and blank-looking imagery.
|
|
|
|
## Known limitations
|
|
|
|
- The contact sheet now makes visual label inspection possible, but it also proves that the current clean AOI1024 dataset still contains blank-looking pure-empty negative tiles.
|
|
- Those blank/low-variance negatives should not be used blindly for the next training run. They can distort the background corpus and do not represent realistic aerial false-positive pressure.
|
|
- The current inactive model/promotion state remains unchanged.
|
|
|
|
## Next recommended pass
|
|
|
|
- Add no-data/low-variance filtering to the operator YOLO tile export path, regenerate the clean AOI1024 dataset, rerun the contact-sheet QA, and only then consider another training attempt.
|
|
|
|
# Sprint 168 - Operator YOLO low-variance negative filtering
|
|
|
|
## What changed
|
|
|
|
- Added opt-in low-variance negative filtering to `scripts/export_operator_yolo_tile_dataset.py`.
|
|
- Added CLI/env controls:
|
|
- `--drop-low-variance-negatives` / `OPERATOR_YOLO_DROP_LOW_VARIANCE_NEGATIVES`;
|
|
- `--blank-range-threshold` / `OPERATOR_YOLO_BLANK_RANGE_THRESHOLD`.
|
|
- The filter evaluates the rendered raster tile image and skips only negative tiles when enabled.
|
|
- Positive/labeled tiles are never removed by this variance gate.
|
|
- Kept tile records now include `low_visual_variance`.
|
|
- Skipped blank/no-data negative records use `skip_reason="low_visual_variance_negative"`.
|
|
- Dataset summaries now include:
|
|
- `drop_low_variance_negatives`;
|
|
- `blank_range_threshold`;
|
|
- `skipped_low_variance_negative_tile_count`.
|
|
- Updated operator documentation with the refreshed AOI1024 cleanpx export command.
|
|
- Added design and execution plan docs under `docs/superpowers/`.
|
|
|
|
## Local validation
|
|
|
|
- RED: `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py::test_export_can_skip_low_variance_negative_tiles -q` failed because `export_sample_tiles()` did not accept `drop_low_variance_negatives`.
|
|
- GREEN: same targeted test passed after adding the filter.
|
|
- Ran `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py -q`: 8 passed.
|
|
- Ran `python -m pytest backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint167_operator_yolo_label_qa_contact_sheets.py backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_for_runtime_use backend/tests/test_docker_runtime_config.py::test_all_in_one_dockerfile_copies_operator_scripts_after_dependency_install -q`: 11 passed.
|
|
- Ran `python scripts/export_operator_yolo_tile_dataset.py --help`: the CLI exposes `--drop-low-variance-negatives`, `--no-drop-low-variance-negatives` and `--blank-range-threshold` without loading GIS dependencies.
|
|
- Ran `bash scripts/run_readiness_check.sh`: 460 backend tests passed, frontend typecheck passed, frontend build passed, readiness passed.
|
|
|
|
## Tower runtime evidence
|
|
|
|
- Pushed commit `a159370` and redeployed the all-in-one Tower runtime at `http://192.168.10.150:1202`.
|
|
- Deploy validation passed:
|
|
- live migration smoke passed;
|
|
- browser runtime verification passed;
|
|
- container exposed `0.0.0.0:1202->80/tcp`.
|
|
- Regenerated the AOI1024 cleanpx dataset with low-variance negative filtering enabled:
|
|
- dataset: `/app/storage/operator-data/yolo-building-aoi1024-cleanpx12vis035`
|
|
- `drop_low_variance_negatives=true`
|
|
- `blank_range_threshold=3`
|
|
- tile count: `135`
|
|
- positive tiles: `114`
|
|
- negative tiles: `21`
|
|
- skipped negative tiles: `9`
|
|
- skipped low-variance negative tiles: `9`
|
|
- labels: `14632`
|
|
- train tiles: `108`
|
|
- validation tiles: `27`
|
|
- Dataset audit:
|
|
- report: `/app/artifacts/operator-yolo-dataset-audit/aoi1024-cleanpx12vis035-lowvarfilter/operator_yolo_dataset_quality_audit.json`
|
|
- status: `ok`
|
|
- invalid labels: `0`
|
|
- missing label files: `0`
|
|
- median normalized box area: `0.001373291016`
|
|
- small-box share: `0.0`
|
|
- Visual label QA:
|
|
- report: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035-lowvarfilter/operator_yolo_label_qa_summary.json`
|
|
- contact sheet: `/app/artifacts/operator-yolo-label-qa/aoi1024-cleanpx12vis035-lowvarfilter/contact_sheet_001.png`
|
|
- selected tiles: `32`
|
|
- rendered tiles: `32`
|
|
- valid labels: `7670`
|
|
- invalid labels: `0`
|
|
- missing images: `0`
|
|
- missing label files: `0`
|
|
- low-variance rendered tiles: `0`
|
|
- Visual inspection confirmed that the previous blank white `arendonk_heide` negatives are no longer present in the review sheet. The remaining selected pure-empty negatives are real visible orthophoto/context tiles.
|
|
|
|
## Known limitations
|
|
|
|
- The low-variance gate is deliberately simple and only identifies visually blank/no-data-looking negative tiles.
|
|
- Operator visual contact-sheet review remains required before any new training run.
|
|
- The filtered dataset is now a cleaner input candidate, but model training is still not guaranteed to improve QA/QC; another training run must be gated through the existing positive-AOI and background promotion reports.
|
|
|
|
## Next recommended pass
|
|
|
|
- Train one inactive candidate from the filtered AOI1024 cleanpx dataset, then run the existing positive-AOI matrix and split-background promotion workflow before considering default activation.
|
|
# Sprint 171 - Positive AOI expansion and split safety
|
|
|
|
## What changed
|
|
|
|
- Converted the Sprint 170 false-negative evidence into a guarded data action instead of another blind training run.
|
|
- Added Olen, Lille, Oud-Turnhout and Kasterlee center as explicit real-reference training AOIs.
|
|
- Kept Turnhout, Retie, Westerlo and Arendonk-heide as the documented validation holdouts.
|
|
- Added generated `recommended_split` provenance and tile-export validation that rejects unknown samples and manifest-backed holdout leakage.
|
|
- Hardened false-negative portfolio comparison to require identical reference feature identities, not only matching AOI names.
|
|
|
|
## Local validation
|
|
|
|
- RED tests proved the expansion/split constants and validation guard were absent before implementation.
|
|
- `python -m pytest backend/tests/test_sprint131_operator_sample_expansion.py backend/tests/test_sprint130_operator_yolo_tile_dataset.py backend/tests/test_sprint156_background_corpus_classification.py backend/tests/test_sprint170_detection_false_negative_audit.py -q`: 23 passed.
|
|
- Official GRB OGC API probes returned building features at all four new AOI centers.
|
|
- Local sample generation was attempted but correctly stopped because the workstation Python lacks the existing GIS runtime extras; the all-in-one Tower runtime is the supported execution environment.
|
|
|
|
## Tower runtime evidence
|
|
|
|
- Pushed `0f49c98` and redeployed the all-in-one runtime at `http://192.168.10.150:1202`; live migration and browser proxy verification passed.
|
|
- Refreshed `/app/storage/operator-data/operator-samples-1024/operator_samples_manifest.json` to schema version 2 with 20 sources.
|
|
- Newly fetched real GRB reference counts: Olen 1,952, Lille 1,839, Oud-Turnhout 2,691 and Kasterlee 1,831; all are `recommended_split=train`.
|
|
- Exported `/app/storage/operator-data/yolo-building-aoi1024-expanded-minpx4vis035`:
|
|
- 171 retained tiles;
|
|
- 153 positive and 18 negative tiles;
|
|
- 45,892 labels;
|
|
- 144 train and 27 validation tiles;
|
|
- 9 low-variance negatives skipped;
|
|
- validation holdouts Turnhout, Retie, Westerlo and Arendonk-heide recorded in summary provenance.
|
|
- Dataset audit status `ok`: no warnings, invalid labels or missing files; median normalized box area `0.000694274766`, small-box share `0.3832694151486098`.
|
|
- Improved the visual contact-sheet selector after the first live sheet overrepresented dense AOIs. The balanced rerun selected 40 tiles across all 19 retained source samples with zero invalid labels, missing images, missing label files or low-variance selections.
|
|
- Started one 50-epoch CPU YOLOv8s candidate as inactive runtime evidence: `geointel-building-yolov8s-aoi1024expandedminpx4vis035e50.pt`.
|
|
|
|
## Known limitations
|
|
|
|
- No new model has been trained or activated.
|
|
- The inactive training run must finish and pass positive-AOI plus split-background promotion gates before it can be considered for activation.
|
|
|
|
## Next recommended pass
|
|
|
|
- Finish the inactive candidate, run fixed-threshold positive evidence and split pure-empty/sparse-context background matrices, and preserve the current production default unless the promotion report passes every gate.
|
|
# Sprint 172 - CPU AI image build hardening
|
|
|
|
## What changed
|
|
|
|
- Reordered `deploy/unraid/Dockerfile.all-in-one` so `pyproject.toml` and minimal package metadata are installed before the complete backend source is copied.
|
|
- Code-only backend changes can now reuse the expensive GIS/AI dependency layer; dependency metadata changes still invalidate it.
|
|
- The opt-in CPU AI build now installs the same validated PyTorch `2.13.0` / torchvision `0.28.0` versions from the official CPU wheel index before installing the `ai` extra.
|
|
- Kept the full GIS and YOLO import/preflight smoke after the complete backend source copy.
|
|
|
|
## Validation so far
|
|
|
|
- RED: the new Docker ordering/CPU-wheel regression test failed against the old Dockerfile.
|
|
- GREEN: `python -m pytest backend/tests/test_docker_runtime_config.py -q`: 26 passed.
|
|
- `python -m pip index versions` confirmed `torch 2.13.0+cpu` and `torchvision 0.28.0+cpu` are available from the configured CPU index for the workstation platform.
|
|
|
|
## Remaining validation
|
|
|
|
- Build the AI-enabled all-in-one image on Tower after the current inactive model training run finishes, verify Torch reports a CPU build and rerun live migration/browser smokes before replacing the runtime.
|
|
|
|
# Sprint 171.1 - Validation coverage provenance
|
|
|
|
- The expanded live export exposed that Arendonk-heide remained configured as a holdout while all of its low-variance tiles were correctly filtered out.
|
|
- Added `retained_validation_sample_slugs` and `empty_validation_sample_slugs` to tile dataset summaries so configured and actual validation coverage cannot be confused.
|
|
- Added a focused regression test and kept filtering behavior unchanged; no blank tile was reintroduced.
|
|
|
|
# Sprint 173 - Expanded building model promotion
|
|
|
|
## Runtime evidence
|
|
|
|
- Completed inactive 50-epoch CPU training for `geointel-building-yolov8s-aoi1024expandedminpx4vis035e50.pt` from the expanded 20-source real-data corpus.
|
|
- Trained-model SHA256: `a8a79cf5b0bdc19a0245acc322cf77232c335e222bd5f3c00a17d5f29402c196`.
|
|
- Training summary recorded 45,892 labels across 171 retained tiles; final training metrics were precision `0.55558`, recall `0.35206`, mAP50 `0.27440` and mAP50-95 `0.10107`.
|
|
- The persisted seven-AOI positive matrix recommended tile size `512`, overlap `64` and threshold `0.15`: mean precision `0.6470590036`, recall `0.4699913837`, F1 `0.5432865391`, and minimum per-AOI F1 `0.4897494305`.
|
|
- The strict pure-empty gate covered Postel, Lommel and Arendonk across all tested thresholds and produced zero detections. Sparse-building contextual AOIs remained review-only evidence because real GRB buildings are present there.
|
|
- Fixed-threshold persisted false-negative comparison used identical reference feature populations and reduced the false-negative rate in all seven positive AOIs versus the previous active `0.35` profile.
|
|
- The guarded activation helper first returned `ready_to_apply`, resolved the exact local model asset and reported no downloads or inference. The reviewed `--apply` pass updated only `GEOINTEL_INSTALL_AI`, `YOLO_ENABLED`, `YOLO_MODELS_DIR` and `YOLO_MODEL_PATH` in the Tower environment.
|
|
|
|
## UI and operator behavior
|
|
|
|
- Detection Lab now recommends the promoted expanded-AOI model at threshold `0.15` and surfaces its seven-AOI coverage plus QA metrics.
|
|
- The previous AOI1024 background-aware model remains mounted as an explicit legacy high-precision `0.35` review profile.
|
|
- Profile selection remains deliberate: it selects a mounted local asset and run threshold only; it does not mutate runtime environment, download a model or start inference automatically.
|
|
|
|
## Remaining limitation
|
|
|
|
- Persistent small-building misses remain the weakest quality bucket. Continue with targeted evidence review and data coverage before considering another training run; do not infer production accuracy from aggregate F1 alone.
|
|
|
|
## Tower deployment verification
|
|
|
|
- Pushed commit `1c16313` and rebuilt the all-in-one runtime at `http://192.168.10.150:1202`.
|
|
- The opt-in AI image installed `torch 2.13.0+cpu` and `torchvision 0.28.0+cpu` from the official CPU wheel index; runtime CUDA availability is `false`. Ultralytics reports version `8.4.93`.
|
|
- The live model catalog exposes 23 local files and marks only `geointel-building-yolov8s-aoi1024expandedminpx4vis035e50-pt` active with the expected SHA256.
|
|
- Explicit `--check-model-load` preflight passed against an existing nine-tile manifest: dependencies, model load, manifest structure, tile paths and tile limit were all valid; no inference or download ran.
|
|
- Embedded PostGIS live migration smoke passed with PostGIS `3.6`, required tables/indexes, database collation and the single Alembic head `202606120900`.
|
|
- Browser validation confirmed that the recommended profile selects the exact active asset and threshold `0.15`, while live preflight displays CPU dependency/model readiness and the expected missing-manifest guard before dataset handoff.
|
|
- Browser console warnings/errors: `0`.
|
|
|
|
# Sprint 174 - Focused small-building recovery and promotion
|
|
|
|
## Data and training evidence
|
|
|
|
- Converted the Sprint 173 persistent false-negative audit into one focused real-data experiment instead of extending the same corpus blindly.
|
|
- Added Beerse, Rijkevorsel, Hoogstraten and Vorselaar as training AOIs and Vosselaar/Grobbendonk as independent tile-level validation AOIs.
|
|
- Kept Turnhout, Retie and Westerlo outside the tile corpus as operation-level holdouts.
|
|
- Exported `/app/storage/operator-data/yolo-building-aoi1024-smallbld-minpx3vis035` from an explicit 23-sample manifest subset:
|
|
- 198 retained tiles;
|
|
- 180 positive and 18 negative tiles;
|
|
- 58,820 real GRB-derived labels;
|
|
- 48 visually reviewed tiles;
|
|
- zero invalid labels, missing images, missing label files or low-variance review selections.
|
|
- The accepted `min-label-px=3` corpus retained 1,228 more genuine small-building labels than the comparable `min-label-px=4` export.
|
|
- Trained one inactive 30-epoch CPU candidate from the previous active local model:
|
|
- model: `geointel-building-yolov8s-smallbld-minpx3-img640-ft30.pt`;
|
|
- model SHA256: `a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`;
|
|
- dataset-summary SHA256: `49b2a07d2105d08356431757b83eafc1498eaf1fb76965b1efe05b776824942a`;
|
|
- dataset-YAML SHA256: `3a2ea97c35a18072a1ab6738cd673c0ecec5344b19461c91d72a15e138d46e8d`;
|
|
- no model download and no fake training or QA data.
|
|
|
|
## Persisted promotion evidence
|
|
|
|
- Evaluated the exact fixed profile `tile=512`, `overlap=64`, `confidence=0.15` with QA match IoU explicitly fixed at `0.25`.
|
|
- Seven positive AOIs produced:
|
|
- mean precision `0.5898197518`;
|
|
- mean recall `0.5769921004`;
|
|
- mean F1 `0.5824578632`;
|
|
- minimum per-AOI F1 `0.5527837436`.
|
|
- Every AOI improved F1 relative to the previous balanced model. Turnhout improved from `0.4897494305` to `0.5527837436`.
|
|
- The strict pure-empty gate covered Postel, Lommel and Arendonk and produced zero detections for every sample.
|
|
- The formal promotion report recommended the exact key `geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt|512|64|0.15`.
|
|
- Fixed-reference object evidence used identical GRB feature identities and reduced false negatives from `7,753` to `6,182`:
|
|
- 1,571 fewer total false negatives;
|
|
- 745 fewer misses in the 25-100 m2 bucket;
|
|
- 181 fewer misses below 25 m2;
|
|
- all seven AOIs improved.
|
|
- Remaining persistent misses total `5,838`, concentrated in Turnhout, Herentals and Geel and still dominated by small buildings.
|
|
- Mean precision decreased from `0.6470590036` to `0.5898197518`. The new profile is therefore a recall-balanced operator default with a higher false-positive review load, not ground truth.
|
|
|
|
## Repository hardening
|
|
|
|
- Added explicit `--samples` / `OPERATOR_YOLO_SAMPLES` corpus selection with selected/excluded sample provenance and unknown-sample rejection.
|
|
- Added persistent false-negative area statistics, size buckets and combined GeoJSON review evidence.
|
|
- Copied the complete operator evaluation/promotion toolchain into the all-in-one image and added a regression that rejects every Docker `COPY scripts/...` source that does not exist.
|
|
- Removed Pydantic protected-namespace warnings for legitimate `model_*` API fields while preserving all schema field names and response contracts.
|
|
- Updated Detection Lab profiles: the new small-building profile is recommended, the previous expanded profile remains the higher-precision legacy choice, and the background-aware `0.35` profile remains conservative.
|
|
- Live `2560x1080` inspection found that two-panel AI/QA/Export workspaces inherited four ultrawide columns and left half of the main canvas empty. Those workspaces now remain explicit two-column grids while the three-panel Data workspace keeps three columns.
|
|
- The all-in-one dependency layer now copies only `pyproject.toml` plus a stable build-only package README before installation; the real `backend/README.md` still enters with the complete backend source, so documentation-only edits no longer invalidate Torch/GIS dependencies.
|
|
- Guarded activation first returned `ready_to_apply`; the reviewed `--apply` pass updated only `GEOINTEL_INSTALL_AI`, `YOLO_ENABLED`, `YOLO_MODELS_DIR` and `YOLO_MODEL_PATH` in the Tower environment.
|
|
|
|
## Local validation
|
|
|
|
- `python -m compileall backend/app`: passed.
|
|
- `python -m pytest`: 472 passed.
|
|
- `python -m ruff check` for all changed Python modules/tests: passed.
|
|
- `npm run typecheck`: passed.
|
|
- `npm run build`: passed; app bundle `215.64 kB`, MapLibre bundle `801.82 kB` before gzip.
|
|
- `bash scripts/run_readiness_check.sh`: passed with 472 tests.
|
|
- `python -m alembic heads`: one head, `202606120900`.
|
|
- `python -m alembic upgrade head --sql`: complete migration chain rendered successfully.
|
|
- Shell syntax checks passed for live migration and the full operator evaluation/promotion chain.
|
|
|
|
## Next recommended pass
|
|
|
|
- After redeploy, verify the active model SHA, local model-load preflight, live PostGIS migration smoke and browser profile selection. Then review false-positive evidence and the remaining 5,838 persistent misses before any further training.
|
|
|
|
# Sprint 175 - Detection result scale and false-positive evidence review
|
|
|
|
## UI hardening
|
|
|
|
- Confirmed that Detection Lab rendered every persisted detection row at once; Westerlo alone produced 1,172 body rows in the browser.
|
|
- Added local 25/50/100-row pagination with a default of 50 rows, bounded page controls and automatic page-one reset after run/filter/result changes.
|
|
- Live visual inspection exposed tall rows caused by full container paths; source-tile cells now show the filename and retain the full persisted path as a tooltip.
|
|
- Kept the full persisted collection unchanged for the existing MapLibre GeoJSON overlay and detection QA/QC. No endpoint, response envelope or persistence contract changed.
|
|
|
|
## Persisted false-positive evidence
|
|
|
|
- Added `scripts/audit_detection_false_positive_evidence.py` as a read-only evidence consumer.
|
|
- The audit validates FeatureCollection/polygon geometry, compares declared portfolio role counts with actual evidence, computes WGS84 geodesic areas, preserves original feature provenance and emits combined `false_positives.geojson`.
|
|
- Source tile summaries are qualified by AOI because `tile_index` is local to each raster manifest.
|
|
- The active fixed-threshold seven-AOI portfolio produced:
|
|
- 5,568 false positives among 13,613 candidate detections (`0.4090` false-positive share);
|
|
- median false-positive geometry area `184.5 m2`, p90 `607.7 m2`;
|
|
- 52 below 25 m2, 1,382 between 25-100 m2, 3,412 between 100-500 m2 and 722 at or above 500 m2;
|
|
- largest AOI review volumes: Turnhout `1,102`, Herentals `917`, Geel `913`;
|
|
- largest AOI-qualified tile hotspot: `turnhout:0` with 236 false positives.
|
|
- Existing persisted QA evidence carries run threshold and tile index but no per-detection confidence. The audit reports confidence coverage `0/5,568` and does not invent confidence statistics.
|
|
- Added focused regression coverage, readiness compilation and all-in-one image inclusion. No training, inference, provider fetch, model download or activation occurred.
|
|
|
|
## Local validation
|
|
|
|
- `python -m compileall backend/app`: passed.
|
|
- `python -m pytest`: 475 passed.
|
|
- `python -m ruff check` for the new audit/test modules: passed.
|
|
- `npm run typecheck`: passed.
|
|
- `npm run build`: passed; app bundle `217.00 kB`, MapLibre bundle `801.82 kB` before gzip.
|
|
- `bash scripts/run_readiness_check.sh`: passed with 475 tests and all release-critical syntax gates.
|
|
- `python -m alembic heads`: one head, `202606120900`.
|
|
- `python -m alembic upgrade head --sql`: complete migration chain rendered successfully.
|
|
- Local `docker compose config` could not run because Docker CLI is not installed on the Windows host; live image/PostGIS validation is delegated to the Docker-enabled Tower deployment.
|
|
|
|
## Tower deployment verification
|
|
|
|
- Pushed runtime commits `4455e24` and `d188014`; Tower rebuilt the all-in-one image from `main` with the GIS/CPU-AI dependency layer cached.
|
|
- Live PostGIS migration smoke passed with PostGIS `3.6`, required runtime tables/indexes and single Alembic head `202606120900`.
|
|
- The active local model remained `/app/models/geointel-building-yolov8s-smallbld-minpx3-img640-ft30.pt` with SHA256 `a9088b8491dfae36694b53e9e9406cb4e3511d334a5712fa34f75078a47759c1`.
|
|
- Local model-load preflight passed against a persisted nine-tile manifest with Torch `2.13.0+cpu`, Ultralytics `8.4.93`, CUDA disabled, no inference and no download.
|
|
- The new false-positive audit ran successfully inside the live all-in-one container against the persisted seven-AOI evidence portfolio.
|
|
- Browser verification against the persisted Westerlo run confirmed 1,172 loaded detections, 50 rendered rows, `1-50 of 1172`, page `1/24`, working next-page and 100-row controls, and reset back to the 50-row default.
|
|
- Final cells display `tile_0001.tif` while the full persisted path remains available through the `title` attribute.
|
|
- At `2560x1080`, the Detection results surface used about 871 px, its table client/scroll widths both measured 827 px and the document did not overflow the viewport.
|
|
- Browser console warnings/errors: `0`.
|
|
|
|
## Next recommended pass
|
|
|
|
- Visually classify a stratified false-positive sample from Turnhout, Herentals and Geel before deciding whether any confirmed examples belong in a new hard-negative corpus. Review the remaining 5,838 persistent false negatives in the same evidence-led pass; do not start another blind training run.
|
|
|
|
# Sprint 176 - Detection false-positive visual review gate
|
|
|
|
## Persisted provenance
|
|
|
|
- Extended the existing read-only QA evidence GeoJSON conversion so detection-backed evidence carries the persisted detection id, job id, confidence, model name/version, source tile path and pixel bbox.
|
|
- Added equivalent persisted segmentation provenance fields without changing the endpoint, canonical envelope, ORM or migration chain.
|
|
- Historical `QualityCheck` evidence can be re-exported against existing persisted `Detection` rows; no QA rerun or data rewrite is required.
|
|
|
|
## Manual visual review
|
|
|
|
- Added a storage-root-confined contact-sheet renderer for persisted detection false-positive evidence.
|
|
- The renderer validates portfolio role counts, polygon geometry, source imagery and persisted provenance, then selects deterministically across AOI, WGS84 area bucket and confidence band.
|
|
- Source imagery is rendered with the candidate pixel bbox plus persisted matched-reference and missed-reference overlays.
|
|
- Live orthophoto inspection exposed stretched non-square edge tiles and overly distant full-tile context; the renderer now uses candidate-centred crops, preserves aspect ratio and limits reference overlays to the crop.
|
|
- Added an explicit five-state review CSV: `confirmed_model_false_positive`, `reference_gap_or_change`, `qa_alignment_mismatch`, `uncertain` and `unreviewed`.
|
|
- Added a separate validator that rejects missing, duplicate, unexpected or invalid decisions. `--require-complete` exits with code `2` while any record remains unreviewed.
|
|
- Only explicitly confirmed model false-positives are emitted to `confirmed_model_false_positives.geojson`; no QA result is automatically converted into a model label or training artifact.
|
|
|
|
## Validation
|
|
|
|
- `python -m compileall backend/app`: passed.
|
|
- `python -m pytest`: 478 passed.
|
|
- Focused provenance/render/path-confinement/incomplete-review/export tests: passed.
|
|
- `python -m ruff check` for changed Python services, scripts and tests: passed.
|
|
- Generated fixture contact sheet was visually inspected at 128 px thumbnails; candidate/reference/missed-reference overlays and header provenance remained readable.
|
|
- `npm run typecheck`: passed.
|
|
- `npm run build`: passed; app bundle `217.00 kB`, MapLibre bundle `801.82 kB` before gzip.
|
|
- `bash scripts/run_readiness_check.sh`: passed with 478 tests and the new operator-script compile gates.
|
|
- `python -m alembic heads`: one head, `202606120900`.
|
|
- `python -m alembic upgrade head --sql`: complete migration chain rendered successfully.
|
|
- Local Docker validation remains unavailable because Docker CLI is not installed on the Windows host; live all-in-one/PostGIS validation follows on Tower after deployment.
|
|
|
|
## Tower deployment evidence
|
|
|
|
- Deployed commit `1322a5d` through the repository-driven all-in-one build with the existing CPU-AI dependency layer cached.
|
|
- PostGIS `3.6`, required tables/indexes, Alembic head `202606120900`, frontend proxy, API proxy and icon checks passed.
|
|
- Re-exported all seven persisted fixed-threshold QualityChecks into an enriched portfolio containing 27,840 evidence features.
|
|
- The false-positive audit now reports persisted confidence coverage `5,568/5,568` rather than inventing values for the older static export.
|
|
- Rendered an initial 48-case Turnhout/Herentals/Geel review spanning all four area buckets and all three confidence bands with no missing provenance or source tiles.
|
|
- All 48 decisions remain explicitly `unreviewed`; validation status is `review_required` and the confirmed-model-false-positive GeoJSON is empty.
|
|
|
|
## Next recommended pass
|
|
|
|
- Re-export the seven-AOI evidence portfolio from the deployed backend, render the Mol/Geel/Turnhout sheets and inspect the real orthophoto evidence. Keep all CSV decisions `unreviewed` until an operator makes an explicit visual classification; do not start another model training run yet.
|
|
|
|
# Sprint 177 - Mol-first operating context
|
|
|
|
## Product and workbench focus
|
|
|
|
- Declared Mol as the primary operational context while preserving the broader Kempen for cross-area validation and regional interoperability.
|
|
- Centralized Mol coordinates, default region and a compact 1 km EPSG:4326 MultiPolygon AOI in the frontend focus configuration.
|
|
- Initial project discovery now recognizes Mol from project metadata or persisted dataset provenance and inspects those candidates first.
|
|
- Explicitly selected and newly created projects still take precedence, so the focus rule does not fight operator intent.
|
|
- The empty MapLibre workbench starts over Mol; loaded AOIs and data continue to determine map bounds normally.
|
|
|
|
## Operator provenance
|
|
|
|
- Moved Mol to the first position in default real-data sample preparation without dropping any existing Kempen sample.
|
|
- Multi-sample quality runs now forward their AOI slug, and future persisted matrix project names retain that origin instead of appearing as anonymous model/tile runs.
|
|
- No provider fetch, AI output, QA metric, migration, API contract or model configuration was changed.
|
|
|
|
## Validation
|
|
|
|
- Focused Mol-primary regression coverage: `3 passed`.
|
|
- `bash scripts/run_readiness_check.sh`: passed with `481` backend tests, API-contract audit, single Alembic head, frontend typecheck and production build.
|
|
- Frontend production bundles: app `217.69 kB`, React vendor `140.74 kB`, MapLibre `801.82 kB` before gzip.
|
|
- `python -m alembic upgrade head --sql`: complete migration chain rendered through single head `202606120900`.
|
|
- Both changed matrix scripts pass `bash -n`; no migration file changed.
|
|
|
|
## Tower deployment evidence
|
|
|
|
- Deployed commit `e8eecb2` through the repository-driven all-in-one build on port `1202` with the existing CPU-AI dependency layer cached.
|
|
- Live migration smoke passed with PostGIS `3.6`, required runtime tables/indexes and single Alembic head `202606120900`.
|
|
- Renamed persisted project `d25206c0-dcba-46e5-aba9-66ace1122a30` to an explicit Mol operating context and added a real 1 km Mol AOI (`999,796.69 m2`) without changing its datasets or analysis results.
|
|
- Live Mol context contains a ready EPSG:31370 1024x1024 orthophoto and `1,993` ready EPSG:4326 GRB reference buildings.
|
|
- Persisted detection QA remains unchanged: precision `0.6011`, recall `0.5891`, F1 `0.5950`, mean IoU `0.4554`, `779` false positives and `819` false negatives.
|
|
- In-app browser verification selected the Mol project, Mol AOI and Mol reference dataset automatically. MapLibre rendered the existing `1,953`-feature detection layer over the road basemap with one canvas, zero browser warnings/errors and no horizontal document overflow at `1265x720`.
|
|
|
|
# Sprint 178 - Mol multi-zone operational validation
|
|
|
|
## Implementation
|
|
|
|
- Added Mol center, Achterbos, Gompel, Donk and Postel as explicit operator contexts with municipality and operational-zone provenance.
|
|
- Marked Achterbos, Gompel, Donk and Postel as validation holdouts to keep them outside future training exports unless the split policy is deliberately changed.
|
|
- Kept Postel-bos as a separate background control, so an empty/sparse context is never assigned fabricated precision, recall or F1.
|
|
- Extended the existing real-data workflow with optional project region and EPSG:4326 AOI bounds; manifest-backed positive and background projects now open map-ready with persisted Areas.
|
|
- Added `run_mol_operational_validation.sh` to compose the existing positive QA matrix and background detection-pressure matrix and emit one evidence summary.
|
|
- Included the new runner in the all-in-one runtime and release-readiness syntax gate. No API route, ORM model, migration, model activation or frontend behavior changed.
|
|
|
|
## Initial validation
|
|
|
|
- Focused Mol/operator/workflow/Docker regressions: `43 passed`.
|
|
- Changed Python operator preparer compiled successfully.
|
|
- All five affected shell workflows passed `bash -n`.
|
|
- `git diff --check`: clean apart from the existing Windows line-ending notice for the all-in-one Dockerfile.
|
|
- `bash scripts/run_readiness_check.sh`: passed with `485` backend tests, contract audit, single Alembic head, frontend typecheck and production build.
|
|
- Frontend bundle sizes are unchanged: app `217.69 kB`, React vendor `140.74 kB` and MapLibre `801.82 kB` before gzip.
|
|
|
|
## Live Tower completion
|
|
|
|
- Deployed through Gitea commit `720ad71`; the all-in-one browser/API smoke, PostGIS `3.6`, required runtime schema and Alembic head `202606120900` passed.
|
|
- Prepared real 1024 px orthophoto/GRB samples for Mol center, Achterbos, Gompel, Donk, Postel and Postel-bos. Positive reference counts are `1,993`, `1,388`, `1,070`, `1,234` and `137`; the background control contains exactly `0` GRB buildings.
|
|
- Fixed two manifest-path type errors found by the first live multi-sample aggregation and moved all-in-one evidence defaults from the replaceable container layer to `/app/storage/operator-evidence`.
|
|
- Completed the persistent four-zone configured-YOLO benchmark at confidence `0.15`, tile `512`, overlap `64` and QA IoU `0.25`: mean precision `0.6286`, mean recall `0.5370`, mean F1 `0.5768`, `2,239` matches, `1,215` false positives and `1,590` false negatives.
|
|
- Per-zone F1 is Achterbos `0.6433`, Gompel `0.6333`, Donk `0.5823` and Postel `0.4483`. Postel is the explicit weakest-zone review priority; no automatic promotion or retraining decision was made.
|
|
- The real empty Postel-bos control produced `0` detections and therefore `0.0` false-positive pressure without fabricated QA metrics.
|
|
- Verified through the browser that the complete Postel project wins initial selection over the newer raster-only background project, with one AOI, one ready raster, one ready GRB reference, one detection run, one QualityCheck and one export.
|
|
- MapLibre rendered the OSM road basemap, AOI and `95` persisted detections without browser warnings or horizontal overflow at `1280x720` and `2560x1080`; the ultrawide canvas measured `1772x830` CSS pixels.
|
|
- Ran the interactive PostGIS AOI query against `vector_features`: `126` GRB buildings returned with no truncation. The guided GIS smoke persisted a derived dataset and GeoJSON export, then produced candidate/reference QA F1 `0.9582` with `126` matches, `0` false positives and `11` false negatives; `263` persisted evidence features rendered back on the map.
|
|
- Remaining evidence caveat: the dataset QA result reports weak CRS-assumption warnings, so those geometry metrics remain explicitly approximate until CRS provenance handling is reviewed.
|
|
- Host observation outside GeoIntel: Unraid recovered after reboot and serves the app, but still reports one disabled/invalid array device. Storage administration should resolve that independently of application development.
|
|
|
|
# Sprint 179 - Mol Donk/Postel detection evidence diagnosis
|
|
|
|
## Implementation
|
|
|
|
- Added `render_detection_false_negative_review_contact_sheets.py` as a read-only counterpart to the persisted false-positive review workflow.
|
|
- Resolved each sample's exact persisted tile manifest from its fixed-threshold run summary; manifests and source tiles are confined to `/app/storage`.
|
|
- Projected WGS84 missed-reference geometry onto the real inference tiles and rendered nearby persisted candidate detections plus matched-reference context.
|
|
- Added deterministic AOI/area stratification and an explicit five-state manual decision CSV. No decision is inferred and no training input is exported automatically.
|
|
- Separated references outside every persisted source tile into `false_negatives_outside_tile_coverage.geojson` instead of hiding them or calling them model misses.
|
|
- Added focused rendering, manifest, source-coverage and storage-confinement regression tests; wired the script into readiness compilation and the all-in-one image.
|
|
|
|
## Live Mol evidence
|
|
|
|
- Re-exported the four-zone fixed-threshold portfolio from existing persisted QualityChecks without rerunning inference or mutating application data.
|
|
- The complete portfolio contains `7,283` evidence features. Donk contributes `424` false positives and `553` false negatives; Postel contributes `43` and `85`.
|
|
- Rendered and inspected 48 stratified false-positive cases over Donk/Postel. Explicit decisions: `37` QA alignment mismatches, `7` confirmed model false positives, `3` reference gaps/changes and `1` uncertain. The existing validator passed with status `complete`.
|
|
- Rendered and inspected 48 stratified false-negative cases. Explicit decisions: `27` QA alignment mismatches, `6` confirmed model false negatives, `3` reference gaps/changes and `12` imagery-obscured/uncertain.
|
|
- Found `41/638` false-negative evidence records outside every persisted inference tile: Donk `28`, Postel `13`. Directionally excluding those records raises Donk recall from `0.5519` to `0.5647` and Postel from `0.3796` to `0.4194`; these are audit diagnostics only and no persisted metric was changed.
|
|
- The dominant visual mode is rectangle-to-footprint mismatch on large industrial roofs and dense residential blocks, often with a blue persisted candidate already overlapping the red missed GRB footprint. Postel additionally contains many tiny/vegetation-obscured references.
|
|
- Persistent evidence and the assessment are stored below `/app/storage/operator-evidence/mol-operational-review/20260713`.
|
|
|
|
## Decision and next pass
|
|
|
|
- **NO-GO for immediate retraining.** Only `7/48` reviewed false positives and `6/48` reviewed false negatives were confirmed model errors; evaluation alignment and coverage defects dominate the selected evidence.
|
|
- Next harden detection QA to restrict candidate/reference populations to persisted raster/tile coverage and expose best-IoU/overlap/unmatched diagnostics. Rerun Donk/Postel QA against the unchanged persisted detections before deciding whether the confirmed model-error subset justifies curated training.
|
|
|
|
## Validation
|
|
|
|
- Focused false-positive/false-negative audit and visual-review coverage: `8 passed`.
|
|
- Ruff passed for the changed Python renderer and regression tests.
|
|
- `bash scripts/run_readiness_check.sh`: passed with `487` backend tests, the 81-route API contract audit, one Alembic head, frontend typecheck and production build.
|
|
- `python -m alembic upgrade head --sql` rendered the complete migration chain through `202606120900`; no migration changed.
|
|
- Local `docker compose config` was unavailable because the Windows workstation has no Docker CLI. The repository-driven Tower deployment remains the required live Docker validation.
|
|
|
|
## Tower deployment evidence
|
|
|
|
- Pushed and deployed commits `50952e1` and `191d7aa` through the repository-driven all-in-one flow with the existing CPU AI dependency layer cached.
|
|
- Tower `docker compose -f docker-compose.unraid.yml config`, live PostGIS migration smoke, required schema/index checks and Alembic head `202606120900` passed.
|
|
- The definitive image contains the false-negative renderer under `/app/scripts`; persisted review artifacts remained available after both container replacements.
|
|
- Precomputing source-tile footprints in EPSG:4326 reduced the deployed 12-card Donk/Postel smoke from minutes to `2.06 s` while preserving `638` evidence records, `597` reviewable records and `41` explicit outside-coverage exclusions.
|
|
- Browser verification on `http://192.168.10.150:1202` selected the complete Mol Postel workspace and showed ready project/AOI/dataset/QA/export state.
|
|
- The Map workspace rendered one `1033x542` MapLibre canvas, the OSM road basemap, AOI and `95` persisted detections with no browser warnings/errors, no visible dialog and no horizontal document overflow at a `1280 px` viewport.
|
|
- No API contract, migration, QualityCheck/Metric row, Detection row, model asset, active-model configuration or inference result changed in this pass.
|
|
|
|
# Sprint 180 - Premium workbench UX hardening
|
|
|
|
## Implementation
|
|
|
|
- Reworked the workbench shell into a calmer operational hierarchy with grouped Workspace, Analyze and Deliver navigation, a compact Mol/Kempen context header and a centered content canvas.
|
|
- Replaced the permanently reserved inspector column with an on-demand detail drawer. Dataset inspection still invokes the existing detail-loading callback and now opens the drawer explicitly.
|
|
- Made Data operational at scale by disclosing create/upload forms on demand, arranging Project, AOI and Dataset panels side by side on desktop and bounding their long collections with internal scrolling.
|
|
- Promoted MapLibre to the primary Map surface. Layer provenance and low-frequency BBox/raw-feature controls remain available in collapsed detail surfaces without removing any GIS action.
|
|
- Reordered Detection and Segmentation Labs around run controls and result review; model registry and YOLO preflight remain fully available as secondary disclosures.
|
|
- Added a dedicated `premium.css` presentation layer with a restrained neutral/teal palette, consistent controls, stable panel dimensions and an actual full-width mobile shell. The former narrow sidebar-plus-content split is removed below `920 px`.
|
|
- Added focused static regression coverage for navigation grouping, optional inspector wiring, scalable Data panels, map hierarchy, AI Lab ordering and responsive behavior.
|
|
|
|
## Behavior preservation
|
|
|
|
- Existing React state hooks, API clients, service calls, map callbacks, QA actions, uploads, exports and AI run handlers were retained.
|
|
- No backend application code, endpoint contract, database model, migration, geospatial algorithm, model configuration or persisted evidence was changed.
|
|
- Legacy UI contract strings and callback signatures remain present for the existing Sprint 1-179 regression suite.
|
|
|
|
## Visual audit
|
|
|
|
- At `1280x720`, the Overview now uses one compact readiness row and no permanent inspector reservation; Map controls, canvas and secondary layer details follow the intended visual order.
|
|
- At `2560x1080`, the main workbench uses a centered `1680 px` maximum content width while the map can expand independently; the previous permanently empty `416 px` inspector column is gone.
|
|
- At `390x844`, navigation becomes a horizontal full-width rail and the main workspace occupies the viewport instead of sharing it with a `152 px` sidebar.
|
|
- A live 50-project Data state now keeps project and dataset panels within the viewport with internal scrolling instead of pushing the Dataset panel thousands of pixels below the fold.
|
|
- Detection and Segmentation run controls appear before registry diagnostics, reducing the distance to the primary task while retaining honest model-state detail.
|
|
|
|
## Local validation
|
|
|
|
- React best-practice review found no new conditional hooks, effect synchronization, unstable list keys or non-semantic interactive controls.
|
|
- Focused premium-workbench and legacy UI regression coverage passed.
|
|
- `python -m compileall backend/app`: passed.
|
|
- Full backend suite: `492 passed`.
|
|
- Frontend typecheck and production build: passed; app bundle `219.76 kB`, React vendor `140.74 kB` and MapLibre `801.82 kB` before gzip.
|
|
- `bash scripts/run_readiness_check.sh`: passed with the 81-route API contract audit, single Alembic head, all backend tests, frontend typecheck/build and live-smoke syntax gate.
|
|
|
|
## Tower deployment evidence
|
|
|
|
- Pushed and deployed commit `a2d9cef` through the repository-driven all-in-one flow on port `1202`.
|
|
- The production image rebuilt the frontend successfully and retained the existing cached CPU AI/GIS dependency layer.
|
|
- Live PostGIS `3.6`, database collation, required tables/indexes and the single Alembic head `202606120900` passed the migration smoke.
|
|
- The frontend, proxied projects API and Unraid icon passed the deployed browser runtime smoke.
|
|
- At `1280x720`, the populated Data workspace rendered three bounded columns; the 50-project panel remained `552 px` high with internal scrolling instead of extending its `8575 px` content into the document.
|
|
- At `2560x1080`, MapLibre rendered one nonblank `1678x734` canvas in a centered `1680 px` map surface with no permanently reserved inspector.
|
|
- At `390x844`, the sidebar became a `375 px` full-width horizontal navigation rail, the main workspace remained `375 px` wide and document width stayed within the viewport. The detail drawer opened full-screen and closed correctly.
|
|
- The live Map workspace preserved layer selection, AOI controls and persisted 95-feature detection overlay. Browser console verification returned zero warnings and zero errors across desktop, ultrawide, mobile, Data, Map and AI Labs checks.
|
|
|
|
## Next pass
|
|
|
|
- Harden detection QA coverage and matching diagnostics before making a retraining decision; keep that work separate from this presentation-only sprint.
|
|
|
|
# Sprint 181 - Complete Mol municipality workspace
|
|
|
|
## Implementation
|
|
|
|
- Added an explicit operator provisioner for the official VRBG `Refgem`
|
|
municipality geometry for Mol (NIS `13025`) and the complete paged GRB `GBG`
|
|
building collection clipped to that exact boundary.
|
|
- Added deterministic persistent source artefacts and a manifest containing
|
|
source URLs, checksums, page/feature counts, boundary bounds and area, and an
|
|
explicit truncation flag. Pagination and identity checks fail closed.
|
|
- Declared EPSG:4326 in both generated GeoJSON FeatureCollections so the
|
|
importer records `crs_assumed=false` for this known official OGC source.
|
|
- Added bounded retries for safe source GETs after the live refresh exposed a
|
|
transient GRB HTTP 500 on page 66. Mutating GeoIntel API requests are not
|
|
retried automatically.
|
|
- Kept the provider boundary honest: provisioning is an operator action and
|
|
imports through canonical Project, Area and Dataset HTTP routes. It does not
|
|
enable the dormant live GRB provider or write directly to `vector_features`.
|
|
- Optimized the existing vector persistence path by replacing one ORM refresh
|
|
per feature with a single flush and commit. Persistence shape and API
|
|
behavior remain unchanged.
|
|
- Made `Mol Municipality Workbench` the preferred fresh-session context once
|
|
its official boundary is ready. The boundary opens first; the much larger
|
|
building layer remains explicitly selectable from the Map database-layer
|
|
control.
|
|
- Replaced spread-based map extent calculations with a streaming, memoized
|
|
GeoJSON bounds helper and added municipality/building layer styling. This
|
|
avoids large coordinate arrays while keeping the existing MapLibre path.
|
|
|
|
## Initial validation
|
|
|
|
- Exact clipping, pagination/truncation, persistence-scaling and frontend
|
|
wiring regression coverage passed locally.
|
|
- Final full backend suite and readiness gate passed with `497` tests and the
|
|
`81`-route API contract audit. Frontend typecheck/build, Ruff, shell syntax,
|
|
single Alembic head `202606120900` and the complete offline migration chain
|
|
also passed.
|
|
- No API route, ORM model, migration, QA metric, detection result or model
|
|
configuration changed.
|
|
|
|
## Tower and live GIS evidence
|
|
|
|
- Pushed and deployed the implementation through commits `490325e`, `480634c`
|
|
and `e8a35b9`; live PostGIS `3.6`, required
|
|
schema/indexes, Alembic head, frontend proxy, API proxy and icon checks
|
|
passed.
|
|
- Provisioned definitive project `d74c1f87-29c0-4c67-adfc-560764f2b80e` and
|
|
official area `6551ee61-0bc7-47cd-8e38-94aefb815997` with bounds
|
|
`[5.035357, 51.1534809, 5.24218961, 51.32265943]` and projected area
|
|
`114.546677 km2`.
|
|
- The complete source bbox returned `89,153` unique GRB candidates over `90`
|
|
pages. Exact municipality clipping retained `36,941` buildings, rejected
|
|
`52,212` outside features and clipped `24` boundary-crossing features. The
|
|
manifest is complete and explicitly reports `reference_truncated=false`.
|
|
- PostGIS contains `36,941` vector feature rows and `36,941` distinct source
|
|
IDs, with zero invalid geometries and zero non-EPSG:4326 geometries. Strict
|
|
coverage reports 18 floating-point boundary artefacts whose combined
|
|
outside area is only `5.75e-9 m2`; no material building area falls outside
|
|
Mol.
|
|
- The definitive boundary dataset `a8be7de2-e9e1-437a-959c-a18d9a454886`
|
|
and building dataset `9b829bcb-eff5-48c2-baf9-ff6f8675fe7e` both persist
|
|
`EPSG:4326` with `crs_assumed=false`. The retry-hardened refresh completed all
|
|
90 pages after the earlier transient source failure.
|
|
- A fresh browser session selected the municipality project, official area
|
|
and one-feature boundary automatically instead of a historical Postel
|
|
validation context. Selecting the complete building layer rendered all
|
|
`36,941` features and a live AOI PostGIS query returned `250` persisted
|
|
features with dataset/export actions enabled.
|
|
- Browser verification at `1280x720` and `2560x1080` found one nonblank
|
|
MapLibre canvas, no console warnings/errors and no horizontal overflow. The
|
|
widescreen map canvas measured `1678x734` CSS pixels.
|
|
|
|
## Next pass
|
|
|
|
- Add viewport/zoom-aware vector delivery before expanding beyond Mol or
|
|
layering additional municipality-wide reference classes. Then acquire and
|
|
tile georeferenced imagery only for an explicitly selected Mol analysis zone
|
|
and run configured-YOLO plus QA/QC against the persisted GRB reference.
|
|
|
|
# Sprint 182 - Municipality viewport delivery and bounded AI handoff
|
|
|
|
## Implementation
|
|
|
|
- Added `useViewportVectorLayer` with centralized feature threshold, minimum
|
|
zoom, response limit and debounce policy. Vector datasets above 5,000
|
|
features now use the existing canonical PostGIS bbox-selection endpoint from
|
|
zoom level 14 instead of downloading the full stored GeoJSON file.
|
|
- Added MapLibre `moveend` viewport reporting, stale-request protection and
|
|
disabled automatic fit for viewport slices. AOI changes still frame the map;
|
|
small vectors and persisted analysis/evidence layers retain their existing
|
|
complete-layer fit behavior.
|
|
- Added explicit low-zoom, loading, visible/total, truncation and error states.
|
|
A selected large database layer is no longer presented as an empty or missing
|
|
layer before the first detail request.
|
|
- Extended the real-data detection/QA workflow with validated existing-project
|
|
reuse, safe distinct upload filenames and actual Area linkage for the raster
|
|
and matching reference vector. The workflow still consumes operator-provided
|
|
files and an existing local model asset; it performs no provider fetch or
|
|
model download.
|
|
|
|
## Local validation
|
|
|
|
- `python -m compileall backend/app` passed.
|
|
- Full backend suite passed: `500` tests.
|
|
- `npm run typecheck` and `npm run build` passed; the production build contains
|
|
85 transformed modules and preserves the dedicated MapLibre chunk.
|
|
- `bash scripts/run_readiness_check.sh` passed with the 81-route API contract
|
|
audit, one Alembic head `202606120900`, shell syntax checks and full frontend
|
|
build.
|
|
- Internal browser validation against the local frontend and live Tower API
|
|
selected `mol_grb_gbg_buildings.geojson`, showed the zoom-14 PostGIS delivery
|
|
guard, suppressed the incorrect empty-layer state and loaded an actual
|
|
viewport slice with `2 visible of 36,941 total features`. No browser warnings
|
|
or errors were recorded.
|
|
|
|
# Sprint 183 - Mol map source clarity and live AI validation
|
|
|
|
## Runtime evidence
|
|
|
|
- Deployed commit `1ba479b` to the all-in-one Tower runtime at
|
|
`http://192.168.10.150:1202`. The deployment rebuilt the updated
|
|
`postgres:16-bookworm` GIS runtime and installed the configured CPU AI image.
|
|
- Live migration smoke passed against PostGIS `3.6` with required runtime
|
|
schema objects, geometry indexes and the single Alembic head
|
|
`202606120900`. `/health` reported service, backend version and database as
|
|
healthy.
|
|
- Reused project `d74c1f87-29c0-4c67-adfc-560764f2b80e` and created bounded
|
|
area `0a38dd7a-d55a-4453-ae59-db8579ed8170` for
|
|
`Mol Centrum - AI analysezone 500m`.
|
|
- Persisted raster dataset `e9a292e2-57d2-4a85-9df6-32580ba6ea98` and GRB
|
|
reference dataset `bfc83882-2acc-470d-baec-ee828b4428f8` through the
|
|
canonical upload and vector persistence paths.
|
|
- Ran local model asset
|
|
`geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt` against the
|
|
persisted tile manifest. Analysis run
|
|
`97c204cc-bc75-4139-aab5-25397dd0d87e` persisted `36` detections.
|
|
- Quality check `1edb81ec-6799-41c1-ae25-9145f52407bd` persisted `17` matches,
|
|
`19` false positives and `357` false negatives. Persisted metrics were
|
|
precision `0.472222`, recall `0.045455`, F1 `0.082927` and mean IoU
|
|
`0.595803`; the workflow works, but these figures do not justify production
|
|
model acceptance.
|
|
- Detection export `d18946c4-b7c5-4262-83ad-6c406c6aa59c` was persisted.
|
|
|
|
## Map-source correction
|
|
|
|
- Live browser review exposed that the automatically selected detection run
|
|
could mask an explicitly selected database layer and leave the header at
|
|
`36 features`. Added a Database/Analysis result source mode and made
|
|
database selection return to Database mode.
|
|
- Local browser verification then selected the complete
|
|
`mol_grb_gbg_buildings.geojson` dataset, rendered a real `1,000`-feature
|
|
PostGIS viewport slice over the OpenStreetMap road basemap, displayed the
|
|
truncation/zoom guidance, switched to the 36-feature detection overlay and
|
|
back without state loss, and recorded no console errors.
|
|
|
|
## Next pass
|
|
|
|
- Clip detection QA reference populations to actual raster/tile coverage and
|
|
add box-to-building-footprint matching diagnostics before any further model
|
|
training or promotion decision.
|
|
|
|
# Sprint 184 - Detection QA coverage and matching diagnostics
|
|
|
|
## Implementation
|
|
|
|
- Added a focused detection-QA coverage service that resolves the persisted
|
|
tile manifest from the analysis run, validates dataset ownership and CRS,
|
|
transforms each tile footprint to EPSG:4326 and unions the exact inference
|
|
coverage.
|
|
- Configured-YOLO QA now fails closed when persisted manifest provenance is
|
|
absent, invalid or belongs to another raster. Explicit fixture and legacy
|
|
runs without a manifest retain their existing unbounded QA behavior.
|
|
- Candidate Detection geometries and persisted reference VectorFeature
|
|
geometries are clipped to inference coverage before the existing one-to-one
|
|
IoU matcher runs. Raw, evaluated, excluded-outside and boundary-clipped
|
|
counts are persisted in `quality_checks.findings_json`.
|
|
- Canonical precision, recall, F1, mean IoU and metric rows remain strict
|
|
candidate-polygon versus reference-footprint results. A second
|
|
candidate-polygon versus reference-envelope pass is persisted and displayed
|
|
as diagnostic-only evidence; it never replaces or promotes canonical
|
|
metrics.
|
|
- Detection Lab now explains which reference population was evaluated and
|
|
clearly separates possible box-to-footprint artifacts from the canonical
|
|
scorecard.
|
|
- Updated the real-data operator smoke assertions, API/AI/database contracts
|
|
and backend/frontend operator documentation. No migration, request contract,
|
|
provider fetch or model dependency changed.
|
|
|
|
## Local validation
|
|
|
|
- `bash scripts/run_readiness_check.sh` passed end to end.
|
|
- Full backend suite passed: `509` tests after the final coverage, manifest
|
|
provenance, framework-contract and frontend-toolchain regression additions.
|
|
- API contract audit passed with `81` implemented routes and the two documented
|
|
non-envelope endpoints.
|
|
- Alembic reports one head: `202606120900`.
|
|
- Frontend TypeScript checking and Vite 7 production build passed with `82` modules;
|
|
the dedicated MapLibre chunk remains intact.
|
|
|
|
## Next pass
|
|
|
|
- Deploy Sprint 184 to Tower, rerun QA for the persisted Mol-center analysis
|
|
run and verify both persisted coverage evidence and Detection Lab rendering
|
|
against live PostGIS before making any model-training decision.
|
|
|
|
## Tower and live Mol evidence
|
|
|
|
- Pushed commit `948e50b` and rebuilt the all-in-one runtime at
|
|
`http://192.168.10.150:1202`. PostGIS `3.6`, required schema/indexes,
|
|
database collation, Alembic head `202606120900`, frontend proxy, API proxy
|
|
and icon checks passed.
|
|
- Reused persisted analysis run `97c204cc-bc75-4139-aab5-25397dd0d87e`,
|
|
raster `e9a292e2-57d2-4a85-9df6-32580ba6ea98` and GRB reference dataset
|
|
`bfc83882-2acc-470d-baec-ee828b4428f8`; no duplicate inference or fake data
|
|
was produced.
|
|
- The persisted EPSG:31370 tile manifest contained one inference tile. All 36
|
|
detections were evaluated; one was clipped at the tile boundary. Of 374
|
|
persisted reference footprints, 304 intersected coverage, 70 were excluded
|
|
outside coverage and 34 were clipped at the boundary.
|
|
- Quality check `a25290ed-14ea-4c84-aff7-20672f568b1c` persisted 19 strict
|
|
footprint-IoU matches, 17 false positives and 285 false negatives. Its six
|
|
canonical Metric rows contain precision `0.527778`, recall `0.0625`, F1
|
|
`0.111765`, mean IoU `0.599302` and the two error counts.
|
|
- The diagnostic-only reference-envelope pass found 33 matches and identifies
|
|
14 possible box-to-footprint matching artifacts. Those values are persisted
|
|
only in findings and do not alter canonical Metric rows.
|
|
- Internal-browser validation reran the comparison from Detection Lab and
|
|
displayed `304 of 374 reference features evaluated`, coverage exclusions,
|
|
boundary clipping and the explicitly labelled diagnostic. The deployed page
|
|
produced no console warnings or errors for the live Tower URL.
|
|
- A later clean framework resolution exposed FastAPI `0.139.0`'s grouped
|
|
top-level router representation: `app.routes` contains 17 groups while the
|
|
canonical OpenAPI schema still contains all 81 operations. Updated the API
|
|
contract audit to read the OpenAPI path map and added a regression guard so
|
|
clean framework installations cannot produce a false 81-route drift report.
|
|
- Starlette `1.3.1` also deprecates the existing `httpx` TestClient path in
|
|
favor of `httpx2`. Kept Starlette on the still-supported
|
|
`>=0.46.0,<1.0.0` line until that test-client migration receives a dedicated
|
|
compatibility pass; the release gate consequently remains warning-free.
|
|
- The cache-free frontend dependency install exposed one high and one moderate
|
|
Vite/esbuild development-server advisory. Upgraded only the build toolchain
|
|
to Vite `7.3.6` and React plugin `5.2.0`, retained React 18 and MapLibre 4,
|
|
and declared Node `^20.19.0 || >=22.12.0`. `npm audit`, TypeScript checking
|
|
and the production build then passed with zero known vulnerabilities.
|
|
- A full cache-free Tower image build installed the complete Debian GIS stack,
|
|
Starlette `0.52.1`, FastAPI `0.139.0`, PyTorch `2.13.0+cpu`, TorchVision
|
|
`0.28.0+cpu` and Ultralytics `8.4.95`. GIS import smoke, model-file
|
|
discovery, PostGIS `3.6`, schema/index checks, migration head and browser
|
|
proxy verification all passed from the newly built image.
|
|
- Deployed final release-hardening commit `68ee228` to Tower. The image build
|
|
used Vite `7.3.6`, reported zero npm vulnerabilities, and the all-in-one
|
|
container became healthy on port `1202`. A fresh live QA comparison created
|
|
quality check `6a0fba57-fae8-4963-95ce-3ba89a3587a4`; the browser rendered
|
|
the corrected single-tile wording, persisted coverage counts and diagnostic
|
|
separation without console warnings or errors from the Tower host.
|
|
|
|
## Decision
|
|
|
|
- Sprint 184 is operationally complete. Coverage bias is now explicit and the
|
|
observed 14-match geometry gap confirms that model quality must not be judged
|
|
from envelope diagnostics. The next safe model step is an evidence-led
|
|
footprint-label/matching review, followed by a fresh bounded Mol multi-zone
|
|
benchmark before any activation or retraining decision.
|
|
|
|
# Sprint 185 - Coverage-aware Mol operational benchmark
|
|
|
|
## Implementation
|
|
|
|
- Extended the existing real-data quality matrix summaries with the exact
|
|
persisted tile-coverage population, raw/evaluated/excluded/clipped candidate
|
|
and reference counts, CRS/tile provenance and diagnostic-only
|
|
box-to-footprint match gap.
|
|
- Preserved manifest-backed Mol municipality, operational-zone, validation
|
|
split, WGS84 bounds and source reference counts in the multi-sample summary.
|
|
- Added `build_mol_operational_benchmark_report.py`. It groups evidence by the
|
|
exact model asset, tile size, overlap and confidence threshold and refuses to
|
|
blend different candidate configurations.
|
|
- Added explicit gates for four independent positive Mol holdouts, one
|
|
pure-empty background control, complete coverage provenance, minimum 90%
|
|
reference coverage, mean F1 `0.25`, minimum per-zone F1 `0.10` and zero
|
|
background detections. Envelope matches remain diagnostic and cannot satisfy
|
|
the canonical F1 gates.
|
|
- Wired the report into the persistent Mol operator runner, readiness gate and
|
|
all-in-one image. The report does not activate, replace, download or train a
|
|
model and does not change APIs, migrations or inference behavior.
|
|
|
|
## Local validation
|
|
|
|
- Focused Sprint 126/127/178/184/185 regression set passed: `14` tests.
|
|
- Full readiness passed with `512` backend tests, one Alembic head
|
|
`202606120900`, 81 audited API operations, frontend typecheck and Vite 7.3.6
|
|
production build.
|
|
- `npm audit --audit-level=high` reported zero vulnerabilities.
|
|
- Shell syntax checks passed for the single-, multi-sample and Mol operational
|
|
runners. Fixture tests prove both an accepted four-zone result and explicit
|
|
rejection for missing coverage, one-zone collapse and background pressure.
|
|
|
|
## Next pass
|
|
|
|
- Deploy the operator tooling to Tower and run the current active local model
|
|
over Achterbos, Gompel, Donk, Postel and the Postel-bos pure-empty control.
|
|
Record the coverage-aware operational decision before changing model state.
|
|
|
|
## Tower benchmark evidence
|
|
|
|
- Deployed implementation commit `9f61037`, completed the real benchmark and
|
|
then deployed final evidence/default-gate commit `5db93a9` to the healthy
|
|
all-in-one runtime with active asset
|
|
`geointel-building-yolov8s-smallbld-minpx3-img640-ft30-pt`, tile `512`,
|
|
overlap `64`, confidence `0.15` and canonical QA IoU `0.25`.
|
|
- Canonical results by holdout were: Achterbos F1 `0.6694`, Gompel `0.6564`,
|
|
Donk `0.5894` and Postel `0.4749`. Macro precision/recall/F1 were
|
|
`0.6296` / `0.5726` / `0.5975`; micro precision/recall/F1 were `0.6494` /
|
|
`0.6189` / `0.6338`.
|
|
- Across all positive zones, 3,624 of 3,829 raw reference features intersected
|
|
exact tile coverage. The runner excluded 205 outside features, clipped 60 at
|
|
boundaries and kept 101 additional envelope matches diagnostic-only.
|
|
- Postel-bos produced zero detections. All model, per-zone, coverage provenance,
|
|
diagnostic-separation and pure-empty-background gates passed. The minimum
|
|
coverage ratio was `0.9051`; the integrity gate was set to `0.90` because
|
|
OGC bbox retrieval intentionally retains complete edge geometries outside the
|
|
projected raster polygon.
|
|
- The final report under
|
|
`/app/storage/operator-evidence/mol-operational-validation/coverage-aware-20260714T1000Z`
|
|
is `accepted` with recommendation `retain_or_promote_candidate`. No model
|
|
state or weights were changed by the report.
|
|
- Exported a four-zone portfolio with 7,078 persisted QA evidence features.
|
|
Audits found 1,381 canonical false negatives and 1,211 false positives.
|
|
False-negative medians were about 24-28 m2 and tiny/small buildings dominate;
|
|
Postel and Donk are the weakest zones.
|
|
- Rendered and visually inspected 48 false-negative plus 48 false-positive
|
|
review cards across every holdout. No source tile was missing. The cards show
|
|
a mixture of genuine small-building/model errors, temporal GRB/orthophoto
|
|
differences and box-versus-footprint matching artifacts, so raw FP/FN counts
|
|
must not be converted directly into new labels.
|
|
|
|
## Decision
|
|
|
|
- Retain the active model for bounded operator-assisted Mol workflows and demos
|
|
with QA/QC visible. It is not ground truth and should not be auto-exported as
|
|
authoritative buildings. Do not start another blind training run; first
|
|
complete the generated manual review decisions and curate confirmed small-
|
|
building positives plus confirmed visual hard negatives from the weakest
|
|
zones.
|
|
- Final internal-browser verification loaded the official Mol municipality
|
|
boundary over the OpenStreetMap road basemap from the persisted PostGIS
|
|
workspace. The live Tower page reported no console warnings or errors.
|
|
## Sprint 186 Map-first Mol geographic explorer (2026-07-14)
|
|
|
|
Changed:
|
|
- Made Map the default application workspace and reduced the primary task to data theme, rectangle selection and evidence review.
|
|
- Added a dedicated three-column explorer with explicit available/missing themes, full-Mol scope, true MapLibre drag selection, automatic PostGIS queries, exact totals, density, property evidence and GeoJSON handoff.
|
|
- Preserved the previous technical workflow behind `Geavanceerde werkbank` and kept QA, AI and export persistence unchanged.
|
|
- Added exact `total_feature_count` to the vector bbox-selection response while retaining the existing 1,000-feature geometry cap.
|
|
- Added `provision_mol_context_layers.py` for official GRB roads (`Wegsegment`), water (`WTZ`, `WLAS`, `WGR`) and parcels (`ADP`) clipped to NIS 13025 and imported through the public dataset API.
|
|
|
|
Validated:
|
|
- Frontend typecheck and production build passed.
|
|
- Focused map, orchestration and new explorer tests passed.
|
|
- Official fetch-only smoke produced 8,444 Mol road features, 3,668 water features and 32,961 parcels with complete pagination and no truncation.
|
|
- Full backend suite passed 517 tests; one legacy component-boundary guard initially failed and was resolved by moving theme API orchestration into `useMapThemeSelectionInsights`.
|
|
- Tower was rebuilt from `main`; live PostGIS 3.6 connectivity, Alembic head `202606120900`, required geometry tables/indexes and browser proxy health all passed.
|
|
- The official provisioner persisted 8,444 roads, 3,668 water features and 32,961 parcels alongside the existing 36,941 GRB buildings for the complete Mol municipality.
|
|
- In-app browser validation of `Volledige gemeente` reported 114.55 km2, 36,941 buildings and a building density of 322.5/km2; the bounded preview clearly reported 1,000 of 36,941 features.
|
|
- Dataset-detail loading now ignores stale asynchronous responses, preventing a slower municipality-boundary request from overwriting the currently selected building layer or its map context.
|
|
- Theme switching reuses the same analyzed rectangle and keeps GeoJSON download/copy actions bound to the active theme result instead of a stale technical dataset selection.
|
|
|
|
Limitations:
|
|
- Population and forest/green remain unavailable rather than simulated until suitable authoritative sources and semantics are selected.
|
|
|
|
Next:
|
|
- Define authoritative population and land-cover source adapters, then reuse the proven municipality provisioner and bbox analysis flow for the complete Kempen.
|
|
|
|
## Sprint 187 Temporal Mol explorer (2026-07-14)
|
|
|
|
Implemented:
|
|
- Added observation/validity/source-version fields to datasets and immutable provenance fields to dataset versions, with one Alembic head and indexed temporal/source identity lookups.
|
|
- Persisted dataset version 1 atomically for uploads, demo fixtures and derived vector/raster operations.
|
|
- Added source-governed PostGIS selection summaries for object count, area, length, numeric sum and area-weighted sum.
|
|
- Added project temporal-series discovery and same-series bbox comparison with honest metric deltas and stable-identity-only object changes.
|
|
- Added the map-first `Laatste toestand` / `Evolutie` workflow with period selection, automatic rectangle analysis and change overlays.
|
|
- Added explicit, idempotent Statbel population (2021-2025) and Digitaal Vlaanderen historical land-use (1778/1873/1969) provisioners for Mol.
|
|
- Kept all source fetching operator-triggered; application startup and user queries never fabricate or silently download source data.
|
|
|
|
Methodology:
|
|
- Population totals are source-published per statistical sector. Intersections with partial sectors are labelled area-weighted estimates.
|
|
- Historical land-use classes are clipped from official editions and measured in EPSG:31370. They do not claim stable cadastral object identity.
|
|
- Every snapshot carries its source URL, observation date, source version, checksum and processing limitations.
|
|
|
|
Validation before live deployment:
|
|
- Script compilation passed.
|
|
- New temporal/API/static regression suite passed 6 tests, including append-only/idempotent temporal provenance updates.
|
|
- Raster and temporal focused suite passed 27 tests after extending existing assertions to require dataset-version persistence.
|
|
- Frontend TypeScript typecheck and production build passed.
|
|
- Offline Alembic SQL generation passed with head `202607140001`.
|
|
|
|
Next:
|
|
- Run the complete readiness gate, deploy to Tower/PostGIS, provision the official snapshots and verify current/evolution selection end to end in the internal browser.
|
|
|
|
Live deployment correction:
|
|
- The first Tower rollout exposed a deployment race: all-in-one startup and `live_migration_smoke.sh` both began `alembic upgrade head` after PostgreSQL became reachable.
|
|
- Startup committed head `202607140001`; the concurrent smoke transaction rolled back on a duplicate first column. Database contents and the successful migration remained healthy.
|
|
- Both Tower deploy entry points now wait for the `geointel` container healthcheck, which includes completed startup migrations and backend readiness, before launching the independent migration smoke.
|
|
- The first Statbel upload exposed 3D sector coordinates (`Z=0`) against the canonical 2D PostGIS vector column. The source is valid; GeoIntel now preserves the original artifact, records the Z-feature count and explicitly drops Z only for the 2D query index.
|
|
- Vector upload persistence is now atomic across Dataset, DatasetVersion and VectorFeature rows, with storage cleanup on rollback. This prevents the failed-indexing orphan state observed during the live import.
|
|
- Tower could read the historical WFS capabilities but its gateway rejected XML `FILTER` content embedded in a GET query. The operator now sends the same read-only WFS 2.0 GetFeature request as XML POST; a live two-feature forest page returned successfully before redeployment.
|
|
- The 1778 edition classifies roads as `weg`, while later editions can use road subclasses. The exact server-side road filter now uses `weg*`, covering the documented class family without admitting unrelated themes.
|
|
- Live population comparison returned 37,015 inhabitants for 2021 and 38,675 for 2025 (+1,660 / +4.48%). Full covered sectors now report exact source totals; only cut sectors are labelled area-weighted estimates.
|
|
- Population sector identity is explicitly unstable across annual geometry editions, so source boundary/code changes are not presented as added/removed population objects.
|
|
|
|
Live operational proof:
|
|
- Provisioned the complete Mol context in the canonical municipality project: 36,941 GRB buildings, 8,444 roads, 3,668 water features and 32,961 parcels.
|
|
- Provisioned five official Statbel population snapshots (2021-2025) and twelve official historical land-use datasets for buildings, forest, water and roads in 1778, 1873 and 1969.
|
|
- The full municipality browser flow reported 114.55 km2, 36,941 buildings, 38,675 inhabitants, 4,203.75 ha historical green/forest class, 3,668 water features, 8,444 roads and 32,961 parcels.
|
|
- The full-Mol population comparison reported 37,015 inhabitants in 2021 and 38,675 in 2025 (+1,660 / +4.48%) as an exact whole-sector total.
|
|
- A drawn 29.99 km2 northern-Mol rectangle returned 540 buildings, an area-weighted estimate of 152 inhabitants, 1,365.74 ha historical green/forest class, 624 water features, 732 roads and 1,715 parcels. Its 2021-2025 population estimate changed from 151 to 152 and was explicitly labelled `Ruimtelijke schatting`.
|
|
- Internal-browser checks at 1280x720 and 3440x1440 confirmed no horizontal overflow. A ResizeObserver plus workbench-scoped absolute map fill keeps the MapLibre canvas equal to its GIS stage after responsive layout changes.
|
|
- The final deployed runtime passed container health, PostGIS 3.6 connectivity, required schema/index checks, Alembic head `202607140001`, frontend proxy verification and browser console inspection with no warnings or errors.
|
|
|
|
Remaining source limitations:
|
|
- The latest official population snapshot in this workspace is 2025.
|
|
- The forest/green evolution series currently represents the available historical land-use editions through 1969; it must not be presented as current forest cover.
|
|
- Partial-sector population results remain area-weighted estimates because no finer authoritative population surface has been ingested.
|
|
|
|
## Sprint 188 Official modern Mol land-use series (2026-07-14)
|
|
|
|
Implemented:
|
|
- Added `provision_official_landuse_timeseries.py` for explicit MercatorNet WCS subsets of the official Departement Omgeving version 3 land-use maps for 2013, 2016, 2019, 2022 and 2025.
|
|
- Validated one-band integer GeoTIFF input, EPSG:31370, 10 m resolution and the documented 1-19 class domain before processing.
|
|
- Preserved every raw raster, exact request URL, catalogue URL, raster/vector checksum, class histogram and processing manifest in operator storage.
|
|
- Polygonized documented class 12 (`Bos`) in metric CRS, clipped it to the official Mol boundary, normalized it to EPSG:4326 and uploaded it through the existing API/DatasetService/vector-feature path.
|
|
- Added source-governed hectare aggregation metadata, 10 m/non-cadastral limitations and unstable raster-polygon identity declarations.
|
|
- Kept `department-omgeving:land-use:forest:mol` separate from the historical land-use series and added a compact frontend series selector.
|
|
|
|
Source validation:
|
|
- Live WCS capabilities exposed all five expected coverages through `lu:lu_landgebruik_vlaa_<year>_v3`.
|
|
- A full-Mol fetch-only run produced 3,768 / 3,823 / 3,953 / 3,615 / 3,676 forest polygons for 2013 / 2016 / 2019 / 2022 / 2025 without truncation.
|
|
- Polygonized forest area was 3,723.17 / 3,615.39 / 3,592.27 / 3,648.98 / 3,626.56 ha. These are measurements within the official 10 m representation, not cadastral forest areas.
|
|
- Every raster cell touching the municipality is considered before exact vector clipping. The 916 NoData edge cells in each WCS subset were explicitly excluded and recorded rather than assigned a class.
|
|
|
|
Validation:
|
|
- The first live import was rejected before any upload because the operator requested 500 datasets while the canonical API limit is 200. The operator now follows every 200-item page and a focused pagination regression test covers 405 records.
|
|
- New focused backend suite passed 5 tests.
|
|
- Full readiness passed 533 backend tests, one Alembic head, frontend TypeScript typecheck/build and all repository smoke gates.
|
|
- Tower rebuilt successfully and passed PostGIS 3.6, migration, container health and browser-proxy validation.
|
|
- All five modern forest snapshots were persisted in the canonical Mol project: 3,768 / 3,823 / 3,953 / 3,615 / 3,676 vector features for 2013 / 2016 / 2019 / 2022 / 2025. A second operator run reused every existing dataset without uploading duplicates.
|
|
- The live full-Mol comparison returned 3,723.17 ha for 2013 and 3,626.56 ha for 2025, a measured change of -96.60 ha (-2.60%) within the documented 10 m raster representation.
|
|
- Internal-browser verification confirmed latest-state source selection, separate modern and historical series, exact warning text, valid MapLibre rendering and an empty warning/error console.
|
|
- A final presentation correction aligned the active MapLibre and legend colors with each data theme and removed the old 105rem content cap only for the primary explorer on ultrawide screens.
|
|
- Deployed presentation commit `3a2d1fa` to Tower. The all-in-one container became healthy, live PostGIS/migration/schema validation passed and the browser proxy remained reachable on port 1202.
|
|
- Final in-app browser measurements showed no horizontal overflow at 1280x720 or 3440x1440. At 3440 px the explorer uses 3,134 px and its MapLibre stage 2,408 px; forest layer/legend colors resolve to the green theme and the console contains zero warnings or errors.
|
|
|
|
Next:
|
|
- Define the exact municipality list or administrative boundary that GeoIntel will call `Kempen`, then reuse the proven source operators and temporal query path for that scope without changing Mol semantics.
|
|
|
|
## Sprint 189 Official Kempen operational scope (2026-07-14)
|
|
|
|
Implemented:
|
|
- Chose the official Vlaamse `Vervoerregio Kempen` as GeoIntel's reproducible operational regional scope. It has 28 municipalities and explicitly includes both Mol and Nijlen.
|
|
- Added `geographic_scopes.py` as the operator registry for scope identity, current municipality names/NIS codes, policy authority and limitation text.
|
|
- Added `provision_geographic_scope.py` to fetch current authoritative VRBG boundaries, validate all members, create a valid regional union and retain boundary/member GeoJSON plus checksums and manifest.
|
|
- The operator creates or reuses `Kempen Regional Workbench`, one regional Area, 28 municipality Areas and two source datasets exclusively through canonical API envelopes.
|
|
- Added a compact Mol/Kempen selector to the primary map flow, scope-aware heading text and the neutral `Volledig werkgebied` action.
|
|
- Kept thematic regional fetching outside the scope pass. No startup fetch, direct PostGIS write, fake metric or unbounded regional theme download was introduced.
|
|
|
|
Source proof:
|
|
- A live read-only VRBG run matched all 28 registered municipalities and current NIS codes.
|
|
- The valid union measured 1,399.2505 km2 in EPSG:31370 with WGS84 bbox `[4.59723873, 51.01047967, 5.26224853, 51.50511313]`.
|
|
- The scope is labelled as the transport-policy region and never as the complete cultural, landscape or historical Kempen.
|
|
|
|
Validation before deployment:
|
|
- New focused scope and map-flow suite passed 9 tests.
|
|
- Python scope/operator compilation and frontend TypeScript typecheck passed.
|
|
|
|
Deployment target:
|
|
- Deploy and run the scope operator on Tower, verify 29 Areas and both boundary datasets, then design bounded regional theme partitions before importing high-volume GRB or WCS data.
|
|
|
|
Live operational proof:
|
|
- Deployed scope foundation commit `bb7310e` and context-isolation/copy corrections through `94f5e01` to the all-in-one Tower runtime on port 1202.
|
|
- Provisioned `Kempen Regional Workbench` through the canonical API with one regional Area, 28 municipality Areas and two VRBG datasets. Repeating the operator returned the same project, Area and dataset identifiers.
|
|
- The live project contains 29 Areas, one valid regional-boundary feature and 28 municipality-boundary features with persisted authority, policy-scope limitation, source URL and checksums.
|
|
- A live Mol -> Kempen switch now clears all project-bound state, rejects stale project-data responses, selects `Vervoerregio Kempen - officiële operationele grens` and opens `kempen_transport_region_boundary_2026-07-14.geojson` without retaining a Mol dataset.
|
|
- The reverse Kempen -> Mol switch restored `Gemeente Mol - officiële grens`, the GRB building layer and all six available Mol themes.
|
|
- Internal-browser checks passed at 1280x720 and 3440x1440. The ultrawide explorer used 3,134 px, its MapLibre canvas used 2,408 px and no horizontal overflow or browser warning/error was present.
|
|
- Full readiness passed 538 backend tests, one Alembic head, frontend typecheck/build and every repository smoke gate. Tower additionally passed container health, PostGIS 3.6, migration/schema/index and frontend proxy checks.
|
|
|
|
Next:
|
|
- Build bounded, idempotent regional theme ingestion in municipality-sized partitions, starting with current buildings and retaining per-member provenance before exposing any Kempen-wide metric in the explorer.
|
|
|
|
## Sprint 194 - Regional official time-series operator foundation (2026-07-14)
|
|
|
|
Implemented:
|
|
- Made the existing Statbel population operator scope-aware while retaining Mol defaults and artifact compatibility.
|
|
- Added approved-scope NIS filtering, per-feature municipality provenance, checksummed scope-boundary resolution and distinct regional temporal-series keys.
|
|
- Added `provision_regional_timeseries.py` as one explicit coordinator for population and modern forest imports into `Kempen Regional Workbench`.
|
|
- Reused the current Dataset upload API and existing forest polygonization path; no migration, public endpoint, direct PostGIS write, startup fetch or fabricated value was added.
|
|
- Corrected regional forest provenance so a 28-member coverage does not claim to be one municipality.
|
|
- Added friendly Statbel/GRB/VRBG/Departement Omgeving labels to the primary map.
|
|
|
|
Validation before deployment:
|
|
- Source contracts were rechecked against the official Statbel sector-population page and the official 2025 Landgebruik Vlaanderen catalogue.
|
|
- The focused regional/temporal suite passed 19 tests.
|
|
- Frontend TypeScript typecheck and production build passed.
|
|
|
|
Next:
|
|
- Deploy the packaged operator, execute the live regional synchronization, verify persisted counts and temporal comparisons, then mark the regional time-series TODO complete only if PostGIS and browser evidence agree.
|
|
|
|
Live source finding:
|
|
- The regional Statbel synchronization imported all five requested snapshots successfully.
|
|
- The first complete-region forest request was rejected by MercatorNet before download because the estimated 97.68 MB response exceeded its 78.12 MB service limit.
|
|
- The operator now uses the 28 retained official municipality boundaries as resumable WCS partitions, merges them locally at the unchanged 10 m grid and applies the exact region-union clip before polygonization. It does not lower resolution or truncate the source.
|
|
- Live full-region analysis returned the correct six-theme result but initially took about 92 seconds because every already-clipped feature was intersected again with a 947 kB regional geometry. Added a provenance-gated full-Area path that skips only this redundant intersection; arbitrary rectangles and untrusted uploads retain the existing exact query.
|
|
|
|
## Sprint 190 Regional Kempen GRB buildings (2026-07-14)
|
|
|
|
Implemented:
|
|
- Added `provision_regional_grb_buildings.py` for the approved 28-member scope, with complete OGC pagination, per-member safety caps, resumable checksummed artifacts and an explicit total cap.
|
|
- Added deterministic cross-boundary ownership by maximum member intersection and NIS-code tie breaking, preventing duplicate source identities without cutting the final geometry at internal municipality borders.
|
|
- Added streaming managed-artifact copy plus batch-wise partition indexing through DatasetService and VectorFeatureService. One regional Dataset remains compatible with existing viewport, selection, QA and export ownership boundaries.
|
|
- Added exact manifest/index count enforcement, duplicate source-feature rejection and rollback/storage cleanup on import failure.
|
|
- Kept fetching operator-triggered and local service import container-only. No startup fetch, browser provider fetch, direct vector-feature SQL or public API change was introduced.
|
|
|
|
Validation before deployment:
|
|
- Backend/application and operator compilation passed.
|
|
- New focused operator/storage/batch-index suite passed 6 tests.
|
|
- Dataset, vector persistence, Mol municipality and Kempen scope regression selection passed 37 tests.
|
|
|
|
Live operational proof:
|
|
- Deployed the regional operator and its interior-feature fast path through commit `a879c74`, then completed all 28 municipality partitions against the official GRB `GBG` collection.
|
|
- The immutable `2026-07-14` snapshot contains 466,078 buildings from 879 source pages. The combined retained artifact is 478,143,249 bytes and records `reference_truncated=false`.
|
|
- Dataset `e236fa03-7fce-4b37-bc7d-8bddd4d50346` is `ready`, `reference`, `source_name=grb`, `reference_layer_name=buildings`, EPSG:4326 and has one DatasetVersion. A repeat operator run returned `reused=true` in 3.5 seconds.
|
|
- Direct PostGIS verification returned 466,078 rows, 466,078 distinct source IDs, zero missing source IDs, zero invalid/empty geometries and zero non-4326 rows. The geometry mix is 466,069 Polygon and 9 MultiPolygon features.
|
|
- Live browser verification exposed an exact-scope defect: selecting Mol inside the regional dataset initially counted the surrounding bbox and returned 86,222 buildings. Commit `616424e` added optional persisted-Area filtering while retaining backward-compatible bbox selection for drawn rectangles.
|
|
- After redeploy, the complete Vervoerregio Area returned exactly 466,078 buildings and the official Mol Area exactly 36,941. A drawn rectangle returned 308 measured buildings, and zoom-level detail loaded 730 visible features from PostGIS onto MapLibre.
|
|
- Full readiness passed 546 backend tests, one Alembic head, frontend typecheck/build and all repository gates. Tower passed health, PostGIS 3.6, schema/index and proxy checks; browser audits at 1280x720 and 3440x1440 had no console warnings/errors or horizontal overflow.
|
|
|
|
Next:
|
|
- Reuse the bounded regional operator pattern for current roads, water and parcels, then add official population and land-use time series without changing Mol semantics or introducing interactive external fetching.
|
|
|
|
## Sprint 195 - Live guided detection and bounded regional QA (2026-07-14)
|
|
|
|
Implemented:
|
|
- Added one guided Detection Lab action that uploads a georeferenced raster through `DatasetService`, creates or reuses the canonical tile manifest, runs preflight, invokes the configured local YOLO adapter and opens persisted detection geometry on MapLibre.
|
|
- Live Tower validation used the official Mol orthophoto sample, produced nine 512 px tiles and persisted 1,953 detections in analysis run `3912e179-3d1d-4080-8093-f883bbe95d9c`.
|
|
- A real QA attempt against the 466,078-feature regional GRB building dataset revealed that coverage clipping happened after every reference row had already been materialized.
|
|
- Moved configured-YOLO reference bounding into the existing GiST-indexed PostGIS query with the persisted manifest coverage. Full source count, evaluated count and excluded count remain explicit in the persisted evidence.
|
|
- Added a Shapely STRtree candidate index around the unchanged exact IoU matcher and clarified the simple map legend/source whenever AI detections are active.
|
|
|
|
Validation before final deployment:
|
|
- Focused detection, coverage, QA and guided-workflow tests passed.
|
|
- Frontend TypeScript typecheck passed after the source/legend correction.
|
|
|
|
Live operational proof:
|
|
- The final repository readiness gate passed 578 backend tests, API/document audits, frontend typecheck/build, one Alembic head and all shell smoke checks.
|
|
- Deployed commit `0cad8fd` to the Tower all-in-one runtime with AI extras enabled. Health, PostGIS 3.6, the single migration head, required schema/indexes and the port-1202 browser proxy all passed.
|
|
- The bounded QA request completed in 1.82 seconds. It retained a raw GRB population of 466,078, materialized/evaluated 1,826 coverage features and excluded 464,252 outside the nine persisted tiles.
|
|
- At match IoU 0.25, persisted quality check `341b0076-663a-4427-8a36-d5d9247a730e` measured 1,186 matches, precision 0.6073, recall 0.6495, F1 0.6277 and mean IoU 0.4565. No metric was fabricated.
|
|
- A separate UI run at its default stricter match IoU 0.50 also completed and persisted quality check `7a4464cd-152d-424f-a2b9-ed321b91524c`; the lower F1 is now explained by displaying the threshold beside the score.
|
|
- In-app browser inspection confirmed 1,953 persisted rows, map rendering over OpenStreetMap and explicit `AI-detecties`, `Detectierun`, `Gevonden gebouwen` and `controle vereist` labels.
|
|
|
|
Next:
|
|
- Add bounded operator-triggered orthophoto acquisition from a drawn map rectangle, then hand that raster to this proven guided pipeline. Keep external acquisition out of browser/startup paths and retain explicit source licensing/provenance.
|
|
|
|
## Sprint 196 - Map-driven official orthophoto analysis (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added explicit project-scoped acquisition for the official Digitaal
|
|
Vlaanderen `OMWRGBMRVL` WMS `Ortho` layer.
|
|
- Enforced EPSG:4326 input, EPSG:31370 metric bounds, 128-1,024 m side limits,
|
|
optional persisted-Area coverage, 1 m sampling, timeout/response limits and
|
|
24-hour exact-request reuse.
|
|
- Georeferenced the RGB TIFF and persisted it, its DatasetVersion, source,
|
|
request URL/hash, bounds, attribution and latest-mosaic limitation only
|
|
through DatasetService.
|
|
- Added one simple map action chaining acquisition, existing raster tiling,
|
|
configured-YOLO inference, Detection persistence and existing GRB detection
|
|
QA before showing the persisted result on MapLibre.
|
|
- Exposed orthophoto settings through Docker Compose, the all-in-one Unraid
|
|
runtime script and DockerMan template.
|
|
|
|
Validation before deployment:
|
|
- Focused service/API tests passed, including in-memory TIFF georeferencing,
|
|
persistence and the canonical Job envelope.
|
|
- Frontend TypeScript typecheck and production build passed.
|
|
|
|
Next:
|
|
- Deploy to Tower, execute one bounded Mol rectangle through the real WMS,
|
|
local model and GRB QA, and verify persistence plus browser state before
|
|
accepting the flow as operational.
|
|
|
|
Live operational proof:
|
|
- Deployed commits `daccd38` and `ec1e1af` to the Tower all-in-one runtime on
|
|
port 1202. The latter keeps acquisition inside the approved Kempen regional
|
|
Area while municipality selection remains an independent map filter.
|
|
- A rectangle in Mol fetched a 244 x 207, 1 m/pixel RGB GeoTIFF from the
|
|
Digitaal Vlaanderen `OMWRGBMRVL` `Ortho` WMS in EPSG:31370. Acquisition job
|
|
`8f6fd7f6-8f8a-436f-be54-d2c7e28ea250` completed successfully and created
|
|
Dataset `35f9f72e-edec-4053-887c-f4524222756d` with exactly one immutable
|
|
DatasetVersion.
|
|
- Existing canonical tiling produced one persisted inference tile. Analysis
|
|
run `e68a6291-9c46-4f23-832a-3c2b7480f94f` completed with model
|
|
`yolo-configured` and exactly 78 persisted Detection rows.
|
|
- Existing bounded GRB QA materialized 117 of 466,078 reference buildings for
|
|
the persisted tile footprint. Quality check
|
|
`3f1223f0-7dc1-40bb-a48b-910af2a343bf` persisted six Metric rows: 44 matches,
|
|
34 false positives, 73 false negatives, precision 0.5641, recall 0.3761,
|
|
F1 0.4513 and mean IoU 0.6622 at IoU 0.50. The result remains explicitly
|
|
labelled as AI output requiring review.
|
|
- The in-app browser completed the full action, displayed the 78 persisted
|
|
geometries over the road basemap and reported no console errors. Layout
|
|
checks passed at 1280 x 720 and 3440 x 1440 without horizontal overflow; at
|
|
3440 px the MapLibre stage used about 2,408 px.
|
|
- Full readiness passed 588 backend tests, backend compile, one Alembic head,
|
|
API/document audits, frontend typecheck/build and shell smoke checks. The
|
|
Tower runtime passed health, PostGIS 3.6, migration/schema/index and reverse
|
|
proxy validation.
|
|
|
|
Next:
|
|
- Add a review workflow for accepted/rejected detections and use those audited
|
|
labels as the gate for model calibration or retraining. Do not present the
|
|
current F1 score as production-grade accuracy.
|
|
|
|
## Sprint 197 - Measured detection accuracy and durable review (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added first-class `detection_reviews` persistence linked to Project,
|
|
QualityCheck, AnalysisRun, Detection and reference VectorFeature, with
|
|
role-constrained decisions and one durable row per evidence item.
|
|
- Added canonical paginated GET/POST review endpoints and a frontend review
|
|
queue with role/status filters, notes, summary counts and map handoff.
|
|
- Changed map-driven detection QA from generic UI IoU `0.50` to the documented
|
|
box-versus-footprint operational IoU `0.25`.
|
|
- Reworded map output as candidates and exposed persisted matches, precision,
|
|
recall, F1, false positives and false negatives.
|
|
- Bounded evidence lookup to persisted evidence ids; complete regional GRB
|
|
layers are no longer materialized for a small review overlay.
|
|
|
|
Live model evidence before deployment:
|
|
- Re-ran the active model at confidence `0.10` and `0.15` on Mol Achterbos,
|
|
Gompel, Donk and Postel with QA IoU `0.25`.
|
|
- Confidence `0.15` won F1 in all four positive holdouts: `0.6694`, `0.6564`,
|
|
`0.5894` and `0.4749`. Confidence `0.10` measured `0.6287`, `0.6348`,
|
|
`0.5636` and `0.4435` respectively.
|
|
- Both thresholds produced zero detections on the pure-empty Postel forest
|
|
control. The active confidence remains `0.15`; no model or asset was trained,
|
|
downloaded or promoted.
|
|
|
|
Validation before deployment:
|
|
- The complete readiness gate passed 592 backend tests, the API/document audit
|
|
for 88 implemented routes, backend compilation, one Alembic head, offline
|
|
migration SQL generation, frontend typecheck/build and shell smoke checks.
|
|
- Focused review/evidence regressions passed and the production bundle retained
|
|
separate React, application and MapLibre chunks.
|
|
|
|
Live operational proof:
|
|
- Tower deployed commit `d22abe8` with AI extras enabled. The all-in-one
|
|
runtime became healthy, PostGIS 3.6 passed, Alembic reached `202607150001`,
|
|
and the new table exposed 13 columns plus six expected indexes.
|
|
- Quality check `acaca138-dbb9-4498-ae53-d0732b6270e3` returned 206 persisted
|
|
evidence geometries in 338 ms with zero unresolved ids. All 72 reviewable
|
|
false-positive/false-negative features were explicitly `unreviewed`.
|
|
- A new browser-driven 35.57 ha Mol rectangle acquired the official orthophoto,
|
|
ran the configured local YOLO model and persisted analysis run
|
|
`14f3b856-1ff0-4939-892a-3c8347a669f1` plus quality check
|
|
`087d2b6a-6d3c-4508-ab1e-da49734097c2` at IoU `0.25`.
|
|
- The live run produced 100 candidates, 73 matches, precision `0.73`, recall
|
|
`0.4965986395`, F1 `0.5910931174`, mean IoU `0.5361610234`, 27 false
|
|
positives and 74 false negatives. The UI showed these values as measured
|
|
candidates and retained the explicit `controle vereist` source warning.
|
|
- Browser audits at 1280x720 and 3440x1440 had no console warnings/errors or
|
|
horizontal overflow. The review queue rendered 50 paginated rows; the
|
|
ultrawide map used about 2,410 px. A final compact-metric fix was queued after
|
|
the 1280 px screenshot exposed cramped six-column labels.
|
|
|
|
Next:
|
|
- Deploy the migration and UI, verify one live review queue and map QA result,
|
|
then complete representative manual decisions before constructing any new
|
|
model-training corpus.
|
|
|
|
## Sprint 198 - Evidence-closed model review (2026-07-15)
|
|
|
|
Implemented:
|
|
- Generated the previously missing 48-card false-negative contact-sheet bundle
|
|
from the exact persisted inference manifests for Geel, Herentals and
|
|
Turnhout; 731 references outside tile coverage remained explicitly excluded.
|
|
- Completed all 96 FP/FN decisions through orthophoto inspection plus
|
|
persisted geometry-overlap diagnostics. The result is 5 confirmed model FP,
|
|
10 confirmed model FN, 59 QA-alignment cases, 10 reference-gap/change cases
|
|
and 12 uncertain/obscured cases.
|
|
- Added `validate_detection_false_negative_review_decisions.py`, mirroring the
|
|
existing FP safety contract and exporting only explicit confirmed misses.
|
|
- Included the symmetric validator in the all-in-one image and its packaging
|
|
regression list after the first live build exposed the missing COPY entry.
|
|
- Audited confirmed evidence against the active training split. Geel and
|
|
Herentals are existing training sources; Turnhout is an excluded operational
|
|
holdout. The review therefore provides zero novel leakage-free labels and a
|
|
new fine-tuning run was deliberately rejected.
|
|
- Added strict match count plus the existing diagnostic reference-envelope
|
|
result to the map analysis panel. Canonical footprint IoU metrics remain
|
|
unchanged.
|
|
|
|
Validation:
|
|
- Both 48-row decision CSVs completed with zero `unreviewed` records and passed
|
|
their validators with `--require-complete`.
|
|
- Focused validator/readiness/map-contract tests passed.
|
|
- Full readiness passed 596 backend tests, backend compilation, 88 documented
|
|
API routes, one Alembic head, frontend typecheck/build and shell syntax gates.
|
|
|
|
Known limitation:
|
|
- The active local model remains useful but imperfect. A new candidate requires
|
|
independently collected training-only AOIs and complete tile labels; holdout
|
|
review evidence must not be recycled into training.
|
|
|
|
Live operational proof:
|
|
- Deployed commits `919e102` and packaging fix `8758721` to the Tower
|
|
all-in-one runtime. The container became healthy, PostGIS reported 3.6,
|
|
Alembic remained at `202607150001`, and the frontend/API/icon proxy smoke
|
|
passed.
|
|
- Both review validators ran inside the rebuilt image against persistent
|
|
operator data with `status=complete`: 5 confirmed model false positives and
|
|
10 confirmed model false negatives, with zero unreviewed cards.
|
|
- A new 15.31 ha browser-drawn Mol selection persisted 282 candidates and 209
|
|
strict GRB matches. It measured precision `0.741`, recall `0.685`, F1 `0.712`,
|
|
73 false positives and 96 false negatives. The diagnostic reference-envelope
|
|
pass found 212 matches and only 3 possible box/footprint differences.
|
|
- Browser checks at 1280x720 and 3440x1440 showed the strict and diagnostic
|
|
values without horizontal overflow. The ultrawide map used about 2,408 px;
|
|
the browser console contained no warnings or errors.
|
|
|
|
Next:
|
|
- Collect a new training-only small-building/background evidence pack outside
|
|
all operational holdouts, then train an inactive candidate only if the pack
|
|
passes label, leakage and sample-volume audits.
|
|
|
|
## Sprint 199 - Reviewed accuracy expansion and fail-closed challenger (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added six training-only reference AOIs for Arendonk, Dessel, Meerhout,
|
|
Laakdal, Nijlen and Hulshout. Tests enforce explicit training role, unique
|
|
municipality/center pairs and at least 2 km separation from every protected
|
|
Mol, Turnhout, Retie, Westerlo, Vosselaar and Grobbendonk holdout.
|
|
- Prepared 1024 px orthophoto/GRB pairs on Tower. The paged GRB exports contain
|
|
9,964 features in total, use EPSG:31370 rasters and have no truncated,
|
|
invalid or empty reference geometry.
|
|
- Exported `yolo-building-aoi1024-reviewedexp6-minpx3vis035`: 252 tiles, 234
|
|
positive, 18 negative, 234 train, 18 validation and 79,192 labels. The
|
|
dataset audit returned `ok`; the 64-tile visual review covered 28 retained
|
|
sources with no invalid, missing or low-variance selections.
|
|
- Fine-tuned the active local YOLOv8s asset for 20 CPU epochs at image size
|
|
`640`. The best checkpoint came from epoch 16 and was copied as inactive
|
|
`geointel-building-yolov8s-reviewedexp6-minpx3-img640-ft20-pt`, SHA256
|
|
`038f1f97a6afd534f29e1f392a730a58207b928ca01e31ab8d8fed6106705820`.
|
|
- Re-ran both active and challenger assets through the exact current API,
|
|
persistence and coverage-aware QA/QC path at tile `512`, overlap `64`,
|
|
confidence `0.15` and match IoU `0.25`. The evaluated reference populations
|
|
are equal per zone; older unequal-coverage runs were excluded.
|
|
- The active model measured mean precision `0.6141`, recall `0.6062` and F1
|
|
`0.6069`; the challenger measured `0.6251`, `0.6293` and `0.6248`. Challenger
|
|
F1 improved in all seven zones and reduced false negatives from 3,416 to
|
|
3,208.
|
|
- The challenger produced two detections in explicitly empty Postel-bos while
|
|
the active model remained at zero across Postel-bos, Lommel-heide and
|
|
Arendonk-heide. The formal promotion report recommended the existing active
|
|
key. No `.env`, active model path or runtime threshold was changed.
|
|
- Updated the approved Detection Lab profile to the current coverage-aligned
|
|
active evidence and retained the explicit Postel limitation.
|
|
|
|
Validation evidence:
|
|
- Full readiness after the documentation/profile update passed 598 backend
|
|
tests, backend compilation, one Alembic head, frontend typecheck/build and
|
|
the complete shell syntax gate.
|
|
- Persistent promotion evidence lives under
|
|
`/app/storage/operator-data/model-review/reviewed-accuracy-expansion/` and
|
|
`/app/storage/operator-evidence/mol-operational-validation/`.
|
|
|
|
Next:
|
|
- Convert the two confirmed Postel-bos challenger errors into a complete,
|
|
leakage-free hard-negative training sample, add independent empty controls,
|
|
then train a new inactive candidate through the same fail-closed gate.
|
|
|
|
## Sprint 200 - Operational time-series handoff (2026-07-15)
|
|
|
|
Implemented:
|
|
- Reproduced the `Geen tijdreeks beschikbaar` dead end in the deployed regional
|
|
map: Evolution retained the current-only GRB building snapshot even though
|
|
persisted population and forest series were available.
|
|
- Built one temporal-series catalog for every end-user theme from ready
|
|
persisted datasets. Entering Evolution now retains a comparable theme or
|
|
automatically opens the first available series and its latest snapshot.
|
|
- Theme cards now expose observation count and year range for real series.
|
|
Single-snapshot sources remain visible as `Alleen huidige toestand` but are
|
|
disabled for comparison instead of opening empty controls.
|
|
- Kept every comparison behind the existing temporal API and PostGIS
|
|
`vector_features` aggregation. No browser-side metrics, inferred snapshots
|
|
or synthetic historical values were introduced.
|
|
- Updated the top-level Detection documentation values while touching the
|
|
frontend handoff documentation so the active profile no longer shows an
|
|
older superseded benchmark.
|
|
|
|
Validation evidence:
|
|
- A live regional Statbel comparison for a Mol bbox returned 87,868.45
|
|
area-weighted inhabitants in 2021 and 91,497.15 in 2025: +3,628.70 or
|
|
+4.13%. The API correctly marked the result as an estimate and suppressed
|
|
unsupported object lineage.
|
|
- Focused temporal/regional/frontend contract tests passed: 21 tests.
|
|
- Full readiness passed 601 backend tests, backend compilation, 88 documented
|
|
API routes, one Alembic head, frontend typecheck/build and shell gates.
|
|
|
|
Known limitation:
|
|
- The complete regional project currently has real 2021-2025 population and
|
|
2013-2025 modern forest series. Buildings, water, roads and parcels each
|
|
have one current GRB snapshot at regional scope. The official 1778/1873/1969
|
|
historical building, forest, water and road series is persisted for Mol but
|
|
has not yet been partitioned and imported for all 28 municipalities.
|
|
|
|
Next:
|
|
- Generalize the audited historical-land-use operator to the approved regional
|
|
scope, partition source retrieval by municipality and provision the three
|
|
official editions without merging their methodology into current GRB.
|
|
|
|
## Sprint 201 - Semantic area-selection metrics (2026-07-15)
|
|
|
|
Implemented:
|
|
- Extended the canonical persisted-vector selection summary with an additive
|
|
metric set while preserving the existing primary metric fields and envelope.
|
|
- Mapped known data themes to useful units: building footprint, forest, water
|
|
and parcel surfaces in hectares; roads and linear watercourses in kilometres;
|
|
population in inhabitants; and intersecting feature counts as supporting
|
|
evidence.
|
|
- Kept all spatial calculations in PostGIS after transformation to EPSG:31370.
|
|
No browser-side area/length calculation or synthetic source value was added.
|
|
- Added honest domain limits for building floor area, road capacity and water
|
|
volume. The current 2D GRB water source cannot support volume without an
|
|
independently governed depth/bathymetry dataset.
|
|
- Updated the regional GRB operator metadata for future imports and added a
|
|
compact supporting-metric surface to the map-first result panel.
|
|
|
|
Validation evidence:
|
|
- Focused temporal and semantic-selection regression set passed 16 tests.
|
|
- Full readiness passed 605 backend tests, backend compilation, the API
|
|
contract audit, one Alembic head, frontend TypeScript typecheck/build and all
|
|
shell syntax gates.
|
|
|
|
Known limitation:
|
|
- Water volume remains unavailable by design. Adding it requires a compatible
|
|
depth or bathymetry source, coverage validation, units, observation date and
|
|
a documented integration method.
|
|
|
|
Next:
|
|
- Validate the semantic metrics against live Mol PostGIS data and then continue
|
|
the audited regional historical buildings/water/roads import.
|
|
|
|
## Sprint 202 - Source intelligence, complete evolution metrics and Ollama (2026-07-15)
|
|
|
|
Implemented:
|
|
- Audited the current regional PostGIS inventory: current GRB buildings, roads,
|
|
water and parcels; Statbel population 2021-2025; and modern forest
|
|
2013/2016/2019/2022/2025.
|
|
- Extended temporal comparison additively with optional exact persisted-Area
|
|
geometry, every compatible semantic metric and a complete observation
|
|
timeline. Bbox-only rectangle comparison remains supported.
|
|
- Expanded the official 10 m land-use operator with water, built-function and
|
|
transport surfaces. All themes reuse one retained source raster per year;
|
|
their measurements remain separate from current GRB geometry semantics.
|
|
- Added a Sources inventory for loaded themes and audited follow-up sources.
|
|
- Added an optional local Ollama assistant with status/model/query endpoints,
|
|
installed-model validation, compact persisted GIS context and strict missing-
|
|
data behavior. The browser never addresses Ollama directly.
|
|
- Added editable Docker/Unraid Ollama settings and automatic host-gateway
|
|
mapping for the server runtime.
|
|
|
|
Validation evidence:
|
|
- Focused temporal, source, Ollama, navigation and Unraid regression tests
|
|
passed, including direct coverage for multi-metric history and DockerMan host
|
|
mapping.
|
|
- Full readiness passed 617 backend tests, backend compilation, 91 documented
|
|
API routes, one Alembic head, frontend TypeScript typecheck/build and all
|
|
shell syntax gates.
|
|
- Tower deployment passed the all-in-one container health check, live PostGIS
|
|
3.6 migration/schema smoke and frontend/API proxy smoke. The container reached
|
|
host Ollama and exposed 10 installed models through the canonical API.
|
|
- A live historical question revealed that the Dutch verb `evolueerden` did
|
|
not activate history context. The intent stem was corrected to `evolu`, three
|
|
direct regression cases were added and full readiness reran successfully.
|
|
- In-app browser validation on the deployed HTTP LAN URL exposed that
|
|
`crypto.randomUUID()` was unavailable outside a secure browser context. Chat
|
|
message keys now use a session-local monotonic id generator; no persisted or
|
|
security-sensitive identity depends on it. Typecheck, production build and
|
|
full readiness reran successfully.
|
|
- The regional operator then reused the five governed forest snapshots and
|
|
imported 15 new authoritative snapshots: water, built functions and
|
|
transport for 2013, 2016, 2019, 2022 and 2025. A live exact-Area Mol water
|
|
comparison returned 606.16 ha in 2013 and 641.75 ha in 2025 across all five
|
|
observations; the API retained the 10 m raster and object-lineage warnings.
|
|
- Browser QA also reproduced an Ollama `done_reason=length`: the previous
|
|
implicit 4096-token context left only 86 tokens after a 4010-token grounded
|
|
prompt. GeoIntel now requests a configurable 16,384-token context and rejects
|
|
any future length-truncated response instead of presenting a partial answer.
|
|
- A complete live response then exposed two unsupported model interpretations:
|
|
an estimated population was called exact and an ungrounded demographic cause
|
|
was suggested. The system contract now mandates estimate wording, forbids
|
|
causal/forecast claims not present in context, uses deterministic temperature
|
|
zero and requests plain text. Full readiness remained green at 617 tests.
|
|
- Assistant context now carries explicit measurement quality for current and
|
|
historical values. A deterministic response guard prepends the estimate
|
|
limitation whenever an answer discusses estimated population data, while the
|
|
model prompt forbids independent averages, rates and derived trends. Focused
|
|
tests and full readiness passed with 619 backend tests.
|
|
- Browser review identified that the source summary visually combined current
|
|
GRB geometry with historical 10 m land-use observations. The UI now reports
|
|
comparable observations per temporal-series key and labels singleton/current
|
|
datasets as a separate source method. Full readiness remained green.
|
|
|
|
Known limitations:
|
|
- Water volume remains unavailable until a governed depth/bathymetry source is
|
|
integrated. VMM station water levels or flows alone do not establish volume
|
|
for every selected polygon.
|
|
- Available follow-up sources are catalogued but are not labelled as loaded
|
|
until a controlled import and provenance validation have completed.
|
|
|
|
Next:
|
|
- Integrate Waterinfo/VMM station observations as point time series and add
|
|
historical orthophoto acquisition, while preserving their spatial and
|
|
methodological limitations.
|
|
|
|
## Sprint 203 - Governed Waterinfo history and historical orthophotos (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added `provision_waterinfo_station_history.py` using the documented Waterinfo
|
|
KiWIS annual water-level/discharge groups. The operator filters station
|
|
points against the exact persisted Area, retains raw JSON plus SHA256
|
|
manifests and writes only through the canonical upload API.
|
|
- Persisted one Point Dataset per station/year and kept station identities in
|
|
separate temporal series. Added backend `mean` aggregation without combining
|
|
stations or inferring area-wide water level/volume.
|
|
- Added a governed orthophoto allowlist for the most-recent product, annual
|
|
winter mosaics 2012-2025, older winter periods, RGB 1979-1990 and
|
|
panchromatic 1971. Arbitrary WMS URLs/layers remain impossible.
|
|
- Added raster temporal metadata and a constrained PNG rendering endpoint for
|
|
persisted orthophoto datasets. The map can select and display official
|
|
historical imagery over the same bounded rectangle.
|
|
- Historical products explicitly bypass configured-YOLO and current-GRB QA;
|
|
only `most_recent` retains that path.
|
|
- Updated source inventory behavior and wrote a governed source backlog with
|
|
acceptance criteria for BWK/Natura 2000, agricultural parcels, Buildings
|
|
Register, DHMV and bathymetry.
|
|
|
|
Validation evidence:
|
|
- Focused Waterinfo/orthophoto/semantic metric tests and frontend typecheck/build
|
|
passed before the full repository gate.
|
|
- Full readiness passed 627 backend tests, 93 documented routes, one Alembic
|
|
head and the frontend production build. Tower passed PostGIS 3.6 migration,
|
|
schema, frontend proxy and icon smokes.
|
|
- Live Mol import persisted 26 annual water-level observations across two
|
|
independent stations for 2013-2025. Two stale station series contained no
|
|
observations and no in-area annual discharge station was available.
|
|
- Live temporal comparison for station `L10_089` returned 30.460 m in 2013 and
|
|
30.455 m in 2025 with all 13 observations, `mean` aggregation and the point-
|
|
measurement/no-volume warning.
|
|
- Live 2020 bounded WMS acquisition persisted an EPSG:31370 raster with annual
|
|
temporal metadata; its constrained PNG endpoint returned HTTP 200.
|
|
- Browser validation then exposed that 26 new annual datasets pushed the
|
|
project above the frontend's implicit 50-row dataset page. The central API
|
|
client now exhausts canonical 200-row pages and fails if totals drift or a
|
|
page is incomplete, so older population/parcel sources remain visible.
|
|
|
|
Remaining operational step:
|
|
- Redeploy the pagination correction and verify complete source visibility plus
|
|
the historical overlay through the LAN browser.
|
|
|
|
Final live UI correction:
|
|
- Browser inspection at 1280x720 found the premium desktop grid overriding the
|
|
source catalog's full-width placement and constraining it to a 282px internal
|
|
scroller. A final, component-specific CSS rule restores the catalog as a
|
|
full-width overview band without changing the remaining data-workspace grid.
|
|
|
|
Live completion evidence:
|
|
- Tower was redeployed from commit `d3a09e8`; the all-in-one runtime passed its
|
|
health check, PostGIS 3.6 migration/schema smoke, frontend proxy and icon
|
|
checks.
|
|
- The source catalog measured 1,027px wide at a 1,280x720 viewport, with six
|
|
readable source cards, natural height and no internal overflow. The remaining
|
|
source-management panels start on the next grid row.
|
|
- A browser-driven Mol workflow drew a bounded 63.19ha selection, returned
|
|
persisted metrics for all six available themes and loaded the governed 2020
|
|
winter orthophoto as a MapLibre overlay. The UI retained its explicit warning
|
|
that historical imagery is not validated against the current GRB state.
|
|
- The browser console contained no warnings or errors after the complete flow.
|
|
|
|
## Sprint 204 - Governed BWK and Natura 2000 for Mol (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added `provision_mol_bwk_natura2000.py` for the official INBO
|
|
`BWK:Bwkhab` state-2025 WFS. The explicit operator follows complete
|
|
pagination, retains every raw page with URL and SHA256 evidence, and refuses
|
|
silent replacement when a persisted checksum differs.
|
|
- Clipped source polygons against the exact persisted Mol Area in EPSG:31370,
|
|
transformed valid polygonal output to EPSG:4326 and imported only through
|
|
the canonical Dataset upload route. Original BWK evaluation, mapping-unit,
|
|
origin, habitat and PHAB fields remain present on persisted features.
|
|
- Added server-owned filtered PostGIS metrics for the official `EVAL` classes,
|
|
area-weighted Natura 2000 habitat, regionally important biotopes and
|
|
uncertain `ohab` knowledge gaps. PHAB-derived values remain labelled as
|
|
estimates.
|
|
- Added the Natuurwaarde map theme and source-inventory state. The 2025 product
|
|
is presented as one current map state and not as a fabricated annual trend.
|
|
|
|
Validation evidence:
|
|
- The source fetch retained 11 checksummed WFS pages containing 10,332 bbox
|
|
features. Exact Mol clipping retained 4,668 features, rejected 5,664 outside
|
|
features, clipped 476 boundary-crossing features and found no duplicates.
|
|
- Full local readiness passed backend compilation, 637 backend tests, one
|
|
Alembic head, frontend TypeScript typecheck/build and shell syntax gates.
|
|
- Tower deployment passed container health, PostGIS 3.6 migration/schema smoke,
|
|
frontend proxy and icon verification.
|
|
- The first live upload correctly failed before persistence because
|
|
`observed_at` lacked the required `temporal_series_key`. The operator now
|
|
records `inbo-bwk-natura2000:mol` with snapshot granularity; the focused
|
|
regression suite passed and the corrected deployment imported exactly 4,668
|
|
`vector_features`. A second operator run returned `existing` for the same
|
|
Dataset and checksum.
|
|
- A live exact-Area PostGIS selection measured 11,447.996 ha mapped BWK area:
|
|
960.446 ha very valuable, 3,082.023 ha valuable, 4,697.797 ha less valuable
|
|
and 2,707.730 ha mixed. Source-faithful weighted results were 821.690 ha
|
|
Natura 2000 habitat, 143.565 ha regional biotope and 104.557 ha uncertain
|
|
habitat/knowledge gap.
|
|
- Browser validation at 1280x720 selected Mol, activated Natuurwaarde and ran
|
|
the full-work-area analysis. All seven available themes returned persisted
|
|
metrics, the MapLibre layer rendered, no horizontal document overflow was
|
|
present and the console contained no warnings or errors. The source inventory
|
|
reports one current BWK state and 4,668 loaded map polygons.
|
|
|
|
Known limitations:
|
|
- The official state-2025 edition is the best available map state, not one
|
|
uniform 2025 field survey. Origin fields remain necessary for interpretation.
|
|
- PHAB percentages can be theoretical source allocations and may differ from
|
|
field reality. GeoIntel does not reinterpret them as exact field coverage.
|
|
- Regional Kempen coverage and historical comparison remain separate future
|
|
work; this sprint proves the governed Mol path only.
|
|
|
|
Next:
|
|
- Implement P2 annual agricultural-use parcels, beginning with an edition and
|
|
code-list comparability audit before any import or trend is exposed.
|
|
|
|
## Sprint 205 - Governed annual agricultural-use parcels (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added an explicit ALZ operator for the definitive 2008-2025 annual
|
|
agricultural-use parcel GeoPackages. The operator uses a fixed official
|
|
allowlist, streamed archive limits, ZIP path validation and native
|
|
EPSG:31370 schema checks before exact regional clipping.
|
|
- Retained each official archive, annual crop-code list and checksummed
|
|
manifest. Derived GeoJSON enters the application only through the canonical
|
|
Dataset upload service and ordinary Dataset/vector_feature persistence.
|
|
- Added server-owned exact PostGIS metrics for total declared-use hectares,
|
|
official main-crop groups and feature count. Comma-separated official group
|
|
labels found during the live source audit remain compatible with the first
|
|
persisted import.
|
|
- Added the Landbouw map theme, readable source labels, the 18-edition source
|
|
inventory and historical comparison UI. Parcel-level object lineage remains
|
|
disabled because declarations and boundaries are not stable identities.
|
|
|
|
Validation evidence:
|
|
- Full local readiness passed backend compilation, 647 backend tests, one
|
|
Alembic head, frontend TypeScript typecheck/build and shell syntax gates.
|
|
- Tower deployment passed container health, PostGIS 3.6 migration/schema
|
|
smoke, frontend proxy and icon verification.
|
|
- All 18 definitive editions were imported into one governed regional series
|
|
for the exact 28-municipality Kempen transport-region Area. Database audit
|
|
found 18 ready Datasets, 18 DatasetVersions, matching feature/source-id
|
|
counts, EPSG:4326 throughout and zero invalid geometries.
|
|
- The 2025 source contained 594,730 Flanders features. Exact regional clipping
|
|
retained 54,071 features, including 411 clipped at the boundary, measuring
|
|
60,087.0667 ha. The 2008 edition retained 43,343 regional features and
|
|
measured 63,272.1152 ha.
|
|
- Live Mol analysis measured 3,223.4983 ha declared use across 2,374 features
|
|
for 2025, including 934.2348 ha grassland, 874.7350 ha maize and 329.1544 ha
|
|
potatoes. All configured crop/use groups returned explicit hectare values.
|
|
- The exact Mol 2008-2025 comparison returned all 18 observations: 3,343.2441
|
|
ha to 3,223.4983 ha, a change of -119.7458 ha (-3.5817%). The UI states that
|
|
declarations are not cadastral ownership and individual objects cannot be
|
|
followed reliably between years.
|
|
- Browser validation confirmed the current-state metrics, complete historical
|
|
chart, 18-edition inventory and readable authority labels through the live
|
|
LAN runtime.
|
|
|
|
Known limitations:
|
|
- The current campaign remains provisional and is deliberately excluded until
|
|
a definitive annual edition is published.
|
|
- Declared agricultural use includes non-crop elements such as water, hedges,
|
|
buildings and infrastructure; it is not ownership, cadastral area or a
|
|
direct measure of agricultural production.
|
|
- Cross-year comparison is valid for governed category and area totals only,
|
|
not for parcel identity, causal attribution or forecasts.
|
|
|
|
Next:
|
|
- Implement P3 Buildings and Addresses Register through a governed snapshot
|
|
operator, preserving register lifecycle semantics separately from GRB
|
|
footprint geometry and excluding personal data.
|
|
|
|
## Sprint 206 - Governed Buildings and Addresses Register snapshot (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added `scripts/provision_buildings_addresses_register.py` for the official
|
|
Digitaal Vlaanderen `Gebouw`, `Gebouweenheid` and `Adres` OGC API Features
|
|
collections. It retains raw pages and SHA256 checksums, uses retries and
|
|
limits, clips exactly in EPSG:31370 and persists only through DatasetService.
|
|
- Added service-window-safe pagination. Live audit proved that the address
|
|
collection omitted `next` after 52,000 bbox rows while offset 52,000 still
|
|
returned data. The first incomplete snapshot was rejected and transactionally
|
|
removed; the corrected operator continued with `startIndex` to a short final
|
|
page at 52,110 rows.
|
|
- Kept building lifecycle status separate from GRB footprint evidence. Official
|
|
building-unit relations use `GebouwObjectId`; address relations are accepted
|
|
only for an exact unambiguous unit position or unambiguous building
|
|
containment. Ambiguous and unmatched relations remain diagnostics.
|
|
- Persisted building polygons with aggregate relation counts only. Full address,
|
|
street, house-number and box-number fields remain in restricted raw operator
|
|
evidence and never enter queryable feature properties.
|
|
- Added exact footprint, lifecycle, unit, address-status and GRB-match metrics.
|
|
Fixed filtered `feature_count` execution without changing the existing
|
|
unfiltered count contract.
|
|
- Added Map and Sources presentation. The exact Mol Area prefers the register;
|
|
another municipality and the full Kempen scope retain complete regional GRB.
|
|
Evidence paths and manifest reuse are isolated by Area key and boundary hash.
|
|
|
|
Live evidence:
|
|
- Source bbox acquisition completed with 111,696 buildings (112 pages), 67,239
|
|
building units (68 pages) and 52,110 addresses (53 pages), with zero duplicate
|
|
source ids. Exact Mol clipping retained 43,945 buildings and 23,788 addresses.
|
|
- Dataset `c3a47680-cea1-406c-aebb-8ebfa8d7a359` is the only ready Mol register
|
|
snapshot for 2026-07-15. It has one DatasetVersion, 43,945 vector rows,
|
|
43,945 distinct source ids, SRID 4326, polygonal geometry only and zero invalid
|
|
geometries.
|
|
- Lifecycle counts are 37,159 realized, 14 under construction, 155 planned,
|
|
22 not realized and 6,595 historical buildings. Exact footprint area is
|
|
502.1434 ha.
|
|
- Aggregate relations contain 30,382 building units, 21,388 linked addresses
|
|
and 19,513 addresses in use. There are 1,687 ambiguous and 713 unmatched
|
|
address positions. Queryable properties contain zero prohibited address
|
|
label/number fields.
|
|
- GRB reconciliation produced 36,588 matched, 5 review, 664 ambiguous and 6,688
|
|
unmatched register buildings. There are 353 GRB footprints without a confirmed
|
|
target and 329 duplicate-target groups. Realized-building match rate is
|
|
98.4526%; total register match rate is lower because historical and other
|
|
lifecycle records are retained.
|
|
- Live exact-Area API returned all eleven governed metrics. Browser validation
|
|
confirmed Mol register selection, Geel GRB fallback, readable metric/warning
|
|
output, source inventory, MapLibre rendering, no horizontal overflow and no
|
|
browser console warnings/errors.
|
|
|
|
Validation evidence:
|
|
- Full local readiness passed backend compilation, 656 backend tests, one
|
|
Alembic head, frontend TypeScript typecheck/build and shell syntax gates.
|
|
- Tower deployment passed all-in-one health, PostGIS 3.6 migration/schema smoke,
|
|
frontend proxy and icon verification.
|
|
|
|
Known limitations:
|
|
- This is a current continuously updated snapshot, not a fabricated historical
|
|
series. GRB remains the complete regional geometry source outside Mol.
|
|
- Address counts are not households, dwellings, residents or population.
|
|
Building footprint is ground area, not floor area, height or volume.
|
|
- Ambiguous address links, duplicate GRB targets and review/unmatched geometry
|
|
remain explicit evidence for later regional expansion; no nearest match is
|
|
forced.
|
|
|
|
Next:
|
|
- Implement P4 Digitaal Hoogtemodel Vlaanderen with a governed DTM/DSM product,
|
|
vertical reference, bounded raster storage and measured elevation/slope
|
|
statistics. Do not infer water depth or volume from terrain height alone.
|
|
|
|
## Sprint 207 - Governed DHMV II terrain foundation (2026-07-15)
|
|
|
|
Implemented:
|
|
- Added a fixed `dtm_1m`/`dsm_1m` product registry for the official Digitaal
|
|
Vlaanderen production WCS and rejected arbitrary coverage identifiers.
|
|
- Added bounded, rate-limited 10 km WCS tiling, multipart GeoTIFF extraction,
|
|
georeferenced mosaicking, exact persisted-Area clipping and validation of
|
|
EPSG:31370, one Float32 band, resolution, nodata and valid cells before
|
|
canonical DatasetService persistence.
|
|
- Retained response, coverage and normalized-output SHA256 evidence plus native
|
|
1 m resolution, default 5 m analysis resolution, TAW and period 2013-2015.
|
|
- Added exact masked terrain selection with governed height, relief and slope
|
|
metrics. DTM and DSM semantics remain separate; water depth/volume remain
|
|
explicit unsupported metrics and drainage is not calculated.
|
|
- Added the `Hoogte & reliëf` map theme, MapLibre colour-relief overlay,
|
|
readable period/source labels and raster-aware rectangle/full-Area queries.
|
|
- Added `scripts/provision_mol_dhmv.py`, Docker/Unraid settings and documentation.
|
|
|
|
Validation evidence:
|
|
- Full local readiness passed compilation, 669 backend tests, one Alembic head
|
|
(`202607150001`), frontend TypeScript typecheck/build and shell syntax gates.
|
|
- The production WCS required an explicit `Accept: */*` header and rejected
|
|
rapid municipality tile requests. The governed client now waits two seconds
|
|
between requests and retries a transient provider status once. Four real
|
|
requests produced one `2928 x 3794`, EPSG:31370, 5 m Mol bounding raster
|
|
before exact municipality masking.
|
|
- Tower deployment passed all-in-one health, PostGIS 3.6 migration/schema
|
|
smoke, frontend proxy and icon verification.
|
|
- Live DTM Dataset `eccb9174-edb9-4867-928c-9acc5d686fa5` is ready with
|
|
checksum `9c5f06f5ce2070128aca06e3cbcec00ec8f698bf592c1745245280cd0488ad9b`.
|
|
Live DSM Dataset `34d1be8e-3a80-4ea3-98e5-238739000524` is ready with
|
|
checksum `54e36a2aada03f52eb24a7b6af505871d33ed52a5c2af04500793dbfca72f7af`.
|
|
Both contain 4,581,867 valid 5 m cells and report complete Mol coverage.
|
|
- Full-Mol DTM metrics are mean 31.8985 m TAW, minimum 20.8598 m TAW,
|
|
maximum 48.4594 m TAW, relief 27.5996 m and mean slope 1.2377 degrees.
|
|
Re-running the operator reused both immutable datasets.
|
|
- Browser validation selected Mol, activated DTM by default, rendered the
|
|
colour-relief overlay, returned all terrain metrics, retained the explicit
|
|
water-depth/volume limitation, showed no horizontal overflow and emitted no
|
|
browser console warnings/errors.
|
|
- No migration, direct raster database write, LiDAR point-cloud processing or
|
|
water-volume inference was introduced.
|
|
|
|
Known limitations:
|
|
- DHMV II represents acquisition period 2013-2015 and is not a current or
|
|
annual height series. Exact flight-day contours are not yet joined.
|
|
- The 5 m analysis copy is operationally bounded; sub-5 m detail requires an
|
|
explicit smaller acquisition. DSM-DTM building height remains future work.
|
|
|
|
Next:
|
|
- Audit P5 official bathymetry and water-depth sources before enabling any
|
|
water depth or volume metric. Keep DHMV as height/relief input only.
|
|
|
|
## Sprint 208 - Governed VMM flood-hazard scenarios (2026-07-15)
|
|
|
|
Implemented:
|
|
- Audited official Vlaamse water-depth/bathymetry sources. Public coastal and
|
|
North Sea bathymetry does not cover Mol; Waterinfo is station-based and GRB
|
|
water geometry is two-dimensional. Permanent waterbody volume therefore
|
|
remains unavailable.
|
|
- Added a fixed VMM OGRK registry for twelve water-depth scenarios: fluvial and
|
|
pluvial, current climate and climate projection 2050, each at T10/T100/T1000.
|
|
- Added bounded WCS 1.1 retrieval, provider-safe 5 km tiling, multipart GeoTIFF extraction,
|
|
georeferenced mosaicking, exact persisted-Area clipping and conversion from
|
|
positive source centimetres to normalized metres.
|
|
- Persisted scenario, probability, units, bounds, request URLs and response,
|
|
coverage and normalized checksums through the existing Job, Dataset and
|
|
DatasetVersion architecture. No migration or direct raster DB write exists.
|
|
- Added exact raster selection metrics for mapped inundated area/share,
|
|
mean/P90/maximum modeled depth and a maximum-depth area integral. The API,
|
|
UI and Ollama context all prohibit interpreting that integral as actual,
|
|
permanent or concurrent water volume.
|
|
- Added an `Overstroming` map theme, explicit scenario selector, transparent
|
|
MapLibre overlay, source-inventory presentation and full-Mol operator.
|
|
- Added Docker/Unraid configuration, packaging and focused backend/frontend/AI
|
|
contracts without a new dependency.
|
|
|
|
Validation evidence:
|
|
- Official WCS capabilities and DescribeCoverage confirmed EPSG:31370, 2 m
|
|
GridOffsets, Float32 values, null value 0 and `image/tiff` multipart output.
|
|
- A live bounded VMM request returned valid georeferenced data; official
|
|
catalog metadata confirms depth is expressed in centimetres between modeled
|
|
water surface and terrain.
|
|
- Synthetic GIS fixtures confirmed 200 positive 5 m cells at 1 m depth produce
|
|
0.5 ha mapped area and exactly 5,000 m3 maximum-depth area integral.
|
|
- Full local readiness passed compilation, 679 backend tests, one Alembic head
|
|
(`202607150001`), frontend TypeScript typecheck/build and shell syntax gates.
|
|
- The first full-Mol WCS smoke exposed the provider's 4.88 MB generated-output
|
|
limit for a 10 km request. The client now uses 5 km tiles (one million cells
|
|
at 5 m) and surfaces the official XML exception text. The focused suite
|
|
directly guards both constraints.
|
|
- Tower deployment passed all-in-one health, PostGIS 3.6 migration/schema
|
|
smoke, frontend proxy and icon verification.
|
|
- Live provisioning created twelve unique ready Mol Datasets with all governed
|
|
product keys, twelve DatasetVersions and no `observed_at` values. Every
|
|
dataset is attached to Area `7233f575-dbac-4e83-b9fa-e3fd99453722`; a second
|
|
complete run reused all twelve immutable outputs.
|
|
- Full-Mol current pluvial T100 reports 451.0425 ha mapped inundation (3.9376%),
|
|
mean local maximum depth 0.2419 m, P90 0.4318 m and maximum 2.4809 m. Current
|
|
fluvial T100 reports 98.805 ha (0.8626%). These remain scenario-bound values,
|
|
not actual event measurements or permanent water volumes.
|
|
- The raster image endpoint returned a valid 273,330-byte PNG generated from
|
|
persisted Dataset `5d37d9cb-cad5-446e-bfd9-ec30e308d4bd`; provenance records
|
|
twelve bounded WCS requests for that scenario.
|
|
- Browser validation confirmed twelve scenario choices, MapLibre overlay,
|
|
exact full-Area metrics and the volume limitation. A live-discovered theme
|
|
collision was fixed so `Water` retains GRB water geometry while only
|
|
`Overstroming` can activate VMM flood rasters. The final 1280 px and 3440 px
|
|
checks had no horizontal overflow or console warnings/errors.
|
|
|
|
Known limitations:
|
|
- The scenario raster represents modeled local maximum depth at 1:100,000
|
|
application scale, not a measurement of a current flood event.
|
|
- No public municipality-wide inland bathymetry was identified. Waterbody
|
|
bottom elevation, uncertainty and permanent content remain open.
|
|
- Scenario alternatives are not observations in time and are intentionally not
|
|
exposed as a historical temporal series.
|
|
|
|
Next:
|
|
- Treat permanent inland bathymetry as an external-data prerequisite, not as a
|
|
value derivable from DHMV or flood-hazard maps. Continue the independent P5
|
|
waterbody linkage/uncertainty design only when governed depth profiles exist;
|
|
otherwise proceed with the regional historical buildings/water/roads backlog.
|
|
|
|
## Sprint 209 - Regional historical land-use expansion (2026-07-15)
|
|
|
|
Implemented:
|
|
- Audited the production Digitaal Vlaanderen Historical Land Use WFS and its
|
|
1778, 1873 and 1969 collections. Broad regional hit counts stop at 10,000,
|
|
confirming that one transport-region bounding request is not a defensible
|
|
completeness strategy.
|
|
- Added `scripts/provision_regional_historical_landuse.py` for historical
|
|
building, water and road land-use classes over the 28 registered transport-
|
|
region municipalities. It requires the checksummed VRBG scope artifact and
|
|
never infers or broadens the regional definition.
|
|
- Added resumable municipality partitions with exact WFS JSON response bytes
|
|
retained as deterministic gzip artifacts, source/artifact SHA256, request
|
|
pagination evidence, boundary hashes, clipping diagnostics and combined
|
|
regional output manifests.
|
|
- Clipped and validated polygonal geometry per municipality, suffixing source
|
|
ids with NIS codes to prevent cross-boundary collisions. Source identities
|
|
remain explicitly unstable and do not enable object lineage.
|
|
- Added one canonical Dataset upload per theme/year with regional Area scope,
|
|
immutable temporal identity, hectare aggregation and methodological warning.
|
|
No API route, database migration or direct PostGIS write was added.
|
|
- Extended `provision_regional_timeseries.py`, all-in-one packaging, readiness
|
|
compilation and operator/source documentation.
|
|
|
|
Validation and live evidence:
|
|
- The final release gate passes with 685 backend tests, frontend typecheck and
|
|
production build, one Alembic head (`202607150001`) and script syntax checks.
|
|
- Live Tower/PostGIS provisioning completed all 252 municipality/theme/year
|
|
partitions and nine combined snapshots. No snapshot has an empty municipal
|
|
partition. The retained regional historical audit cache is about 518 MB.
|
|
- Persisted feature counts are: buildings 14,245 / 22,036 / 69,869; water
|
|
9,010 / 13,571 / 8,188; roads 6,305 / 33,578 / 30,915 for 1778 / 1873 /
|
|
1969 respectively. Total persisted geometry count is 207,717.
|
|
- Each of the nine ready Datasets belongs to regional Area
|
|
`46ab5614-fcf8-4e63-8ea8-6049f34c14fd`, exposes one immutable
|
|
DatasetVersion/checksum and participates in one of three ordered temporal
|
|
series. A second complete operator run returned `existing` for all nine with
|
|
identical audited feature counts.
|
|
- The live audit found that the full-Area fast path did not yet recognize the
|
|
new clipped operator. Adding its governed provenance reduced exact complete-
|
|
Kempen comparisons from multi-minute redundant clipping to 4.0 s for
|
|
buildings, 0.9 s for water and 1.9 s for roads.
|
|
- Complete-Kempen 1778 -> 1969 results are 481.62 -> 1,833.23 ha built land,
|
|
1,546.89 -> 2,198.80 ha mapped water and 4,382.49 -> 6,978.12 ha mapped
|
|
roads. These are map-class surfaces, not modern footprints, centreline
|
|
lengths, bathymetry or volume.
|
|
- Browser validation selected the historical buildings series, analyzed the
|
|
complete persisted Kempen Area and then municipality Mol. Mol returned
|
|
24.35 / 32.45 / 120.81 ha for 1778 / 1873 / 1969 and explicitly disclosed
|
|
the cartographic-method and unstable-object-identity limitations.
|
|
- The irrelevant generic `0 km` metrics for polygon-only historical water and
|
|
road classes were removed. The workbench source context now follows the
|
|
selected Evolution series. Normal and 3440 x 1440 visual checks found no
|
|
overlap; browser console warnings/errors were empty.
|
|
- Each deployment passed live PostGIS 3.6 connectivity, required schema/index
|
|
checks, database collation, Alembic head `202607150001`, frontend/API proxy
|
|
and icon verification.
|
|
|
|
Known limitations:
|
|
- Historical `buildings` are mapped built land-use surfaces, not individual
|
|
building footprints. Historical `roads` and `water` are mapped surfaces,
|
|
not current centerlines, bathymetry or volume.
|
|
- The three editions use different source maps and cartographic methods.
|
|
Comparisons are exploratory hectare changes within one series, never a
|
|
continuous equivalent to modern GRB or 10 m land-use products.
|
|
|
|
Next:
|
|
- Keep permanent inland bathymetry blocked on a governed depth source. The
|
|
next implementation pass can focus on current-only nature/terrain/flood
|
|
regional expansion or on a benchmarked data-refresh scheduler; it must not
|
|
reinterpret these historical surfaces as modern object lineage.
|
|
|
|
## Sprint 210 - Regional BWK/Natura 2000 expansion (2026-07-16)
|
|
|
|
Implemented:
|
|
- Added `provision_regional_bwk_natura2000.py` for the approved 28-municipality
|
|
Kempen transport-region scope, using the existing official INBO state-2025
|
|
source contract and governed Mol feature normalization.
|
|
- Added per-municipality WFS pagination, exact EPSG:31370 clipping,
|
|
municipality-suffixed feature identities, deterministic gzip retention of
|
|
every exact source response and checksum-bound partition/snapshot reuse.
|
|
- Kept one canonical regional Dataset upload behind DatasetService. No direct
|
|
`vector_features` write, migration, endpoint or fabricated time series was
|
|
introduced.
|
|
- Made map theme selection prefer an exact selected-Area Dataset over a broader
|
|
compatible regional Dataset. Updated the source inventory to present
|
|
overlapping BWK snapshots as area coverages without double-counting their
|
|
feature totals.
|
|
- Added focused tests for source evidence, GIS clipping, context fields,
|
|
regional assembly, semantic metrics, upload metadata, packaging and UI
|
|
selection behavior.
|
|
- Added the expression index
|
|
`ix_vector_features_dataset_municipality` and a matching ORM declaration so
|
|
municipality-filtered reads from the regional snapshot remain interactive.
|
|
|
|
Live result:
|
|
- Tower imported all 28/28 municipality partitions from 147 retained source
|
|
responses: 134,249 source features were clipped to 72,933 persisted regional
|
|
features in Dataset `11606922-c84b-4bfe-84cb-9372c0799b62`.
|
|
- The canonical output checksum is
|
|
`45c42f60ed51e0f860c79c060b9c5ed6cb59b9352732e75e02cb9ef34b66f6b9`.
|
|
The idempotency rerun returned the same existing Dataset without adding a
|
|
DatasetVersion or duplicate features.
|
|
- PostGIS contains 71,991 Polygon and 942 MultiPolygon rows in SRID 4326, with
|
|
72,933 distinct source ids, zero invalid geometries, zero empty geometries
|
|
and zero duplicate source ids.
|
|
- Complete-region metrics report 138,701.35 ha BWK-mapped surface (99.1% of
|
|
the selected 1,399.25 km2), including 6,580.43 ha Natura 2000 habitat,
|
|
1,467.68 ha regionally important biotope and 384.10 ha uncertain habitat.
|
|
- Mol resolves to its exact 4,668-feature snapshot. The fallback regional
|
|
partition returns the same 4,668 municipality features in 0.989 seconds;
|
|
Geel returns 5,392 features in 0.529 seconds after the new index.
|
|
|
|
Validation:
|
|
- Full local readiness passes backend compilation, 690 backend tests, the
|
|
101-route API contract audit, frontend typecheck/build, script syntax gates
|
|
and one Alembic head (`202607160001`).
|
|
- Tower deployment and live migration smoke pass with PostGIS 3.6, the same
|
|
Alembic head and all required schema/index checks.
|
|
- Live browser verification confirms the regional and Mol metrics, separate
|
|
overlapping area coverages in the source inventory, no console warnings or
|
|
errors and no horizontal overflow at 3440x1440.
|
|
|
|
Boundaries and next step:
|
|
- This Dataset is the official current BWK/Natura 2000 state dated 2025. It is
|
|
not a historical nature time series, and BWK-mapped surface must not be
|
|
presented as total natural habitat.
|
|
- Regional flood-hazard coverage is the next bounded data pass. It should use
|
|
governed municipality partitions and measured VMM scenario rasters before
|
|
attempting a full-resolution regional DHMV expansion; no water volume or
|
|
inland bathymetry may be inferred from flood depth.
|
|
|
|
## Sprint 211 - Regional VMM flood-hazard provisioning support (2026-07-16)
|
|
|
|
Implemented:
|
|
- Added `scripts/provision_regional_flood_hazards.py`, an explicit operator
|
|
that resolves the approved geographic scope, validates the fixed twelve-item
|
|
VMM flood-hazard registry and acquires scenarios per persisted municipality
|
|
Area through the existing canonical API endpoints.
|
|
- Kept all raster persistence inside the existing `FloodHazardAcquisitionService`
|
|
and Dataset/DatasetVersion/Job flow. The operator performs no direct WCS
|
|
requests, no direct PostGIS writes and no browser/startup provider fetching.
|
|
- Added `--dry-run`, `--members`, `--products`, `--force` and
|
|
`--stop-on-error` controls so Mol, selected municipalities or the complete
|
|
28-member scope can be run safely and resumed.
|
|
- Added packaging/readiness support so the operator is compiled and available
|
|
in the all-in-one Unraid image.
|
|
- Documented that regional flood coverage remains municipality-partitioned
|
|
because one monolithic Kempen raster would exceed practical WCS/pixel limits.
|
|
|
|
Validation:
|
|
- Focused tests cover product/member resolution, dry-run planning, canonical
|
|
acquisition/selection calls and release packaging.
|
|
|
|
Known limitations:
|
|
- This pass adds the operational regional provisioner and validation contract.
|
|
A complete live run of all 336 municipality/scenario acquisitions is long
|
|
operator work and should be launched deliberately on Tower after reviewing
|
|
the dry-run matrix.
|
|
- VMM flood depth remains modelled local maximum scenario depth. GeoIntel still
|
|
must not expose permanent waterbody volume, current water level or
|
|
bathymetry from this source.
|
|
|
|
Next:
|
|
- Run the regional VMM operator first for Mol/Geel with `pluviaal_current_t100`
|
|
on Tower, then expand to the full twelve-scenario municipality matrix if the
|
|
provider remains stable. After that, prioritize regional DHMV DTM/DSM with
|
|
the same partitioned operator discipline.
|
|
|
|
Follow-up validation:
|
|
- Fixed the regional flood operator's Area list pagination after the live
|
|
Tower API correctly rejected `limit=500`; the operator now uses canonical
|
|
`limit=200` paging.
|
|
- Redeployed Tower, reran live migration/browser runtime checks and validated
|
|
the live operator path with Mol `pluviaal_current_t100`. The run reused
|
|
Dataset `5d37d9cb-cad5-446e-bfd9-ec30e308d4bd` and reported 451.04 ha
|
|
modeled pluvial T100 positive-depth area for Mol with no failures.
|
|
- Added `docs/DATAVINDPLAATS_SOURCE_ROADMAP.md` with recommended public
|
|
Vlaanderen Datavindplaats follow-up sources for hydrology, soil, planning
|
|
and accessibility.
|
|
|
|
## Sprint 212 - Platform-wide official source portfolio (2026-07-16)
|
|
|
|
Implemented:
|
|
- Audited the official public-source roadmap across the complete GeoIntel
|
|
product instead of continuing with a water-first backlog.
|
|
- Added one central frontend portfolio covering space/buildings,
|
|
nature/agriculture, soil/relief, mobility/accessibility,
|
|
population/services and climate/living environment.
|
|
- Each source records official ownership/coverage, the metrics it can honestly
|
|
support, an implementation priority and a real catalogue URL. Runtime status
|
|
is derived only from matching ready Datasets.
|
|
- Reworked the Sources inventory into six compact domain summaries. Detailed
|
|
theme/timeline state, active source limitations and candidate-source cards
|
|
remain accessible through progressive disclosure.
|
|
- Live responsive review found and fixed a narrow-viewport title-row squeeze;
|
|
the status badge now stacks below the source introduction on mobile.
|
|
- Rewrote the Datavindplaats roadmap around a cross-domain area profile. The
|
|
next recommended implementation is a fixed allowlist of Mercator thematic
|
|
rasters plus the digital soil map, not another isolated water connector.
|
|
|
|
Boundaries:
|
|
- No backend provider, live fetch, API contract, database model or migration
|
|
changed in this pass.
|
|
- Audited sources remain visibly non-operational until a governed import and
|
|
metric-validation pass has completed. No source data or historical trend was
|
|
fabricated.
|
|
|
|
Validation:
|
|
- Frontend typecheck passes after the source-portfolio extraction and Sources
|
|
workspace refactor.
|
|
- Focused source-portfolio tests and the complete readiness gate are recorded
|
|
in the delivery handoff for this sprint.
|
|
|
|
Next:
|
|
- Implement the governed Flemish thematic-raster registry Wave 1 for space
|
|
occupation, open space, population density, node value and total service
|
|
level, followed by the digital soil map vector operator.
|
|
|
|
## Sprint 221 - Governed source freshness audit (2026-07-16)
|
|
|
|
Implemented:
|
|
- Added a project-level canonical source-freshness endpoint derived exclusively
|
|
from persisted Dataset, DatasetVersion and local storage evidence.
|
|
- Defined explicit policies for GRB/VRBG/register snapshots, annual
|
|
population/land-use/agriculture/nature/water series, fixed DHMV/soil/thematic
|
|
editions, VMM scenarios, historical archives and local artifacts.
|
|
- Added fail-closed classification for unknown sources and local integrity
|
|
checks for missing versions, checksum mismatches, missing files and stored
|
|
size disagreement.
|
|
- Added a compact Status workspace panel and packaged
|
|
`audit_source_freshness.py` for explicit Unraid cron/reporting use.
|
|
|
|
Boundaries:
|
|
- No migration, provider call, background daemon, source download or automatic
|
|
Dataset refresh was introduced.
|
|
- A fixed historical edition/scenario remains current evidence of that edition;
|
|
old publication dates alone are not treated as corruption or staleness.
|
|
|
|
Validation:
|
|
- Focused service, policy, canonical envelope, CLI and UI contract tests added.
|
|
- Full release, live deployment and browser validation are recorded in the
|
|
delivery result after this implementation entry.
|
|
|
|
Next:
|
|
- Add machine-readable catalogue release probes only for providers with stable
|
|
official version endpoints, keeping every acquisition an explicit bounded
|
|
operator action.
|
|
|
|
## Sprint 222 - Official source edition probes (2026-07-16)
|
|
|
|
Implemented:
|
|
- Verified the public machine-readable contracts published by Digitaal
|
|
Vlaanderen. GRB WFS and OMWRGBMRVL WMS capabilities expose stable ISO 19139
|
|
CSW records containing real source editions and metadata dates.
|
|
- Added a canonical project endpoint that reads only allowlisted capabilities
|
|
and linked `metadata.vlaanderen.be` `GetRecordById` responses. It confirms
|
|
GRB `GBG/WBN/WGO/ADP` and orthophoto `Ortho/Vliegdagcontour` availability,
|
|
then reports the official edition beside local `source_version` evidence.
|
|
- Added strict HTTP(S)/metadata allowlists, timeout and byte limits,
|
|
post-redirect validation, SHA-256 capabilities evidence, provider-isolated
|
|
degraded/unavailable states and a short in-memory cache.
|
|
- Added an explicit Status action and operator CLI flags. Neither path runs on
|
|
page load, starts a job, downloads source data or changes a Dataset.
|
|
- Added Compose, all-in-one Unraid and DockerMan controls for the bounded probe.
|
|
|
|
Validation:
|
|
- Deterministic tests cover official XML parsing, edition comparison,
|
|
provider failure isolation, metadata URL rejection, size limits, caching,
|
|
disabled mode and canonical envelopes.
|
|
- Frontend typecheck/build and targeted backend tests pass before the full
|
|
release and Tower validation recorded in the delivery result.
|
|
|
|
Boundaries:
|
|
- The general Datavindplaats API requires authentication and is not scraped or
|
|
silently bypassed. GeoIntel uses only public metadata records advertised by
|
|
the official OGC services.
|
|
- A version difference remains a manual provenance-review signal. No automatic
|
|
import, refresh scheduler, GRB feature fetch or orthophoto pixel request was
|
|
added.
|
|
|
|
Next:
|
|
- Extend the same probe pattern only where another official source publishes a
|
|
stable machine-readable edition. Do not infer release versions from service
|
|
protocol versions, ETags or HTTP modification dates.
|
|
|
|
## Sprint 223 - Governed regional GRB refresh (2026-07-16)
|
|
|
|
Implemented:
|
|
- Added a canonical read-only refresh plan that maps the official dated GRB
|
|
edition onto the immutable regional buildings, roads, water and parcel
|
|
temporal series.
|
|
- Added explicit per-layer current/update/review states, existing Dataset,
|
|
feature/storage impact and non-destructive action guidance.
|
|
- Added `manage_grb_refresh.py` with separate plan, fetch-only stage and
|
|
checksum-confirmed apply phases. Stage validates all retained artifacts and
|
|
municipality partitions; apply revalidates the exact staged plan before
|
|
calling the existing regional DatasetService-based operators.
|
|
- Added a compact Status surface after the explicit official catalog check.
|
|
|
|
Boundaries:
|
|
- No migration, scheduler, arbitrary provider URL, browser-triggered heavy
|
|
import, direct vector-feature write or automatic Dataset replacement was
|
|
introduced.
|
|
- Existing GRB snapshots remain immutable historical observations. Exact
|
|
remote deltas are reported only after full staging, never guessed from
|
|
capabilities metadata.
|
|
|
|
Validation so far:
|
|
- Backend compile and frontend typecheck passed.
|
|
- Source-catalog plus governed-refresh focused suite passed: 20 tests.
|
|
- Full `scripts/run_readiness_check.sh` passed: 781 backend tests, 110
|
|
documented route contracts, one Alembic head, frontend typecheck/build and
|
|
all packaged smoke checks.
|
|
- Live Tower staging completed all 112 municipality/theme partitions for
|
|
official edition `2026-07-15`: 1,054,223 features, 1,250,873,313 artifact
|
|
bytes and staged plan SHA-256
|
|
`7a772b70771c223b14d16c097c917fb160883349df61e24a2730e8c9591d56fd`.
|
|
- Checksum-confirmed apply created four new Datasets with exact PostGIS counts:
|
|
466,092 buildings, 84,513 roads, 88,330 water features and 415,288 parcels.
|
|
The four `2026-07-14` snapshots remain present; total project datasets moved
|
|
from 632 to 636 and source integrity remains zero-error.
|
|
- Live browser validation exposed and fixed additive theme ranking where two
|
|
extra old water features could outweigh a newer observation date. Ranking is
|
|
now priority, observation date, import date, then feature-count tie-breaker.
|
|
- Post-deploy browser validation also exposed a stale map-theme remount state:
|
|
returning from Status to Map could retain the Water legend while the viewport
|
|
query and zoom guidance reverted to Buildings. Map state now initializes from
|
|
the selected persisted dataset so all four signals remain aligned.
|
|
|
|
Next:
|
|
- Run the full release gate, deploy to Tower, execute a live read-only plan and
|
|
then stage/apply only the exact confirmed official edition.
|
|
|
|
## Sprint 224 - Governed GRB evolution (2026-07-16)
|
|
|
|
Implemented:
|
|
- Audited the two retained regional editions for buildings, roads, water and
|
|
parcels. All 2,108,425 persisted VectorFeatures have an official
|
|
`source_feature_id`; none uses the former 64-character geometry-hash
|
|
fallback.
|
|
- Added a fail-closed GRB identity contract to temporal object comparison.
|
|
Legacy snapshots require authoritative, area-clipped, non-truncated
|
|
provenance from an approved regional operator and every selected identifier
|
|
must be present, unique and match its collection prefix.
|
|
- Made future regional GRB operators abort when an official OGC feature ID is
|
|
missing and persist their identity scheme/prefixes explicitly.
|
|
- Replaced ambiguous same-year labels such as `2026-2026` with daily edition
|
|
ranges and explained that registration changes do not date physical change.
|
|
- Documented an honest refresh-readiness matrix for all major source families;
|
|
no additional provider fetch or automatic refresh was enabled.
|
|
|
|
Boundaries:
|
|
- No migration, new persistence model, scheduler, browser-triggered import or
|
|
parallel temporal engine was added.
|
|
- Object comparison remains bounded to 5,000 selected features and falls back
|
|
to metric-only output whenever source identity cannot be proven.
|
|
|
|
Validation so far:
|
|
- Focused GRB operator and temporal tests pass: 39 tests in the first pass.
|
|
- The complete readiness gate passes after the legacy regression addition: 786
|
|
backend tests, 110 documented routes, one Alembic head, frontend typecheck,
|
|
production build and packaged script checks.
|
|
- The first Tower API comparison correctly exposed that the 14 July snapshots
|
|
predated the explicit `geometry_clipped_to_area` marker. Those snapshots are
|
|
now accepted only when their actual retained legacy evidence is complete:
|
|
the exact Kempen scope, 28 members and partitions, an approved assignment
|
|
strategy, manifest/source evidence, artifact checksum and 28 valid partition
|
|
checksums. A focused negative test proves incomplete partition evidence still
|
|
fails closed.
|
|
|
|
Live validation:
|
|
- Tower was redeployed at commit `2d0e02e`; the all-in-one container became
|
|
healthy on port 1202, PostGIS 3.6 answered, Alembic remained at the single
|
|
`202607160001` head and the frontend/API proxy smoke passed.
|
|
- A read-only PostGIS edition audit found real official-ID changes between 14
|
|
and 15 July: buildings 46 added/32 removed, roads 22/13, water 4/6 and
|
|
parcels 27/27 across the region.
|
|
- Four bounded canonical API comparisons returned
|
|
`object_changes.available=true` and persisted GeoJSON evidence for every GRB
|
|
theme. The checked selections returned 8 building, 20 road, 4 water and 20
|
|
parcel changes respectively.
|
|
- The live browser exposes `Bebouwing · GRB (14-15 jul 2026)`, both exact
|
|
observation dates and the registration-versus-physical-change warning. The
|
|
workbench remained visually coherent and produced no browser warnings or
|
|
errors.
|
|
|
|
Next:
|
|
- Evaluate ALZ as the next governed edition probe only after its official
|
|
machine-readable release version and schema stability are verified.
|
|
## Sprint 230 - Governed orthophoto release preflight (2026-07-17)
|
|
|
|
Implemented:
|
|
- Added `scripts/orthophoto_release_preflight.py` as a read-only gate for one
|
|
bounded current-orthophoto selection in the approved Kempen regional scope.
|
|
- Bound the existing canonical product registry and source-catalog result to
|
|
the exact official WMS 1.3.0 capabilities hash, ISO metadata identifier and
|
|
`YYYY.NN` edition. Redirects, hosts, paths and response sizes fail closed.
|
|
- Added metadata-only WCS `DescribeCoverage` validation for `Ortho`,
|
|
EPSG:31370, the complete raster domain, 15 cm rectified grid, three bands and
|
|
TIFF native format. Selection limits remain identical to acquisition:
|
|
128-1,024 m per projected side.
|
|
- Added a maximum 64-point deterministic `Vliegdagcontour` grid with exact
|
|
GeoJSON layer/date/year validation and hashed sample evidence. WCS domain
|
|
containment proves raster-domain coverage; point samples remain honestly
|
|
labelled as flight-date evidence rather than polygon-union geometry.
|
|
- Kept all pixel requests, filesystem staging, Jobs, Dataset/PostGIS writes,
|
|
migrations and browser behavior out of this sprint. Local
|
|
`most_recent_at_*` acquisition markers remain non-comparable and blocked.
|
|
|
|
Validation so far:
|
|
- 42 focused preflight/runtime-packaging tests passed. Coverage includes
|
|
release ordering, legacy provenance, hash drift, product variant, official
|
|
host/path, queryable/GeoJSON capability, WCS identity/resolution, missing
|
|
flight coverage, flight-year mismatch, selection bounds and no-write rules.
|
|
- Target Python compilation and Ruff passed.
|
|
- A read-only compatibility run against Tower for bbox
|
|
`5.110,51.180,5.117,51.185` verified official edition `2025.04`, exact Ortho
|
|
WCS domain containment and 20/20 flight-day samples from 2025. It performed
|
|
zero pixel requests and correctly returned `staging_permitted=false` because
|
|
local version `most_recent_at_2026-07-15` is not an official edition.
|
|
- Complete local readiness passed with 866 backend tests, 110 documented
|
|
routes, one Alembic head `202607160001`, static full-chain migration SQL,
|
|
frontend typecheck and production build. Target Ruff, docs/contracts,
|
|
compile, shell syntax and diff checks also passed. Docker is not installed in
|
|
the Codex Windows environment; Compose and PostGIS were therefore validated
|
|
on Tower rather than marked locally complete.
|
|
- Commit `f48fe7b` was pushed to Gitea and deployed through the repository's
|
|
all-in-one DockerMan flow. The rebuilt image packaged the preflight, became
|
|
healthy on port 1202, passed live PostGIS 3.6/schema/collation/Alembic smoke
|
|
and passed the frontend/API/icon browser-proxy smoke.
|
|
- The packaged container repeated the Mol preflight with regional Dataset
|
|
total unchanged at `636 -> 636`: WCS domain containment passed, all 20
|
|
deterministic points reported flight year 2025, zero pixel requests ran and
|
|
`staging_permitted=false` remained enforced for local legacy version
|
|
`most_recent_at_2026-07-15`. Tower checkout and Gitea were synchronized.
|
|
|
|
Boundary:
|
|
- This sprint supplies preflight only. It deliberately does not promote or
|
|
backfill existing orthophotos. A future stage/review/apply coordinator must
|
|
revalidate and retain the exact preflight identity before creating a new
|
|
immutable raster Dataset with official `YYYY.NN` source version.
|
|
|
|
## Sprint 231 - Governed orthophoto release promotion (2026-07-17)
|
|
|
|
Implemented:
|
|
- Added `scripts/manage_orthophoto_release.py` with separate `plan`, `stage`,
|
|
named `review` and checksum-confirmed `apply` actions for one bounded current
|
|
orthophoto selection. No action is scheduled or browser-triggered.
|
|
- Reused the complete Sprint 230 preflight identity. Stage performs exactly one
|
|
allowlisted WMS `Ortho` GetMap, retains the exact source response, creates a
|
|
three-band EPSG:31370 GeoTIFF plus PNG preview and mutates no Dataset.
|
|
- Bound remote catalog/WMS/WCS/flight evidence, request identity, all staged
|
|
file hashes, reviewer and the final Dataset checksum into atomic persistent
|
|
evidence. Host/path drift, oversize responses, modified bytes, stale local
|
|
state and missing exact confirmations fail closed.
|
|
- Added a double-confirmed first official baseline transition for legacy
|
|
`most_recent_at_*` markers. Apply retains every legacy raster and uses only
|
|
the existing canonical upload/DatasetService transaction.
|
|
- Made the latest official `YYYY.NN` Dataset authoritative for source-catalog
|
|
comparison even when a newer-imported rolling marker also exists. No API,
|
|
migration, release table, Job type or frontend behavior changed.
|
|
- Made stage, review and applied evidence write-once. Exact apply retries reuse
|
|
the original checksum-bound evidence and never rewrite approval history.
|
|
|
|
Validation so far:
|
|
- 48 focused Sprint 221/222/230/231 tests pass. The release-management suite
|
|
contains 11 direct plan/stage/review/apply tests. Coverage includes
|
|
official-edition ordering in both catalog and source-freshness views,
|
|
first-baseline authorization, GetMap allowlisting/limits, RGB/CRS
|
|
normalization, preview generation, tampering, preflight drift, named review,
|
|
loopback-only apply and complete upload provenance.
|
|
- Complete local readiness passed with 878 backend tests, 110 documented
|
|
routes, one Alembic head `202607160001`, frontend typecheck and production
|
|
build. Target compile and Ruff checks also passed.
|
|
|
|
Live validation:
|
|
- Commits `06d05f0`, `0c2ebc1` and final status correction `0befca0` were
|
|
pushed, deployed through the repository to Tower and validated against
|
|
PostGIS 3.6. The all-in-one container became healthy on port 1202 and passed
|
|
migrations, schema/collation, API proxy and browser shell checks.
|
|
- The packaged operator staged official edition `2025.04` for bbox
|
|
`5.110,51.180,5.117,51.185`: one 921,848-byte official WMS response became a
|
|
496x562, three-band EPSG:31370 GeoTIFF at an approximately one-metre grid.
|
|
All 20 deterministic flight-day samples reported `2025-04-05`.
|
|
- The generated preview was visually inspected as a complete RGB orthophoto
|
|
with expected Mol roads, buildings and sports context. Named review bound
|
|
plan SHA-256 `c6cefd19dc60d7469b4885d9d1f5eff4e52a88808bbe3ac21879c5650e558662`
|
|
and review SHA-256
|
|
`9f0a3a65eb184fea7f91daa5018c0980ed03b792bd465a98c65f2add2413d90b`.
|
|
- Checksum-confirmed apply created exactly one immutable raster Dataset
|
|
`3cf14603-03da-42f5-b152-4bbecad7d006`; project total moved from 636 to 637.
|
|
Dataset, version and staged GeoTIFF share SHA-256
|
|
`f87701de6680f8ad25407996ddfd08056bf0533c05572ae489e9d478a72e7e46`.
|
|
The canonical API returns the complete request, catalog, flight-date,
|
|
reviewer and artifact provenance.
|
|
- An idempotent retry retained 637 Datasets and the same Dataset ID. A live
|
|
audit exposed that the applied-evidence timestamp could still be rewritten;
|
|
write-once validation was added, tested, redeployed and the evidence file
|
|
remained byte-identical on the next retry at SHA-256
|
|
`f4ae7b2315aed9370c8ca04dcb57eabf0c0e49c40cf63a862bb4e69bacd4b16d`.
|
|
- Browser validation at 1920x911 showed a correctly rendered MapLibre canvas,
|
|
coherent regional layer workflow, no horizontal overflow and no console
|
|
warnings or errors. A final live API audit found the generic source status
|
|
still selected a retained rolling label; source-freshness now prefers the
|
|
official edition without changing the 180-day review policy. The redeployed
|
|
Status workspace visibly reports `Editie: 2025.04`; a fresh official catalog
|
|
probe reports `local=remote=2025.04` and comparison `same`.
|
|
|
|
Next:
|
|
- Surface the existing release evidence read-only in the operator Status
|
|
workspace so official/local edition, review identity, Dataset ID and hash
|
|
integrity can be inspected without shell access. Keep all refresh actions
|
|
operator-only and outside the browser.
|
|
|
|
## Sprint 232 - V1 user-flow completion (2026-07-17)
|
|
|
|
Implemented:
|
|
- Re-audited the live platform from the end-user entry point instead of
|
|
continuing operator release work. The normal path is now explicitly Map ->
|
|
area -> measurements/evolution -> AI question or download.
|
|
- Added the semantic metric label to every row in the 15-theme result summary.
|
|
This distinguishes, for example, a Waterinfo station level from mapped
|
|
surface-water area or watercourse length.
|
|
- Kept vector output as GeoJSON and added complete JSON downloads for governed
|
|
raster analyses and historical comparisons. Copy actions now copy the full
|
|
analysis response rather than an empty raster GeoJSON shell.
|
|
- Added direct post-analysis handoff buttons to `AI-vragen` and `Downloads`.
|
|
The selected Area or drawn bbox remains in shared App state.
|
|
- Kept the geographic explorer mounted while other workspaces are active so
|
|
its selected theme, analysis result and temporal comparison survive the
|
|
round trip to AI or Downloads. Hidden workspaces remain non-interactive and
|
|
MapLibre resizes through its existing `ResizeObserver` when shown again.
|
|
- Added one current V1 completion board to `docs/TODO.md`; legacy checklists
|
|
remain preserved but no longer define product readiness.
|
|
|
|
Validation so far:
|
|
- Live pre-change acceptance selected the persisted Mol municipality Area and
|
|
completed all 15 PostGIS-backed themes. Representative results included
|
|
502.14 ha building footprint, 3,626.56 ha forest/green, 36,783 inhabitants,
|
|
3,223.5 ha agricultural use and 1,015.51 km roads.
|
|
- Live historical acceptance compared the official modern forest series from
|
|
2013 to 2025 over Mol: five observations, 3,723.17 ha to 3,626.56 ha,
|
|
absolute change -96.6 ha and relative change -2.6%.
|
|
- Ollama status and installed-model discovery were live. A grounded assistant
|
|
query returned HTTP 200; a separate `qwen3.5:4b` request completed in 13.01
|
|
seconds. Model answers remain advisory and source evidence remains primary.
|
|
- Focused end-user, temporal/Ollama and Sprint 232 tests pass: 38 tests.
|
|
- Frontend typecheck and production build pass.
|
|
- Complete readiness passes with 881 backend tests, 110 documented routes,
|
|
one Alembic head `202607160001`, frontend typecheck and production build.
|
|
|
|
Live validation:
|
|
- Commit `161564f` was pushed to Gitea and deployed through the repository's
|
|
all-in-one Unraid flow. The rebuilt container became healthy on port 1202
|
|
and passed PostGIS 3.6 connectivity, collation, runtime schema and the single
|
|
Alembic head `202607160001` through the live migration smoke.
|
|
- The redeployed browser flow selected the official Mol Area, analysed the
|
|
complete 114.55 km2 municipality and returned all 15 themes. The governed
|
|
space-occupation result remained 3,638.41 ha / 31.8%, with every summary row
|
|
carrying a semantic metric name including `Jaargemiddelde waterstand`.
|
|
- The result survived a complete Map -> AI -> Map and Map -> Downloads -> Map
|
|
round trip. The same 15-theme result and `Download analyse` action remained
|
|
available after both returns; no browser console warning or error occurred.
|
|
- Visual acceptance passed at 3440x1440 with theme, MapLibre map and results
|
|
visible side by side. At a 390x844 mobile viewport the document stayed at
|
|
its 375 px client width with no horizontal page overflow.
|
|
|
|
V1 completion status:
|
|
- No release-critical V1 blocker remains in the intended end-user flow. Future
|
|
source editions, additional independent AI review data and real segmentation
|
|
models are optional controlled expansions, not prerequisites for using the
|
|
current map, measurement, evolution, local-AI and export workflow.
|
|
|
|
## Sprint 233 - Operational correctness and result completion (2026-07-17)
|
|
|
|
Implemented:
|
|
- Corrected the map-first selection scope after live API evidence showed that a
|
|
drawn bbox plus the active `area_id` returned the complete municipality
|
|
vector population. The shared selection contract now resolves `bbox ∩ Area`;
|
|
a bbox enclosing the complete Area keeps the exact persisted geometry and
|
|
only that case may use the full-Area fast path.
|
|
- Applied the same constrained geometry to current theme queries, temporal
|
|
comparisons, derived selection datasets and vector exports, including
|
|
boundary-crossing rectangle regressions.
|
|
- Added Area-aware persistent vector selection exports and
|
|
`POST /api/v1/exports/map-result`. Current vector, governed raster and
|
|
temporal exports are recomputed server-side from persisted data before an
|
|
export row and artifact are written.
|
|
- Replaced the passive `Open downloads` handoff with `Bewaar in downloads`.
|
|
Downloads opens only after successful persistence and surfaces current map
|
|
analyses, historical comparisons and vector selections among the latest
|
|
artifacts.
|
|
- Added exact project-name filtering and merged the canonical
|
|
`Kempen Regional Workbench` into the frontend project page. This remains
|
|
stable even with more than 1,000 retained operator/benchmark projects.
|
|
- Named failed map sources and their API reasons, reset the actual scrolling
|
|
workbench container on navigation, bounded the desktop map layout, removed
|
|
nested desktop scrolling from Detection Lab and added plain-language QA/F1
|
|
interpretation.
|
|
- Limited Detection Lab's luchtbeeld selector to ready imagery rasters;
|
|
terrain, flood-hazard and thematic policy rasters remain available only in
|
|
their correct map workflows.
|
|
- Translated the visible Downloads workflow and moved technical identifiers to
|
|
the existing history disclosure.
|
|
|
|
Validation:
|
|
- Focused operational-correctness, map-selection, export and V1 flow tests
|
|
passed after the corrected selection semantics were applied.
|
|
- The final complete readiness gate passed 899 backend tests, backend compilation,
|
|
documentation smoke, API contract audit, the single Alembic head
|
|
`202607160001`, frontend TypeScript typecheck and the production build.
|
|
- Commit `a29c787` passed the first live deployment, exact bbox/full-Area API
|
|
comparison and persistent Downloads browser handoff. The final constrained
|
|
boundary and luchtbeeld-filter follow-up is validated below before its
|
|
replacement deployment.
|
|
|
|
Final live acceptance:
|
|
- Commit `d652db6` was pushed to Gitea and deployed through the repository's
|
|
all-in-one Unraid flow. Container health, PostGIS 3.6, required
|
|
schema/indexes, Alembic head `202607160001`, frontend proxy and icon checks
|
|
passed.
|
|
- A bbox crossing Mol's western boundary returned 358 buildings / 3.73 ha
|
|
without an Area constraint and 203 buildings / 2.27 ha with the official
|
|
Mol Area. The same constrained contract returned 1,249 / 16.71 ha for a
|
|
bbox inside Mol and 36,941 / 425.95 ha only for a bbox enclosing the complete
|
|
municipality.
|
|
- A browser-drawn Mol selection returned 14 buildings / 0.33 ha rather than
|
|
municipality totals. `Bewaar in downloads` persisted the server-computed
|
|
GeoJSON and opened Downloads with the new ready artifact selected among the
|
|
latest files.
|
|
- Detection Lab now reports seven ready orthophotos. Its selector contains no
|
|
VMM flood, DHMV terrain or thematic policy raster, while PyTorch remains
|
|
ready, the local building model remains active and F1 `0.607` is explicitly
|
|
labelled exploratory/review-required.
|
|
- Browser acceptance passed at 3440x1440 and 390x844 without horizontal page
|
|
overflow. Workspace navigation reset the real scrolling main container and
|
|
the browser console contained no warning or error.
|
|
|
|
## Sprint 234 - Full audit closure and workspace lifecycle cleanup (2026-07-17)
|
|
|
|
Implemented:
|
|
- Closed the remaining P1 project-pollution finding with reversible
|
|
`active`/`archived` lifecycle filtering. The default API and frontend load
|
|
active workspaces only; archived workspaces and all dependent persistence
|
|
remain available.
|
|
- Added `scripts/archive_technical_projects.py`. It is dry-run-first, uses a
|
|
strict technical-name allowlist and always preserves the canonical Kempen
|
|
and Mol workspaces. The all-in-one image packages it and readiness compiles
|
|
it.
|
|
- Added an explicit archive action for a selected non-canonical workspace.
|
|
The UI explains that data and results are retained and protects the two
|
|
canonical workspaces.
|
|
- Extracted Overview orchestration, Detection model management and pure Map
|
|
helpers from the largest frontend components. Shared state and API clients
|
|
remain unchanged.
|
|
- Reworded remaining visible operator terminology in the Map, Quality,
|
|
Detection, Segmentation and dataset controls. Technical UUIDs, file paths,
|
|
hashes and raw runtime states remain available only under labelled details.
|
|
- Added ultrawide AI-workspace constraints so controls use the available width
|
|
without creating multiple narrow nested columns.
|
|
|
|
Validation:
|
|
- The local readiness gate passed 907 backend tests, backend compilation,
|
|
documentation and API contract audits, the single Alembic head
|
|
`202607160001`, frontend TypeScript typecheck and the production build.
|
|
- The first live dry-run correctly made no database change but exposed that a
|
|
direct `/app/scripts/archive_technical_projects.py` invocation resolved the
|
|
wrong Python package. The script now bootstraps the packaged `/app` backend
|
|
root, and a subprocess regression executes the documented command shape.
|
|
- The corrected all-in-one deployment became healthy, reported PostGIS 3.6,
|
|
passed required schema/index checks and retained Alembic head
|
|
`202607160001`.
|
|
- The live cleanup dry-run matched 1,091 allowlisted technical projects and
|
|
preserved both canonical workspaces. Apply archived the same 1,091 rows.
|
|
The active-project API now reports exactly two rows (`Kempen Regional
|
|
Workbench` and `Mol Municipality Workbench`), the archived API reports
|
|
1,091, and a repeated dry-run reports zero remaining matches.
|
|
- Live browser inspection found no console errors or document overflow at
|
|
3440x1440. Map and Detection workspaces keep a bounded 1,680 px operational
|
|
canvas, technical disclosures are collapsed and persisted AI internals now
|
|
use friendly labels in the ordinary result tables. At 390x844 the document
|
|
stays within its 375 px client width; the theme inventory is bounded to a
|
|
compact scrollable selector so it no longer pushes the map behind all 15
|
|
theme cards.
|
|
|
|
## Sprint 235 - Governed bathymetry profiles and Belgian expansion model (2026-07-17)
|
|
|
|
Implemented:
|
|
- Audited official VHA, MDK Belgian Continental Shelf and SPW Walloon
|
|
bathymetry services and separated profile evidence, continuous bed models
|
|
and time-specific water depth/volume.
|
|
- Added bounded, paged VHA profile acquisition with exact persisted-Area
|
|
clipping, official watercourse names, document links, checksums and standard
|
|
Dataset/VectorFeature persistence.
|
|
- Added reusable vector-byte import and governed `min`/`max` property metrics
|
|
to the existing selection aggregation path.
|
|
- Added the bathymetry source and acquisition API, environment controls, Mol
|
|
operator command, map theme, profile inspector and source inventory.
|
|
- Kept municipality-only bathymetry hidden for a regional active Area until a
|
|
complete partition manifest exists; the Mol operator targets the canonical
|
|
Kempen workspace so selecting Mol in the ordinary map exposes the layer.
|
|
- Documented partitioned Flanders scaling and federated Belgium/maritime
|
|
scaling with explicit territorial sea, EEZ/continental shelf and
|
|
TAW/LAT/mDNG semantics.
|
|
|
|
Validation:
|
|
- Backend compilation, 914 backend tests, documentation/contract audits,
|
|
Alembic head `202607160001`, frontend TypeScript typecheck and the production
|
|
Vite build passed in the complete readiness gate.
|
|
- Commits `5b4e180`, `00fcdcb` and `efdcaec` were pushed to Gitea and deployed
|
|
through the all-in-one Unraid flow. Container health, PostGIS 3.6, required
|
|
schema/indexes, Alembic head `202607160001`, frontend proxy and icon checks
|
|
passed after the final deployment.
|
|
- The canonical Kempen workspace now contains one reusable Mol bathymetry
|
|
partition with 828 persisted profiles, 715 official source documents,
|
|
26 named watercourses and observations dated from 1877-01-01 through
|
|
2020-07-03. A repeated operator run returned `reused=true` for dataset
|
|
`25d0f189-aa0c-4467-adc7-2fc684277bf9`, proving deployment-safe
|
|
idempotency.
|
|
- Persisted metrics report 1.08 m mean recorded depth for 112 structured
|
|
observations, a 0.3-3.0 m recorded range, 2.79 m mean crown width and
|
|
1.30 m mean floor width. The UI keeps these historical profile statistics
|
|
separate from continuous bed elevation and current water volume.
|
|
- Live browser acceptance selected the official Mol boundary, exposed the
|
|
Waterbodem theme as `828 profielen / 715 bronbladen`, displayed the
|
|
historical survey range `1877-2020` and loaded all 828 persisted objects.
|
|
The browser console contained no warnings or errors and the 1265x720
|
|
viewport had no horizontal document overflow.
|
|
|
|
## Sprint 236 - Flemish bathymetry partitions and safe North Sea probe (2026-07-17)
|
|
|
|
Implemented:
|
|
- Added a dynamic Flanders scope operator that discovers the complete current
|
|
official VRBG RefGem municipality inventory instead of freezing a fragile
|
|
hand-maintained list. It validates unique NIS codes/names, polygonal source
|
|
features and a 270..300 safety range before creating artifacts or API state.
|
|
- Added an atomic, resumable VHA municipality coordinator. Every partition is
|
|
exact-Area clipped and persisted through the existing acquisition endpoint;
|
|
failures, explicit zero-profile results and ready Dataset ids remain in the
|
|
checksum-bound coverage manifest.
|
|
- Added server-side partition finalization. Regional activation requires exact
|
|
accounting for every expected municipality, one ready VHA Dataset per
|
|
non-empty Area and explicit no-profile Area ids. Dataset and DatasetVersion
|
|
metadata retain the manifest identity and completeness flags.
|
|
- Added a strict-TLS, response-bounded MDK WCS GetCapabilities probe and a
|
|
concise Sources-workspace control. It never performs GetCoverage, never
|
|
disables certificate verification and always leaves acquisition disabled.
|
|
- Packaged the Flanders scope, VHA coordinator and MDK probe operators in the
|
|
all-in-one image and exposed their bounded runtime settings in Compose and
|
|
the Unraid deployment configuration.
|
|
|
|
Validation:
|
|
- Official VRBG fetch-only validation returned 285 unique municipalities, an
|
|
exact union of 13,625.73 km2 and WGS84 bounds
|
|
`[2.54132923, 50.68749237, 5.9111094, 51.50511313]`.
|
|
- The live MDK metadata endpoint currently fails strict hostname validation:
|
|
`bathy.agentschapmdk.be` presents a certificate for `*.l27powered.eu`.
|
|
Diagnostic requests to both the previously configured and current metadata
|
|
paths returned HTTP 404 after an explicitly external, non-application
|
|
insecure inspection. GeoIntel itself reports `tls_error` and provides no
|
|
bypass.
|
|
- The complete readiness gate passed 920 backend tests, backend compilation,
|
|
documentation/contract audits, 115 documented API routes, Alembic head
|
|
`202607160001`, frontend TypeScript typecheck and the production Vite build.
|
|
|
|
Final live acceptance:
|
|
- Commit `132c4fe` was pushed to Gitea and deployed through the all-in-one
|
|
Unraid flow. Container health, PostGIS 3.6, required schema/indexes,
|
|
Alembic head `202607160001`, frontend proxy and icon checks passed.
|
|
- The dynamic VRBG operator provisioned one exact Flanders land boundary,
|
|
all 285 current municipality Areas and both governed boundary Datasets in
|
|
`Flanders Regional Workbench`. The live union covers 13,625.73 km2 with
|
|
WGS84 bounds `[2.54132923, 50.68749237, 5.9111094, 51.50511313]`.
|
|
A repeated run returned the same project, Area and Dataset ids.
|
|
- The complete VHA coordinator accounted for all 285 municipality partitions:
|
|
269 contain data, 16 explicitly contain no profile and none failed. It
|
|
persisted 128,913 exact-Area profile points from 79,398 source documents;
|
|
49,628 observations have a structured depth field and the recorded survey
|
|
dates range from 1877-01-01 through 2026-04-15.
|
|
- Regional finalization became true only after the complete inventory passed.
|
|
The checksum-bound manifest is
|
|
`7f7fd94ecd64e8fc978b3f14e8f21efb471f9c63f89f16fafc23f9fded89aaa2`.
|
|
A second complete run returned the same hash, counts and Dataset ids without
|
|
creating duplicate data.
|
|
- The live MDK probe returned `tls_error`: the configured official hostname
|
|
does not present a matching trusted certificate. Acquisition remains
|
|
disabled and no insecure fallback or North Sea raster request was made.
|
|
- Live volume exposed a frontend scalability defect: Area loading stopped at
|
|
the first API page and the Data workspace rendered every Dataset card at
|
|
once. The follow-up now exhaustively pages all Areas, renders bounded
|
|
searchable Area/Dataset pages and mounts collapsed Area/history catalogs
|
|
only when opened. This preserves the complete regional inventory without
|
|
overwhelming the browser DOM.
|
|
|
|
## Sprint 236 follow-up - manifest-aware regional VHA analysis (2026-07-17)
|
|
|
|
Implemented:
|
|
- Corrected the Map explorer so a complete Flemish VHA manifest is never
|
|
represented or queried as one arbitrary municipality Dataset.
|
|
- Added one canonical bounded partition-selection endpoint. It selects the
|
|
latest internally complete manifest generation, prefilters Dataset bounds
|
|
and runs a single PostGIS intersection across the contributing persisted
|
|
`vector_features`.
|
|
- Kept exact municipality selection on its own Area partition and added
|
|
manifest-aware server-side GeoJSON export with complete contributing
|
|
Dataset provenance.
|
|
- Made the regional Waterbodem card report the sum of all 269 data-bearing
|
|
partitions: 128,913 profiles and 79,398 source documents. The rendered
|
|
GeoJSON remains capped at 1,000 objects while counts and configured metrics
|
|
cover the complete spatial result.
|
|
|
|
Validation:
|
|
- The complete readiness gate passed: 927 backend tests, backend compilation,
|
|
116 documented API routes, Alembic head `202607160001`, frontend TypeScript
|
|
typecheck and the production Vite build.
|
|
- Tower deployment passed container health, PostGIS 3.6, schema/index,
|
|
Alembic, frontend proxy and icon checks. Live selection returned 828
|
|
profiles from the exact Mol partition in 0.59 seconds, 8,806 profiles from
|
|
22 intersecting partitions in 1.95 seconds and 128,913 profiles from all
|
|
269 data-bearing partitions in 15.89 seconds.
|
|
- The live partitioned export persisted a canonical 1,000-feature GeoJSON
|
|
sample with `total_feature_count=8806`, all 22 contributing Dataset ids and
|
|
`server_recomputed=true`.
|
|
- Browser acceptance showed the complete Flemish metric result, all five
|
|
configured structured depth/width measurements and the explicit 1,000-item
|
|
display limit. A final shell follow-up replaces the representative
|
|
municipality count with regional profile/partition totals and uses the
|
|
end-user label `Vlaanderen (285 gemeenten)`.
|
|
|
|
## Sprint 237 - Bounded Flanders cross-domain profile (2026-07-17)
|
|
|
|
Implemented:
|
|
- Reused the governed MercatorNet registry and canonical acquisition,
|
|
Dataset, analysis and image endpoints instead of introducing a second
|
|
provider path.
|
|
- Added a Flanders-only `Op aanvraag` state for ruimtebeslag 2025, open ruimte
|
|
2022, inwonersdichtheid 2019, knooppuntwaarde 2022 and
|
|
voorzieningenniveau 2022.
|
|
- Made one explicit map rectangle or municipality analysis acquire/reuse all
|
|
five products for the same `bbox ∩ Area`, refresh the Dataset catalog and
|
|
keep the measured results available for switching, export and AI context.
|
|
- Kept the whole Flanders Area disabled for policy-raster acquisition under
|
|
the existing 60 km/30 million cell safety limits. Full-region vectors and
|
|
manifest-aware VHA analysis are unchanged.
|
|
- Corrected `coverage_scope` so only canonically named municipality Areas are
|
|
labelled municipality coverage; regional clipping remains a bounded
|
|
selection.
|
|
|
|
Validation:
|
|
- A pre-implementation live probe acquired and persisted a real 143 x 224
|
|
ruimtebeslag raster for a bounded Mol rectangle through Tower. The canonical
|
|
analysis returned 275.4 ha, 88.4563 percent and 311.34 ha valid raster area
|
|
with complete selection coverage.
|
|
- The complete local release gate passed 932 backend tests, backend
|
|
compilation, the 116-route contract audit, Alembic head `202607160001`,
|
|
frontend TypeScript typecheck and the production Vite build.
|
|
|
|
Final live acceptance:
|
|
- The Sprint 237 commits were pushed to Gitea and deployed through the
|
|
all-in-one Tower flow. Container health, PostGIS 3.6, collation,
|
|
required schema/indexes, Alembic head `202607160001`, frontend proxy and
|
|
icon checks passed.
|
|
- Browser acceptance exposed and then closed one usability gap: entering
|
|
Flanders with an unavailable prior theme now automatically selects the
|
|
usable ruimtebeslag profile.
|
|
- A complete exact-Area Mol run persisted all five official policy rasters and
|
|
returned six real themes including the existing VHA partition: 3,638.41 ha
|
|
ruimtebeslag, 8,139.67 ha open space, an estimated 36,783 inhabitants for
|
|
2019, mean node value 0.68, mean service level 0.16 and 828 bathymetry
|
|
profiles. A repeated run reused the retained raster requests.
|
|
- The five new full-Mol Datasets are `ready`, linked to the canonical Mol Area
|
|
and labelled `coverage_scope=municipality`. The UI reports product reference
|
|
years directly so an end-of-year UTC timestamp cannot roll 2025 into a
|
|
misleading local `1 jan 2026` label.
|
|
|
|
## Sprint 238 - Governed Flanders terrain and flood selection (2026-07-17)
|
|
|
|
Implemented:
|
|
- Replaced the policy-raster-specific map query field with a typed official
|
|
raster acquisition contract for `thematic_raster`, `dhmv` and
|
|
`flood_hazard`.
|
|
- Added a single Flanders product hook that loads all three fixed backend
|
|
registries in parallel. The browser still uses only canonical GeoIntel API
|
|
routes and has no external WCS URL.
|
|
- Added an end-user DTM/DSM selector and the full twelve-product VMM scenario
|
|
selector. Existing exact-area Datasets are reused; absent products are
|
|
acquired, persisted, analyzed and reflected back into the Dataset catalog.
|
|
- Extended the all-theme selection run with terrain and flood hazard while
|
|
preserving area intersection and the no-whole-Flanders-raster guard.
|
|
- Fixed the invalid default flood key spelling in the Pydantic request schema.
|
|
|
|
Validation:
|
|
- Focused DHMV, VMM, map-orchestration and source-contract tests passed
|
|
(49 tests).
|
|
- Frontend TypeScript typecheck and production build passed before the full
|
|
release gate.
|
|
|
|
## Sprint 239 - Governed bounded GRB map acquisition (2026-07-17)
|
|
|
|
Implemented:
|
|
- Added a server-allowlisted GRB OGC API Features registry for `GBG`,
|
|
`Wegsegment`, `WTZ`/`WLAS`/`WGR` and `ADP`, exposed as buildings, roads,
|
|
water and parcels.
|
|
- Added canonical product and acquisition endpoints. Acquisition validates
|
|
EPSG:4326 and metric side lengths, intersects the request with the persisted
|
|
Area, follows only trusted pagination, fails instead of truncating and
|
|
retains response/artifact checksums.
|
|
- Routed all output through the existing synchronous Job and
|
|
`DatasetService.import_vector_bytes` flow. The resulting reference Datasets,
|
|
DatasetVersions and PostGIS VectorFeatures use the same persistence boundary
|
|
as uploaded data.
|
|
- Added explicit semantic selection metadata for hectares and kilometres.
|
|
Water volume, legal parcel certainty and traffic semantics remain
|
|
unsupported.
|
|
- Extended the Flanders map product catalog with the four GRB products and
|
|
kept all provider traffic behind the backend.
|
|
- Browser acceptance caught and closed a coverage-label defect: a newly
|
|
acquired small Mol Dataset no longer makes a complete-Flanders theme card
|
|
appear fully loaded. In the Flanders workspace GRB stays `Op aanvraag`;
|
|
the exact bounded result is shown only after selection and an older active
|
|
vector layer is not rendered behind it. The top-level map context follows
|
|
the same state and cannot retain an unrelated Dataset count.
|
|
|
|
Validation:
|
|
- Focused provider, acquisition, route-envelope and frontend contract tests
|
|
cover complete pagination, exact clipping, official identity, hostile next
|
|
links, feature limits, persistent metadata and the canonical Job response.
|
|
- A read-only live provider probe over one small Mol rectangle returned 8
|
|
`GBG` buildings, 23 `Wegsegment` roads, 20 water features across
|
|
`WTZ`/`WLAS`/`WGR` and 96 `ADP` parcels. Every collection completed without
|
|
truncation using explicit OGC CRS84 bbox and output negotiation.
|
|
- The complete readiness gate passed 943 backend tests, backend compilation,
|
|
the 118-route API contract audit, Alembic head `202607160001`, frontend
|
|
TypeScript typecheck and the production Vite build.
|
|
- Tower live acceptance persisted bounded Mol GRB results through PostGIS:
|
|
339 building contours (5.953873 ha footprint), one road segment
|
|
(0.0137304 km), two water features (0.0027273 ha plus 0.0050982 km
|
|
supporting linework) and eight parcels (1.2566294 ha). A repeated exact
|
|
building request reused the same Dataset and reported `reused=true`.
|
|
- The rebuilt all-in-one deployment remained healthy on port 1202 with
|
|
PostGIS 3.6 and Alembic head `202607160001`. Live browser acceptance showed
|
|
all four Flanders GRB cards as `Op aanvraag` and no stale vector content
|
|
underneath an unmeasured on-demand theme.
|
|
|
|
## Sprint 240 - Operational forest, agriculture, nature and soil (2026-07-17)
|
|
|
|
Implemented:
|
|
- Added governed Landgebruik Vlaanderen 2025 class masks for forest and
|
|
agricultural use while keeping definitive ALZ parcels as a separate source.
|
|
- Added one allowlisted official-vector service for BWK/Natura 2000 2025 and
|
|
DOV soil types with exact Area intersection, complete pagination, metric CRS
|
|
clipping, checksums, cache identity and canonical Dataset persistence.
|
|
- Added source-faithful selection metadata for forest/agricultural hectares,
|
|
biological value, estimated PHAB habitat areas and historical soil classes.
|
|
- Connected all four themes to the existing Flanders product catalog and
|
|
all-theme selection flow without introducing browser-side provider traffic.
|
|
- Browser acceptance exposed and closed a coverage-state defect: a bounded
|
|
municipality raster can no longer make a complete-Flanders theme appear
|
|
preloaded. Governed thematic cards remain `Op aanvraag`.
|
|
- Added Compose and editable Unraid runtime settings for provider endpoints
|
|
and transfer/feature guardrails.
|
|
|
|
Validation:
|
|
- Live read-only provider probes confirmed the BWK `BWK:Bwkhab` WFS collection,
|
|
stable `UIDN` paging and expected `EVAL`/`HAB`/`PHAB` fields.
|
|
- Live DOV WFS probing confirmed `bodemkaart:bodemtypes`, stable
|
|
`numberMatched`/`numberReturned` and expected soil attributes.
|
|
- Direct service probing over a small Mol rectangle completed without
|
|
truncation: 75 BWK source/retained polygons and 29 DOV candidates resulting
|
|
in 28 clipped soil polygons.
|
|
- The complete readiness gate passed 950 backend tests, backend compilation,
|
|
the 120-route contract audit, Alembic head `202607160001`, frontend
|
|
TypeScript typecheck and the production Vite build.
|
|
- Tower deployment and browser acceptance results are recorded after the
|
|
rebuilt all-in-one runtime is verified.
|
|
## 2026-07-17 - Belgium/North Sea RC-3 local gate
|
|
|
|
- Removed automatic selection and every `rasterDatasets[0]` execution fallback
|
|
from Detection Lab. Normal detection, guided preparation and calibration now
|
|
require an explicit raster selection.
|
|
- Added `TemporalCompatibilityService`. Official raster editions carrying
|
|
`supports_detection=false` fail before Job/AnalysisRun creation with
|
|
`DETECTION_SOURCE_TEMPORALLY_UNSUPPORTED`.
|
|
- Detection QA now loads the persisted source Dataset and rejects historical
|
|
candidate/reference periods that are absent or non-overlapping with
|
|
`DETECTION_QA_TEMPORAL_MISMATCH`.
|
|
- Compatible temporal evidence is persisted under
|
|
`parameters_json.temporal_compatibility` and
|
|
`findings_json.temporal_compatibility` on the existing QualityCheck.
|
|
- Added sanitized request IDs, request-duration logs and correlated detection
|
|
start/QA completion logs.
|
|
- Added `scripts/runtime_state_report.py`; default mode is read-only and
|
|
mutation requires `--reconcile --confirm reconcile-interrupted-runtime`.
|
|
|
|
Validation:
|
|
|
|
- `python -m compileall backend/app`: passed.
|
|
- `python -m pytest backend`: 974 passed.
|
|
- `npm run typecheck`: passed.
|
|
- `npm run build`: passed.
|
|
- `bash scripts/run_readiness_check.sh`: passed, including 122-route API audit,
|
|
974 tests, Alembic head, typecheck and build.
|
|
- `python -m alembic heads`: one head, `202607160001`.
|
|
- `python -m alembic upgrade head --sql`: passed.
|
|
- backup/restore/live-smoke shell syntax checks: passed.
|
|
- Local `docker compose config` was unavailable because this Windows
|
|
workstation has no Docker CLI; Tower validation remains mandatory before
|
|
RC-3 completion.
|
|
|
|
## 2026-07-17 - Belgium/North Sea RC-3 live acceptance
|
|
|
|
- Deployed immutable commit
|
|
`a28f2497e8d0da7d108828875ba9ffedda3c1688` to Tower.
|
|
- Live readiness reports PostgreSQL, PostGIS 3.6, Alembic head
|
|
`202607160001`, writable storage and the configured local YOLO capability as
|
|
ready. The health response exposes the exact commit and build time.
|
|
- `scripts/runtime_state_report.py` executed inside the all-in-one container
|
|
and found zero running Jobs and zero running AnalysisRuns.
|
|
- Live migration smoke completed against the retained PostGIS volume.
|
|
- In-app browser acceptance opened Detection Lab on port 1202 and confirmed
|
|
that the raster selector starts empty, exposes nine choices including its
|
|
placeholder, keeps the primary detection action disabled and produces no
|
|
console errors or warnings.
|
|
- RC-3 is complete. RC-4 national and maritime coverage contracts are the
|
|
active roadmap phase.
|
|
|
|
## 2026-07-18 - Belgium/North Sea RC-6 supply-chain gate
|
|
|
|
Implemented:
|
|
|
|
- Replaced smoke-only automation with equivalent Gitea and GitHub release
|
|
workflows covering readiness, offline Alembic SQL, Compose resolution,
|
|
Python/npm audits, immutable image build, SPDX SBOM and container scan.
|
|
- Added hash-pinned Linux/Python 3.11 runtime and CI locks with verified input
|
|
fingerprints. Optional PyTorch, torchvision and Ultralytics remain outside
|
|
the base and CI locks.
|
|
- Added timeboxed, machine-validated Starlette advisory exceptions with
|
|
request-target and form-content compensating controls.
|
|
- Replaced the final runtime `gosu` path with a minimal `setpriv` exec wrapper
|
|
and upgraded packaged setuptools/wheel metadata.
|
|
- Fixed the Trivy policy wrapper so its generated ignore policy is mounted
|
|
read-only into the scanner. The policy pass excludes the shadowed Go
|
|
executable path only; the complete report preserves every base-layer
|
|
finding for review.
|
|
|
|
Validation:
|
|
|
|
- Local readiness passed backend compilation, 1,005 backend tests, the
|
|
124-route contract audit, Alembic head `202607160001`, frontend typecheck
|
|
and production build.
|
|
- Python dependency policy passed with seven explicitly documented Starlette
|
|
aliases ignored through 2026-08-31; frontend npm audit reported zero
|
|
vulnerabilities.
|
|
- Immutable AI image
|
|
`geointel-all-in-one:6a22fcd1f87fbf4921e16b1baa9367c6e24dc8a3-ai`
|
|
deployed healthy and passed live PostGIS 3.6 migration plus browser proxy
|
|
verification.
|
|
- Tower retained `artifacts/geointel-ai-sbom.spdx.json` (25,653,538 bytes)
|
|
and the complete Trivy JSON report. The executable policy pass reported
|
|
zero reachable fixed HIGH/CRITICAL findings.
|
|
- Direct runtime proof returned PostgreSQL UID `999` through the final
|
|
`/usr/local/bin/gosu` wrapper. Runtime packaging reports setuptools
|
|
`83.0.0` and wheel `0.47.0`.
|
|
|
|
Decision:
|
|
|
|
- RC-6 is complete. RC-7 critical API response typing and OpenAPI validation
|
|
is active.
|
|
|
|
## 2026-07-18 - Belgium/North Sea RC-7 API contract hardening
|
|
|
|
Implemented:
|
|
|
|
- Replaced every remaining free-form successful JSON response model with a
|
|
concrete Pydantic schema while retaining the canonical `{"data": ...}`
|
|
payload shape.
|
|
- Added reusable generic envelope, list, pagination and GeoJSON schemas plus
|
|
concrete project, area, assistant, YOLO preflight and QA evidence models.
|
|
- Corrected Area serialization so persisted PostGIS geometries are converted
|
|
to GeoJSON before response validation.
|
|
- Added an executable OpenAPI contract audit and focused regression tests that
|
|
reject missing, free-dictionary or undocumented non-envelope responses.
|
|
- Tracked exactly eight deliberate exceptions: three health probes, four
|
|
persisted PNG endpoints and the streamed export download.
|
|
|
|
Validation:
|
|
|
|
- The API audit passed 124 implemented routes and 228 OpenAPI component
|
|
schemas.
|
|
- Repository readiness passed backend compilation, 1,008 backend tests,
|
|
Alembic head `202607160001`, frontend typecheck and production build.
|
|
- Offline `alembic upgrade head --sql` completed and generated 31,548 bytes of
|
|
migration evidence.
|
|
|
|
Decision:
|
|
|
|
- RC-7 is complete. RC-8 automated frontend and browser release journeys are
|
|
active.
|
|
|
|
## 2026-07-18 - Belgium/North Sea RC-8 release journey automation
|
|
|
|
Implemented:
|
|
|
|
- Added four Vitest suites with 12 executable tests for map selection,
|
|
coverage resolution, temporal comparison and bootstrap state.
|
|
- Added a dry-run-first golden-area operator. It reuses Mol/Kempen and creates
|
|
only missing bounded Wallonia, Brussels, language-boundary, coast and
|
|
offshore Areas through canonical APIs, with deterministic geometry hashes.
|
|
- Added a Playwright release runner and shell wrapper covering every golden
|
|
area, governed Mol metrics/provenance, compatible forest history, no-data,
|
|
partial coverage, unsupported maritime metrics, simulated provider failure,
|
|
persisted map export, real local Ollama context and explicit configured
|
|
local YOLO execution.
|
|
- Fixed explicit demo seeding so its archived technical project is reactivated
|
|
before reuse. A direct service regression protects this lifecycle behavior.
|
|
- Added the frontend tests and static E2E/script checks to readiness and copied
|
|
the golden-area operator into the all-in-one image.
|
|
|
|
Validation:
|
|
|
|
- Repository readiness passed backend compilation, 1,012 backend tests, four
|
|
frontend test files with 12 tests, frontend typecheck/build and Alembic head
|
|
`202607160001`.
|
|
- The live runner completed all Belgium/North Sea journeys against PostGIS
|
|
3.6. It persisted an export, received a grounded Ollama answer and completed
|
|
configured local YOLO execution through Job and AnalysisRun persistence.
|
|
- The successful evidence manifest reports no unexpected browser-console or
|
|
failed-request events. Re-running the area operator created zero duplicate
|
|
Areas and retained seven unique geometry fingerprints.
|
|
|
|
Decision:
|
|
|
|
- RC-8 is complete. RC-9 loading, accessibility and performance hardening is
|
|
active.
|
|
|
|
## 2026-07-18 - Belgium/North Sea RC-9 UX and performance acceptance
|
|
|
|
Implemented:
|
|
|
|
- Replaced transient false missing-source states with explicit bootstrap
|
|
loading and propagated loading/error state to the map and source panels.
|
|
- Added keyboard-correct analysis tabs, reliable skip-link focus, accessible
|
|
map/status regions and stable focus handling across workspace changes.
|
|
- Added measured four-second coverage and fifteen-second map-analysis budgets
|
|
with visible completion and over-budget feedback.
|
|
- Hardened 390-pixel mobile, 1366-pixel desktop and 2560-pixel ultrawide
|
|
layouts without changing map-first behavior.
|
|
- Added a Playwright UX audit for every workspace, accessible control names,
|
|
delayed bootstrap, keyboard behavior, layout overflow and timing feedback.
|
|
|
|
Validation:
|
|
|
|
- Repository readiness passed backend compilation, 1,015 backend tests,
|
|
16 frontend tests, frontend typecheck/build and Alembic head
|
|
`202607160001`.
|
|
- Immutable image
|
|
`9b0239747d1f12d0c5dd3b7fa2d03cf672884dbd-ai` deployed healthy with
|
|
PostGIS 3.6 and the expected migration head.
|
|
- The live UX audit passed at 390x844, 1366x768 and 2560x1080 with no
|
|
horizontal overflow, console errors or failed requests. All seven
|
|
workspaces passed accessible-name checks and the delayed-loading and
|
|
performance-feedback assertions were green.
|
|
|
|
Decision:
|
|
|
|
- RC-9 is complete. RC-10 data operations and retention is active.
|
|
|
|
## 2026-07-18 - Belgium/North Sea RC-10 data operations and retention
|
|
|
|
Implemented:
|
|
|
|
- Added a read-only data-operations audit for storage lifecycle, disk pressure,
|
|
persisted artifact references, failed work and national, regional and
|
|
maritime source families.
|
|
- Added dry-run-first cleanup restricted to explicit derived/cache categories.
|
|
Apply mode requires an exact confirmation token, a recent checksum-verified
|
|
release backup and a bounded delete ceiling.
|
|
- Mounted release backups read-only and protected release evidence, official
|
|
source editions, originals, uploads, model assets, operator evidence and
|
|
unknown paths from cleanup.
|
|
- Hardened artifact discovery for large production tables with bounded SQL
|
|
counts and distinct-path queries, strict manifest field traversal and
|
|
separate current-reference and historical-manifest findings.
|
|
- Classified fixed NGI and RBINS national/maritime editions so old publication
|
|
dates are not mislabeled as stale.
|
|
|
|
Validation:
|
|
|
|
- Repository readiness passed backend compilation, 1,021 backend tests,
|
|
16 frontend tests, frontend typecheck/build and Alembic head
|
|
`202607160001`; focused final policy tests passed 19/19.
|
|
- Immutable image
|
|
`22fb8d51a9fde39552cf06a789174441631842b6-ai` deployed healthy with
|
|
PostgreSQL/PostGIS and the expected migration.
|
|
- The exact live image passed
|
|
`scripts/run_rc10_data_operations_audit.sh`: disk pressure `ok`, zero missing
|
|
direct database references, 2,002 dry-run candidates totaling 218,263,512
|
|
bytes and unchanged critical table counts.
|
|
- The audit retained 224 unavailable historical manifest intermediates as a
|
|
non-destructive provenance warning. No cleanup was applied.
|
|
- Live national/maritime freshness evidence reports three current sources,
|
|
zero due/review-required sources and zero integrity issues.
|
|
|
|
Decision:
|
|
|
|
- RC-10 is complete. RC-11 final release packaging and acceptance is active.
|
|
|
|
## 2026-07-18 - Belgium/North Sea RC-11 final release candidate
|
|
|
|
Implemented:
|
|
|
|
- Assigned semantic version `1.0.0-rc.1` across backend health, frontend
|
|
package metadata and OCI image identity.
|
|
- Added a final release runbook and a fail-closed package builder requiring a
|
|
clean tagged revision, exact image revision, complete SHA-256 inventory and
|
|
verified detached SSH signature.
|
|
- Updated active repository navigation and product identity to Belgium and the
|
|
Belgian North Sea while preserving Mol/Kempen as golden regression areas.
|
|
- Added a current known-limitations register for federated coverage, selection
|
|
semantics, history, AI and operations.
|
|
|
|
Validation:
|
|
|
|
- Repository readiness passed backend compilation, 1,030 backend tests,
|
|
16 frontend tests, frontend typecheck/build and Alembic head
|
|
`202607160001`. Offline migration SQL generation also passed.
|
|
- The pre-tag `1.0.0-rc.1` image passed live PostGIS migration and proxy smoke,
|
|
Docker Compose validation and an isolated fresh install.
|
|
- A full SHA-256 storage/model inventory plus PostgreSQL dump verified
|
|
read-only. Separate isolated restore and upgrade drills matched all retained
|
|
table counts, PostGIS and Alembic, then removed their temporary databases.
|
|
- Real rollback started immutable build `22fb8d5` against the unchanged
|
|
volumes, passed live migration/readiness and returned successfully to the
|
|
RC image.
|
|
- All seven Belgium/North Sea release journeys passed. The UX audit passed at
|
|
390x844, 1366x768 and 2560x1080 with zero horizontal overflow, browser
|
|
console errors or failed requests.
|
|
- Python dependency policy and npm audit passed. SPDX SBOM generation and the
|
|
complete container scan passed the executable policy with zero reachable
|
|
fixed HIGH/CRITICAL findings.
|
|
- RC10 remained read-only: zero missing current database references, unchanged
|
|
critical table counts and 2,002 reported zero-delete cleanup candidates.
|
|
|
|
Decision:
|
|
|
|
- RC-11 is complete subject only to binding the accepted final commit and
|
|
immutable image in the signed/checksummed package. No RC-12 phase is used.
|
|
|
|
## 2026-07-19 - Stitch-guided complete Atlas Workbench redesign
|
|
|
|
Implemented:
|
|
|
|
- Created the private Stitch project `GeoIntel Complete Workbench Redesign`
|
|
and the `GeoIntel Atlas Workbench` design system.
|
|
- Generated production references for the Map Explorer, Sources catalog and
|
|
local AI Questions workspaces.
|
|
- Rebuilt the application shell so the brand lives in one compact navigation
|
|
rail and the active work context lives in one 56-pixel top bar.
|
|
- Added Lucide icons for the eight workspace destinations and removed the
|
|
duplicate command navigation.
|
|
- Restyled Map, Sources, AI Questions, Quality, Image Analysis, Downloads,
|
|
Status and Administration under one restrained token system.
|
|
- Restored the desktop map to a theme/map/insight row, allowed the map to
|
|
absorb ultrawide width and bounded the mobile theme list.
|
|
- Replaced the old `visual-polish.css` layer with the documented
|
|
`atlas-workbench.css` implementation.
|
|
|
|
Validated during implementation:
|
|
|
|
- the complete readiness gate passed: 1,052 backend tests, 22 frontend tests,
|
|
frontend TypeScript typecheck, the production build and Alembic head
|
|
`202607160001`;
|
|
- the browser UX audit passed at 390x844, 1366x768 and 2560x1080, including
|
|
accessible control names, loading states, keyboard tabs, the advanced
|
|
coverage flow, zero horizontal overflow, zero console errors and zero failed
|
|
API requests;
|
|
- all seven Belgium/North Sea release journeys passed against the live backend
|
|
through the redesigned frontend;
|
|
- browser review passed at 1280x720, 3440x1200 and 390x844 with no horizontal
|
|
document overflow or clipped button/select text;
|
|
- Map, Sources, AI Questions, Quality, Image Analysis, Downloads, Status and
|
|
Administration were each reviewed against live backend data.
|
|
|
|
Behavior preservation:
|
|
|
|
- No API contract, migration, database model, provider behavior, GIS
|
|
calculation, model runtime or persistence flow changed.
|
|
- Existing map selection, source loading, local Ollama, QA, analysis and
|
|
download actions remain wired through their original hooks and services.
|
|
|
|
## 2026-07-19 - Stitch screen parity and responsive refinement
|
|
|
|
Implemented:
|
|
|
|
- Extended the Stitch project with dedicated Quality, Image Analysis,
|
|
Downloads and Status/Administration screens, all using design-system asset
|
|
`4667932515738184526`.
|
|
- Replaced the residual dark navigation treatment with the white 88-pixel
|
|
Stitch rail, pale context bar and locally bundled Public Sans/Manrope
|
|
typography.
|
|
- Made Sources a deterministic three-pane workspace after removing a legacy
|
|
full-width catalogue override.
|
|
- Added the Stitch control/result/inspector structure to the empty Quality
|
|
state and kept the existing populated evidence/history implementation.
|
|
- Compacted detection model administration into one row above the existing
|
|
configuration/result/QA workbench.
|
|
- Reorganized Downloads as artifact actions, current/history results and a
|
|
preview inspector without changing export callbacks or formats.
|
|
- Kept the MapLibre map full-height with icon controls and stable rails on
|
|
normal and ultrawide displays.
|
|
|
|
Visual verification:
|
|
|
|
- compared Map, Sources, AI Questions, Quality, Image Analysis, Downloads,
|
|
Status and Administration against their dedicated Stitch references;
|
|
- reviewed live-backend states at 1280x720, 1920x1080 and 390x844;
|
|
- confirmed that the widescreen map absorbs the additional width and that the
|
|
mobile workspaces remain vertically navigable without document overflow.
|
|
|
|
Validation:
|
|
|
|
- the complete readiness gate passed 1,052 backend tests, 22 frontend tests,
|
|
frontend typecheck/build and Alembic head `202607160001`;
|
|
- the final UX browser audit passed at 390x844, 1366x768 and 2560x1080 with
|
|
zero horizontal overflow, console errors or failed requests;
|
|
- all seven Belgium/North Sea release journeys passed through the redesigned
|
|
local frontend against the live backend;
|
|
- the journey retry caught and verified a fixed Sources grid containment issue
|
|
that previously allowed the Area panel to intercept a scrolled Project
|
|
management control.
|
|
|
|
Behavior preservation:
|
|
|
|
- no backend, API contract, migration, persistence, GIS metric, provider,
|
|
Ollama or model-runtime behavior changed;
|
|
- all existing actions continue through the original React hooks and service
|
|
layer.
|
|
|
|
## 2026-07-19 - Complete bounded rectangle analysis
|
|
|
|
Implemented:
|
|
|
|
- removed the active-theme-only acquisition filter from Map rectangle
|
|
analysis;
|
|
- added a deterministic acquisition queue capped at three concurrent official
|
|
provider requests;
|
|
- restricted the result list to themes that are operational or have an
|
|
applicable bounded adapter in the resolved coverage zone;
|
|
- replaced misleading `Bron ontbreekt` rows with explicit provider-failure
|
|
states and counts results per theme rather than per contributing layer;
|
|
- registered VMM VHA historical profile points as an operational bounded
|
|
Flanders bathymetry source and connected its existing acquire/select routes
|
|
to the Map product catalogue;
|
|
- treated both Dutch municipality names and release-golden
|
|
`... municipality` Areas as local scope after live verification exposed a
|
|
false regional flood-raster partition path;
|
|
- extended the shared bounded-dataset guard to raster
|
|
`coverage_scope=bounded_selection` metadata after live full-Mol verification
|
|
proved that a previously acquired small raster could otherwise be
|
|
misrepresented as municipality-wide evidence.
|
|
|
|
Live evidence:
|
|
|
|
- a bounded Mol rectangle persisted and selected all relevant source families:
|
|
NGI, Statbel, four GRB products, seven Flemish thematic rasters, DHMV,
|
|
VMM flood hazard, BWK/Natura 2000, DOV soil and VHA profile points;
|
|
- semantic results included building/forest/agricultural/open-space/water/
|
|
parcel/soil/nature hectares, road kilometres, inhabitants, terrain height,
|
|
flood area, accessibility/service scores and historical profile counts;
|
|
- no browser-direct provider request, fabricated value or vertical-datum
|
|
conversion was introduced.
|
|
|
|
Validation:
|
|
|
|
- the complete readiness gate passed: 1,053 backend tests, 25 frontend tests,
|
|
backend compile, frontend TypeScript/typecheck, production build, Alembic
|
|
head `202607160001` and script smoke checks;
|
|
- a live API smoke over `5.10,51.17,5.11,51.18` proved all 17 applicable Mol
|
|
themes through acquisition or persisted national data and semantic
|
|
selection metrics;
|
|
- browser verification follows against the deployed commit on port 1202.
|
|
|
|
## 2026-07-21 - Scale-aware rectangle analysis
|
|
|
|
Implemented:
|
|
|
|
- reproduced the reported provider failure with an approximately 11,468 km2
|
|
cross-region rectangle: twenty high-resolution adapters received one unsafe
|
|
bbox and returned their governed size limits;
|
|
- added explicit detail, regional and overview selection tiers before any
|
|
provider acquisition starts;
|
|
- regional selections up to 50 by 50 km now split compatible vector and point
|
|
sources into at most sixteen 18 km tiles, resolve the authority zone per tile
|
|
and combine each provider product as one persisted PostGIS result;
|
|
- added a canonical multi-partition vector selection route with project,
|
|
readiness and source-product consistency guards and source-feature count
|
|
deduplication;
|
|
- allowed terrain and flood partition analysis to receive an explicit set of
|
|
freshly acquired Dataset ids, avoiding accidental reuse of overlapping old
|
|
bounded rasters;
|
|
- kept 5 m raster analysis behind its real pixel budget and 10 m thematic
|
|
analysis behind its 50 km scale budget;
|
|
- changed overview selections to query only national or already provisioned
|
|
scale-compatible datasets and explain the scale choice once, instead of
|
|
presenting one provider error for every unavailable detail source.
|
|
|
|
Validation:
|
|
|
|
- backend import/compile and frontend TypeScript passed after the contract
|
|
change;
|
|
- focused selection-partition, DHMV and VMM tests passed (30 tests);
|
|
- focused frontend scale, tiling and concurrency tests passed (12 tests).
|
|
|
|
Remaining in this pass:
|
|
|
|
- run the complete readiness gate, deploy the immutable revision and repeat
|
|
both the large overview rectangle and a regional partitioned rectangle in
|
|
the live browser.
|
|
|
|
## 2026-07-21 - Persisted raster scale guard
|
|
|
|
Live verification of the first scale-aware release showed that on-demand
|
|
provider fan-out was resolved, but already persisted 5 m terrain and flood
|
|
rasters could still enter a Belgium-scale analysis. The shared selection guard
|
|
now applies the same real pixel budgets to persisted rasters: country-scale
|
|
overview analysis keeps PostGIS vector and national statistical sources, while
|
|
high-resolution rasters require detail or safely bounded regional extent.
|
|
Overview responses now return at most 25 representative GeoJSON features per
|
|
source (250 regionally, 1,000 in detail) while `total_feature_count` and every
|
|
PostGIS metric continue to cover the complete selection. This prevents large
|
|
profile properties from blocking the browser without weakening measurements.
|
|
|
|
Validation:
|
|
|
|
- frontend typecheck, 29 unit tests and the production build passed;
|
|
- the map acquisition, component-boundary and density regression contracts
|
|
passed (12 tests);
|
|
- live redeployment and a repeated Belgium-scale rectangle follow on the
|
|
immutable patch revision.
|
|
## 2026-07-22 - National scope and governed Brussels land cover
|
|
|
|
Implemented:
|
|
|
|
- made the persisted Belgium/North Sea project the unconditional startup
|
|
workspace and changed the initial basemap extent and form defaults from Mol
|
|
to Belgium plus its legally labelled maritime scope;
|
|
- retained Mol/Kempen provisioning and tests strictly as golden regression
|
|
evidence instead of product routing;
|
|
- live-validated the UrbIS WFS `Blocks` contract and added bounded land-cover,
|
|
FO/GB forest/park and WB permanent-water products with clipped PostGIS area
|
|
metrics and source-class provenance;
|
|
- restricted the production model picker to the explicit active model asset
|
|
and surfaced that the current building model is locally, not nationally,
|
|
validated.
|
|
|
|
Validated during implementation:
|
|
|
|
- backend compile and frontend typecheck passed;
|
|
- 31 frontend unit tests passed;
|
|
- focused national coverage, model catalog, project lifecycle and Mol golden
|
|
regression tests passed;
|
|
- live resolver checks covered Brussels, Wallonia and all three Belgian
|
|
maritime legal zones. WALOUS, Walloon flood analytics and multi-epoch marine
|
|
bathymetry remain real open source-integration work and were not simulated.
|
|
- the final repository readiness gate passed with 1,084 backend tests, 31
|
|
frontend unit tests, compile, typecheck, production build, Alembic head
|
|
`202607160001` and all script syntax/contract checks.
|
|
|
|
## 2026-07-22 - Live national deployment verification follow-up
|
|
|
|
Implemented after deploying the national-scope revision:
|
|
|
|
- reproduced and removed a national UI ambiguity where the Belgium land scope
|
|
displayed the final Brussels UrbIS catalog item as the active country-wide
|
|
source; multiple applicable regional products are now grouped as an official
|
|
source-per-region contract until the actual rectangle resolves its zones;
|
|
- added exclusive locks for release deployment and container replacement plus
|
|
a bounded wait for Docker's asynchronous removal, after an interrupted Codex
|
|
session exposed a concurrent replacement race on Tower.
|
|
|
|
Live evidence:
|
|
|
|
- immutable AI image `0aff8e3b8c551a9d1aa29a8495a17e5a858205ab-ai` became healthy on
|
|
port 1202 and applied Alembic migrations during startup;
|
|
- `scripts/live_migration_smoke.sh` passed against the embedded PostGIS 3.6
|
|
runtime with all required tables/indexes and single head `202607160001`;
|
|
- the production model catalog exposes exactly the configured approved local
|
|
model instead of 25 training artifacts;
|
|
- coverage resolution is zone-correct for Brussels, Wallonia and all three
|
|
Belgian maritime legal zones. Unimplemented WALOUS, Walloon raster and MDK
|
|
acquisition stays explicitly `not_configured`.
|
|
|
|
## 2026-07-22 - Live WALOUS provisioning and signed-raster correction
|
|
|
|
Implemented and verified:
|
|
|
|
- provisioned the official WALOUS 2020 and 2023 land-cover GeoTIFFs in the
|
|
live source cache, including source checksums, EPSG:3812 metadata and the
|
|
official class-code validation set `1,2,3,4,5,6,7,8,9,80,90`;
|
|
- corrected masked reads of the signed `int8` 2023 source, whose official
|
|
nodata value is `-128`, by widening valid raster values before assigning the
|
|
internal `uint8` nodata value `255`; acquisition, persisted analysis and map
|
|
rendering now share the safe conversion;
|
|
- added a regression test that exercises all official class codes through an
|
|
`int8`/`-128` source and verifies the derived `uint8`/`255` output contract;
|
|
- removed the redundant downloaded 2020 archive after checksum-verified
|
|
provisioning while retaining the normalized source raster and report.
|
|
|
|
Live release evidence:
|
|
|
|
- the complete readiness gate passed with 1,097 backend tests and 31 frontend
|
|
tests, backend compile, API/documentation contract audit, frontend typecheck,
|
|
production build, Alembic head `202607160001` and all script checks;
|
|
- commit `e15a8200fd5c21f95a8cc1f5d6fdc6c5cb2aae3c` was deployed as immutable AI
|
|
image `geointel-all-in-one:e15a8200fd5c21f95a8cc1f5d6fdc6c5cb2aae3c-ai`;
|
|
- live migration smoke passed against embedded PostGIS 3.6 with the complete
|
|
required schema and exactly one Alembic head;
|
|
- live 2023 acquisition created Dataset
|
|
`92561bc3-3f1c-4d30-b926-58dea7793e43`; the same test area measured 315.86 ha
|
|
total, including 110.91 ha forest, 7.57 ha water and 135.79 ha artificial
|
|
land, with full raster coverage;
|
|
- live 2020 acquisition created Dataset
|
|
`bdd19377-bfc9-425c-a7d4-6ac71c02bc1f`; a persisted 2020-2023 comparison
|
|
returned two observations and seven class-aware metric comparisons;
|
|
- the live browser on port 1202 rendered the 2023 WALOUS overlay and real
|
|
hectare metrics for a Walloon rectangle. The evolution workspace exposed
|
|
distinct dated observations and the browser console remained error-free.
|
|
|
|
Known governed boundaries remain explicit: WALOUS covers Wallonia rather than
|
|
all Belgium, no unstable 2018 artifact is accepted without a verifiable source
|
|
checksum, and the current local building model is not represented as nationally
|
|
trained or validated.
|