Add split background promotion workflow
This commit is contained in:
@@ -278,6 +278,24 @@ only strict background gate for default promotion, while
|
||||
`sparse_building_context` remains review-only evidence in the report. This keeps
|
||||
contextual buildings from being treated as empty-background false positives.
|
||||
|
||||
For the Tower/runtime pass, run the split matrix and promotion report together:
|
||||
|
||||
```bash
|
||||
PROMOTION_POSITIVE_PORTFOLIO_PATH=artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-positive/multi_sample_quality_summary.json \
|
||||
OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json \
|
||||
QUALITY_MODEL_ASSET_IDS="geointel-building-yolov8s-aoi1024bg512r3e50-pt" \
|
||||
QUALITY_TILE_SIZES="512" \
|
||||
QUALITY_TILE_OVERLAPS="64" \
|
||||
QUALITY_THRESHOLDS="0.35 0.15" \
|
||||
BACKGROUND_SPLIT_OUTPUT_DIR=artifacts/detection-hard-negatives/aoi1024bg512r3e50-split \
|
||||
PROMOTION_OUTPUT_DIR=artifacts/detection-model-promotion/aoi1024bg512r3e50-split-aware \
|
||||
bash scripts/run_split_background_promotion_workflow.sh http://192.168.10.150:1202
|
||||
```
|
||||
|
||||
The wrapper keeps the same safety boundary: existing dataset upload, configured
|
||||
YOLO detection and report tooling only. It does not change model configuration
|
||||
or bypass the persisted QA/QC evidence requirement.
|
||||
|
||||
The underlying single-category matrix remains available:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -6312,3 +6312,41 @@ Open:
|
||||
- After redeploy, run `scripts/run_background_corpus_split_matrix.sh` on Tower for `geointel-building-yolov8s-aoi1024bg512r3e50-pt`.
|
||||
- Feed the generated `background_corpus_split_summary.json` into `scripts/build_detection_model_promotion_report.py --background-split-summary` together with the seven-AOI positive summary.
|
||||
- If pure-empty false-positive pressure still fails, retrain or recalibrate before any default activation. If pure-empty passes, inspect sparse-context review evidence before deciding whether to keep the model operator-only or prepare a guarded default-candidate decision.
|
||||
|
||||
# Sprint 159 - Split-background promotion workflow wrapper
|
||||
|
||||
## What changed
|
||||
|
||||
- Added `scripts/run_split_background_promotion_workflow.sh`.
|
||||
- The wrapper runs `scripts/run_background_corpus_split_matrix.sh`, verifies the emitted `background_corpus_split_summary.json`, then calls `scripts/build_detection_model_promotion_report.py --background-split-summary`.
|
||||
- The wrapper requires `PROMOTION_POSITIVE_PORTFOLIO_PATH` and exposes promotion gate overrides through environment variables.
|
||||
- Added readiness syntax coverage for the new wrapper.
|
||||
- Updated operator pipeline docs, TODO and changelog.
|
||||
|
||||
## What was tested
|
||||
|
||||
- Red step: `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q` failed because the wrapper script and readiness syntax check did not exist.
|
||||
- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py -q`: 2 passed.
|
||||
- Ran `bash -n scripts/run_split_background_promotion_workflow.sh`.
|
||||
- Ran `python -m pytest tests/test_sprint159_split_promotion_workflow.py tests/test_sprint158_promotion_report_split_background.py tests/test_sprint157_background_split_matrix_runner.py -q`: 7 passed.
|
||||
- Ran `bash -n scripts/run_readiness_check.sh`.
|
||||
- Ran `python -m compileall backend/app`.
|
||||
- Ran `python -m pytest` in `backend`: 446 passed, 17 existing Pydantic namespace warnings.
|
||||
- Ran `cd frontend && npm run typecheck`.
|
||||
- Ran `cd frontend && npm run build`.
|
||||
- Ran `cd backend && python -m alembic heads`: `202606120900 (head)`.
|
||||
- Ran `cd backend && python -m alembic upgrade head --sql`.
|
||||
- Ran `bash -n scripts/live_migration_smoke.sh`.
|
||||
- Ran `bash scripts/run_readiness_check.sh`: passed.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- This pass adds operator orchestration only. It does not run live inference locally, retrain YOLO, activate a model default, change backend API contracts, change migrations, fetch providers, fetch weights or create fake detections.
|
||||
- `http://192.168.10.150:1202` is reachable from this workstation, but `ssh -p 222 root@192.168.10.150` returned `Permission denied (publickey)`. Port `222` appears to be the Gitea SSH endpoint rather than an Unraid shell endpoint, so Docker rebuild/restart still needs the existing Unraid deployment path or a separate shell-access route.
|
||||
|
||||
## Next recommended pass
|
||||
|
||||
- Redeploy/pull commit on Tower through the existing Gitea/Unraid flow.
|
||||
- Run:
|
||||
- `PROMOTION_POSITIVE_PORTFOLIO_PATH=... bash scripts/run_split_background_promotion_workflow.sh http://192.168.10.150:1202`
|
||||
- Review the generated split-aware promotion report before any retraining, recalibration or default-model decision.
|
||||
|
||||
@@ -124,6 +124,7 @@ This file now starts with the current implementation status. Older preparation/b
|
||||
- [x] Add pure-empty versus sparse-building contextual background corpus classification to operator manifests, hard-negative matrix filters and YOLO tile provenance.
|
||||
- [x] Add a split background-corpus matrix runner and report builder that runs pure-empty and sparse-context matrices separately.
|
||||
- [x] Teach the model promotion report to consume split background summaries so only `pure_empty_negative` blocks default promotion and `sparse_building_context` stays review-only.
|
||||
- [x] Add one-command operator workflow to run split background matrices and immediately build the split-aware promotion report.
|
||||
- [ ] Rerun split background matrices on Tower after rebuild, then retrain or recalibrate against the cleaner pure-empty gate plus separate sparse-context inspection matrix.
|
||||
- [ ] Promote a V1 default building detector only after it passes seven positive AOIs, clean hard-negative/background gates and persisted QA/QC evidence without fake detections or model downloads.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user