Add split background promotion workflow
GeoIntel CI / docs-smoke (push) Has been cancelled
GeoIntel CI / contract-smoke (push) Has been cancelled

This commit is contained in:
Codex
2026-07-10 12:12:50 +02:00
parent dea65666ec
commit 947ad6784d
8 changed files with 228 additions and 0 deletions
+19
View File
@@ -699,6 +699,25 @@ summary as the strict default-promotion false-positive gate. The
`sparse_building_context` source remains visible in the JSON/Markdown report as
review evidence only and is not counted as a default-promotion gate.
To run both steps after one redeploy, use the workflow wrapper:
```bash
PROMOTION_POSITIVE_PORTFOLIO_PATH=/mnt/user/appdata/geointel/artifacts/detection-quality-matrix/multi-sample/aoi1024bg512r3e50-positive/multi_sample_quality_summary.json \
OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json \
QUALITY_MODEL_ASSET_IDS="geointel-building-yolov8s-aoi1024bg512r3e50-pt" \
QUALITY_TILE_SIZES="512" \
QUALITY_TILE_OVERLAPS="64" \
QUALITY_THRESHOLDS="0.35 0.15" \
BACKGROUND_SPLIT_OUTPUT_DIR=artifacts/detection-hard-negatives/aoi1024bg512r3e50-split \
PROMOTION_OUTPUT_DIR=artifacts/detection-model-promotion/aoi1024bg512r3e50-split-aware \
bash scripts/run_split_background_promotion_workflow.sh http://192.168.10.150:1202
```
The wrapper first calls `run_background_corpus_split_matrix.sh`, then feeds the
generated `background_corpus_split_summary.json` into the split-aware promotion
report. It still uses only existing upload, detection and report paths; it does
not fetch providers, fetch model weights or activate a default.
If a legacy positive evidence portfolio records `model_asset_id` at portfolio
level but does not include per-run tile size/overlap, pass explicit tile
defaults instead of letting the report guess:
+1
View File
@@ -69,6 +69,7 @@ bash -n scripts/run_detection_quality_matrix.sh
bash -n scripts/run_multi_sample_detection_quality_matrix.sh
bash -n scripts/run_operator_hard_negative_detection_matrix.sh
bash -n scripts/run_background_corpus_split_matrix.sh
bash -n scripts/run_split_background_promotion_workflow.sh
bash -n scripts/train_operator_yolo_detector.sh
bash -n scripts/verify_workbench_default_state.sh
bash -n scripts/verify_workbench_interactions.sh
@@ -0,0 +1,115 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat >&2 <<'EOF'
Usage:
PROMOTION_POSITIVE_PORTFOLIO_PATH=artifacts/detection-quality-matrix/multi-sample/aoi1024-positive/multi_sample_quality_summary.json \
OPERATOR_SAMPLE_MANIFEST_PATH=storage/operator-data/operator_samples_manifest.json \
QUALITY_MODEL_ASSET_IDS="geointel-building-yolov8s-aoi1024bg512r3e50-pt" \
QUALITY_TILE_SIZES="512" \
QUALITY_TILE_OVERLAPS="64" \
QUALITY_THRESHOLDS="0.35 0.15" \
bash scripts/run_split_background_promotion_workflow.sh [base_url]
Optional environment:
BACKGROUND_SPLIT_OUTPUT_DIR Split matrix output directory.
PROMOTION_OUTPUT_DIR Promotion report output directory.
PROMOTION_POSITIVE_PORTFOLIO_PATH Positive AOI portfolio or multi-sample summary.
PROMOTION_MIN_POSITIVE_SAMPLES Default: 7.
PROMOTION_MIN_BACKGROUND_SAMPLES Default: 2.
PROMOTION_MIN_MEAN_F1 Default: 0.25.
PROMOTION_MAX_BACKGROUND_DETECTIONS_PER_SAMPLE Default: 0.
PROMOTION_DEFAULT_POSITIVE_TILE_SIZE Optional fallback for legacy positive portfolios.
PROMOTION_DEFAULT_POSITIVE_TILE_OVERLAP Optional fallback for legacy positive portfolios.
Runs the split background matrix, then builds a split-aware promotion report.
The pure-empty background summary is the strict default gate. Sparse-context
background evidence is included for operator review only.
The script uses existing runtime/API flows and report tooling only. It does not
fetch providers, fetch weights, mutate model settings or activate a default.
EOF
}
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT"
BASE_URL="${1:-${GE_INTEL_BASE_URL:-http://localhost:1202}}"
if [ "${BASE_URL}" = "-h" ] || [ "${BASE_URL}" = "--help" ]; then
usage
exit 0
fi
if [ -z "${PROMOTION_POSITIVE_PORTFOLIO_PATH:-}" ]; then
echo "PROMOTION_POSITIVE_PORTFOLIO_PATH is required" >&2
usage
exit 1
fi
if [ ! -s "${PROMOTION_POSITIVE_PORTFOLIO_PATH}" ]; then
echo "Positive portfolio is not readable: ${PROMOTION_POSITIVE_PORTFOLIO_PATH}" >&2
exit 1
fi
if [ -n "${PYTHON_BIN:-}" ]; then
PYTHON_BIN="${PYTHON_BIN}"
else
PYTHON_BIN=""
for candidate in python3 python.exe python; do
if command -v "${candidate}" >/dev/null 2>&1 && "${candidate}" -c "import json, sys" >/dev/null 2>&1; then
PYTHON_BIN="${candidate}"
break
fi
done
fi
if [ -z "${PYTHON_BIN}" ]; then
echo "A Python interpreter is required for split-background promotion reporting" >&2
exit 1
fi
stamp="$(date -u +%Y%m%dT%H%M%SZ)"
BACKGROUND_SPLIT_OUTPUT_DIR="${BACKGROUND_SPLIT_OUTPUT_DIR:-artifacts/detection-hard-negatives/background-split/${stamp}}"
PROMOTION_OUTPUT_DIR="${PROMOTION_OUTPUT_DIR:-artifacts/detection-model-promotion/split-aware/${stamp}}"
echo "== GeoIntel split-background promotion workflow =="
echo "Base URL: ${BASE_URL}"
echo "Positive portfolio: ${PROMOTION_POSITIVE_PORTFOLIO_PATH}"
echo "Background split output: ${BACKGROUND_SPLIT_OUTPUT_DIR}"
echo "Promotion output: ${PROMOTION_OUTPUT_DIR}"
BACKGROUND_SPLIT_OUTPUT_DIR="${BACKGROUND_SPLIT_OUTPUT_DIR}" \
bash scripts/run_background_corpus_split_matrix.sh "${BASE_URL}"
split_summary="${BACKGROUND_SPLIT_OUTPUT_DIR}/background_corpus_split_summary.json"
if [ ! -s "${split_summary}" ]; then
echo "Split summary was not produced: ${split_summary}" >&2
exit 1
fi
mkdir -p "${PROMOTION_OUTPUT_DIR}"
promotion_args=(
scripts/build_detection_model_promotion_report.py
--positive-portfolio "${PROMOTION_POSITIVE_PORTFOLIO_PATH}"
--background-split-summary "${split_summary}"
--output-dir "${PROMOTION_OUTPUT_DIR}"
--min-positive-samples "${PROMOTION_MIN_POSITIVE_SAMPLES:-7}"
--min-background-samples "${PROMOTION_MIN_BACKGROUND_SAMPLES:-2}"
--min-mean-f1 "${PROMOTION_MIN_MEAN_F1:-0.25}"
--max-background-detections-per-sample "${PROMOTION_MAX_BACKGROUND_DETECTIONS_PER_SAMPLE:-0}"
)
if [ -n "${PROMOTION_DEFAULT_POSITIVE_TILE_SIZE:-}" ]; then
promotion_args+=(--default-positive-tile-size "${PROMOTION_DEFAULT_POSITIVE_TILE_SIZE}")
fi
if [ -n "${PROMOTION_DEFAULT_POSITIVE_TILE_OVERLAP:-}" ]; then
promotion_args+=(--default-positive-tile-overlap "${PROMOTION_DEFAULT_POSITIVE_TILE_OVERLAP}")
fi
"${PYTHON_BIN}" "${promotion_args[@]}"
echo "Split summary: ${split_summary}"
echo "Promotion report: ${PROMOTION_OUTPUT_DIR}/detection_model_promotion_report.json"