Harden operator YOLO training runtime
This commit is contained in:
+2
-1
@@ -283,7 +283,7 @@ candidates are too weak for the target imagery. It is not a browser feature and
|
|||||||
does not change API contracts:
|
does not change API contracts:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker exec -it geointel python /app/scripts/export_operator_yolo_dataset.py \
|
docker exec -it geointel python3 /app/scripts/export_operator_yolo_dataset.py \
|
||||||
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
|
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
|
||||||
--output-dir /app/storage/operator-data/yolo-building-dataset \
|
--output-dir /app/storage/operator-data/yolo-building-dataset \
|
||||||
--val-samples turnhout \
|
--val-samples turnhout \
|
||||||
@@ -302,6 +302,7 @@ docker exec \
|
|||||||
-e TRAIN_BATCH=2 \
|
-e TRAIN_BATCH=2 \
|
||||||
-e TRAIN_WORKERS=0 \
|
-e TRAIN_WORKERS=0 \
|
||||||
-e TRAIN_DEVICE=cpu \
|
-e TRAIN_DEVICE=cpu \
|
||||||
|
-e PYTHON_BIN=python3 \
|
||||||
geointel bash /app/scripts/train_operator_yolo_detector.sh
|
geointel bash /app/scripts/train_operator_yolo_detector.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -233,7 +233,7 @@ When repeated public model benchmarks remain too weak, the operator can convert
|
|||||||
the prepared real-data samples into a local YOLO training dataset:
|
the prepared real-data samples into a local YOLO training dataset:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker exec -it geointel python /app/scripts/export_operator_yolo_dataset.py \
|
docker exec -it geointel python3 /app/scripts/export_operator_yolo_dataset.py \
|
||||||
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
|
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
|
||||||
--output-dir /app/storage/operator-data/yolo-building-dataset \
|
--output-dir /app/storage/operator-data/yolo-building-dataset \
|
||||||
--val-samples turnhout \
|
--val-samples turnhout \
|
||||||
@@ -258,6 +258,7 @@ docker exec \
|
|||||||
-e TRAIN_BATCH=2 \
|
-e TRAIN_BATCH=2 \
|
||||||
-e TRAIN_WORKERS=0 \
|
-e TRAIN_WORKERS=0 \
|
||||||
-e TRAIN_DEVICE=cpu \
|
-e TRAIN_DEVICE=cpu \
|
||||||
|
-e PYTHON_BIN=python3 \
|
||||||
geointel bash /app/scripts/train_operator_yolo_detector.sh
|
geointel bash /app/scripts/train_operator_yolo_detector.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ Changed:
|
|||||||
- requires `OPERATOR_YOLO_DATASET_DIR`
|
- requires `OPERATOR_YOLO_DATASET_DIR`
|
||||||
- requires an existing `YOLO_BASE_MODEL_PATH`
|
- requires an existing `YOLO_BASE_MODEL_PATH`
|
||||||
- writes a local `TRAIN_MODEL_OUTPUT_PATH`
|
- writes a local `TRAIN_MODEL_OUTPUT_PATH`
|
||||||
|
- uses `PYTHON_BIN=python3` by default for the all-in-one container
|
||||||
- writes `training_summary.json`.
|
- writes `training_summary.json`.
|
||||||
- Added readiness coverage for exporter compile and train-wrapper shell syntax.
|
- Added readiness coverage for exporter compile and train-wrapper shell syntax.
|
||||||
- Added regression coverage in `backend/tests/test_sprint129_operator_yolo_training_dataset.py`.
|
- Added regression coverage in `backend/tests/test_sprint129_operator_yolo_training_dataset.py`.
|
||||||
|
|||||||
+2
-1
@@ -265,7 +265,7 @@ Export the same operator samples to a local YOLO detection dataset when the
|
|||||||
public model candidates are not strong enough for the target imagery:
|
public model candidates are not strong enough for the target imagery:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker exec -it geointel python /app/scripts/export_operator_yolo_dataset.py \
|
docker exec -it geointel python3 /app/scripts/export_operator_yolo_dataset.py \
|
||||||
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
|
--manifest-path /app/storage/operator-data/operator_samples_manifest.json \
|
||||||
--output-dir /app/storage/operator-data/yolo-building-dataset \
|
--output-dir /app/storage/operator-data/yolo-building-dataset \
|
||||||
--val-samples turnhout \
|
--val-samples turnhout \
|
||||||
@@ -291,6 +291,7 @@ docker exec \
|
|||||||
-e TRAIN_BATCH=2 \
|
-e TRAIN_BATCH=2 \
|
||||||
-e TRAIN_WORKERS=0 \
|
-e TRAIN_WORKERS=0 \
|
||||||
-e TRAIN_DEVICE=cpu \
|
-e TRAIN_DEVICE=cpu \
|
||||||
|
-e PYTHON_BIN=python3 \
|
||||||
geointel bash /app/scripts/train_operator_yolo_detector.sh
|
geointel bash /app/scripts/train_operator_yolo_detector.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ Environment variables:
|
|||||||
TRAIN_BATCH Batch size. Default: 2
|
TRAIN_BATCH Batch size. Default: 2
|
||||||
TRAIN_WORKERS Data-loader workers. Default: 0
|
TRAIN_WORKERS Data-loader workers. Default: 0
|
||||||
TRAIN_DEVICE Device passed to Ultralytics. Default: cpu
|
TRAIN_DEVICE Device passed to Ultralytics. Default: cpu
|
||||||
|
PYTHON_BIN Python executable. Default: python3
|
||||||
|
|
||||||
This helper is an operator/runtime smoke wrapper. It requires an existing local
|
This helper is an operator/runtime smoke wrapper. It requires an existing local
|
||||||
base model and an existing local dataset.yaml. It does not create app features.
|
base model and an existing local dataset.yaml. It does not create app features.
|
||||||
@@ -42,6 +43,7 @@ TRAIN_IMGSZ="${TRAIN_IMGSZ:-512}"
|
|||||||
TRAIN_BATCH="${TRAIN_BATCH:-2}"
|
TRAIN_BATCH="${TRAIN_BATCH:-2}"
|
||||||
TRAIN_WORKERS="${TRAIN_WORKERS:-0}"
|
TRAIN_WORKERS="${TRAIN_WORKERS:-0}"
|
||||||
TRAIN_DEVICE="${TRAIN_DEVICE:-cpu}"
|
TRAIN_DEVICE="${TRAIN_DEVICE:-cpu}"
|
||||||
|
PYTHON_BIN="${PYTHON_BIN:-python3}"
|
||||||
|
|
||||||
DATASET_YAML="${OPERATOR_YOLO_DATASET_DIR%/}/dataset.yaml"
|
DATASET_YAML="${OPERATOR_YOLO_DATASET_DIR%/}/dataset.yaml"
|
||||||
SUMMARY_PATH="${TRAIN_OUTPUT_DIR%/}/${TRAIN_RUN_NAME}/training_summary.json"
|
SUMMARY_PATH="${TRAIN_OUTPUT_DIR%/}/${TRAIN_RUN_NAME}/training_summary.json"
|
||||||
@@ -55,6 +57,7 @@ export TRAIN_IMGSZ
|
|||||||
export TRAIN_BATCH
|
export TRAIN_BATCH
|
||||||
export TRAIN_WORKERS
|
export TRAIN_WORKERS
|
||||||
export TRAIN_DEVICE
|
export TRAIN_DEVICE
|
||||||
|
export PYTHON_BIN
|
||||||
export SUMMARY_PATH
|
export SUMMARY_PATH
|
||||||
|
|
||||||
if [[ ! -f "${DATASET_YAML}" ]]; then
|
if [[ ! -f "${DATASET_YAML}" ]]; then
|
||||||
@@ -69,7 +72,7 @@ fi
|
|||||||
|
|
||||||
mkdir -p "${TRAIN_OUTPUT_DIR}" "$(dirname "${TRAIN_MODEL_OUTPUT_PATH}")"
|
mkdir -p "${TRAIN_OUTPUT_DIR}" "$(dirname "${TRAIN_MODEL_OUTPUT_PATH}")"
|
||||||
|
|
||||||
python - <<'PY'
|
"${PYTHON_BIN}" - <<'PY'
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
|||||||
Reference in New Issue
Block a user