fix(release): make deployment backup and rollback immutable
This commit is contained in:
@@ -160,8 +160,10 @@ COPY scripts/migrate_runtime_model_provenance.py /app/scripts/migrate_runtime_mo
|
||||
COPY scripts/archive_technical_projects.py /app/scripts/archive_technical_projects.py
|
||||
COPY scripts/runtime_state_report.py /app/scripts/runtime_state_report.py
|
||||
COPY scripts/release_backup_guard.py /app/scripts/release_backup_guard.py
|
||||
COPY scripts/release_backup_snapshot.py /app/scripts/release_backup_snapshot.py
|
||||
COPY scripts/audit_data_operations.py /app/scripts/audit_data_operations.py
|
||||
COPY scripts/cleanup_storage_artifacts.py /app/scripts/cleanup_storage_artifacts.py
|
||||
COPY scripts/restore_storage_quarantine.py /app/scripts/restore_storage_quarantine.py
|
||||
COPY deploy/unraid/nginx-all-in-one.conf /etc/nginx/conf.d/default.conf
|
||||
COPY deploy/unraid/all-in-one-start.sh /usr/local/bin/geointel-all-in-one-start
|
||||
COPY --from=frontend-build /frontend/dist/ /usr/share/nginx/html/
|
||||
|
||||
@@ -11,9 +11,8 @@ met de naam `geointel`, bereikbaar op `http://192.168.10.150:1202`.
|
||||
Kopieer de **volledige** map `C:\Projects\geointel` naar
|
||||
`/mnt/user/appdata/geointel` op de server. Verder niets uitzoeken.
|
||||
|
||||
`.dockerignore` regelt de rest: `node_modules/`, `.git/`, de dubbele
|
||||
`geointel/`-map, `docs/`, `artifacts/` en testoutput gaan niet mee de
|
||||
build-context in, ook al staan ze in de map.
|
||||
`.dockerignore` regelt de rest: `node_modules/`, `.git/`, `docs/`, `artifacts/`
|
||||
en testoutput gaan niet mee de build-context in, ook al staan ze in de map.
|
||||
|
||||
Eén waarschuwing bij het overschrijven: laat `storage/`, `postgres-data/`,
|
||||
`backups/` en `models/` op de server **staan**. Dat is je bestaande data, en
|
||||
@@ -71,7 +70,8 @@ die hostname in `GEOINTEL_CORS_ORIGINS` staan — anders blokkeert de browser de
|
||||
API-calls vanaf het publieke adres. De backend doet geen host-validatie, dus
|
||||
verder is er niets nodig aan applicatiekant.
|
||||
|
||||
Voor GPU-inferentie (optioneel, kan ook later):
|
||||
De productie-image bevat altijd de gepinde AI-runtime. Inferentie zelf kan
|
||||
uitblijven totdat een lokaal, gevalideerd model beschikbaar is:
|
||||
|
||||
```env
|
||||
GEOINTEL_INSTALL_AI=true
|
||||
@@ -81,8 +81,8 @@ YOLO_REQUIRE_CUDA=true
|
||||
YOLO_MODEL_PATH=/app/models/<jouw-model>.pt
|
||||
```
|
||||
|
||||
Laat `GEOINTEL_INSTALL_AI=false` staan als je eerst gewoon wilt dat de app
|
||||
draait — dat scheelt een paar GB aan PyTorch-lagen in de build.
|
||||
Laat `YOLO_ENABLED=false` zolang er geen geschikt modelbestand is. Het
|
||||
release-deployscript weigert bewust een GIS-only productie-image.
|
||||
|
||||
---
|
||||
|
||||
@@ -134,12 +134,21 @@ bash deploy/unraid/deploy-release.sh
|
||||
|
||||
Het script:
|
||||
|
||||
1. ruimt een eventueel achtergebleven Compose-stack op (ook de oude
|
||||
1. bindt het exacte huidige image-ID aan een unieke backup-specifieke
|
||||
`rollback-predeploy-*`-tag;
|
||||
2. bouwt of hergebruikt de AI-candidate terwijl de huidige release beschikbaar
|
||||
blijft, legt het exacte lokale image-ID vast en maakt daarop SBOM- en
|
||||
Trivy-evidence;
|
||||
3. controleert vóór het pauzeren de vrije ruimte en maakt vervolgens een
|
||||
byte-complete, SHA-256-geverifieerde database-, storage- en modelsnapshot in
|
||||
`/mnt/user/appdata/geointel/backups`; ongewijzigde bestanden mogen alleen
|
||||
vanuit een oudere geverifieerde backup worden gehardlinkt;
|
||||
4. ruimt een eventueel achtergebleven Compose-stack op (ook de oude
|
||||
3-container dev-stack die óók poort 1202 pakt);
|
||||
2. bewaart de huidige image als `geointel-all-in-one:previous`;
|
||||
3. bouwt `deploy/unraid/Dockerfile.all-in-one`;
|
||||
4. start één container `geointel` met `-p 1202:80` en `--gpus all`;
|
||||
5. rolt automatisch terug naar `:previous` als de healthcheck of smoke faalt.
|
||||
5. start één container `geointel` met `-p 1202:80` en `--gpus all`;
|
||||
6. bewijst een rollbackdump eerst in een geïsoleerde tijdelijke database,
|
||||
bewaart de oude productiedatabase als herstelpad en start pas daarna
|
||||
automatisch het image-ID dat cryptografisch in die pre-deploybackup staat.
|
||||
|
||||
De eerste build duurt lang (PostGIS + GDAL + npm build). Volgende deploys
|
||||
hergebruiken de Docker-layercache.
|
||||
@@ -165,24 +174,13 @@ docker logs --tail 200 geointel
|
||||
|
||||
---
|
||||
|
||||
## 7. Belangrijk: dubbele projectmap lokaal
|
||||
## 7. Geretireerde dubbele projectmap
|
||||
|
||||
In `C:\Projects\geointel` staat een tweede, volledige kopie van het project
|
||||
onder `C:\Projects\geointel\geointel\`. Die bevat dezelfde bestanden en
|
||||
dezelfde datum, maar staat buiten git. Zolang die er staat:
|
||||
|
||||
- wordt de Docker build-context onnodig verdubbeld;
|
||||
- weet je bij het bewerken van bijvoorbeeld `docker-compose.unraid.yml` niet
|
||||
welke versie je te pakken hebt.
|
||||
|
||||
`.dockerignore` sluit hem nu uit, maar ruim hem op zodra je zeker weet dat er
|
||||
niets unieks in staat. Vergelijk eerst:
|
||||
|
||||
```powershell
|
||||
robocopy C:\Projects\geointel\geointel C:\Projects\geointel /L /E /NJH /NJS /NDL /XF *.pyc
|
||||
```
|
||||
|
||||
Regels die als `New File` verschijnen bestaan alleen in de kopie.
|
||||
De vroegere geneste mirror `C:\Projects\geointel\geointel` is geretireerd. De
|
||||
immutable Git-herkomst, niet-getrackte recoverybestanden en verificatiegrens
|
||||
staan in `docs/accuracy-program/13-nested-mirror-retirement.md`. Behandel die
|
||||
evidence als herstelreferentie; deze handleiding vraagt geen extra kopieer- of
|
||||
opruimactie.
|
||||
|
||||
---
|
||||
|
||||
@@ -190,11 +188,15 @@ Regels die als `New File` verschijnen bestaan alleen in de kopie.
|
||||
|
||||
```bash
|
||||
cd /mnt/user/appdata/geointel
|
||||
bash deploy/unraid/rollback-dockerman-container.sh
|
||||
bash deploy/unraid/rollback-dockerman-container.sh \
|
||||
--backup-dir /mnt/user/appdata/geointel/backups/<predeploy-release-id> \
|
||||
--confirm-production-database-restore
|
||||
```
|
||||
|
||||
Rollback hergebruikt dezelfde PostGIS- en storage-paden en draait nooit een
|
||||
Alembic-downgrade.
|
||||
Rollback hergebruikt dezelfde storage-paden, bewijst de geverifieerde dump
|
||||
eerst geïsoleerd, wisselt daarna databases via no-clobber namen en draait nooit
|
||||
een Alembic-downgrade. De oude productiedatabase blijft staan totdat een
|
||||
operator haar na controle expliciet opruimt.
|
||||
|
||||
---
|
||||
|
||||
@@ -202,7 +204,7 @@ Alembic-downgrade.
|
||||
|
||||
| Bestand | Aanpassing |
|
||||
|---|---|
|
||||
| `.dockerignore` | Sluit root-`node_modules`, de dubbele `geointel/`-map, `.git`, `docs/`, `artifacts/` en testoutput uit de build-context |
|
||||
| `.dockerignore` | Sluit root-`node_modules`, `.git`, `docs/`, `artifacts/` en testoutput uit de build-context |
|
||||
| `deploy/unraid/deploy-release.sh` | `git rev-parse HEAD` crashte op een kopie zonder `.git`. Valt nu terug op `GEOINTEL_BUILD_SHA`, een `RELEASE_SHA`-bestand of een content-hash van de broncode |
|
||||
| `deploy/unraid/deploy-release.sh` | Smoke-scripts worden op bestaan getest in plaats van op de execute-bit, die bij een Windows-kopie verloren gaat |
|
||||
| `deploy/unraid/run-dockerman-container.sh` | Ruimt expliciet zowel `docker-compose.yml` (3 containers) als `docker-compose.unraid.yml` op, zodat poort 1202 gegarandeerd vrij is |
|
||||
|
||||
+30
-22
@@ -106,27 +106,25 @@ customer or operational data. Deploy a separate demo container and storage
|
||||
root for public or recruiter-facing access.
|
||||
|
||||
The repository deploy scripts run the same flow automatically. They validate
|
||||
the Compose reference, preserve the current image as
|
||||
`geointel-all-in-one:previous`, build an immutable `<commit-sha>-ai` or
|
||||
`<commit-sha>-gis` tag plus `latest`, install the DockerMan metadata and start
|
||||
the immutable image. An existing matching tag is reused, never rebuilt. A
|
||||
failed start, live migration smoke or browser/API smoke automatically attempts
|
||||
the previous image without changing the configured PostGIS or storage paths.
|
||||
the Compose reference, preserve the current image under a unique
|
||||
backup-specific `rollback-predeploy-*` tag, build the immutable production
|
||||
`<commit-sha>-ai` tag plus `latest`, attest its exact local image ID, generate
|
||||
an SBOM and enforce the Trivy policy before starting that same ID. An existing
|
||||
matching tag is reused, never rebuilt. A failed start, live migration smoke or
|
||||
browser/API smoke automatically attempts the previous image without changing
|
||||
the configured PostGIS or storage paths.
|
||||
|
||||
`scripts/deploy_tower.sh` and `scripts/deploy_tower.ps1` source the remote
|
||||
`.env` before building the image. That means `GEOINTEL_INSTALL_AI=true` in
|
||||
`/mnt/user/appdata/geointel/.env` is enough for the automatic deploy to build
|
||||
the AI-enabled image. Set `GEOINTEL_INSTALL_AI` in the local shell or pass
|
||||
`-InstallAi true/false` to the PowerShell wrapper only when you intentionally
|
||||
want to override the remote `.env` for that deploy.
|
||||
`.env` before building the image. Production deployment requires
|
||||
`GEOINTEL_INSTALL_AI=true`; an explicit false value fails closed before the
|
||||
image or running container is replaced.
|
||||
|
||||
Database credentials are runtime configuration, not image metadata. The
|
||||
all-in-one image does not bake `GEOINTEL_POSTGRES_PASSWORD` into the Dockerfile;
|
||||
set it through `.env`, the Unraid template or `docker run -e`.
|
||||
|
||||
AI dependencies are opt-in. Leave `GEOINTEL_INSTALL_AI=false` for the default
|
||||
GIS-only image. Set `GEOINTEL_INSTALL_AI=true`, mount models through
|
||||
`GEOINTEL_MODELS_PATH` and configure `YOLO_ENABLED=true` plus
|
||||
Production images always include the pinned AI dependencies. Mount models
|
||||
through `GEOINTEL_MODELS_PATH` and configure `YOLO_ENABLED=true` plus
|
||||
`YOLO_MODELS_DIR=/app/models` and `YOLO_MODEL_PATH=/app/models/<model>.pt` only
|
||||
when you have a local model file.
|
||||
The AI-enabled image installs PyTorch/Ultralytics plus the native OpenCV runtime
|
||||
@@ -264,7 +262,8 @@ git reset --hard origin/main
|
||||
bash deploy/unraid/deploy-release.sh
|
||||
```
|
||||
|
||||
The equivalent low-level build remains available for debugging:
|
||||
The equivalent low-level GIS-only build remains available only for local
|
||||
debugging; it is not a production deployment path:
|
||||
|
||||
```bash
|
||||
docker build --build-arg GEOINTEL_INSTALL_AI=${GEOINTEL_INSTALL_AI:-false} -f deploy/unraid/Dockerfile.all-in-one -t geointel-all-in-one:latest .
|
||||
@@ -305,20 +304,27 @@ bash scripts/verify_release_upgrade_smoke.sh \
|
||||
Return to the image that was active immediately before the latest deployment:
|
||||
|
||||
```bash
|
||||
bash deploy/unraid/rollback-dockerman-container.sh
|
||||
bash deploy/unraid/rollback-dockerman-container.sh \
|
||||
--backup-dir /mnt/user/appdata/geointel/backups/<predeploy-release-id> \
|
||||
--confirm-production-database-restore
|
||||
```
|
||||
|
||||
For an older retained commit, select its immutable tag explicitly:
|
||||
|
||||
```bash
|
||||
GEOINTEL_ROLLBACK_IMAGE=geointel-all-in-one:<commit-sha>-ai \
|
||||
bash deploy/unraid/rollback-dockerman-container.sh
|
||||
bash deploy/unraid/rollback-dockerman-container.sh \
|
||||
--backup-dir /mnt/user/appdata/geointel/backups/<matching-predeploy-release-id> \
|
||||
--confirm-production-database-restore
|
||||
```
|
||||
|
||||
Rollback reuses the configured PostGIS and storage mounts and never runs an
|
||||
Alembic downgrade. If a future release has a backward-incompatible migration,
|
||||
restore its verified pre-release backup instead of forcing an older app
|
||||
against a newer schema.
|
||||
Rollback restores and verifies the selected dump in an isolated proof database
|
||||
before any production replacement. It then swaps database names, retains the
|
||||
pre-restore production database for operator recovery, reuses the configured
|
||||
storage mount and never runs an Alembic downgrade or an older app against an
|
||||
unknown newer schema. Remove the retained recovery database and old backup
|
||||
directories only in a separately reviewed operator retention step; deployment
|
||||
never deletes them automatically.
|
||||
|
||||
The configured upload limit is shared by FastAPI and the generated nginx
|
||||
runtime configuration. Values outside `1..2048` MiB are rejected before the
|
||||
@@ -339,7 +345,9 @@ docker exec geointel python /app/scripts/cleanup_storage_artifacts.py
|
||||
```
|
||||
|
||||
The full backup, confirmation, candidate-limit and apply sequence is in
|
||||
`docs/DATA_OPERATIONS_RUNBOOK.md`. GeoIntel installs no automatic cleanup
|
||||
`docs/DATA_OPERATIONS_RUNBOOK.md`. Apply moves bytes to protected,
|
||||
checksum-bound quarantine rather than deleting them; a separate confirmed
|
||||
restore command reverses the move. GeoIntel installs no automatic cleanup
|
||||
schedule.
|
||||
|
||||
## Safe cleanup
|
||||
|
||||
+372
-22
@@ -26,7 +26,11 @@ if [ -n "${DEPLOY_GEOINTEL_INSTALL_AI:-}" ]; then
|
||||
GEOINTEL_INSTALL_AI="$DEPLOY_GEOINTEL_INSTALL_AI"
|
||||
fi
|
||||
|
||||
GEOINTEL_INSTALL_AI="${GEOINTEL_INSTALL_AI:-false}"
|
||||
GEOINTEL_INSTALL_AI="${GEOINTEL_INSTALL_AI:-true}"
|
||||
if [ "$GEOINTEL_INSTALL_AI" != "true" ]; then
|
||||
echo "Production release deployment requires the gated AI image (GEOINTEL_INSTALL_AI=true)." >&2
|
||||
exit 2
|
||||
fi
|
||||
GEOINTEL_APP_VERSION="$(tr -d '[:space:]' < VERSION)"
|
||||
if ! [[ "$GEOINTEL_APP_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]]; then
|
||||
echo "Invalid semantic version in VERSION: ${GEOINTEL_APP_VERSION}" >&2
|
||||
@@ -57,7 +61,52 @@ source_tree_hash() {
|
||||
}
|
||||
|
||||
resolve_build_sha() {
|
||||
local head="" content=""
|
||||
local head="" content="" controller_sha="" controller_source=""
|
||||
|
||||
if [ -n "${GITEA_COMMIT_SHA:-}" ]; then
|
||||
controller_sha="$GITEA_COMMIT_SHA"
|
||||
controller_source="GITEA_COMMIT_SHA"
|
||||
fi
|
||||
if [ -n "${GITHUB_SHA:-}" ]; then
|
||||
if ! [[ "$GITHUB_SHA" =~ ^[0-9A-Fa-f]{40}$ ]]; then
|
||||
echo "GITHUB_SHA must contain one full 40-character Git commit SHA." >&2
|
||||
return 2
|
||||
fi
|
||||
if [ -n "$controller_sha" ] && [ "${controller_sha,,}" != "${GITHUB_SHA,,}" ]; then
|
||||
echo "Controller commit variables disagree." >&2
|
||||
return 2
|
||||
fi
|
||||
controller_sha="$GITHUB_SHA"
|
||||
controller_source="${controller_source:-GITHUB_SHA}"
|
||||
fi
|
||||
if [ -n "$controller_sha" ]; then
|
||||
if ! [[ "$controller_sha" =~ ^[0-9A-Fa-f]{40}$ ]]; then
|
||||
echo "${controller_source} must contain one full 40-character Git commit SHA." >&2
|
||||
return 2
|
||||
fi
|
||||
controller_sha="${controller_sha,,}"
|
||||
if [ -n "${GEOINTEL_BUILD_SHA:-}" ] && [ "${GEOINTEL_BUILD_SHA,,}" != "$controller_sha" ]; then
|
||||
echo "Explicit build revision differs from the controller revision." >&2
|
||||
return 2
|
||||
fi
|
||||
if command -v git >/dev/null 2>&1 && git rev-parse --git-dir >/dev/null 2>&1; then
|
||||
head="$(git rev-parse HEAD 2>/dev/null || true)"
|
||||
if [ "${head,,}" != "$controller_sha" ]; then
|
||||
echo "Prepared Git checkout does not match the controller revision." >&2
|
||||
return 2
|
||||
fi
|
||||
if [ -n "$(git status --porcelain 2>/dev/null)" ]; then
|
||||
echo "Prepared Git checkout contains changes outside the controller revision." >&2
|
||||
return 2
|
||||
fi
|
||||
fi
|
||||
printf '%s' "$controller_sha"
|
||||
return 0
|
||||
fi
|
||||
if [ -n "${GITEA_REPOSITORY:-}" ] || [ -n "${GITHUB_REPOSITORY:-}" ]; then
|
||||
echo "Automated deployment context is missing GITEA_COMMIT_SHA/GITHUB_SHA." >&2
|
||||
return 2
|
||||
fi
|
||||
|
||||
# 1. Explicit override wins.
|
||||
if [ -n "${GEOINTEL_BUILD_SHA:-}" ]; then
|
||||
@@ -107,7 +156,13 @@ if [ -z "$GEOINTEL_BUILD_SHA" ]; then
|
||||
echo "Could not determine a build revision for this deployment." >&2
|
||||
exit 2
|
||||
fi
|
||||
export GEOINTEL_BUILD_SHA
|
||||
echo "Build revision: ${GEOINTEL_BUILD_SHA}"
|
||||
GEOINTEL_RELEASE_TOKEN="$(printf '%s' "$GEOINTEL_BUILD_SHA" | tr -c 'A-Za-z0-9._-' '_' | cut -c1-48)"
|
||||
if [ -z "$GEOINTEL_RELEASE_TOKEN" ]; then
|
||||
echo "Could not derive a safe release evidence identifier." >&2
|
||||
exit 2
|
||||
fi
|
||||
GEOINTEL_BUILD_TIME="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
GEOINTEL_IMAGE_REPOSITORY="${GEOINTEL_IMAGE_REPOSITORY:-geointel-all-in-one}"
|
||||
if [ "$GEOINTEL_INSTALL_AI" = "true" ]; then
|
||||
@@ -116,8 +171,23 @@ else
|
||||
GEOINTEL_RELEASE_VARIANT="gis"
|
||||
fi
|
||||
GEOINTEL_RELEASE_IMAGE="${GEOINTEL_IMAGE_REPOSITORY}:${GEOINTEL_BUILD_SHA}-${GEOINTEL_RELEASE_VARIANT}"
|
||||
GEOINTEL_PREVIOUS_IMAGE="${GEOINTEL_IMAGE_REPOSITORY}:previous"
|
||||
FRONTEND_URL="${FRONTEND_URL:-http://127.0.0.1:${GEOINTEL_FRONTEND_PORT:-1202}}"
|
||||
GEOINTEL_BACKUPS_PATH="${GEOINTEL_BACKUPS_PATH:-/mnt/user/appdata/geointel/backups}"
|
||||
GEOINTEL_STORAGE_PATH="${GEOINTEL_STORAGE_PATH:-/mnt/user/appdata/geointel/storage}"
|
||||
GEOINTEL_MODELS_PATH="${GEOINTEL_MODELS_PATH:-/mnt/user/appdata/geointel/models}"
|
||||
GEOINTEL_POSTGIS_DATA_PATH="${GEOINTEL_POSTGIS_DATA_PATH:-/mnt/user/appdata/geointel/postgres-data}"
|
||||
GEOINTEL_DEPLOY_EVIDENCE_DIR="${GEOINTEL_DEPLOY_EVIDENCE_DIR:-artifacts/release-evidence/deploy/${GEOINTEL_RELEASE_TOKEN}-ai}"
|
||||
GEOINTEL_PREDEPLOY_BACKUP_DIR=""
|
||||
GEOINTEL_RELEASE_IMAGE_ID=""
|
||||
GEOINTEL_BACKUP_LINK_DEST=""
|
||||
GEOINTEL_PREDEPLOY_ROLLBACK_TAG=""
|
||||
|
||||
case "$GEOINTEL_DEPLOY_EVIDENCE_DIR" in
|
||||
/*|*..*)
|
||||
echo "Deployment evidence directory must be repository-relative and must not contain '..'." >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
|
||||
wait_for_geointel_health() {
|
||||
local status=""
|
||||
@@ -141,35 +211,312 @@ wait_for_geointel_health() {
|
||||
|
||||
start_image() {
|
||||
local image="$1"
|
||||
local running_image_id=""
|
||||
local running_revision=""
|
||||
local running_ai=""
|
||||
GEOINTEL_IMAGE="$image" bash deploy/unraid/run-dockerman-container.sh
|
||||
wait_for_geointel_health
|
||||
running_image_id="$(docker inspect --format '{{.Image}}' geointel)"
|
||||
if [ "$running_image_id" != "$image" ]; then
|
||||
echo "Running container image ${running_image_id} differs from attested image ${image}." >&2
|
||||
return 1
|
||||
fi
|
||||
running_revision="$(docker inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' geointel)"
|
||||
running_ai="$(docker inspect --format '{{index .Config.Labels "io.geointel.ai.enabled"}}' geointel)"
|
||||
if [ "$running_revision" != "$GEOINTEL_BUILD_SHA" ] || [ "$running_ai" != "true" ]; then
|
||||
echo "Running container labels do not match the attested AI revision." >&2
|
||||
return 1
|
||||
fi
|
||||
echo "Running container matches attested image: ${running_image_id}"
|
||||
}
|
||||
|
||||
scan_release_image() {
|
||||
local scanned_image_id=""
|
||||
local current_image_id=""
|
||||
local inspect_output="${GEOINTEL_DEPLOY_EVIDENCE_DIR}/image-inspect.json"
|
||||
local sbom_output="${GEOINTEL_DEPLOY_EVIDENCE_DIR}/geointel-sbom.spdx.json"
|
||||
local vulnerability_output="${GEOINTEL_DEPLOY_EVIDENCE_DIR}/geointel-container-vulnerabilities.json"
|
||||
local attestation_output="${GEOINTEL_DEPLOY_EVIDENCE_DIR}/deployment-attestation.json"
|
||||
|
||||
scanned_image_id="$(docker image inspect --format '{{.Id}}' "$GEOINTEL_RELEASE_IMAGE")"
|
||||
test -n "$scanned_image_id"
|
||||
mkdir -p "$ROOT/$GEOINTEL_DEPLOY_EVIDENCE_DIR"
|
||||
docker image inspect "$GEOINTEL_RELEASE_IMAGE" > "$ROOT/$inspect_output"
|
||||
bash scripts/generate_container_sbom.sh "$GEOINTEL_RELEASE_IMAGE" "$sbom_output"
|
||||
bash scripts/scan_container_image.sh "$GEOINTEL_RELEASE_IMAGE" "$vulnerability_output"
|
||||
current_image_id="$(docker image inspect --format '{{.Id}}' "$GEOINTEL_RELEASE_IMAGE")"
|
||||
if [ "$current_image_id" != "$scanned_image_id" ]; then
|
||||
echo "Release image tag changed while SBOM/scan evidence was being generated." >&2
|
||||
return 1
|
||||
fi
|
||||
test -s "$ROOT/$inspect_output"
|
||||
test -s "$ROOT/$sbom_output"
|
||||
test -s "$ROOT/$vulnerability_output"
|
||||
GEOINTEL_RELEASE_IMAGE_ID="$scanned_image_id"
|
||||
python3 - \
|
||||
"$ROOT/$attestation_output" \
|
||||
"$GEOINTEL_RELEASE_IMAGE" \
|
||||
"$GEOINTEL_RELEASE_IMAGE_ID" \
|
||||
"$GEOINTEL_BUILD_SHA" \
|
||||
"$inspect_output" \
|
||||
"$sbom_output" \
|
||||
"$vulnerability_output" <<'PY'
|
||||
import datetime
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
output, image_tag, image_id, revision, inspect_path, sbom_path, vulnerability_path = sys.argv[1:]
|
||||
payload = {
|
||||
"schema_version": 1,
|
||||
"attested_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
"image_tag": image_tag,
|
||||
"image_id": image_id,
|
||||
"image_config_digest": image_id,
|
||||
"revision": revision,
|
||||
"variant": "ai",
|
||||
"evidence": {
|
||||
"image_inspect": inspect_path,
|
||||
"sbom": sbom_path,
|
||||
"vulnerabilities": vulnerability_path,
|
||||
},
|
||||
}
|
||||
path = pathlib.Path(output)
|
||||
temporary = path.with_suffix(".json.partial")
|
||||
temporary.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
temporary.replace(path)
|
||||
PY
|
||||
test -s "$ROOT/$attestation_output"
|
||||
echo "Exact deployment image scanned: ${GEOINTEL_RELEASE_IMAGE_ID}"
|
||||
}
|
||||
|
||||
preflight_backup_capacity() {
|
||||
local database_name=""
|
||||
local database_user=""
|
||||
local database_size_bytes=""
|
||||
|
||||
database_name="$(docker exec geointel sh -c 'printf %s "${POSTGRES_DB:-${GEOINTEL_POSTGRES_DB:-geointel}}"')"
|
||||
database_user="$(docker exec geointel sh -c 'printf %s "${POSTGRES_USER:-${GEOINTEL_POSTGRES_USER:-geointel}}"')"
|
||||
database_size_bytes="$(docker exec geointel psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$database_user" -d "$database_name" -Atqc \
|
||||
'SELECT pg_database_size(current_database());')"
|
||||
mkdir -p "$GEOINTEL_BACKUPS_PATH"
|
||||
python3 - \
|
||||
"$GEOINTEL_BACKUPS_PATH" \
|
||||
"$GEOINTEL_STORAGE_PATH" \
|
||||
"$GEOINTEL_MODELS_PATH" \
|
||||
"$database_size_bytes" <<'PY'
|
||||
import os
|
||||
import pathlib
|
||||
import shutil
|
||||
import stat
|
||||
import sys
|
||||
|
||||
backup_root = pathlib.Path(sys.argv[1]).expanduser().resolve()
|
||||
sources = [pathlib.Path(value).expanduser().resolve() for value in sys.argv[2:4]]
|
||||
database_bytes = int(sys.argv[4])
|
||||
|
||||
def retained_bytes(root: pathlib.Path) -> int:
|
||||
if not root.is_dir():
|
||||
raise SystemExit(f"Mandatory snapshot source is not a directory: {root}")
|
||||
total = 0
|
||||
for current, directories, files in os.walk(root, topdown=True, followlinks=False):
|
||||
current_path = pathlib.Path(current)
|
||||
for name in [*directories, *files]:
|
||||
path = current_path / name
|
||||
details = path.lstat()
|
||||
if stat.S_ISLNK(details.st_mode):
|
||||
raise SystemExit(f"Mandatory snapshot refuses symlinked content: {path}")
|
||||
if name in directories and not stat.S_ISDIR(details.st_mode):
|
||||
raise SystemExit(f"Snapshot directory changed during capacity preflight: {path}")
|
||||
if name in files:
|
||||
if not stat.S_ISREG(details.st_mode):
|
||||
raise SystemExit(f"Mandatory snapshot refuses non-regular content: {path}")
|
||||
total += details.st_size
|
||||
return total
|
||||
|
||||
source_bytes = sum(retained_bytes(source) for source in sources)
|
||||
# Reflink clones are used when the backing filesystem supports them. Budget for
|
||||
# a complete copy plus two uncompressed database sizes (dump and isolated
|
||||
# restore/cutover recovery) so fallback still fails before the live backend is
|
||||
# quiesced rather than midway through the snapshot.
|
||||
required = source_bytes + (2 * database_bytes)
|
||||
headroom = max(5 * 1024**3, required // 10)
|
||||
free = shutil.disk_usage(backup_root).free
|
||||
if free < required + headroom:
|
||||
raise SystemExit(
|
||||
"Insufficient free space for a fail-safe predeploy snapshot: "
|
||||
f"required={required + headroom} free={free} source={source_bytes} database={database_bytes}"
|
||||
)
|
||||
print(
|
||||
"Predeploy snapshot capacity: "
|
||||
f"source_bytes={source_bytes} database_bytes={database_bytes} free_bytes={free}"
|
||||
)
|
||||
PY
|
||||
}
|
||||
|
||||
select_verified_link_dest() {
|
||||
local candidate=""
|
||||
GEOINTEL_BACKUP_LINK_DEST=""
|
||||
while IFS= read -r candidate; do
|
||||
if (
|
||||
cd "$candidate" \
|
||||
&& sha256sum -c CHECKSUMS.sha256 >/dev/null \
|
||||
&& python3 "$ROOT/scripts/release_backup_snapshot.py" verify-backup --backup-dir "$candidate"
|
||||
); then
|
||||
GEOINTEL_BACKUP_LINK_DEST="$candidate"
|
||||
echo "Using verified prior byte snapshot as link-dest: ${candidate}"
|
||||
return 0
|
||||
fi
|
||||
echo "Skipping unusable prior backup link-dest: ${candidate}" >&2
|
||||
done < <(
|
||||
python3 - "$GEOINTEL_BACKUPS_PATH" <<'PY'
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
root = pathlib.Path(sys.argv[1]).expanduser().resolve()
|
||||
candidates = sorted(
|
||||
(
|
||||
path
|
||||
for path in root.iterdir()
|
||||
if path.is_dir() and not path.name.startswith(".") and (path / "manifest.json").is_file()
|
||||
),
|
||||
key=lambda path: path.stat().st_mtime_ns,
|
||||
reverse=True,
|
||||
)
|
||||
for candidate in candidates:
|
||||
print(candidate)
|
||||
PY
|
||||
)
|
||||
echo "No verified prior byte snapshot found; this deployment will create a full first snapshot."
|
||||
}
|
||||
|
||||
create_predeploy_backup() {
|
||||
local container_exists="false"
|
||||
local container_running="false"
|
||||
local release_id=""
|
||||
local backup_link_args=()
|
||||
local current_image_id=""
|
||||
local existing_rollback_id=""
|
||||
|
||||
if docker ps -a --format '{{.Names}}' | grep -Fxq geointel; then
|
||||
container_exists="true"
|
||||
fi
|
||||
if [ "$(docker inspect -f '{{.State.Running}}' geointel 2>/dev/null || true)" = "true" ]; then
|
||||
container_running="true"
|
||||
fi
|
||||
|
||||
if [ "$container_exists" = "false" ]; then
|
||||
if [ -f "$GEOINTEL_POSTGIS_DATA_PATH/PG_VERSION" ]; then
|
||||
echo "PostGIS data exists without a running GeoIntel container; refusing an unbacked migration." >&2
|
||||
return 1
|
||||
fi
|
||||
echo "No existing GeoIntel state found; pre-deploy backup is not required for this fresh install."
|
||||
return 0
|
||||
fi
|
||||
if [ "$container_running" != "true" ]; then
|
||||
echo "Existing GeoIntel container is not running; refusing deployment because a consistent backup cannot be created." >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
current_image_id="$(docker inspect --format '{{.Image}}' geointel)"
|
||||
if ! [[ "$current_image_id" =~ ^sha256:[0-9a-f]{64}$ ]]; then
|
||||
echo "Running release does not expose one immutable Docker image ID." >&2
|
||||
return 1
|
||||
fi
|
||||
release_id="predeploy-${GEOINTEL_RELEASE_TOKEN:0:24}-$(date -u +%Y%m%dT%H%M%SZ)-$$"
|
||||
GEOINTEL_PREDEPLOY_ROLLBACK_TAG="${GEOINTEL_IMAGE_REPOSITORY}:rollback-${release_id}"
|
||||
existing_rollback_id="$(docker image inspect --format '{{.Id}}' "$GEOINTEL_PREDEPLOY_ROLLBACK_TAG" 2>/dev/null || true)"
|
||||
if [ -n "$existing_rollback_id" ] && [ "$existing_rollback_id" != "$current_image_id" ]; then
|
||||
echo "Backup-specific rollback tag already identifies different image bytes." >&2
|
||||
return 1
|
||||
fi
|
||||
docker tag "$current_image_id" "$GEOINTEL_PREDEPLOY_ROLLBACK_TAG"
|
||||
|
||||
# This conservative full-copy fallback estimate runs while the existing
|
||||
# release is still healthy. Verified backup-to-backup hardlinks normally
|
||||
# avoid recopying unchanged bytes, but are never assumed for this fail-closed
|
||||
# capacity decision.
|
||||
preflight_backup_capacity
|
||||
select_verified_link_dest
|
||||
if [ -n "$GEOINTEL_BACKUP_LINK_DEST" ]; then
|
||||
backup_link_args=(--link-dest-backup "$GEOINTEL_BACKUP_LINK_DEST")
|
||||
fi
|
||||
|
||||
echo "Quiescing the current backend so the rollback point cannot miss concurrent writes..."
|
||||
if ! docker exec -i geointel python - <<'PY'
|
||||
import os
|
||||
import pathlib
|
||||
import signal
|
||||
import time
|
||||
|
||||
matches = []
|
||||
for item in pathlib.Path("/proc").iterdir():
|
||||
if not item.name.isdigit() or int(item.name) in {os.getpid(), os.getppid()}:
|
||||
continue
|
||||
try:
|
||||
command = (item / "cmdline").read_bytes().replace(b"\0", b" ")
|
||||
except (OSError, PermissionError):
|
||||
continue
|
||||
if b"uvicorn" in command and b"app.main:app" in command:
|
||||
matches.append(int(item.name))
|
||||
if not matches:
|
||||
raise SystemExit("Could not identify the running GeoIntel backend")
|
||||
for process_id in matches:
|
||||
os.kill(process_id, signal.SIGTERM)
|
||||
deadline = time.monotonic() + 60
|
||||
remaining = matches
|
||||
while remaining and time.monotonic() < deadline:
|
||||
time.sleep(0.25)
|
||||
remaining = [process_id for process_id in remaining if pathlib.Path(f"/proc/{process_id}").exists()]
|
||||
if remaining:
|
||||
raise SystemExit(f"Backend did not stop cleanly: {remaining}")
|
||||
print(f"Stopped {len(matches)} backend process(es)")
|
||||
PY
|
||||
then
|
||||
echo "Could not quiesce the current backend; refusing a potentially inconsistent backup." >&2
|
||||
docker restart geointel >/dev/null || true
|
||||
wait_for_geointel_health || true
|
||||
return 1
|
||||
fi
|
||||
|
||||
GEOINTEL_PREDEPLOY_BACKUP_DIR="${GEOINTEL_BACKUPS_PATH%/}/${release_id}"
|
||||
echo "Creating mandatory pre-deploy backup ${release_id}..."
|
||||
if ! bash scripts/backup_release_state.sh \
|
||||
--container geointel \
|
||||
--output-root "$GEOINTEL_BACKUPS_PATH" \
|
||||
--release-id "$release_id" \
|
||||
--storage-path "$GEOINTEL_STORAGE_PATH" \
|
||||
--models-path "$GEOINTEL_MODELS_PATH" \
|
||||
--inventory-mode sha256 \
|
||||
--rollback-image-tag "$GEOINTEL_PREDEPLOY_ROLLBACK_TAG" \
|
||||
"${backup_link_args[@]}" \
|
||||
|| ! bash scripts/verify_release_backup.sh \
|
||||
--container geointel \
|
||||
--backup-dir "$GEOINTEL_PREDEPLOY_BACKUP_DIR"; then
|
||||
echo "Pre-deploy backup failed; restarting the unchanged current release." >&2
|
||||
docker restart geointel >/dev/null || true
|
||||
wait_for_geointel_health || true
|
||||
GEOINTEL_PREDEPLOY_BACKUP_DIR=""
|
||||
return 1
|
||||
fi
|
||||
echo "Pre-deploy backup verified: ${GEOINTEL_PREDEPLOY_BACKUP_DIR}"
|
||||
}
|
||||
|
||||
rollback_previous() {
|
||||
if ! docker image inspect "$GEOINTEL_PREVIOUS_IMAGE" >/dev/null 2>&1; then
|
||||
echo "Automatic rollback unavailable: ${GEOINTEL_PREVIOUS_IMAGE} does not exist." >&2
|
||||
if [ -z "$GEOINTEL_PREDEPLOY_BACKUP_DIR" ]; then
|
||||
echo "Automatic rollback unavailable: no verified pre-deploy database backup was created." >&2
|
||||
return 1
|
||||
fi
|
||||
echo "Rolling back to ${GEOINTEL_PREVIOUS_IMAGE}..."
|
||||
start_image "$GEOINTEL_PREVIOUS_IMAGE"
|
||||
echo "Rolling back database and image to the verified pre-deploy state..."
|
||||
GEOINTEL_DEPLOY_LOCK_HELD=true \
|
||||
bash deploy/unraid/rollback-dockerman-container.sh \
|
||||
--backup-dir "$GEOINTEL_PREDEPLOY_BACKUP_DIR" \
|
||||
--confirm-production-database-restore
|
||||
}
|
||||
|
||||
docker compose -f docker-compose.unraid.yml config >/dev/null
|
||||
|
||||
current_image_id="$(docker inspect --format '{{.Image}}' geointel 2>/dev/null || true)"
|
||||
release_image_id="$(
|
||||
docker image inspect --format '{{.Id}}' "$GEOINTEL_RELEASE_IMAGE" 2>/dev/null || true
|
||||
)"
|
||||
if (
|
||||
[ -n "$current_image_id" ] &&
|
||||
[ "$current_image_id" != "$release_image_id" ] &&
|
||||
docker image inspect "$current_image_id" >/dev/null 2>&1
|
||||
); then
|
||||
docker tag "$current_image_id" "$GEOINTEL_PREVIOUS_IMAGE"
|
||||
elif [ -n "$current_image_id" ] && [ "$current_image_id" = "$release_image_id" ]; then
|
||||
echo "Current container already uses ${GEOINTEL_RELEASE_IMAGE}; preserving the existing previous image."
|
||||
fi
|
||||
|
||||
if docker image inspect "$GEOINTEL_RELEASE_IMAGE" >/dev/null 2>&1; then
|
||||
stored_revision="$(
|
||||
docker image inspect \
|
||||
@@ -208,7 +555,10 @@ else
|
||||
.
|
||||
fi
|
||||
|
||||
if ! start_image "$GEOINTEL_RELEASE_IMAGE"; then
|
||||
scan_release_image
|
||||
create_predeploy_backup
|
||||
|
||||
if ! start_image "$GEOINTEL_RELEASE_IMAGE_ID"; then
|
||||
rollback_previous || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -32,11 +32,17 @@
|
||||
<Config Name="Postgres Password" Target="GEOINTEL_POSTGRES_PASSWORD" Default="change-me-before-shared-use" Mode="" Description="Embedded PostGIS database password. Change before shared use." Type="Variable" Display="advanced" Required="true" Mask="true">change-me-before-shared-use</Config>
|
||||
<Config Name="CORS Origins" Target="GEOINTEL_CORS_ORIGINS" Default="http://localhost:1202,http://127.0.0.1:1202,http://192.168.10.150:1202" Mode="" Description="Comma-separated browser origins allowed to call the backend directly." Type="Variable" Display="advanced" Required="false" Mask="false">http://localhost:1202,http://127.0.0.1:1202,http://192.168.10.150:1202</Config>
|
||||
<Config Name="Max Upload MB" Target="GEOINTEL_MAX_UPLOAD_MB" Default="500" Mode="" Description="Maximum upload size in MiB enforced consistently by nginx and the backend (1-2048)." Type="Variable" Display="advanced" Required="true" Mask="false">500</Config>
|
||||
<Config Name="Max In-memory Vector MB" Target="GEOINTEL_MAX_IN_MEMORY_VECTOR_MB" Default="64" Mode="" Description="Maximum decompressed vector payload processed fully in memory by the backend (1-256 MiB)." Type="Variable" Display="advanced" Required="true" Mask="false">64</Config>
|
||||
<Config Name="Operator Login Enabled" Target="GEOINTEL_AUTH_ENABLED" Default="false" Mode="" Description="Require the single configured operator login before the browser may access workbench APIs." Type="Variable" Display="advanced" Required="true" Mask="false">false</Config>
|
||||
<Config Name="Operator Username" Target="GEOINTEL_AUTH_USERNAME" Default="" Mode="" Description="Exact username for the single operator account." Type="Variable" Display="advanced" Required="false" Mask="false"></Config>
|
||||
<Config Name="Operator Password Hash" Target="GEOINTEL_AUTH_PASSWORD_HASH" Default="" Mode="" Description="PBKDF2-SHA256 password hash. Never enter a plaintext password." Type="Variable" Display="advanced" Required="false" Mask="true"></Config>
|
||||
<Config Name="Operator Session Secret" Target="GEOINTEL_AUTH_SESSION_SECRET" Default="" Mode="" Description="Random secret of at least 32 characters used only to sign browser sessions." Type="Variable" Display="advanced" Required="false" Mask="true"></Config>
|
||||
<Config Name="Operator Session TTL" Target="GEOINTEL_AUTH_SESSION_TTL_SECONDS" Default="43200" Mode="" Description="Session lifetime in seconds (900-604800)." Type="Variable" Display="advanced" Required="true" Mask="false">43200</Config>
|
||||
<Config Name="Public Base URL" Target="GEOINTEL_PUBLIC_BASE_URL" Default="http://localhost:1202" Mode="" Description="Public browser origin. Must be HTTPS when Authentik is configured." Type="Variable" Display="advanced" Required="true" Mask="false">http://localhost:1202</Config>
|
||||
<Config Name="Authentik Issuer" Target="GEOINTEL_AUTHENTIK_ISSUER" Default="" Mode="" Description="Optional HTTPS OIDC issuer. Configure all Authentik values together." Type="Variable" Display="advanced" Required="false" Mask="false"></Config>
|
||||
<Config Name="Authentik Client ID" Target="GEOINTEL_AUTHENTIK_CLIENT_ID" Default="" Mode="" Description="OIDC client identifier for GeoIntel." Type="Variable" Display="advanced" Required="false" Mask="false"></Config>
|
||||
<Config Name="Authentik Client Secret" Target="GEOINTEL_AUTHENTIK_CLIENT_SECRET" Default="" Mode="" Description="OIDC client secret; never commit this value." Type="Variable" Display="advanced" Required="false" Mask="true"></Config>
|
||||
<Config Name="Authentik Allowed Email" Target="GEOINTEL_AUTHENTIK_ALLOWED_EMAIL" Default="" Mode="" Description="Exact verified operator e-mail address allowed to create an operator session." Type="Variable" Display="advanced" Required="false" Mask="false"></Config>
|
||||
<Config Name="Guest Demo Enabled" Target="GEOINTEL_GUEST_ACCESS_ENABLED" Default="true" Mode="" Description="Show a guest button that opens only the seeded, restricted demo workspace. Enabled by default when operator login is active; set false on non-demo instances." Type="Variable" Display="advanced" Required="true" Mask="false">true</Config>
|
||||
<Config Name="Guest Display Name" Target="GEOINTEL_GUEST_DISPLAY_NAME" Default="Gast" Mode="" Description="Label shown for the temporary guest session." Type="Variable" Display="advanced" Required="true" Mask="false">Gast</Config>
|
||||
<Config Name="Guest Session TTL" Target="GEOINTEL_GUEST_SESSION_TTL_SECONDS" Default="7200" Mode="" Description="Temporary guest session lifetime in seconds (900-86400)." Type="Variable" Display="advanced" Required="true" Mask="false">7200</Config>
|
||||
|
||||
@@ -34,6 +34,8 @@ GEOINTEL_CORS_ORIGINS=https://geointel.itworx.tech,http://geointel.itworx.tech,h
|
||||
|
||||
# Upload guard in MiB. The same 1-2048 limit is applied by nginx and FastAPI.
|
||||
GEOINTEL_MAX_UPLOAD_MB=500
|
||||
# Maximum decompressed vector payload processed fully in memory (1-256 MiB).
|
||||
GEOINTEL_MAX_IN_MEMORY_VECTOR_MB=64
|
||||
GEOINTEL_AOI_WORKER_ENABLED=true
|
||||
GEOINTEL_AOI_WORKER_POLL_SECONDS=2
|
||||
|
||||
@@ -46,6 +48,14 @@ GEOINTEL_AUTH_PASSWORD_HASH=
|
||||
GEOINTEL_AUTH_SESSION_SECRET=
|
||||
GEOINTEL_AUTH_SESSION_TTL_SECONDS=43200
|
||||
|
||||
# Optional additive Authentik OIDC login. Configure all fields together. The
|
||||
# local operator credentials above remain the recovery login.
|
||||
GEOINTEL_PUBLIC_BASE_URL=http://localhost:1202
|
||||
GEOINTEL_AUTHENTIK_ISSUER=
|
||||
GEOINTEL_AUTHENTIK_CLIENT_ID=
|
||||
GEOINTEL_AUTHENTIK_CLIENT_SECRET=
|
||||
GEOINTEL_AUTHENTIK_ALLOWED_EMAIL=
|
||||
|
||||
# Guest access is enabled by default whenever operator authentication is active.
|
||||
# It opens the seeded GeoIntel demo in a temporary, API-enforced restricted
|
||||
# session. Set this to false on installations containing private project data.
|
||||
@@ -153,8 +163,9 @@ SPW_TERRAIN_ANALYSIS_RESOLUTION_M=5
|
||||
SPW_TERRAIN_MAX_SIDE_M=20000
|
||||
SPW_TERRAIN_MAX_PIXELS=12000000
|
||||
|
||||
# Optional configured-YOLO runtime. Keep disabled unless a local model is mounted.
|
||||
GEOINTEL_INSTALL_AI=false
|
||||
# Production releases always contain the pinned AI dependencies. Inference may
|
||||
# remain disabled until an integrity-bound local model is mounted.
|
||||
GEOINTEL_INSTALL_AI=true
|
||||
YOLO_ENABLED=false
|
||||
YOLO_MODELS_DIR=/app/models
|
||||
YOLO_MODEL_PATH=
|
||||
|
||||
@@ -1,3 +1,18 @@
|
||||
geo $geointel_trusted_forwarder {
|
||||
default 0;
|
||||
127.0.0.0/8 1;
|
||||
::1/128 1;
|
||||
# The outer Nginx Proxy Manager reaches this container through Docker's
|
||||
# internal bridge; public/LAN clients are not trusted forwarders.
|
||||
172.16.0.0/12 1;
|
||||
}
|
||||
|
||||
map "$geointel_trusted_forwarder:$http_x_forwarded_proto" $geointel_forwarded_proto {
|
||||
default $scheme;
|
||||
"1:https" https;
|
||||
"1:http" http;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
@@ -5,21 +20,42 @@ server {
|
||||
proxy_read_timeout 600s;
|
||||
proxy_send_timeout 600s;
|
||||
|
||||
add_header Content-Security-Policy "frame-ancestors 'none'" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always;
|
||||
|
||||
root /usr/share/nginx/html;
|
||||
index index.html;
|
||||
|
||||
location = /index.html {
|
||||
add_header Cache-Control "no-cache";
|
||||
add_header Content-Security-Policy "frame-ancestors 'none'" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always;
|
||||
try_files /index.html =404;
|
||||
}
|
||||
|
||||
location = /geointel-icon.svg {
|
||||
add_header Cache-Control "public, max-age=3600";
|
||||
add_header Content-Security-Policy "frame-ancestors 'none'" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always;
|
||||
try_files /geointel-icon.svg =404;
|
||||
}
|
||||
|
||||
location = /geointel-icon.png {
|
||||
add_header Cache-Control "public, max-age=3600";
|
||||
add_header Content-Security-Policy "frame-ancestors 'none'" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always;
|
||||
# Preserve the stable DockerMan/public URL while the frontend keeps
|
||||
# its explicit 32px and 180px icon variants.
|
||||
try_files /geointel-icon-180.png =404;
|
||||
@@ -27,6 +63,11 @@ server {
|
||||
|
||||
location /assets/ {
|
||||
add_header Cache-Control "no-cache";
|
||||
add_header Content-Security-Policy "frame-ancestors 'none'" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
|
||||
add_header Permissions-Policy "camera=(), microphone=(), geolocation=()" always;
|
||||
try_files $uri =404;
|
||||
}
|
||||
|
||||
@@ -36,7 +77,7 @@ server {
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Forwarded-Proto $geointel_forwarded_proto;
|
||||
}
|
||||
|
||||
location = /health {
|
||||
@@ -45,7 +86,7 @@ server {
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Forwarded-Proto $geointel_forwarded_proto;
|
||||
}
|
||||
|
||||
location = /health/live {
|
||||
|
||||
@@ -0,0 +1,323 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Restore the production database from a verified pre-deploy dump while the
|
||||
# normal GeoIntel container is stopped. This is intentionally a separate,
|
||||
# explicitly confirmed operation: starting an older image against a schema
|
||||
# migrated by a newer image is not a safe rollback strategy.
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
GEOINTEL_DEPLOY_LOCK_FILE="${GEOINTEL_DEPLOY_LOCK_FILE:-/tmp/geointel-release-deploy.lock}"
|
||||
if [ "${GEOINTEL_DEPLOY_LOCK_HELD:-false}" != "true" ]; then
|
||||
command -v flock >/dev/null 2>&1 || {
|
||||
echo "GeoIntel database restore requires flock to prevent concurrent deployment." >&2
|
||||
exit 2
|
||||
}
|
||||
exec 9>"$GEOINTEL_DEPLOY_LOCK_FILE"
|
||||
if ! flock -n 9; then
|
||||
echo "Another GeoIntel deployment or rollback is already running." >&2
|
||||
exit 3
|
||||
fi
|
||||
fi
|
||||
|
||||
BACKUP_DIR=""
|
||||
CONFIRMED="false"
|
||||
RESTORE_IMAGE="${GEOINTEL_ROLLBACK_IMAGE:-}"
|
||||
GEOINTEL_CONTAINER_NAME="${GEOINTEL_CONTAINER_NAME:-geointel}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: bash deploy/unraid/restore-predeploy-database.sh \
|
||||
--backup-dir PATH --confirm-production-database-restore [options]
|
||||
|
||||
Stops the normal GeoIntel container, starts an isolated PostGIS recovery
|
||||
container on the same persistent database path, restores the checksum-verified
|
||||
custom-format dump, validates Alembic/table counts, and stops recovery again.
|
||||
The caller must start the rollback image after this command succeeds.
|
||||
|
||||
Options:
|
||||
--image IMAGE Recovery image containing PostgreSQL/PostGIS tools
|
||||
--container NAME Normal application container (default: geointel)
|
||||
EOF
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--backup-dir) BACKUP_DIR="$2"; shift 2 ;;
|
||||
--confirm-production-database-restore) CONFIRMED="true"; shift ;;
|
||||
--image) RESTORE_IMAGE="$2"; shift 2 ;;
|
||||
--container) GEOINTEL_CONTAINER_NAME="$2"; shift 2 ;;
|
||||
--help|-h) usage; exit 0 ;;
|
||||
*) echo "Unknown argument: $1" >&2; usage >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ "$CONFIRMED" != "true" ] || [ -z "$BACKUP_DIR" ]; then
|
||||
echo "Explicit --confirm-production-database-restore and --backup-dir are required." >&2
|
||||
exit 2
|
||||
fi
|
||||
for required in docker python3 sha256sum; do
|
||||
command -v "$required" >/dev/null 2>&1 || {
|
||||
echo "Missing required command: $required" >&2
|
||||
exit 2
|
||||
}
|
||||
done
|
||||
|
||||
if [ -f .env ]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1091
|
||||
. ./.env
|
||||
set +a
|
||||
fi
|
||||
|
||||
GEOINTEL_BACKUPS_PATH="${GEOINTEL_BACKUPS_PATH:-/mnt/user/appdata/geointel/backups}"
|
||||
GEOINTEL_POSTGIS_DATA_PATH="${GEOINTEL_POSTGIS_DATA_PATH:-/mnt/user/appdata/geointel/postgres-data}"
|
||||
GEOINTEL_POSTGRES_DB="${GEOINTEL_POSTGRES_DB:-geointel}"
|
||||
GEOINTEL_POSTGRES_USER="${GEOINTEL_POSTGRES_USER:-geointel}"
|
||||
GEOINTEL_POSTGRES_PASSWORD="${GEOINTEL_POSTGRES_PASSWORD:-}"
|
||||
|
||||
if ! [[ "$GEOINTEL_POSTGRES_DB" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] \
|
||||
|| ! [[ "$GEOINTEL_POSTGRES_USER" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]]; then
|
||||
echo "Configured PostGIS database and user names must be simple SQL identifiers." >&2
|
||||
exit 2
|
||||
fi
|
||||
case "$GEOINTEL_POSTGRES_PASSWORD" in
|
||||
''|geointel|postgres|password|changeme|change-me-before-shared-use)
|
||||
echo "Refusing database restore with an empty or known-default PostGIS password." >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
test -f "$GEOINTEL_POSTGIS_DATA_PATH/PG_VERSION" || {
|
||||
echo "Persistent PostGIS data path is not initialized: $GEOINTEL_POSTGIS_DATA_PATH" >&2
|
||||
exit 3
|
||||
}
|
||||
|
||||
GEOINTEL_BACKUPS_PATH="$(python3 -c 'import pathlib,sys; print(pathlib.Path(sys.argv[1]).expanduser().resolve())' "$GEOINTEL_BACKUPS_PATH")"
|
||||
BACKUP_DIR="$(python3 -c 'import pathlib,sys; print(pathlib.Path(sys.argv[1]).expanduser().resolve())' "$BACKUP_DIR")"
|
||||
python3 - "$GEOINTEL_BACKUPS_PATH" "$BACKUP_DIR" <<'PY'
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
root = pathlib.Path(sys.argv[1])
|
||||
backup = pathlib.Path(sys.argv[2])
|
||||
try:
|
||||
backup.relative_to(root)
|
||||
except ValueError as exc:
|
||||
raise SystemExit(f"Backup directory must be below {root}") from exc
|
||||
if backup == root:
|
||||
raise SystemExit("Backup directory must identify one immutable backup")
|
||||
PY
|
||||
|
||||
for required_file in manifest.json database.dump database.list database-metadata.tsv table-counts.tsv CHECKSUMS.sha256; do
|
||||
test -s "$BACKUP_DIR/$required_file" || {
|
||||
echo "Missing or empty backup artifact: $required_file" >&2
|
||||
exit 3
|
||||
}
|
||||
done
|
||||
(
|
||||
cd "$BACKUP_DIR"
|
||||
sha256sum -c CHECKSUMS.sha256
|
||||
)
|
||||
python3 "$ROOT/scripts/release_backup_snapshot.py" verify-backup --backup-dir "$BACKUP_DIR"
|
||||
|
||||
IFS=$'\t' read -r BACKUP_DB BACKUP_USER BACKUP_IMAGE_ID BACKUP_RELEASE_ID < <(
|
||||
python3 - "$BACKUP_DIR/manifest.json" <<'PY'
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
|
||||
payload = json.loads(pathlib.Path(sys.argv[1]).read_text(encoding="utf-8"))
|
||||
if payload.get("schema_version") != 1 or payload.get("read_only_source") is not True:
|
||||
raise SystemExit("Unsupported or unsafe backup manifest")
|
||||
print(
|
||||
f"{payload.get('database_name', '')}\t{payload.get('database_user', '')}\t"
|
||||
f"{payload.get('image_id', '')}\t{payload.get('release_id', '')}"
|
||||
)
|
||||
PY
|
||||
)
|
||||
if [ "$BACKUP_DB" != "$GEOINTEL_POSTGRES_DB" ] || [ "$BACKUP_USER" != "$GEOINTEL_POSTGRES_USER" ]; then
|
||||
echo "Backup database identity does not match the configured production database." >&2
|
||||
exit 3
|
||||
fi
|
||||
if ! [[ "$BACKUP_IMAGE_ID" =~ ^sha256:[0-9a-f]{64}$ ]]; then
|
||||
echo "Backup manifest does not contain one immutable Docker image ID." >&2
|
||||
exit 3
|
||||
fi
|
||||
if [ -z "$RESTORE_IMAGE" ]; then
|
||||
RESTORE_IMAGE="$BACKUP_IMAGE_ID"
|
||||
fi
|
||||
docker image inspect "$RESTORE_IMAGE" >/dev/null
|
||||
RESTORE_IMAGE_ID="$(docker image inspect --format '{{.Id}}' "$RESTORE_IMAGE")"
|
||||
if [ -z "$BACKUP_IMAGE_ID" ] || [ "$BACKUP_IMAGE_ID" != "$RESTORE_IMAGE_ID" ]; then
|
||||
echo "Backup image identity does not match the retained rollback image." >&2
|
||||
exit 3
|
||||
fi
|
||||
case "$BACKUP_RELEASE_ID" in
|
||||
predeploy-*) ;;
|
||||
*) echo "Production rollback requires a predeploy backup." >&2; exit 3 ;;
|
||||
esac
|
||||
|
||||
if docker ps -a --format '{{.Names}}' | grep -Fxq "$GEOINTEL_CONTAINER_NAME"; then
|
||||
docker rm -f "$GEOINTEL_CONTAINER_NAME" >/dev/null
|
||||
fi
|
||||
|
||||
RECOVERY_CONTAINER="geointel-db-restore-$(date -u +%Y%m%d%H%M%S)-$$"
|
||||
RESTORE_PROOF_DB="geointel_restore_proof_$(date -u +%Y%m%d%H%M%S)_$$"
|
||||
RECOVERY_DB="geointel_pre_restore_$(date -u +%Y%m%d%H%M%S)_$$"
|
||||
FAILED_RESTORE_DB="geointel_failed_restore_$(date -u +%Y%m%d%H%M%S)_$$"
|
||||
SWAP_COMPLETE="false"
|
||||
cleanup_recovery() {
|
||||
if [ "$SWAP_COMPLETE" != "true" ] \
|
||||
&& [ "$(docker inspect -f '{{.State.Running}}' "$RECOVERY_CONTAINER" 2>/dev/null || true)" = "true" ]; then
|
||||
docker exec "$RECOVERY_CONTAINER" dropdb --if-exists --force \
|
||||
-U "$GEOINTEL_POSTGRES_USER" "$RESTORE_PROOF_DB" >/dev/null 2>&1 || true
|
||||
fi
|
||||
docker rm -f "$RECOVERY_CONTAINER" >/dev/null 2>&1 || true
|
||||
}
|
||||
trap cleanup_recovery EXIT
|
||||
|
||||
docker run -d \
|
||||
--name "$RECOVERY_CONTAINER" \
|
||||
--restart no \
|
||||
-e PGDATA=/var/lib/postgresql/data \
|
||||
-e PGPASSWORD="$GEOINTEL_POSTGRES_PASSWORD" \
|
||||
-v "$GEOINTEL_POSTGIS_DATA_PATH:/var/lib/postgresql/data" \
|
||||
-v "$BACKUP_DIR:/restore:ro" \
|
||||
--entrypoint /bin/bash \
|
||||
"$RESTORE_IMAGE" \
|
||||
-c 'set -euo pipefail; chown postgres:postgres "$PGDATA"; exec gosu postgres postgres' \
|
||||
>/dev/null
|
||||
|
||||
for attempt in $(seq 1 180); do
|
||||
if docker exec "$RECOVERY_CONTAINER" pg_isready -h 127.0.0.1 -U "$GEOINTEL_POSTGRES_USER" -d postgres >/dev/null 2>&1; then
|
||||
break
|
||||
fi
|
||||
if [ "$(docker inspect -f '{{.State.Running}}' "$RECOVERY_CONTAINER" 2>/dev/null || true)" != "true" ]; then
|
||||
echo "Database recovery container exited before PostGIS became ready." >&2
|
||||
docker logs "$RECOVERY_CONTAINER" >&2 || true
|
||||
exit 4
|
||||
fi
|
||||
if [ "$attempt" -eq 180 ]; then
|
||||
echo "PostGIS recovery did not become ready within six minutes." >&2
|
||||
exit 4
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
|
||||
RESTORED_LIST="$(mktemp)"
|
||||
trap 'rm -f -- "$RESTORED_LIST"; cleanup_recovery' EXIT
|
||||
docker exec "$RECOVERY_CONTAINER" pg_restore --list /restore/database.dump > "$RESTORED_LIST"
|
||||
cmp -s "$RESTORED_LIST" "$BACKUP_DIR/database.list" || {
|
||||
echo "Recovery image reads a different PostgreSQL archive listing." >&2
|
||||
exit 4
|
||||
}
|
||||
|
||||
if ! docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -Atqc \
|
||||
"SELECT 1 FROM pg_database WHERE datname = '${GEOINTEL_POSTGRES_DB}';" | grep -Fxq 1; then
|
||||
echo "Configured production database does not exist; refusing replacement." >&2
|
||||
exit 4
|
||||
fi
|
||||
for generated_database in "$RESTORE_PROOF_DB" "$RECOVERY_DB" "$FAILED_RESTORE_DB"; do
|
||||
if docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -Atqc \
|
||||
"SELECT 1 FROM pg_database WHERE datname = '${generated_database}';" | grep -Fxq 1; then
|
||||
echo "Generated recovery database already exists: ${generated_database}" >&2
|
||||
exit 4
|
||||
fi
|
||||
done
|
||||
|
||||
# Prove the complete archive in a separate database before touching production.
|
||||
docker exec "$RECOVERY_CONTAINER" createdb \
|
||||
-U "$GEOINTEL_POSTGRES_USER" "$RESTORE_PROOF_DB"
|
||||
docker exec "$RECOVERY_CONTAINER" pg_restore \
|
||||
--exit-on-error \
|
||||
--no-owner \
|
||||
--no-privileges \
|
||||
-U "$GEOINTEL_POSTGRES_USER" \
|
||||
-d "$RESTORE_PROOF_DB" \
|
||||
/restore/database.dump
|
||||
|
||||
EXPECTED_HEAD="$(awk -F $'\t' '$1 == "alembic_head" { print $2 }' "$BACKUP_DIR/database-metadata.tsv")"
|
||||
validate_restored_database() {
|
||||
local database_name="$1"
|
||||
local restored_head=""
|
||||
restored_head="$(docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d "$database_name" -Atqc \
|
||||
'SELECT version_num FROM alembic_version;')"
|
||||
if [ -z "$EXPECTED_HEAD" ] || [ "$restored_head" != "$EXPECTED_HEAD" ]; then
|
||||
echo "Restored Alembic head '$restored_head' differs from backup head '$EXPECTED_HEAD'." >&2
|
||||
return 1
|
||||
fi
|
||||
while IFS=$'\t' read -r table expected; do
|
||||
[[ "$table" =~ ^[a-z_]+$ ]] || {
|
||||
echo "Unsafe table name in retained counts: $table" >&2
|
||||
return 1
|
||||
}
|
||||
actual="$(docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d "$database_name" -Atqc \
|
||||
"SELECT count(*) FROM public.${table};")"
|
||||
if [ "$actual" != "$expected" ]; then
|
||||
echo "Restored count mismatch for $table: expected $expected, got $actual." >&2
|
||||
return 1
|
||||
fi
|
||||
done < "$BACKUP_DIR/table-counts.tsv"
|
||||
}
|
||||
|
||||
validate_restored_database "$RESTORE_PROOF_DB"
|
||||
echo "Isolated predeploy restore proof passed: ${RESTORE_PROOF_DB}"
|
||||
|
||||
docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -c \
|
||||
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname IN ('${GEOINTEL_POSTGRES_DB}', '${RESTORE_PROOF_DB}') AND pid <> pg_backend_pid();" \
|
||||
>/dev/null
|
||||
docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -c \
|
||||
"ALTER DATABASE ${GEOINTEL_POSTGRES_DB} RENAME TO ${RECOVERY_DB};"
|
||||
if ! docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -c \
|
||||
"ALTER DATABASE ${RESTORE_PROOF_DB} RENAME TO ${GEOINTEL_POSTGRES_DB};"; then
|
||||
echo "Restored database cutover failed; restoring the untouched production database name." >&2
|
||||
docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -c \
|
||||
"ALTER DATABASE ${RECOVERY_DB} RENAME TO ${GEOINTEL_POSTGRES_DB};"
|
||||
exit 4
|
||||
fi
|
||||
SWAP_COMPLETE="true"
|
||||
|
||||
if ! validate_restored_database "$GEOINTEL_POSTGRES_DB"; then
|
||||
echo "Post-cutover validation failed; restoring the retained pre-restore database." >&2
|
||||
docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -c \
|
||||
"SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '${GEOINTEL_POSTGRES_DB}' AND pid <> pg_backend_pid();" \
|
||||
>/dev/null
|
||||
docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -c \
|
||||
"ALTER DATABASE ${GEOINTEL_POSTGRES_DB} RENAME TO ${FAILED_RESTORE_DB};"
|
||||
docker exec "$RECOVERY_CONTAINER" psql -X -v ON_ERROR_STOP=1 \
|
||||
-U "$GEOINTEL_POSTGRES_USER" -d postgres -c \
|
||||
"ALTER DATABASE ${RECOVERY_DB} RENAME TO ${GEOINTEL_POSTGRES_DB};"
|
||||
SWAP_COMPLETE="false"
|
||||
echo "Original production database was restored; failed restore retained as ${FAILED_RESTORE_DB}." >&2
|
||||
exit 4
|
||||
fi
|
||||
|
||||
if [ -z "$RECOVERY_DB" ]; then
|
||||
echo "Recovery database identity was not retained." >&2
|
||||
exit 4
|
||||
fi
|
||||
|
||||
while IFS=$'\t' read -r table expected; do
|
||||
[[ "$table" =~ ^[a-z_]+$ ]] || {
|
||||
echo "Unsafe table name in retained counts: $table" >&2
|
||||
exit 4
|
||||
}
|
||||
done < "$BACKUP_DIR/table-counts.tsv"
|
||||
|
||||
rm -f -- "$RESTORED_LIST"
|
||||
cleanup_recovery
|
||||
trap - EXIT
|
||||
echo "Production database restored and verified from: $BACKUP_DIR"
|
||||
echo "Pre-restore production database retained for operator recovery as: $RECOVERY_DB"
|
||||
@@ -4,14 +4,78 @@ set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
GEOINTEL_ROLLBACK_IMAGE="${GEOINTEL_ROLLBACK_IMAGE:-geointel-all-in-one:previous}"
|
||||
GEOINTEL_DEPLOY_LOCK_FILE="${GEOINTEL_DEPLOY_LOCK_FILE:-/tmp/geointel-release-deploy.lock}"
|
||||
if [ "${GEOINTEL_DEPLOY_LOCK_HELD:-false}" != "true" ]; then
|
||||
command -v flock >/dev/null 2>&1 || {
|
||||
echo "GeoIntel rollback requires flock to prevent concurrent deployment." >&2
|
||||
exit 2
|
||||
}
|
||||
exec 9>"$GEOINTEL_DEPLOY_LOCK_FILE"
|
||||
if ! flock -n 9; then
|
||||
echo "Another GeoIntel deployment or rollback is already running." >&2
|
||||
exit 3
|
||||
fi
|
||||
GEOINTEL_DEPLOY_LOCK_HELD="true"
|
||||
export GEOINTEL_DEPLOY_LOCK_HELD
|
||||
fi
|
||||
|
||||
if ! docker image inspect "$GEOINTEL_ROLLBACK_IMAGE" >/dev/null 2>&1; then
|
||||
echo "Rollback image does not exist: ${GEOINTEL_ROLLBACK_IMAGE}" >&2
|
||||
GEOINTEL_ROLLBACK_IMAGE="${GEOINTEL_ROLLBACK_IMAGE:-}"
|
||||
BACKUP_DIR="${GEOINTEL_ROLLBACK_BACKUP_DIR:-}"
|
||||
CONFIRM_RESTORE="false"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: bash deploy/unraid/rollback-dockerman-container.sh \
|
||||
--backup-dir PATH --confirm-production-database-restore
|
||||
|
||||
Restores the verified pre-deploy PostgreSQL dump first and only then starts the
|
||||
retained previous image. Image-only rollback against an unknown migrated
|
||||
schema is deliberately not supported.
|
||||
EOF
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--backup-dir) BACKUP_DIR="$2"; shift 2 ;;
|
||||
--confirm-production-database-restore) CONFIRM_RESTORE="true"; shift ;;
|
||||
--help|-h) usage; exit 0 ;;
|
||||
*) echo "Unknown argument: $1" >&2; usage >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -z "$BACKUP_DIR" ] || [ "$CONFIRM_RESTORE" != "true" ]; then
|
||||
echo "Rollback requires a verified pre-deploy backup and explicit database-restore confirmation." >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
echo "Starting rollback image ${GEOINTEL_ROLLBACK_IMAGE} without changing persistent volumes..."
|
||||
restore_image_args=()
|
||||
if [ -n "$GEOINTEL_ROLLBACK_IMAGE" ]; then
|
||||
restore_image_args=(--image "$GEOINTEL_ROLLBACK_IMAGE")
|
||||
fi
|
||||
echo "Restoring the pre-deploy database before starting its checksum-bound image..."
|
||||
bash deploy/unraid/restore-predeploy-database.sh \
|
||||
--backup-dir "$BACKUP_DIR" \
|
||||
"${restore_image_args[@]}" \
|
||||
--confirm-production-database-restore
|
||||
|
||||
if [ -z "$GEOINTEL_ROLLBACK_IMAGE" ]; then
|
||||
GEOINTEL_ROLLBACK_IMAGE="$(python3 - "$BACKUP_DIR/manifest.json" <<'PY'
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
|
||||
image_id = json.loads(pathlib.Path(sys.argv[1]).read_text(encoding="utf-8")).get("image_id", "")
|
||||
if not re.fullmatch(r"sha256:[0-9a-f]{64}", image_id):
|
||||
raise SystemExit("Backup manifest lacks an immutable rollback image ID")
|
||||
print(image_id)
|
||||
PY
|
||||
)"
|
||||
fi
|
||||
docker image inspect "$GEOINTEL_ROLLBACK_IMAGE" >/dev/null
|
||||
|
||||
echo "Starting rollback image ${GEOINTEL_ROLLBACK_IMAGE} with the restored persistent database..."
|
||||
GEOINTEL_IMAGE="$GEOINTEL_ROLLBACK_IMAGE" bash deploy/unraid/run-dockerman-container.sh
|
||||
|
||||
for attempt in $(seq 1 90); do
|
||||
|
||||
@@ -33,6 +33,7 @@ GEOINTEL_POSTGRES_USER="${GEOINTEL_POSTGRES_USER:-geointel}"
|
||||
GEOINTEL_POSTGRES_PASSWORD="${GEOINTEL_POSTGRES_PASSWORD:-}"
|
||||
GEOINTEL_CORS_ORIGINS="${GEOINTEL_CORS_ORIGINS:-http://localhost:${GEOINTEL_FRONTEND_PORT},http://127.0.0.1:${GEOINTEL_FRONTEND_PORT},http://192.168.10.150:${GEOINTEL_FRONTEND_PORT}}"
|
||||
GEOINTEL_MAX_UPLOAD_MB="${GEOINTEL_MAX_UPLOAD_MB:-500}"
|
||||
GEOINTEL_MAX_IN_MEMORY_VECTOR_MB="${GEOINTEL_MAX_IN_MEMORY_VECTOR_MB:-64}"
|
||||
GEOINTEL_AOI_WORKER_ENABLED="${GEOINTEL_AOI_WORKER_ENABLED:-true}"
|
||||
GEOINTEL_AOI_WORKER_POLL_SECONDS="${GEOINTEL_AOI_WORKER_POLL_SECONDS:-2}"
|
||||
GEOINTEL_AUTH_ENABLED="${GEOINTEL_AUTH_ENABLED:-false}"
|
||||
@@ -40,6 +41,11 @@ GEOINTEL_AUTH_USERNAME="${GEOINTEL_AUTH_USERNAME:-}"
|
||||
GEOINTEL_AUTH_PASSWORD_HASH="${GEOINTEL_AUTH_PASSWORD_HASH:-}"
|
||||
GEOINTEL_AUTH_SESSION_SECRET="${GEOINTEL_AUTH_SESSION_SECRET:-}"
|
||||
GEOINTEL_AUTH_SESSION_TTL_SECONDS="${GEOINTEL_AUTH_SESSION_TTL_SECONDS:-43200}"
|
||||
GEOINTEL_PUBLIC_BASE_URL="${GEOINTEL_PUBLIC_BASE_URL:-http://localhost:${GEOINTEL_FRONTEND_PORT}}"
|
||||
GEOINTEL_AUTHENTIK_ISSUER="${GEOINTEL_AUTHENTIK_ISSUER:-}"
|
||||
GEOINTEL_AUTHENTIK_CLIENT_ID="${GEOINTEL_AUTHENTIK_CLIENT_ID:-}"
|
||||
GEOINTEL_AUTHENTIK_CLIENT_SECRET="${GEOINTEL_AUTHENTIK_CLIENT_SECRET:-}"
|
||||
GEOINTEL_AUTHENTIK_ALLOWED_EMAIL="${GEOINTEL_AUTHENTIK_ALLOWED_EMAIL:-}"
|
||||
GEOINTEL_GUEST_ACCESS_ENABLED="${GEOINTEL_GUEST_ACCESS_ENABLED:-true}"
|
||||
GEOINTEL_GUEST_DISPLAY_NAME="${GEOINTEL_GUEST_DISPLAY_NAME:-Gast}"
|
||||
GEOINTEL_GUEST_SESSION_TTL_SECONDS="${GEOINTEL_GUEST_SESSION_TTL_SECONDS:-7200}"
|
||||
@@ -189,6 +195,17 @@ validate_runtime_config() {
|
||||
return 2
|
||||
fi
|
||||
|
||||
case "$GEOINTEL_MAX_IN_MEMORY_VECTOR_MB" in
|
||||
''|*[!0-9]*)
|
||||
echo "GEOINTEL_MAX_IN_MEMORY_VECTOR_MB must be a whole number." >&2
|
||||
return 2
|
||||
;;
|
||||
esac
|
||||
if [ "$GEOINTEL_MAX_IN_MEMORY_VECTOR_MB" -lt 1 ] || [ "$GEOINTEL_MAX_IN_MEMORY_VECTOR_MB" -gt 256 ]; then
|
||||
echo "GEOINTEL_MAX_IN_MEMORY_VECTOR_MB must be between 1 and 256." >&2
|
||||
return 2
|
||||
fi
|
||||
|
||||
case "$GEOINTEL_AUTH_ENABLED" in
|
||||
true|false) ;;
|
||||
*)
|
||||
@@ -212,6 +229,36 @@ validate_runtime_config() {
|
||||
esac
|
||||
fi
|
||||
|
||||
local authentik_count=0
|
||||
local authentik_value
|
||||
for authentik_value in \
|
||||
"$GEOINTEL_AUTHENTIK_ISSUER" \
|
||||
"$GEOINTEL_AUTHENTIK_CLIENT_ID" \
|
||||
"$GEOINTEL_AUTHENTIK_CLIENT_SECRET" \
|
||||
"$GEOINTEL_AUTHENTIK_ALLOWED_EMAIL"; do
|
||||
if [ -n "$authentik_value" ]; then
|
||||
authentik_count=$((authentik_count + 1))
|
||||
fi
|
||||
done
|
||||
if [ "$authentik_count" -ne 0 ] && [ "$authentik_count" -ne 4 ]; then
|
||||
echo "All GEOINTEL_AUTHENTIK_* values must be configured together." >&2
|
||||
return 2
|
||||
fi
|
||||
if [ "$authentik_count" -eq 4 ]; then
|
||||
if [ "$GEOINTEL_AUTH_ENABLED" != "true" ]; then
|
||||
echo "GEOINTEL_AUTH_ENABLED must be true when Authentik is configured." >&2
|
||||
return 2
|
||||
fi
|
||||
case "$GEOINTEL_AUTHENTIK_ISSUER" in
|
||||
https://*) ;;
|
||||
*) echo "GEOINTEL_AUTHENTIK_ISSUER must use HTTPS." >&2; return 2 ;;
|
||||
esac
|
||||
case "$GEOINTEL_PUBLIC_BASE_URL" in
|
||||
https://*) ;;
|
||||
*) echo "GEOINTEL_PUBLIC_BASE_URL must use HTTPS for Authentik." >&2; return 2 ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
case "$GEOINTEL_GUEST_ACCESS_ENABLED" in
|
||||
true|false) ;;
|
||||
*)
|
||||
@@ -314,6 +361,7 @@ docker run -d \
|
||||
-e GEOINTEL_STORAGE_ROOT=/app/storage \
|
||||
-e GEOINTEL_CORS_ORIGINS="$GEOINTEL_CORS_ORIGINS" \
|
||||
-e GEOINTEL_MAX_UPLOAD_MB="$GEOINTEL_MAX_UPLOAD_MB" \
|
||||
-e GEOINTEL_MAX_IN_MEMORY_VECTOR_MB="$GEOINTEL_MAX_IN_MEMORY_VECTOR_MB" \
|
||||
-e GEOINTEL_AOI_WORKER_ENABLED="$GEOINTEL_AOI_WORKER_ENABLED" \
|
||||
-e GEOINTEL_AOI_WORKER_POLL_SECONDS="$GEOINTEL_AOI_WORKER_POLL_SECONDS" \
|
||||
-e GEOINTEL_AUTH_ENABLED="$GEOINTEL_AUTH_ENABLED" \
|
||||
@@ -321,6 +369,11 @@ docker run -d \
|
||||
-e GEOINTEL_AUTH_PASSWORD_HASH="$GEOINTEL_AUTH_PASSWORD_HASH" \
|
||||
-e GEOINTEL_AUTH_SESSION_SECRET="$GEOINTEL_AUTH_SESSION_SECRET" \
|
||||
-e GEOINTEL_AUTH_SESSION_TTL_SECONDS="$GEOINTEL_AUTH_SESSION_TTL_SECONDS" \
|
||||
-e GEOINTEL_PUBLIC_BASE_URL="$GEOINTEL_PUBLIC_BASE_URL" \
|
||||
-e GEOINTEL_AUTHENTIK_ISSUER="$GEOINTEL_AUTHENTIK_ISSUER" \
|
||||
-e GEOINTEL_AUTHENTIK_CLIENT_ID="$GEOINTEL_AUTHENTIK_CLIENT_ID" \
|
||||
-e GEOINTEL_AUTHENTIK_CLIENT_SECRET="$GEOINTEL_AUTHENTIK_CLIENT_SECRET" \
|
||||
-e GEOINTEL_AUTHENTIK_ALLOWED_EMAIL="$GEOINTEL_AUTHENTIK_ALLOWED_EMAIL" \
|
||||
-e GEOINTEL_GUEST_ACCESS_ENABLED="$GEOINTEL_GUEST_ACCESS_ENABLED" \
|
||||
-e GEOINTEL_GUEST_DISPLAY_NAME="$GEOINTEL_GUEST_DISPLAY_NAME" \
|
||||
-e GEOINTEL_GUEST_SESSION_TTL_SECONDS="$GEOINTEL_GUEST_SESSION_TTL_SECONDS" \
|
||||
|
||||
Reference in New Issue
Block a user