Updating, Release, and Rollback

KMP’s managed Azure environments use gated GitHub Actions workflows and immutable image digests. They do not update by pulling a mutable tag on the server, and the web Container App does not run migrations on startup.

← Back to Deployment and Operations

Managed Azure release procedure

Release to POC

  1. Prepare the selected candidate on official dev and open a dev → main PR when the team needs to validate changes before merge. Commit the release version and notes before testing; record manual results against the running SHA. If starting from approved main instead, fast-forward dev to that commit without discarding unmerged dev work. Never force-push the branch.
  2. Push dev. Quality Gates runs the full suites on that exact branch commit, and Nightly / Dev Docker Image waits for successful dev-push evidence for the same SHA. Pull-request runs are not accepted as branch-push evidence.
  3. The image workflow builds ghcr.io/ansteorra/kmp:dev-<short-sha> for AMD64, scans it, and smoke-checks that immutable image digest. Failed or missing quality evidence blocks image publication and POC deployment.
  4. POC / Deploy to Azure resolves the immutable GHCR digest, imports it to POC ACR, captures rollback evidence, canaries the unified worker, runs the ordered migration job, cuts over web, verifies /livez and /health, and aligns retained jobs.
  5. Only after that deployment succeeds, the workflow applies poc-validated-<12-char-sha> to the same digest.
  6. Validate tenant login, host resolution, queue/worker processing, Platform Admin access on its reserved host, backup readiness, and the release’s changed user journeys. Keep the team’s results and sign-off in the PR before merging.

Merge the release PR with an ancestry-preserving merge commit (or fast-forward), not squash or rebase. Production requires successful main-push quality evidence and POC validation for the exact resulting release SHA. Fetch upstream, verify git merge-base --is-ancestor upstream/dev upstream/main, then fast-forward dev with git merge --ff-only upstream/main and push it to repeat image/POC validation for that Main SHA. An earlier Dev image is not merge-commit evidence.

If that ancestry check fails after a squash/rebase or divergent Dev work, preserve both branches. Obtain a reviewed reconciliation merge on main containing the current Dev tip, and use its new SHA as the candidate. Do not reset/force-push Dev or merge Main back into Dev to claim validation of a different SHA. Manual POC image dispatch currently does not record the exact-SHA evidence tag needed for stable promotion.

A scheduled main build requires exact-commit main-push quality evidence and publishes the nightly channel but does not automatically deploy it to POC. POC deployment is triggered by a successful dev image build or an explicit workflow dispatch.

The reusable security workflow needs packages: read and logs in to GHCR with its job token when scanning a published candidate. Local pull-request image scans do not require registry login. An authentication or scanner error blocks deployment just as a failed vulnerability check does.

GitHub app and base image builds target linux/amd64, matching Azure. Runtime OS packages are still refreshed before compiling extensions for each candidate. Local Docker Compose builds docker/Dockerfile.app, and VS Code builds .devcontainer/Dockerfile; neither pulls the nightly application image.

To investigate a failed scan without repeating a successful build, manually run Security Gates with image set to the published ghcr.io/ansteorra/kmp@sha256:... reference. This audits the existing AMD64 image; it does not deploy it or replace the successful build, smoke-test, and POC evidence required for promotion.

Release to production

  1. Update app/CHANGELOG.md before POC validation.
  2. Publish a stable, non-prerelease v* GitHub Release targeting the exact commit validated in POC. Its notes must exactly match that changelog section.
  3. Release Docker Image verifies that the release commit is on official main, has successful quality-gate evidence, and has a poc-validated-<sha> image.
  4. The workflow applies semantic version, SHA, and stable channel tags to that POC-validated digest with docker buildx imagetools create. It does not rebuild the image or rerun the test suites.
  5. Review and approve the protected production environment deployment.
  6. Production imports and deploys the exact same digest, runs the same worker canary and migration contract, cuts over web, probes health, and aligns jobs.
  7. Verify the active Container Apps digest, critical tenant hosts/login, worker executions, Platform Admin, Redis-backed sessions, and new backup execution.

Do not edit release notes after POC validation without treating the result as a new candidate. Do not release another commit or digest.

Repository shorthand

Repository agents interpret:

Migration behavior during deployment

The reusable Azure deployment starts the migration job and requires this chain to finish before web cutover:

bin/cake migrations migrate &&
bin/cake schema_cache clear &&
bin/cake updateDatabase &&
bin/cake platform_migrate migrate &&
bin/cake schema_cache clear --connection platform &&
bin/cake platform secrets import-env &&
bin/cake platform backup-keys ensure --allow-read-only &&
bin/cake tenant migrate --all --include-suspended --fail-fast &&
bin/cake cache clear _cake_model_

Pending active or suspended tenant migrations create their normal recovery marker and encrypted backup. Current tenants are inspected and skipped without another backup. One tenant failure blocks cutover, and rerunning resumes by reinspecting state.

The optional release-manifest and nightly migration-drill tools are not wired into the current deployment workflow. Use them only as explicit staging/pilot steps unless the workflow is changed to generate and consume the manifest.

Rollback boundaries

The deployment workflow captures the current web and Container Apps Job definitions before cutover. The worker cutover helper can restore those runtime definitions and re-enable compatibility schedules. That rollback:

Before shifting traffic back to an older revision, confirm its supported schema and data contract. If data recovery is required, use the managed backup/restore workflow and its audit/TOTP/suspension guardrails. Never assume image rollback is database rollback.

Image tags and evidence

Reference Meaning
dev-<short-sha> Commit-specific release candidate
poc-<sha-or-dispatch>-<run> ACR deployment tag for a POC run
poc-validated-<sha> Digest that completed the automated POC deployment
vX.Y.Z / semantic tags GitHub Release references applied to the validated digest
sha-<short-sha> Release commit reference
dev, nightly, latest, beta Mutable convenience channels; not deployment evidence

Use the digest attached to the GitHub Release and the active Container Apps revision when recording evidence.

Historical self-hosted maintenance

The legacy management executables and updater distribution are retired and removed. Follow the legacy retirement instructions to disable installed copies, updater services and scheduled jobs before maintenance. Existing operators must take verified engine-appropriate backups, pin an immutable image, rehearse in a clone and define schema recovery explicitly. Historical plaintext backups require separate encryption, access review and retention cleanup.