From 2cd64c76766bf53075b7780a6f8d40df89cb35e5 Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Thu, 5 Mar 2026 10:46:07 +0100 Subject: [PATCH 1/8] feat: enhance backlog with Azure deployment planning and README update for Azure deployment option --- BACKLOG.md | 13 ++++++++++++- README.md | 6 ++++++ docs/wiki | 2 +- 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/BACKLOG.md b/BACKLOG.md index 92e2e38..ebf60f8 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -134,6 +134,17 @@ GetCapabilities() : List — declares what the module can do - [x] **~~E1-04~~** *(obsolete — SQLite removed)* ~~Create `data/` directory gitkeep in mate root so local SQLite file path works~~ - [ ] **E1-12** Sample data seeder — auto-create a sample agent, test suite, and 5 test cases on first startup when the database is empty - [ ] **E1-13** Azure Container Apps IaC — Bicep + `azd` template for one-command production deployment; includes WebUI + Worker containers, managed identity, Azure SQL, Service Bus +- [ ] **E1-13a** Full planning document (non-wiki): `docs/concepts/azure-container-landscape-plan.md` — target landscape, profile sizing, scale-to-zero strategy, implementation phases, cost model +- [ ] **E1-13b** Create `infra/azure/` IaC baseline — `main.bicep`, module split (container apps, postgres, storage, service bus, key vault, diagnostics), `azd` environment templates (`dev`, `staging`, `prod`) +- [ ] **E1-13c** Flexible sizing profiles (`XS`,`S`,`M`,`L`) — parameterized min/max replicas, CPU/memory, queue thresholds, and cooldown values (no hard-coded production sizing) +- [ ] **E1-13d** Implement Azure Service Bus `IMessageQueue` provider in `mate.Infrastructure.Azure` for durable cross-instance run execution +- [ ] **E1-13e** Refactor Worker execution path from DB polling to queue consumption (`test-runs`) so KEDA/ACA can scale worker replicas from zero based on queue depth +- [ ] **E1-13f** Add dedicated migration job in deployment flow; remove startup migration race between WebUI and Worker in cloud mode +- [ ] **E1-13g** Configure worker scale-to-zero (`minReplicas=0`) with queue-based scale rules; configure web profile (`prod min>=1`, non-prod optionally `min=0`) +- [ ] **E1-13h** Add non-prod cost mode automation — off-hours schedule to scale web/worker down, with morning warm-up schedule and override controls +- [ ] **E1-13i** Add DLQ and replay operations — dead-letter policy, replay command/runbook, and alerting for poison messages +- [ ] **E1-13j** Add production observability and cost governance — OpenTelemetry export, dashboards, budget alerts, and cost-anomaly alerts +- [ ] **E1-13k** Add production readiness runbooks — deployment/rollback, queue incident handling, backup/restore checks, key rotation, and scale policy tuning; seed with `docs/concepts/container-stack-update-runbook.md` --- @@ -646,4 +657,4 @@ All connectors implement `IAgentConnectorModule` with `ModuleId`, `DisplayName`, --- -*Last updated: 2026-03-04, v0.6.0 released* +*Last updated: 2026-03-05, v0.6.0 released* diff --git a/README.md b/README.md index cc34b9b..6ab4821 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,12 @@ Open ****. No login required in the default `Generic` aut > **PostgreSQL + Azurite** are always started alongside webui and worker — no extra flags required. The default `.env.template` values work out of the box for local development. +### Option C — Deploy to Azure (planned) + +[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](BACKLOG.md#e1--foundation--infrastructure) + +> Azure one-click deployment (Bicep + `azd`) is tracked in backlog item **E1-13** and is not yet shipped. + --- diff --git a/docs/wiki b/docs/wiki index 465a5f5..ccc713b 160000 --- a/docs/wiki +++ b/docs/wiki @@ -1 +1 @@ -Subproject commit 465a5f5782e9668f6b3e32212d02014223514e26 +Subproject commit ccc713b79b7f33995c00f24f5867c0950f625444 From 637d23159f550afd3d793e2050a813152c9acf95 Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Thu, 5 Mar 2026 11:04:25 +0100 Subject: [PATCH 2/8] feat: update E1-13 tasks for Azure deployment with IaC baseline and flexible sizing profiles --- BACKLOG.md | 80 +++++++++++++++++++++++++++++++++++++++++-- infra/azure/README.md | 47 +++++++++++++++++++++++++ 2 files changed, 125 insertions(+), 2 deletions(-) create mode 100644 infra/azure/README.md diff --git a/BACKLOG.md b/BACKLOG.md index ebf60f8..7e7e0d9 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -135,8 +135,8 @@ GetCapabilities() : List — declares what the module can do - [ ] **E1-12** Sample data seeder — auto-create a sample agent, test suite, and 5 test cases on first startup when the database is empty - [ ] **E1-13** Azure Container Apps IaC — Bicep + `azd` template for one-command production deployment; includes WebUI + Worker containers, managed identity, Azure SQL, Service Bus - [ ] **E1-13a** Full planning document (non-wiki): `docs/concepts/azure-container-landscape-plan.md` — target landscape, profile sizing, scale-to-zero strategy, implementation phases, cost model -- [ ] **E1-13b** Create `infra/azure/` IaC baseline — `main.bicep`, module split (container apps, postgres, storage, service bus, key vault, diagnostics), `azd` environment templates (`dev`, `staging`, `prod`) -- [ ] **E1-13c** Flexible sizing profiles (`XS`,`S`,`M`,`L`) — parameterized min/max replicas, CPU/memory, queue thresholds, and cooldown values (no hard-coded production sizing) +- [ ] **E1-13b** Create `infra/azure/` IaC baseline — `main.bicep`, module split (container apps, postgres, storage, service bus, key vault, diagnostics), `azd` templates with installer-driven environment scope selection +- [ ] **E1-13c** Flexible sizing profiles (`XS`,`S`,`M`,`L`) — parameterized min/max replicas, CPU/memory, queue thresholds, and cooldown values; internal development default is `dev` + `S` - [ ] **E1-13d** Implement Azure Service Bus `IMessageQueue` provider in `mate.Infrastructure.Azure` for durable cross-instance run execution - [ ] **E1-13e** Refactor Worker execution path from DB polling to queue consumption (`test-runs`) so KEDA/ACA can scale worker replicas from zero based on queue depth - [ ] **E1-13f** Add dedicated migration job in deployment flow; remove startup migration race between WebUI and Worker in cloud mode @@ -145,6 +145,82 @@ GetCapabilities() : List — declares what the module can do - [ ] **E1-13i** Add DLQ and replay operations — dead-letter policy, replay command/runbook, and alerting for poison messages - [ ] **E1-13j** Add production observability and cost governance — OpenTelemetry export, dashboards, budget alerts, and cost-anomaly alerts - [ ] **E1-13k** Add production readiness runbooks — deployment/rollback, queue incident handling, backup/restore checks, key rotation, and scale policy tuning; seed with `docs/concepts/container-stack-update-runbook.md` +- [ ] **E1-13l** Add installer prompt contract — installation asks for environment scope and size profile, validates allowed combinations, and writes deterministic deployment parameters + +#### E1-13 Implementation TODO Board (Execution Order) + +- [ ] **T1** Backlog hygiene + scope lock (`E1-13a`) + - [ ] Resolve duplicate E1 IDs (`E1-08`, `E1-09`) to unique IDs + - [ ] Freeze internal engineering scope to `dev` only + - [ ] Freeze internal engineering size to `S` + - [ ] Confirm installer options: scope + profile (`XS`,`S`,`M`,`L`) + +- [ ] **T2** IaC baseline (`E1-13b`) + - [ ] Create `infra/azure/` with Bicep modules (ACA, PostgreSQL, Blob, Service Bus, Key Vault, diagnostics) + - [ ] Add `azd` templates that can generate selected scope during installation + - [ ] Add CI validation for Bicep (lint/what-if) + +- [ ] **T3** Flexible sizing and installer prompt (`E1-13c`, `E1-13l`) + - [ ] Add parameter files for `XS`/`S`/`M`/`L` + - [ ] Wire installation prompt for scope and profile selection + - [ ] Validate selected scope/profile combinations and emit deterministic parameter files + - [ ] Document internal default: `dev` + `S` + +- [ ] **T4** Queue provider implementation (`E1-13d`) + - [ ] Implement Azure Service Bus `IMessageQueue` + - [ ] Keep WebUI enqueue path for `test-runs` + - [ ] Add integration tests for enqueue/consume/ack/abandon + +- [ ] **T5** Worker refactor to queue consumption (`E1-13e`) + - [ ] Replace DB polling with queue consume path in Azure mode + - [ ] Implement idempotency and safe retry behavior + - [ ] Validate concurrent processing limits + +- [ ] **T6** Migration job and startup race removal (`E1-13f`) + - [ ] Add dedicated migration step/job in deployment + - [ ] Remove migration race between WebUI and Worker + - [ ] Add deployment gate: app rollout only after migration success + +- [ ] **T7** Scale-to-zero and autoscaling (`E1-13g`) + - [ ] Worker `minReplicas=0` with queue-based scale rule + - [ ] Web profile: `prod min>=1`, non-prod optional `min=0` + - [ ] Tune thresholds/cooldowns from test telemetry + +- [ ] **T8** Non-prod cost mode automation (`E1-13h`) + - [ ] Add off-hours scale-down schedule + - [ ] Add morning warm-up schedule and override + - [ ] Capture baseline vs optimized cost report + +- [ ] **T9** DLQ and replay operations (`E1-13i`) + - [ ] Configure dead-letter + retry limits + - [ ] Add replay workflow/runbook + - [ ] Add alerts for poison-message spikes + +- [ ] **T10** Observability and governance (`E1-13j`) + - [ ] Enable OpenTelemetry + Azure Monitor dashboards + - [ ] Alert on queue lag, run failure rate, cold-start latency + - [ ] Configure budget + cost anomaly alerts + +- [ ] **T11** Operations runbooks (`E1-13k`) + - [ ] Deployment and rollback playbook + - [ ] Queue incident and DLQ replay playbook + - [ ] Backup/restore + key-rotation playbook + +- [ ] **T12** Health and readiness dependency (`E10-01`, `E10-02`) + - [ ] Implement `/health/live` + - [ ] Implement `/health/ready` with DB/blob/queue checks + - [ ] Wire probes into Azure Container Apps + +- [ ] **T13** Security dependency (`E1-08`) + - [ ] Implement Key Vault `ISecretService` + - [ ] Enforce Managed Identity in Azure mode + - [ ] Validate secret rotation and access policies + +- [ ] **T14** Release progression and go-live + - [ ] Validate internally on `dev` with `S` profile + - [ ] Validate installer-selected rollout scope and profile logic + - [ ] Hypercare window with daily performance/cost review + - [ ] Sign-off against E1-13 acceptance criteria --- diff --git a/infra/azure/README.md b/infra/azure/README.md new file mode 100644 index 0000000..6f43d70 --- /dev/null +++ b/infra/azure/README.md @@ -0,0 +1,47 @@ +# Azure Deployment Scaffold + +This folder contains the first implementation scaffold for `E1-13` Azure deployment. + +## Contents + +- `main.bicep`: orchestrates all modules +- `modules/`: modular Azure resources +- `parameters/`: size/profile parameter files +- `scripts/`: PowerShell deployment helpers (`what-if`, deploy) + +## Prerequisites + +- Azure CLI +- Bicep CLI (bundled with recent Azure CLI) +- Rights to create resources in the target subscription/tenant + +## Quick Start (dev + S) + +```powershell +pwsh ./infra/azure/scripts/deploy-azure.ps1 \ + -TenantId "" \ + -SubscriptionId "" \ + -Location "westeurope" \ + -EnvironmentName "dev" \ + -SizeProfile "s" \ + -ResourceGroupName "rg-mate-dev" \ + -DeploymentName "mate-dev-s" +``` + +## What-if + +```powershell +pwsh ./infra/azure/scripts/whatif-azure.ps1 \ + -TenantId "" \ + -SubscriptionId "" \ + -Location "westeurope" \ + -EnvironmentName "dev" \ + -SizeProfile "s" \ + -ResourceGroupName "rg-mate-dev" +``` + +## Notes + +- Internal engineering default remains `dev` + `s`. +- Parameter files for `xs`, `s`, `m`, `l` are included. +- The installer prompt flow (`E1-13l`) will later select scope/profile and emit deterministic parameters. From c08b437f4572b5ec6ea8a9bdcefaabb7fad154fe Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Sat, 7 Mar 2026 16:08:41 +0100 Subject: [PATCH 3/8] initial commit on adding azure infra deployment --- .gitignore | 3 + infra/azure/.env.template | 39 + infra/azure/main.bicep | 145 +++ infra/azure/main.json | 1053 +++++++++++++++++ infra/azure/modules/container-apps.bicep | 252 ++++ infra/azure/modules/keyvault.bicep | 25 + infra/azure/modules/monitoring.bicep | 30 + infra/azure/modules/postgres.bicep | 41 + infra/azure/modules/servicebus.bicep | 29 + infra/azure/modules/storage.bicep | 35 + infra/azure/parameters/profile-l.json | 15 + infra/azure/parameters/profile-m.json | 15 + infra/azure/parameters/profile-s.json | 15 + infra/azure/parameters/profile-xs.json | 15 + infra/azure/scripts/DEPLOYMENT.md | 211 ++++ infra/azure/scripts/check-prerequisites.ps1 | 91 ++ infra/azure/scripts/cleanup-rg.ps1 | 167 +++ infra/azure/scripts/deploy-whatif.ps1 | 197 +++ infra/azure/scripts/deploy.ps1 | 442 +++++++ infra/azure/scripts/setup-env.ps1 | 384 ++++++ .../azure/scripts/setup-keyvault-secrets.ps1 | 250 ++++ 21 files changed, 3454 insertions(+) create mode 100644 infra/azure/.env.template create mode 100644 infra/azure/main.bicep create mode 100644 infra/azure/main.json create mode 100644 infra/azure/modules/container-apps.bicep create mode 100644 infra/azure/modules/keyvault.bicep create mode 100644 infra/azure/modules/monitoring.bicep create mode 100644 infra/azure/modules/postgres.bicep create mode 100644 infra/azure/modules/servicebus.bicep create mode 100644 infra/azure/modules/storage.bicep create mode 100644 infra/azure/parameters/profile-l.json create mode 100644 infra/azure/parameters/profile-m.json create mode 100644 infra/azure/parameters/profile-s.json create mode 100644 infra/azure/parameters/profile-xs.json create mode 100644 infra/azure/scripts/DEPLOYMENT.md create mode 100644 infra/azure/scripts/check-prerequisites.ps1 create mode 100644 infra/azure/scripts/cleanup-rg.ps1 create mode 100644 infra/azure/scripts/deploy-whatif.ps1 create mode 100644 infra/azure/scripts/deploy.ps1 create mode 100644 infra/azure/scripts/setup-env.ps1 create mode 100644 infra/azure/scripts/setup-keyvault-secrets.ps1 diff --git a/.gitignore b/.gitignore index 3f5737a..22bde67 100644 --- a/.gitignore +++ b/.gitignore @@ -47,3 +47,6 @@ Thumbs.db #concept store docs/concepts/ +infra/azure/scripts/.pg-password +.gitignore +infra/azure/scripts/.credentials diff --git a/infra/azure/.env.template b/infra/azure/.env.template new file mode 100644 index 0000000..570eb50 --- /dev/null +++ b/infra/azure/.env.template @@ -0,0 +1,39 @@ +# Azure Deployment Environment Configuration +# Copy this file to .env and fill in your actual values +# IMPORTANT: .env is git-ignored and should NEVER be committed + +# Azure Tenant ID +# Get from: Azure Portal → Azure AD → Properties → Tenant ID +AZURE_TENANT_ID= + +# Azure Subscription ID +# Get from: Azure Portal → Subscriptions → Subscription ID +AZURE_SUBSCRIPTION_ID= + +# Azure Resource Group Name +# Will be created if it doesn't exist +AZURE_RESOURCE_GROUP=mate-dev-rg + +# Azure Region/Location (e.g., eastus, westeurope, australiaeast) +AZURE_LOCATION=eastus + +# Environment Name Prefix (for resource naming) +# Resources will be named: -[type] (e.g., mate-dev-aca, mate-dev-postgres) +AZURE_ENVIRONMENT_NAME=mate-dev + +# Deployment Profile Size: xs, s, m, or l +# s = development default (0.5 CPU, 1GB, 1-3 web replicas, 0-5 worker replicas) +# xs = testing (0.25 CPU, 0.5GB, 0-1 web replicas, 0-2 worker replicas) +# m = growth (1.0 CPU, 2GB, 2-6 web replicas, 0-10 worker replicas) +# l = high throughput (2.0 CPU, 4GB, 3-12 web replicas, 0-20 worker replicas) +AZURE_PROFILE=s + +# Container Image Tag (docker image version at ghcr.io) +AZURE_IMAGE_TAG=latest + +# PostgreSQL Admin Username +AZURE_POSTGRES_ADMIN_USER=pgadmin + +# PostgreSQL Admin Password (will be prompted interactively at deployment if not set here) +# WARNING: Setting this in .env is risky. Use interactive prompt instead. +# AZURE_POSTGRES_ADMIN_PASSWORD= diff --git a/infra/azure/main.bicep b/infra/azure/main.bicep new file mode 100644 index 0000000..5f71bf3 --- /dev/null +++ b/infra/azure/main.bicep @@ -0,0 +1,145 @@ +targetScope = 'resourceGroup' + +@description('Environment name, e.g. dev/staging/prod') +param environmentName string = 'dev' + +@description('Azure region for all resources') +param location string = resourceGroup().location + +@description('Container image tag to deploy') +param imageTag string = 'latest' + +@description('Worker min replicas (scale-to-zero enabled with 0)') +param workerMinReplicas int = 0 + +@description('Worker max replicas') +param workerMaxReplicas int = 5 + +@description('WebUI min replicas') +param webMinReplicas int = 1 + +@description('WebUI max replicas') +param webMaxReplicas int = 3 + +@description('WebUI CPU cores') +param webCpu string = '0.5' + +@description('WebUI memory') +param webMemory string = '1Gi' + +@description('Worker CPU cores') +param workerCpu string = '0.5' + +@description('Worker memory') +param workerMemory string = '1Gi' + +@description('Service Bus queue activation threshold for worker scaling') +param queueActivationThreshold int = 1 + +@description('If true, create PostgreSQL resources. Keep false until secure admin values are provided.') +param deployPostgres bool = false + +@description('PostgreSQL admin login (required only when deployPostgres=true)') +param postgresAdminLogin string = '' + +@secure() +@description('PostgreSQL admin password (required only when deployPostgres=true)') +param postgresAdminPassword string = '' + +@description('Entra ID application (client) ID for WebUI Easy Auth authentication') +param aadClientId string + +@description('Entra ID tenant ID for authentication issuer') +param aadTenantId string = subscription().tenantId + +@description('Azure Blob Storage container name for documents') +param blobContainerName string = 'mate-blobs' + +@description('Service Bus queue name for test runs') +param serviceBusQueueName string = 'test-runs' + +@description('PostgreSQL database name') +param postgresDatabaseName string = 'mate' + +var env = toLower(environmentName) +var baseName = env + +module monitoring './modules/monitoring.bicep' = { + name: 'monitoring-${env}' + params: { + location: location + baseName: baseName + } +} + +module storage './modules/storage.bicep' = { + name: 'storage-${env}' + params: { + location: location + baseName: baseName + containerName: blobContainerName + } +} + +module serviceBus './modules/servicebus.bicep' = { + name: 'servicebus-${env}' + params: { + location: location + baseName: baseName + queueName: serviceBusQueueName + } +} + +module keyVault './modules/keyvault.bicep' = { + name: 'keyvault-${env}' + params: { + location: location + baseName: baseName + } +} + +module postgres './modules/postgres.bicep' = if (deployPostgres) { + name: 'postgres-${env}' + params: { + location: location + baseName: baseName + administratorLogin: postgresAdminLogin + administratorPassword: postgresAdminPassword + databaseName: postgresDatabaseName + } +} + +module containerApps './modules/container-apps.bicep' = { + name: 'aca-${env}' + params: { + location: location + baseName: baseName + imageTag: imageTag + workspaceId: monitoring.outputs.workspaceId + appInsightsConnectionString: monitoring.outputs.appInsightsConnectionString + serviceBusNamespaceName: serviceBus.outputs.namespaceName + queueName: serviceBus.outputs.queueName + keyVaultName: keyVault.outputs.keyVaultName + aadClientId: aadClientId + aadTenantId: aadTenantId + workerMinReplicas: workerMinReplicas + workerMaxReplicas: workerMaxReplicas + webMinReplicas: webMinReplicas + webMaxReplicas: webMaxReplicas + webCpu: webCpu + webMemory: webMemory + workerCpu: workerCpu + workerMemory: workerMemory + queueActivationThreshold: queueActivationThreshold + postgresServerName: deployPostgres ? postgres!.outputs.serverName : '' + postgresDatabaseName: deployPostgres ? postgres!.outputs.databaseName : '' + postgresAdminLogin: postgresAdminLogin + postgresEnabled: deployPostgres + } +} + +output containerAppEnvironmentName string = containerApps.outputs.containerAppEnvironmentName +output webUiUrl string = containerApps.outputs.webUiUrl +output keyVaultName string = keyVault.outputs.keyVaultName +output serviceBusNamespace string = serviceBus.outputs.namespaceName +output storageAccountName string = storage.outputs.accountName diff --git a/infra/azure/main.json b/infra/azure/main.json new file mode 100644 index 0000000..22214b4 --- /dev/null +++ b/infra/azure/main.json @@ -0,0 +1,1053 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.41.2.15936", + "templateHash": "9925893720925210661" + } + }, + "parameters": { + "environmentName": { + "type": "string", + "defaultValue": "dev", + "metadata": { + "description": "Environment name, e.g. dev/staging/prod" + } + }, + "location": { + "type": "string", + "defaultValue": "[resourceGroup().location]", + "metadata": { + "description": "Azure region for all resources" + } + }, + "imageTag": { + "type": "string", + "defaultValue": "latest", + "metadata": { + "description": "Container image tag to deploy" + } + }, + "workerMinReplicas": { + "type": "int", + "defaultValue": 0, + "metadata": { + "description": "Worker min replicas (scale-to-zero enabled with 0)" + } + }, + "workerMaxReplicas": { + "type": "int", + "defaultValue": 5, + "metadata": { + "description": "Worker max replicas" + } + }, + "webMinReplicas": { + "type": "int", + "defaultValue": 1, + "metadata": { + "description": "WebUI min replicas" + } + }, + "webMaxReplicas": { + "type": "int", + "defaultValue": 3, + "metadata": { + "description": "WebUI max replicas" + } + }, + "webCpu": { + "type": "string", + "defaultValue": "0.5", + "metadata": { + "description": "WebUI CPU cores" + } + }, + "webMemory": { + "type": "string", + "defaultValue": "1Gi", + "metadata": { + "description": "WebUI memory" + } + }, + "workerCpu": { + "type": "string", + "defaultValue": "0.5", + "metadata": { + "description": "Worker CPU cores" + } + }, + "workerMemory": { + "type": "string", + "defaultValue": "1Gi", + "metadata": { + "description": "Worker memory" + } + }, + "queueActivationThreshold": { + "type": "int", + "defaultValue": 1, + "metadata": { + "description": "Service Bus queue activation threshold for worker scaling" + } + }, + "deployPostgres": { + "type": "bool", + "defaultValue": false, + "metadata": { + "description": "If true, create PostgreSQL resources. Keep false until secure admin values are provided." + } + }, + "postgresAdminLogin": { + "type": "string", + "defaultValue": "", + "metadata": { + "description": "PostgreSQL admin login (required only when deployPostgres=true)" + } + }, + "postgresAdminPassword": { + "type": "securestring", + "defaultValue": "", + "metadata": { + "description": "PostgreSQL admin password (required only when deployPostgres=true)" + } + }, + "aadClientId": { + "type": "string", + "metadata": { + "description": "Entra ID application (client) ID for WebUI Easy Auth authentication" + } + }, + "aadTenantId": { + "type": "string", + "defaultValue": "[subscription().tenantId]", + "metadata": { + "description": "Entra ID tenant ID for authentication issuer" + } + }, + "blobContainerName": { + "type": "string", + "defaultValue": "mate-blobs", + "metadata": { + "description": "Azure Blob Storage container name for documents" + } + }, + "serviceBusQueueName": { + "type": "string", + "defaultValue": "test-runs", + "metadata": { + "description": "Service Bus queue name for test runs" + } + }, + "postgresDatabaseName": { + "type": "string", + "defaultValue": "mate", + "metadata": { + "description": "PostgreSQL database name" + } + } + }, + "variables": { + "env": "[toLower(parameters('environmentName'))]", + "baseName": "[variables('env')]" + }, + "resources": [ + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('monitoring-{0}', variables('env'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('location')]" + }, + "baseName": { + "value": "[variables('baseName')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.41.2.15936", + "templateHash": "13426955776256582287" + } + }, + "parameters": { + "location": { + "type": "string" + }, + "baseName": { + "type": "string" + } + }, + "variables": { + "workspaceName": "[take(format('{0}-law', parameters('baseName')), 63)]", + "appInsightsName": "[take(format('{0}-appi', parameters('baseName')), 260)]" + }, + "resources": [ + { + "type": "Microsoft.OperationalInsights/workspaces", + "apiVersion": "2023-09-01", + "name": "[variables('workspaceName')]", + "location": "[parameters('location')]", + "properties": { + "sku": { + "name": "PerGB2018" + }, + "retentionInDays": 30 + } + }, + { + "type": "Microsoft.Insights/components", + "apiVersion": "2020-02-02", + "name": "[variables('appInsightsName')]", + "location": "[parameters('location')]", + "kind": "web", + "properties": { + "Application_Type": "web", + "WorkspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces', variables('workspaceName'))]", + "IngestionMode": "LogAnalytics" + }, + "dependsOn": [ + "[resourceId('Microsoft.OperationalInsights/workspaces', variables('workspaceName'))]" + ] + } + ], + "outputs": { + "workspaceId": { + "type": "string", + "value": "[resourceId('Microsoft.OperationalInsights/workspaces', variables('workspaceName'))]" + }, + "appInsightsConnectionString": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Insights/components', variables('appInsightsName')), '2020-02-02').ConnectionString]" + } + } + } + } + }, + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('storage-{0}', variables('env'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('location')]" + }, + "baseName": { + "value": "[variables('baseName')]" + }, + "containerName": { + "value": "[parameters('blobContainerName')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.41.2.15936", + "templateHash": "10059485585129083932" + } + }, + "parameters": { + "location": { + "type": "string" + }, + "baseName": { + "type": "string" + }, + "containerName": { + "type": "string", + "defaultValue": "mate-blobs" + } + }, + "variables": { + "accountName": "[toLower(replace(take(format('{0}st', parameters('baseName')), 24), '-', ''))]" + }, + "resources": [ + { + "type": "Microsoft.Storage/storageAccounts", + "apiVersion": "2023-05-01", + "name": "[variables('accountName')]", + "location": "[parameters('location')]", + "sku": { + "name": "Standard_LRS" + }, + "kind": "StorageV2", + "properties": { + "supportsHttpsTrafficOnly": true, + "minimumTlsVersion": "TLS1_2", + "allowBlobPublicAccess": false + } + }, + { + "type": "Microsoft.Storage/storageAccounts/blobServices", + "apiVersion": "2023-05-01", + "name": "[format('{0}/{1}', variables('accountName'), 'default')]", + "dependsOn": [ + "[resourceId('Microsoft.Storage/storageAccounts', variables('accountName'))]" + ] + }, + { + "type": "Microsoft.Storage/storageAccounts/blobServices/containers", + "apiVersion": "2023-05-01", + "name": "[format('{0}/{1}/{2}', variables('accountName'), 'default', parameters('containerName'))]", + "properties": { + "publicAccess": "None" + }, + "dependsOn": [ + "[resourceId('Microsoft.Storage/storageAccounts/blobServices', variables('accountName'), 'default')]" + ] + } + ], + "outputs": { + "accountName": { + "type": "string", + "value": "[variables('accountName')]" + }, + "blobContainerName": { + "type": "string", + "value": "[parameters('containerName')]" + } + } + } + } + }, + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('servicebus-{0}', variables('env'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('location')]" + }, + "baseName": { + "value": "[variables('baseName')]" + }, + "queueName": { + "value": "[parameters('serviceBusQueueName')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.41.2.15936", + "templateHash": "9629288634804811541" + } + }, + "parameters": { + "location": { + "type": "string" + }, + "baseName": { + "type": "string" + }, + "queueName": { + "type": "string", + "defaultValue": "test-runs" + } + }, + "variables": { + "namespaceName": "[take(format('sbns-{0}', parameters('baseName')), 50)]" + }, + "resources": [ + { + "type": "Microsoft.ServiceBus/namespaces", + "apiVersion": "2023-01-01-preview", + "name": "[variables('namespaceName')]", + "location": "[parameters('location')]", + "sku": { + "name": "Standard", + "tier": "Standard" + } + }, + { + "type": "Microsoft.ServiceBus/namespaces/queues", + "apiVersion": "2023-01-01-preview", + "name": "[format('{0}/{1}', variables('namespaceName'), parameters('queueName'))]", + "properties": { + "lockDuration": "PT5M", + "maxDeliveryCount": 5, + "deadLetteringOnMessageExpiration": true, + "defaultMessageTimeToLive": "P7D" + }, + "dependsOn": [ + "[resourceId('Microsoft.ServiceBus/namespaces', variables('namespaceName'))]" + ] + } + ], + "outputs": { + "namespaceName": { + "type": "string", + "value": "[variables('namespaceName')]" + }, + "queueName": { + "type": "string", + "value": "[parameters('queueName')]" + }, + "namespaceId": { + "type": "string", + "value": "[resourceId('Microsoft.ServiceBus/namespaces', variables('namespaceName'))]" + } + } + } + } + }, + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('keyvault-{0}', variables('env'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('location')]" + }, + "baseName": { + "value": "[variables('baseName')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.41.2.15936", + "templateHash": "17251467704421812372" + } + }, + "parameters": { + "location": { + "type": "string" + }, + "baseName": { + "type": "string" + } + }, + "variables": { + "keyVaultName": "[take(format('{0}-kv', parameters('baseName')), 24)]" + }, + "resources": [ + { + "type": "Microsoft.KeyVault/vaults", + "apiVersion": "2023-07-01", + "name": "[variables('keyVaultName')]", + "location": "[parameters('location')]", + "properties": { + "sku": { + "family": "A", + "name": "standard" + }, + "tenantId": "[tenant().tenantId]", + "enableRbacAuthorization": true, + "enabledForDeployment": false, + "enabledForTemplateDeployment": false, + "enabledForDiskEncryption": false, + "softDeleteRetentionInDays": 90, + "publicNetworkAccess": "Enabled" + } + } + ], + "outputs": { + "keyVaultName": { + "type": "string", + "value": "[variables('keyVaultName')]" + }, + "keyVaultId": { + "type": "string", + "value": "[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]" + } + } + } + } + }, + { + "condition": "[parameters('deployPostgres')]", + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('postgres-{0}', variables('env'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('location')]" + }, + "baseName": { + "value": "[variables('baseName')]" + }, + "administratorLogin": { + "value": "[parameters('postgresAdminLogin')]" + }, + "administratorPassword": { + "value": "[parameters('postgresAdminPassword')]" + }, + "databaseName": { + "value": "[parameters('postgresDatabaseName')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.41.2.15936", + "templateHash": "7628411096591491134" + } + }, + "parameters": { + "location": { + "type": "string" + }, + "baseName": { + "type": "string" + }, + "administratorLogin": { + "type": "string" + }, + "administratorPassword": { + "type": "securestring" + }, + "databaseName": { + "type": "string", + "defaultValue": "mate" + } + }, + "variables": { + "serverName": "[take(format('{0}-pg', parameters('baseName')), 63)]", + "dbName": "[parameters('databaseName')]" + }, + "resources": [ + { + "type": "Microsoft.DBforPostgreSQL/flexibleServers", + "apiVersion": "2023-06-01-preview", + "name": "[variables('serverName')]", + "location": "[parameters('location')]", + "sku": { + "name": "Standard_B1ms", + "tier": "Burstable" + }, + "properties": { + "version": "17", + "administratorLogin": "[parameters('administratorLogin')]", + "administratorLoginPassword": "[parameters('administratorPassword')]", + "storage": { + "storageSizeGB": 32 + }, + "backup": { + "backupRetentionDays": 7, + "geoRedundantBackup": "Disabled" + }, + "network": { + "publicNetworkAccess": "Enabled" + } + } + }, + { + "type": "Microsoft.DBforPostgreSQL/flexibleServers/databases", + "apiVersion": "2023-06-01-preview", + "name": "[format('{0}/{1}', variables('serverName'), variables('dbName'))]", + "dependsOn": [ + "[resourceId('Microsoft.DBforPostgreSQL/flexibleServers', variables('serverName'))]" + ] + } + ], + "outputs": { + "serverName": { + "type": "string", + "value": "[variables('serverName')]" + }, + "databaseName": { + "type": "string", + "value": "[variables('dbName')]" + } + } + } + } + }, + { + "type": "Microsoft.Resources/deployments", + "apiVersion": "2025-04-01", + "name": "[format('aca-{0}', variables('env'))]", + "properties": { + "expressionEvaluationOptions": { + "scope": "inner" + }, + "mode": "Incremental", + "parameters": { + "location": { + "value": "[parameters('location')]" + }, + "baseName": { + "value": "[variables('baseName')]" + }, + "imageTag": { + "value": "[parameters('imageTag')]" + }, + "workspaceId": { + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('monitoring-{0}', variables('env'))), '2025-04-01').outputs.workspaceId.value]" + }, + "appInsightsConnectionString": { + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('monitoring-{0}', variables('env'))), '2025-04-01').outputs.appInsightsConnectionString.value]" + }, + "serviceBusNamespaceName": { + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('servicebus-{0}', variables('env'))), '2025-04-01').outputs.namespaceName.value]" + }, + "queueName": { + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('servicebus-{0}', variables('env'))), '2025-04-01').outputs.queueName.value]" + }, + "keyVaultName": { + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('keyvault-{0}', variables('env'))), '2025-04-01').outputs.keyVaultName.value]" + }, + "aadClientId": { + "value": "[parameters('aadClientId')]" + }, + "aadTenantId": { + "value": "[parameters('aadTenantId')]" + }, + "workerMinReplicas": { + "value": "[parameters('workerMinReplicas')]" + }, + "workerMaxReplicas": { + "value": "[parameters('workerMaxReplicas')]" + }, + "webMinReplicas": { + "value": "[parameters('webMinReplicas')]" + }, + "webMaxReplicas": { + "value": "[parameters('webMaxReplicas')]" + }, + "webCpu": { + "value": "[parameters('webCpu')]" + }, + "webMemory": { + "value": "[parameters('webMemory')]" + }, + "workerCpu": { + "value": "[parameters('workerCpu')]" + }, + "workerMemory": { + "value": "[parameters('workerMemory')]" + }, + "queueActivationThreshold": { + "value": "[parameters('queueActivationThreshold')]" + }, + "postgresServerName": "[if(parameters('deployPostgres'), createObject('value', reference(resourceId('Microsoft.Resources/deployments', format('postgres-{0}', variables('env'))), '2025-04-01').outputs.serverName.value), createObject('value', ''))]", + "postgresDatabaseName": "[if(parameters('deployPostgres'), createObject('value', reference(resourceId('Microsoft.Resources/deployments', format('postgres-{0}', variables('env'))), '2025-04-01').outputs.databaseName.value), createObject('value', ''))]", + "postgresAdminLogin": { + "value": "[parameters('postgresAdminLogin')]" + }, + "postgresEnabled": { + "value": "[parameters('deployPostgres')]" + } + }, + "template": { + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "metadata": { + "_generator": { + "name": "bicep", + "version": "0.41.2.15936", + "templateHash": "96034003002418543" + } + }, + "parameters": { + "location": { + "type": "string" + }, + "baseName": { + "type": "string" + }, + "imageTag": { + "type": "string" + }, + "workspaceId": { + "type": "string" + }, + "appInsightsConnectionString": { + "type": "string" + }, + "serviceBusNamespaceName": { + "type": "string" + }, + "queueName": { + "type": "string" + }, + "keyVaultName": { + "type": "string" + }, + "aadClientId": { + "type": "string" + }, + "aadTenantId": { + "type": "string" + }, + "workerMinReplicas": { + "type": "int" + }, + "workerMaxReplicas": { + "type": "int" + }, + "webMinReplicas": { + "type": "int" + }, + "webMaxReplicas": { + "type": "int" + }, + "webCpu": { + "type": "string" + }, + "webMemory": { + "type": "string" + }, + "workerCpu": { + "type": "string" + }, + "workerMemory": { + "type": "string" + }, + "queueActivationThreshold": { + "type": "int" + }, + "postgresServerName": { + "type": "string", + "defaultValue": "" + }, + "postgresDatabaseName": { + "type": "string", + "defaultValue": "" + }, + "postgresAdminLogin": { + "type": "string", + "defaultValue": "" + }, + "postgresEnabled": { + "type": "bool", + "defaultValue": false + } + }, + "variables": { + "caeName": "[take(format('{0}-cae', parameters('baseName')), 32)]", + "webAppName": "[take(format('{0}-webui', parameters('baseName')), 32)]", + "workerAppName": "[take(format('{0}-worker', parameters('baseName')), 32)]", + "ghcrBase": "ghcr.io/holgerimbery" + }, + "resources": [ + { + "type": "Microsoft.App/managedEnvironments", + "apiVersion": "2024-03-01", + "name": "[variables('caeName')]", + "location": "[parameters('location')]", + "properties": { + "appLogsConfiguration": { + "destination": "log-analytics", + "logAnalyticsConfiguration": { + "customerId": "[reference(parameters('workspaceId'), '2023-09-01').customerId]", + "sharedKey": "[listKeys(parameters('workspaceId'), '2023-09-01').primarySharedKey]" + } + } + } + }, + { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[take(format('{0}-web-mi', parameters('baseName')), 128)]", + "location": "[parameters('location')]" + }, + { + "type": "Microsoft.ManagedIdentity/userAssignedIdentities", + "apiVersion": "2023-01-31", + "name": "[take(format('{0}-worker-mi', parameters('baseName')), 128)]", + "location": "[parameters('location')]" + }, + { + "type": "Microsoft.App/containerApps", + "apiVersion": "2024-03-01", + "name": "[variables('webAppName')]", + "location": "[parameters('location')]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-web-mi', parameters('baseName')), 128)))]": {} + } + }, + "properties": { + "environmentId": "[resourceId('Microsoft.App/managedEnvironments', variables('caeName'))]", + "configuration": { + "ingress": { + "external": true, + "targetPort": 8080, + "transport": "auto" + }, + "activeRevisionsMode": "Single", + "secrets": [ + { + "name": "azuread-client-secret", + "identity": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-web-mi', parameters('baseName')), 128))]", + "keyVaultUrl": "[format('https://{0}.{1}/secrets/azuread-client-secret', parameters('keyVaultName'), environment().suffixes.keyvaultDns)]" + } + ] + }, + "template": { + "containers": [ + { + "name": "webui", + "image": "[format('{0}/mate-webui:{1}', variables('ghcrBase'), parameters('imageTag'))]", + "resources": { + "cpu": "[json(parameters('webCpu'))]", + "memory": "[parameters('webMemory')]" + }, + "env": [ + { + "name": "ASPNETCORE_ENVIRONMENT", + "value": "Production" + }, + { + "name": "ASPNETCORE_FORWARDEDHEADERS_ENABLED", + "value": "true" + }, + { + "name": "ASPNETCORE_URLS", + "value": "http://+:8080" + }, + { + "name": "Authentication__Scheme", + "value": "EntraId" + }, + { + "name": "AzureAd__TenantId", + "value": "[parameters('aadTenantId')]" + }, + { + "name": "AzureAd__ClientId", + "value": "[parameters('aadClientId')]" + }, + { + "name": "AzureAd__Instance", + "value": "[environment().authentication.loginEndpoint]" + }, + { + "name": "AzureAd__ClientSecret", + "secretRef": "azuread-client-secret" + }, + { + "name": "AzureAd__CallbackPath", + "value": "/signin-oidc" + }, + { + "name": "AzureAd__SignedOutCallbackPath", + "value": "/signout-callback-oidc" + }, + { + "name": "Monitoring__Provider", + "value": "ApplicationInsights" + }, + { + "name": "APPLICATIONINSIGHTS_CONNECTION_STRING", + "value": "[parameters('appInsightsConnectionString')]" + }, + { + "name": "Infrastructure__Provider", + "value": "Azure" + }, + { + "name": "AzureInfrastructure__BlobContainerName", + "value": "mate-blobs" + }, + { + "name": "AzureInfrastructure__BlobConnectionString", + "value": "USE-KEYVAULT-REFERENCE" + }, + { + "name": "ConnectionStrings__Default", + "value": "[if(parameters('postgresEnabled'), format('Host={0}.postgres.database.azure.com;Database={1};Username={2};Password=USE-KEYVAULT-REFERENCE;SSL Mode=Require', parameters('postgresServerName'), parameters('postgresDatabaseName'), parameters('postgresAdminLogin')), '')]" + } + ] + } + ], + "scale": { + "minReplicas": "[parameters('webMinReplicas')]", + "maxReplicas": "[parameters('webMaxReplicas')]" + } + } + }, + "dependsOn": [ + "[resourceId('Microsoft.App/managedEnvironments', variables('caeName'))]", + "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-web-mi', parameters('baseName')), 128))]" + ] + }, + { + "type": "Microsoft.App/containerApps/authConfigs", + "apiVersion": "2024-03-01", + "name": "[format('{0}/{1}', variables('webAppName'), 'current')]", + "properties": { + "platform": { + "enabled": false + } + }, + "dependsOn": [ + "[resourceId('Microsoft.App/containerApps', variables('webAppName'))]" + ] + }, + { + "type": "Microsoft.App/containerApps", + "apiVersion": "2024-03-01", + "name": "[variables('workerAppName')]", + "location": "[parameters('location')]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[format('{0}', resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-worker-mi', parameters('baseName')), 128)))]": {} + } + }, + "properties": { + "environmentId": "[resourceId('Microsoft.App/managedEnvironments', variables('caeName'))]", + "configuration": { + "activeRevisionsMode": "Single" + }, + "template": { + "containers": [ + { + "name": "worker", + "image": "[format('{0}/mate-worker:{1}', variables('ghcrBase'), parameters('imageTag'))]", + "resources": { + "cpu": "[json(parameters('workerCpu'))]", + "memory": "[parameters('workerMemory')]" + }, + "env": [ + { + "name": "DOTNET_ENVIRONMENT", + "value": "Production" + }, + { + "name": "Infrastructure__Provider", + "value": "Azure" + }, + { + "name": "AzureInfrastructure__BlobContainerName", + "value": "mate-blobs" + }, + { + "name": "AzureInfrastructure__BlobConnectionString", + "value": "USE-KEYVAULT-REFERENCE" + }, + { + "name": "ConnectionStrings__Default", + "value": "[if(parameters('postgresEnabled'), format('Host={0}.postgres.database.azure.com;Database={1};Username={2};Password=USE-KEYVAULT-REFERENCE;SSL Mode=Require', parameters('postgresServerName'), parameters('postgresDatabaseName'), parameters('postgresAdminLogin')), '')]" + } + ] + } + ], + "scale": { + "minReplicas": "[parameters('workerMinReplicas')]", + "maxReplicas": "[parameters('workerMaxReplicas')]", + "rules": [ + { + "name": "servicebus-queue-rule", + "custom": { + "type": "azure-servicebus", + "metadata": { + "namespace": "[parameters('serviceBusNamespaceName')]", + "queueName": "[parameters('queueName')]", + "messageCount": "[string(parameters('queueActivationThreshold'))]" + }, + "auth": [ + { + "secretRef": "todo-servicebus-connection", + "triggerParameter": "connection" + } + ] + } + } + ] + } + } + }, + "dependsOn": [ + "[resourceId('Microsoft.App/managedEnvironments', variables('caeName'))]", + "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-worker-mi', parameters('baseName')), 128))]" + ] + } + ], + "outputs": { + "containerAppEnvironmentName": { + "type": "string", + "value": "[variables('caeName')]" + }, + "webUiUrl": { + "type": "string", + "value": "[format('https://{0}', reference(resourceId('Microsoft.App/containerApps', variables('webAppName')), '2024-03-01').configuration.ingress.fqdn)]" + }, + "webIdentityPrincipalId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-web-mi', parameters('baseName')), 128)), '2023-01-31').principalId]" + }, + "workerIdentityPrincipalId": { + "type": "string", + "value": "[reference(resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-worker-mi', parameters('baseName')), 128)), '2023-01-31').principalId]" + } + } + } + }, + "dependsOn": [ + "[resourceId('Microsoft.Resources/deployments', format('keyvault-{0}', variables('env')))]", + "[resourceId('Microsoft.Resources/deployments', format('monitoring-{0}', variables('env')))]", + "[resourceId('Microsoft.Resources/deployments', format('postgres-{0}', variables('env')))]", + "[resourceId('Microsoft.Resources/deployments', format('servicebus-{0}', variables('env')))]" + ] + } + ], + "outputs": { + "containerAppEnvironmentName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('aca-{0}', variables('env'))), '2025-04-01').outputs.containerAppEnvironmentName.value]" + }, + "webUiUrl": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('aca-{0}', variables('env'))), '2025-04-01').outputs.webUiUrl.value]" + }, + "keyVaultName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('keyvault-{0}', variables('env'))), '2025-04-01').outputs.keyVaultName.value]" + }, + "serviceBusNamespace": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('servicebus-{0}', variables('env'))), '2025-04-01').outputs.namespaceName.value]" + }, + "storageAccountName": { + "type": "string", + "value": "[reference(resourceId('Microsoft.Resources/deployments', format('storage-{0}', variables('env'))), '2025-04-01').outputs.accountName.value]" + } + } +} \ No newline at end of file diff --git a/infra/azure/modules/container-apps.bicep b/infra/azure/modules/container-apps.bicep new file mode 100644 index 0000000..f889904 --- /dev/null +++ b/infra/azure/modules/container-apps.bicep @@ -0,0 +1,252 @@ +param location string +param baseName string +param imageTag string +param workspaceId string +param appInsightsConnectionString string +param serviceBusNamespaceName string +param queueName string +param keyVaultName string +param aadClientId string +param aadTenantId string +param workerMinReplicas int +param workerMaxReplicas int +param webMinReplicas int +param webMaxReplicas int +param webCpu string +param webMemory string +param workerCpu string +param workerMemory string +param queueActivationThreshold int +param postgresServerName string = '' +param postgresDatabaseName string = '' +param postgresAdminLogin string = '' +param postgresEnabled bool = false + +var caeName = take('${baseName}-cae', 32) +var webAppName = take('${baseName}-webui', 32) +var workerAppName = take('${baseName}-worker', 32) +var ghcrBase = 'ghcr.io/holgerimbery' + +resource cae 'Microsoft.App/managedEnvironments@2024-03-01' = { + name: caeName + location: location + properties: { + appLogsConfiguration: { + destination: 'log-analytics' + logAnalyticsConfiguration: { + customerId: reference(workspaceId, '2023-09-01').customerId + sharedKey: listKeys(workspaceId, '2023-09-01').primarySharedKey + } + } + } +} + +resource webIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { + name: take('${baseName}-web-mi', 128) + location: location +} + +resource workerIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { + name: take('${baseName}-worker-mi', 128) + location: location +} + +resource webApp 'Microsoft.App/containerApps@2024-03-01' = { + name: webAppName + location: location + identity: { + type: 'UserAssigned' + userAssignedIdentities: { + '${webIdentity.id}': {} + } + } + properties: { + environmentId: cae.id + configuration: { + ingress: { + external: true + targetPort: 8080 + transport: 'auto' + } + activeRevisionsMode: 'Single' + secrets: [ + { + name: 'azuread-client-secret' + identity: webIdentity.id + keyVaultUrl: 'https://${keyVaultName}.${environment().suffixes.keyvaultDns}/secrets/azuread-client-secret' + } + ] + } + template: { + containers: [ + { + name: 'webui' + image: '${ghcrBase}/mate-webui:${imageTag}' + resources: { + cpu: json(webCpu) + memory: webMemory + } + env: [ + { + name: 'ASPNETCORE_ENVIRONMENT' + value: 'Production' + } + { + name: 'ASPNETCORE_FORWARDEDHEADERS_ENABLED' + value: 'true' + } + { + name: 'ASPNETCORE_URLS' + value: 'http://+:8080' + } + { + name: 'Authentication__Scheme' + value: 'EntraId' + } + { + name: 'AzureAd__TenantId' + value: aadTenantId + } + { + name: 'AzureAd__ClientId' + value: aadClientId + } + { + name: 'AzureAd__Instance' + value: environment().authentication.loginEndpoint + } + { + name: 'AzureAd__ClientSecret' + secretRef: 'azuread-client-secret' + } + { + name: 'AzureAd__CallbackPath' + value: '/signin-oidc' + } + { + name: 'AzureAd__SignedOutCallbackPath' + value: '/signout-callback-oidc' + } + { + name: 'Monitoring__Provider' + value: 'ApplicationInsights' + } + { + name: 'APPLICATIONINSIGHTS_CONNECTION_STRING' + value: appInsightsConnectionString + } + { + name: 'Infrastructure__Provider' + value: 'Azure' + } + { + name: 'AzureInfrastructure__BlobContainerName' + value: 'mate-blobs' + } + { + name: 'AzureInfrastructure__BlobConnectionString' + value: 'USE-KEYVAULT-REFERENCE' + } + { + name: 'ConnectionStrings__Default' + value: postgresEnabled ? 'Host=${postgresServerName}.postgres.database.azure.com;Database=${postgresDatabaseName};Username=${postgresAdminLogin};Password=USE-KEYVAULT-REFERENCE;SSL Mode=Require' : '' + } + ] + } + ] + scale: { + minReplicas: webMinReplicas + maxReplicas: webMaxReplicas + } + } + } +} + +resource webAppAuthConfig 'Microsoft.App/containerApps/authConfigs@2024-03-01' = { + parent: webApp + name: 'current' + properties: { + platform: { + enabled: false + } + } +} + +resource workerApp 'Microsoft.App/containerApps@2024-03-01' = { + name: workerAppName + location: location + identity: { + type: 'UserAssigned' + userAssignedIdentities: { + '${workerIdentity.id}': {} + } + } + properties: { + environmentId: cae.id + configuration: { + activeRevisionsMode: 'Single' + } + template: { + containers: [ + { + name: 'worker' + image: '${ghcrBase}/mate-worker:${imageTag}' + resources: { + cpu: json(workerCpu) + memory: workerMemory + } + env: [ + { + name: 'DOTNET_ENVIRONMENT' + value: 'Production' + } + { + name: 'Infrastructure__Provider' + value: 'Azure' + } + { + name: 'AzureInfrastructure__BlobContainerName' + value: 'mate-blobs' + } + { + name: 'AzureInfrastructure__BlobConnectionString' + value: 'USE-KEYVAULT-REFERENCE' + } + { + name: 'ConnectionStrings__Default' + value: postgresEnabled ? 'Host=${postgresServerName}.postgres.database.azure.com;Database=${postgresDatabaseName};Username=${postgresAdminLogin};Password=USE-KEYVAULT-REFERENCE;SSL Mode=Require' : '' + } + ] + } + ] + scale: { + minReplicas: workerMinReplicas + maxReplicas: workerMaxReplicas + rules: [ + { + name: 'servicebus-queue-rule' + custom: { + type: 'azure-servicebus' + metadata: { + namespace: serviceBusNamespaceName + queueName: queueName + messageCount: string(queueActivationThreshold) + } + auth: [ + { + secretRef: 'todo-servicebus-connection' + triggerParameter: 'connection' + } + ] + } + } + ] + } + } + } +} + +output containerAppEnvironmentName string = cae.name +output webUiUrl string = 'https://${webApp.properties.configuration.ingress.fqdn}' +output webIdentityPrincipalId string = webIdentity.properties.principalId +output workerIdentityPrincipalId string = workerIdentity.properties.principalId diff --git a/infra/azure/modules/keyvault.bicep b/infra/azure/modules/keyvault.bicep new file mode 100644 index 0000000..62799ca --- /dev/null +++ b/infra/azure/modules/keyvault.bicep @@ -0,0 +1,25 @@ +param location string +param baseName string + +var keyVaultName = take('${baseName}-kv', 24) + +resource keyVault 'Microsoft.KeyVault/vaults@2023-07-01' = { + name: keyVaultName + location: location + properties: { + sku: { + family: 'A' + name: 'standard' + } + tenantId: tenant().tenantId + enableRbacAuthorization: true + enabledForDeployment: false + enabledForTemplateDeployment: false + enabledForDiskEncryption: false + softDeleteRetentionInDays: 90 + publicNetworkAccess: 'Enabled' + } +} + +output keyVaultName string = keyVault.name +output keyVaultId string = keyVault.id diff --git a/infra/azure/modules/monitoring.bicep b/infra/azure/modules/monitoring.bicep new file mode 100644 index 0000000..fdb5ca9 --- /dev/null +++ b/infra/azure/modules/monitoring.bicep @@ -0,0 +1,30 @@ +param location string +param baseName string + +var workspaceName = take('${baseName}-law', 63) +var appInsightsName = take('${baseName}-appi', 260) + +resource workspace 'Microsoft.OperationalInsights/workspaces@2023-09-01' = { + name: workspaceName + location: location + properties: { + sku: { + name: 'PerGB2018' + } + retentionInDays: 30 + } +} + +resource appInsights 'Microsoft.Insights/components@2020-02-02' = { + name: appInsightsName + location: location + kind: 'web' + properties: { + Application_Type: 'web' + WorkspaceResourceId: workspace.id + IngestionMode: 'LogAnalytics' + } +} + +output workspaceId string = workspace.id +output appInsightsConnectionString string = appInsights.properties.ConnectionString diff --git a/infra/azure/modules/postgres.bicep b/infra/azure/modules/postgres.bicep new file mode 100644 index 0000000..6f4e8c9 --- /dev/null +++ b/infra/azure/modules/postgres.bicep @@ -0,0 +1,41 @@ +param location string +param baseName string +param administratorLogin string +@secure() +param administratorPassword string +param databaseName string = 'mate' + +var serverName = take('${baseName}-pg', 63) +var dbName = databaseName + +resource server 'Microsoft.DBforPostgreSQL/flexibleServers@2023-06-01-preview' = { + name: serverName + location: location + sku: { + name: 'Standard_B1ms' + tier: 'Burstable' + } + properties: { + version: '17' + administratorLogin: administratorLogin + administratorLoginPassword: administratorPassword + storage: { + storageSizeGB: 32 + } + backup: { + backupRetentionDays: 7 + geoRedundantBackup: 'Disabled' + } + network: { + publicNetworkAccess: 'Enabled' + } + } +} + +resource db 'Microsoft.DBforPostgreSQL/flexibleServers/databases@2023-06-01-preview' = { + name: dbName + parent: server +} + +output serverName string = server.name +output databaseName string = db.name diff --git a/infra/azure/modules/servicebus.bicep b/infra/azure/modules/servicebus.bicep new file mode 100644 index 0000000..9a1b132 --- /dev/null +++ b/infra/azure/modules/servicebus.bicep @@ -0,0 +1,29 @@ +param location string +param baseName string +param queueName string = 'test-runs' + +var namespaceName = take('sbns-${baseName}', 50) + +resource namespace 'Microsoft.ServiceBus/namespaces@2023-01-01-preview' = { + name: namespaceName + location: location + sku: { + name: 'Standard' + tier: 'Standard' + } +} + +resource queue 'Microsoft.ServiceBus/namespaces/queues@2023-01-01-preview' = { + name: queueName + parent: namespace + properties: { + lockDuration: 'PT5M' + maxDeliveryCount: 5 + deadLetteringOnMessageExpiration: true + defaultMessageTimeToLive: 'P7D' + } +} + +output namespaceName string = namespace.name +output queueName string = queue.name +output namespaceId string = namespace.id diff --git a/infra/azure/modules/storage.bicep b/infra/azure/modules/storage.bicep new file mode 100644 index 0000000..b16a7d7 --- /dev/null +++ b/infra/azure/modules/storage.bicep @@ -0,0 +1,35 @@ +param location string +param baseName string +param containerName string = 'mate-blobs' + +var accountName = toLower(replace(take('${baseName}st', 24), '-', '')) + +resource storage 'Microsoft.Storage/storageAccounts@2023-05-01' = { + name: accountName + location: location + sku: { + name: 'Standard_LRS' + } + kind: 'StorageV2' + properties: { + supportsHttpsTrafficOnly: true + minimumTlsVersion: 'TLS1_2' + allowBlobPublicAccess: false + } +} + +resource blobService 'Microsoft.Storage/storageAccounts/blobServices@2023-05-01' = { + name: 'default' + parent: storage +} + +resource blobContainer 'Microsoft.Storage/storageAccounts/blobServices/containers@2023-05-01' = { + name: containerName + parent: blobService + properties: { + publicAccess: 'None' + } +} + +output accountName string = storage.name +output blobContainerName string = blobContainer.name diff --git a/infra/azure/parameters/profile-l.json b/infra/azure/parameters/profile-l.json new file mode 100644 index 0000000..01d2434 --- /dev/null +++ b/infra/azure/parameters/profile-l.json @@ -0,0 +1,15 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "webMinReplicas": { "value": 3 }, + "webMaxReplicas": { "value": 12 }, + "workerMinReplicas": { "value": 0 }, + "workerMaxReplicas": { "value": 20 }, + "webCpu": { "value": "2.0" }, + "webMemory": { "value": "4Gi" }, + "workerCpu": { "value": "2.0" }, + "workerMemory": { "value": "4Gi" }, + "queueActivationThreshold": { "value": 4 } + } +} diff --git a/infra/azure/parameters/profile-m.json b/infra/azure/parameters/profile-m.json new file mode 100644 index 0000000..514bcdd --- /dev/null +++ b/infra/azure/parameters/profile-m.json @@ -0,0 +1,15 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "webMinReplicas": { "value": 2 }, + "webMaxReplicas": { "value": 6 }, + "workerMinReplicas": { "value": 0 }, + "workerMaxReplicas": { "value": 10 }, + "webCpu": { "value": "1.0" }, + "webMemory": { "value": "2Gi" }, + "workerCpu": { "value": "1.0" }, + "workerMemory": { "value": "2Gi" }, + "queueActivationThreshold": { "value": 2 } + } +} diff --git a/infra/azure/parameters/profile-s.json b/infra/azure/parameters/profile-s.json new file mode 100644 index 0000000..d3d7d34 --- /dev/null +++ b/infra/azure/parameters/profile-s.json @@ -0,0 +1,15 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "webMinReplicas": { "value": 1 }, + "webMaxReplicas": { "value": 3 }, + "workerMinReplicas": { "value": 0 }, + "workerMaxReplicas": { "value": 5 }, + "webCpu": { "value": "0.5" }, + "webMemory": { "value": "1Gi" }, + "workerCpu": { "value": "0.5" }, + "workerMemory": { "value": "1Gi" }, + "queueActivationThreshold": { "value": 1 } + } +} diff --git a/infra/azure/parameters/profile-xs.json b/infra/azure/parameters/profile-xs.json new file mode 100644 index 0000000..62fa601 --- /dev/null +++ b/infra/azure/parameters/profile-xs.json @@ -0,0 +1,15 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "webMinReplicas": { "value": 1 }, + "webMaxReplicas": { "value": 1 }, + "workerMinReplicas": { "value": 0 }, + "workerMaxReplicas": { "value": 2 }, + "webCpu": { "value": "0.25" }, + "webMemory": { "value": "0.5Gi" }, + "workerCpu": { "value": "0.25" }, + "workerMemory": { "value": "0.5Gi" }, + "queueActivationThreshold": { "value": 1 } + } +} diff --git a/infra/azure/scripts/DEPLOYMENT.md b/infra/azure/scripts/DEPLOYMENT.md new file mode 100644 index 0000000..8573c31 --- /dev/null +++ b/infra/azure/scripts/DEPLOYMENT.md @@ -0,0 +1,211 @@ +# Azure Deployment Workflow + +This directory contains PowerShell helper scripts to deploy the Mate infrastructure to Azure. + +## Overview + +The deployment process is split into two phases: + +1. **What-If (Dry-Run)**: Preview resources without creating them +2. **Deploy (Live)**: Create resources in Azure + +## Prerequisites + +**Required Tools:** +- Azure CLI (version 2.50+) — install via `winget install Microsoft.AzureCLI` +- Bicep CLI (included with Azure CLI) +- PowerShell 5.1+ (PowerShell 7+ recommended) + +**Required Access:** +- Azure tenant with admin consent rights +- Ability to create service principals and assign roles +- Subscription with remaining resource quota + +**Information You'll Need:** +- Tenant ID (Azure Portal → Azure AD → Properties) +- Subscription ID (Azure Portal → Subscriptions) +- Resource Group name (will be created if it doesn't exist) +- PostgreSQL admin password (prompted interactively, never stored) + +## Quick Start + +### 1. Setup Environment Variables + +Store your Azure tenant, subscription, and resource group information locally (never in git): + +```powershell +.\setup-env.ps1 +``` + +This interactive wizard creates a `.env` file with your settings: +- **Save location:** `infra/azure/.env` (automatically git-ignored) +- **What gets stored:** Tenant ID, Subscription ID, Resource Group, Location, etc. +- **Security:** `.env` is never committed to git + +See `.env.template` for all available configuration options. + +### 2. Check Prerequisites + +```powershell +.\check-prerequisites.ps1 +``` + +Validates that Azure CLI, Bicep, and PowerShell are installed. + +### 3. Preview Deployment (What-If) + +```powershell +.\deploy-whatif.ps1 +``` + +**This will:** +- Use values from `.env` automatically (no parameters needed) +- NOT create any Azure resources +- NOT modify your Azure account +- Show exactly what WOULD be created +- Display resource names, locations, and estimated costs + +**To override .env values**, pass parameters: + +```powershell +.\deploy-whatif.ps1 -Location 'westeurope' -Profile 'm' +``` + +**Output:** Review carefully for: +- Resource names and locations +- Capacity and scaling settings +- Any errors or missing parameters + +### 4. Deploy to Azure (Live) + +```powershell +.\deploy.ps1 +``` + +**This will:** +- Use values from `.env` automatically (no parameters needed) +- Prompt for PostgreSQL admin password (never stored, never logged) +- Create resource group (if needed) +- Deploy all Azure resources +- Display post-deployment tasks + +**To override .env values**, pass parameters: + +```powershell +.\deploy.ps1 -Location 'westeurope' -Profile 'm' +``` + +**Warning:** This creates real Azure resources and incurs costs. Always run `deploy-whatif.ps1` first. + +## Deployment Profiles + +| Profile | Web Min | Web Max | Worker Max | CPU | Memory | Use Case | +|---------|---------|---------|------------|-----|--------|----------| +| `xs` | 0 | 1 | 2 | 0.25 | 0.5 GB | Testing, lowest cost | +| `s` | 1 | 3 | 5 | 0.5 | 1 GB | **Default for dev** | +| `m` | 2 | 6 | 10 | 1.0 | 2 GB | Growth production | +| `l` | 3 | 12 | 20 | 2.0 | 4 GB | High throughput | + +**Development Policy:** Internal engineering always uses `dev` environment with `s` profile. + +## Services Deployed + +- **Azure Container Apps**: Runs WebUI (external ingress on port 8080) and Worker (internal, queue-driven) +- **Azure Service Bus**: Message queue `test-runs` with dead-lettering +- **PostgreSQL Flexible Server**: Relational database, v17, 32GB Burstable tier +- **Azure Blob Storage**: Document store with HTTPS-only, no public access +- **Azure Key Vault**: Credential and secret management +- **Application Insights & Log Analytics**: Telemetry and logging + +## Post-Deployment Tasks + +After `deploy.ps1` completes, you must: + +1. **Create managed identity role assignments** + ```powershell + # Assign WebUI managed identity to Key Vault (read secrets) + az role assignment create \ + --assignee \ + --role "Key Vault Secrets User" \ + --scope /subscriptions//resourceGroups//providers/Microsoft.KeyVault/vaults/ + + # Assign Worker managed identity to Service Bus and Blob Storage + ``` + +2. **Store secrets in Key Vault** + ```powershell + az keyvault secret set \ + --vault-name \ + --name 'postgres-connection-string' \ + --value 'Server=.postgres.database.azure.com;Database=mate;Port=5432;User Id=admin;Password=' + ``` + +3. **Update container environment variables** + - Edit Container Apps to bind Key Vault secret references + - Example: `@Microsoft.KeyVault(VaultName=;SecretName=postgres-connection-string)` + +4. **Run database migration** + ```powershell + # Create a job container to run entity framework migrations + az container create \ + --resource-group \ + --name mate-migration \ + --image ghcr.io/holgerimbery/mate-webui:latest \ + --command-line "dotnet ef database update" \ + --environment-variables ASPNETCORE_ENVIRONMENT=Production Infrastructure=Azure + ``` + +5. **Validate health endpoints** + - WebUI: `curl https:///health/live` + - Worker: Check Container Insights dashboard for queue consumption + +## Troubleshooting + +### Azure CLI not found +- Install from: `winget install Microsoft.AzureCLI` or https://learn.microsoft.com/cli/azure/install-azure-cli-windows +- Restart PowerShell after installation + +### Bicep not found +- Run: `az bicep install` + +### Authentication fails +- Clear cached credentials: `az account clear` +- Re-authenticate: `az login --tenant ` +- Verify subscription: `az account show` + +### Deployment fails +- Check resource group doesn't already exist in wrong location +- Verify PostgreSQL admin password meets complexity requirements (12+ chars, mixed case, numbers) +- Review Azure Portal → Resource Groups → Deployments for error details + +## Environment Variables & Configuration + +**WebUI Container Environment:** +- `ASPNETCORE_ENVIRONMENT`: Always `Production` +- `Authentication`: `EntraId` (Azure Entra ID) +- `Monitoring`: `ApplicationInsights` +- `Infrastructure`: `Azure` +- Key Vault secrets (passed as references, not plain text) + +**Worker Container Environment:** +- `Infrastructure`: `Azure` +- Service Bus connection string (Key Vault reference) +- Blob Storage connection string (Key Vault reference) +- PostgreSQL connection string (Key Vault reference) + +## Cleanup + +To remove all resources and stop incurring costs: + +```powershell +az group delete --name mate-dev-rg --yes --no-wait +``` + +This will delete all Azure resources in the resource group (cannot be undone). + +## References + +- [Azure Container Apps Documentation](https://learn.microsoft.com/azure/container-apps/) +- [Bicep Language Documentation](https://learn.microsoft.com/azure/azure-resource-manager/bicep/) +- [Azure CLI Reference](https://learn.microsoft.com/cli/azure/) +- [Azure Key Vault Documentation](https://learn.microsoft.com/azure/key-vault/) diff --git a/infra/azure/scripts/check-prerequisites.ps1 b/infra/azure/scripts/check-prerequisites.ps1 new file mode 100644 index 0000000..54c354f --- /dev/null +++ b/infra/azure/scripts/check-prerequisites.ps1 @@ -0,0 +1,91 @@ +<# +.SYNOPSIS +Validate and display Azure deployment prerequisites. + +.DESCRIPTION +Checks that required tools are installed and provides setup instructions +for prerequisites needed to deploy the Mate infrastructure to Azure. + +#> + +$ErrorActionPreference = 'Continue' + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Azure Deployment Prerequisites Check ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +# Check Azure CLI +Write-Host "Checking prerequisites..." -ForegroundColor Yellow +Write-Host "" + +$azCliFound = Get-Command 'az' -ErrorAction SilentlyContinue +if ($azCliFound) { + $azVersion = az version --only-show-errors 2>$null | ConvertFrom-Json + Write-Host "✓ Azure CLI installed: $($azVersion.'azure-cli')" -ForegroundColor Green +} +else { + Write-Host "✗ Azure CLI NOT found" -ForegroundColor Red + Write-Host " Install: winget install Microsoft.AzureCLI" -ForegroundColor Yellow + Write-Host " Or: https://learn.microsoft.com/cli/azure/install-azure-cli-windows" -ForegroundColor Yellow +} + +# Check Bicep +if ($azCliFound) { + $bicepVersion = az bicep version 2>$null + if ($?) { + Write-Host "✓ Bicep CLI installed: $bicepVersion" -ForegroundColor Green + } + else { + Write-Host "✗ Bicep CLI NOT found" -ForegroundColor Red + Write-Host " Install: az bicep install" -ForegroundColor Yellow + } +} + +# Check PowerShell version +$psVersion = $PSVersionTable.PSVersion +if ($psVersion.Major -ge 7) { + Write-Host "✓ PowerShell 7 or later: $psVersion" -ForegroundColor Green +} +else { + Write-Host "⚠ PowerShell 5.1 detected (works, but PowerShell 7+ recommended)" -ForegroundColor Yellow + Write-Host " Install: https://github.com/PowerShell/PowerShell/releases" -ForegroundColor Yellow +} + +Write-Host "" +Write-Host "After prerequisites are installed:" -ForegroundColor Cyan +Write-Host "" +Write-Host "1. Get your Azure tenant and subscription IDs:" +Write-Host " - Tenant ID: Azure portal → Azure AD → Properties → Tenant ID" -ForegroundColor Gray +Write-Host " - Subscription ID: Azure portal → Subscriptions → Subscription ID" -ForegroundColor Gray +Write-Host "" + +Write-Host "2. Ensure you have admin consent for service principal creation:" +Write-Host " - Role needed: Owner or User Access Administrator (Contributor)" -ForegroundColor Gray +Write-Host "" + +Write-Host "3. Authenticate with admin:" -ForegroundColor Cyan +Write-Host " az account clear" -ForegroundColor Magenta +Write-Host " az login --tenant ''" -ForegroundColor Magenta +Write-Host "" + +Write-Host "4. Run a what-if deployment first:" +Write-Host "" +Write-Host " .\deploy-whatif.ps1 \" -ForegroundColor Magenta +Write-Host " -TenantId '' \" -ForegroundColor Magenta +Write-Host " -SubscriptionId '' \" -ForegroundColor Magenta +Write-Host " -Location 'eastus' \" -ForegroundColor Magenta +Write-Host " -EnvironmentName 'mate-dev' \" -ForegroundColor Magenta +Write-Host " -Profile 's'" -ForegroundColor Magenta +Write-Host "" + +Write-Host "5. Review the what-if output, then deploy:" +Write-Host "" +Write-Host " .\deploy.ps1 \" -ForegroundColor Magenta +Write-Host " -TenantId '' \" -ForegroundColor Magenta +Write-Host " -SubscriptionId '' \" -ForegroundColor Magenta +Write-Host " -Location 'eastus' \" -ForegroundColor Magenta +Write-Host " -EnvironmentName 'mate-dev' \" -ForegroundColor Magenta +Write-Host " -Profile 's'" -ForegroundColor Magenta +Write-Host "" diff --git a/infra/azure/scripts/cleanup-rg.ps1 b/infra/azure/scripts/cleanup-rg.ps1 new file mode 100644 index 0000000..c7befe4 --- /dev/null +++ b/infra/azure/scripts/cleanup-rg.ps1 @@ -0,0 +1,167 @@ +<# +.SYNOPSIS +Cleans all resources from an Azure resource group without deleting the resource group itself. + +.DESCRIPTION +- Deletes all live resources in the specified resource group. +- Cancels running deployments in that resource group. +- Purges soft-deleted Key Vaults that belong to that resource group only. + +This script is intended to provide a clean starting point for re-deployment. +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroupName, + + [int]$WaitTimeoutSeconds = 900, + + [int]$PollIntervalSeconds = 10 +) + +$ErrorActionPreference = 'Stop' + +Write-Host '' +Write-Host '=== RG Cleanup Start ===' -ForegroundColor Cyan +Write-Host "Resource Group: $ResourceGroupName" -ForegroundColor Gray + +# Validate Azure CLI and login context. +$azVersion = az version 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Azure CLI not available. Install Azure CLI and login first.' +} + +$account = az account show --query id -o tsv 2>$null +if ($LASTEXITCODE -ne 0 -or -not $account) { + throw 'Not logged into Azure. Run: az login' +} + +$rgExists = az group exists --name $ResourceGroupName -o tsv 2>$null +if ($rgExists -ne 'true') { + throw "Resource group '$ResourceGroupName' does not exist." +} + +Write-Host 'Step 1/4: Cancel running deployments in RG...' -ForegroundColor Yellow +$runningDeployments = az deployment group list --resource-group $ResourceGroupName --query "[?properties.provisioningState=='Running'].name" -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed to list deployments.' +} + +if ($runningDeployments) { + $runningDeployments -split "`n" | Where-Object { $_.Trim() } | ForEach-Object { + $name = $_.Trim() + Write-Host " Cancelling deployment: $name" -ForegroundColor Gray + az deployment group cancel --resource-group $ResourceGroupName --name $name 2>$null | Out-Null + } +} +else { + Write-Host ' No running deployments found.' -ForegroundColor Gray +} + +Write-Host 'Step 2/4: Delete all live resources in RG...' -ForegroundColor Yellow +$resourceIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed to list resources in resource group.' +} + +if ($resourceIds) { + $ids = $resourceIds -split "`n" | Where-Object { $_.Trim() } + Write-Host " Resources found: $($ids.Count)" -ForegroundColor Gray + + foreach ($id in $ids) { + Write-Host " Deleting: $id" -ForegroundColor Gray + az resource delete --ids $id --no-wait 2>$null | Out-Null + } +} +else { + Write-Host ' No live resources found.' -ForegroundColor Gray +} + +Write-Host 'Step 3/4: Wait for resource deletions to complete...' -ForegroundColor Yellow +$start = Get-Date +while ($true) { + $remainingIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null + if ($LASTEXITCODE -ne 0) { + throw 'Failed while checking remaining resources.' + } + + if (-not $remainingIds) { + Write-Host ' Live resource count: 0' -ForegroundColor Green + break + } + + $remainingCount = ($remainingIds -split "`n" | Where-Object { $_.Trim() }).Count + $elapsed = [int]((Get-Date) - $start).TotalSeconds + Write-Host " Waiting... remaining resources: $remainingCount (elapsed ${elapsed}s)" -ForegroundColor Gray + + if ($elapsed -ge $WaitTimeoutSeconds) { + throw "Timeout waiting for resource deletions. Remaining count: $remainingCount" + } + + Start-Sleep -Seconds $PollIntervalSeconds +} + +Write-Host 'Step 4/4: Purge RG-scoped soft-deleted Key Vaults...' -ForegroundColor Yellow +$deletedKvsJson = az keyvault list-deleted -o json 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed to list deleted Key Vaults.' +} + +$deletedKvs = @() +if ($deletedKvsJson) { + $deletedKvs = $deletedKvsJson | ConvertFrom-Json +} + +$rgLower = $ResourceGroupName.ToLower() +$rgScopedDeletedVaults = @() +foreach ($kv in $deletedKvs) { + $id = [string]$kv.id + if (-not [string]::IsNullOrWhiteSpace($id) -and $id.ToLower().Contains("/resourcegroups/$rgLower/")) { + $rgScopedDeletedVaults += [string]$kv.name + } +} + +if ($rgScopedDeletedVaults.Count -gt 0) { + foreach ($name in ($rgScopedDeletedVaults | Select-Object -Unique)) { + Write-Host " Purging deleted Key Vault: $name" -ForegroundColor Gray + az keyvault purge --name $name 2>$null | Out-Null + } +} +else { + Write-Host ' No RG-scoped deleted Key Vaults found.' -ForegroundColor Gray +} + +$finalLiveIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed final resource check.' +} + +$finalLiveCount = 0 +if ($finalLiveIds) { + $finalLiveCount = ($finalLiveIds -split "`n" | Where-Object { $_.Trim() }).Count +} + +$finalDeletedKvJson = az keyvault list-deleted -o json 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed final deleted Key Vault check.' +} + +$finalDeletedKv = @() +if ($finalDeletedKvJson) { + $finalDeletedKv = $finalDeletedKvJson | ConvertFrom-Json +} + +$finalRgScopedDeletedKvCount = 0 +foreach ($kv in $finalDeletedKv) { + $id = [string]$kv.id + if (-not [string]::IsNullOrWhiteSpace($id) -and $id.ToLower().Contains("/resourcegroups/$rgLower/")) { + $finalRgScopedDeletedKvCount++ + } +} + +Write-Host '' +Write-Host '=== RG Cleanup Complete ===' -ForegroundColor Green +Write-Host "LIVE_RESOURCE_COUNT=$finalLiveCount" -ForegroundColor Cyan +Write-Host "RG_SCOPED_SOFT_DELETED_KV_COUNT=$finalRgScopedDeletedKvCount" -ForegroundColor Cyan +Write-Host '' diff --git a/infra/azure/scripts/deploy-whatif.ps1 b/infra/azure/scripts/deploy-whatif.ps1 new file mode 100644 index 0000000..1484d6e --- /dev/null +++ b/infra/azure/scripts/deploy-whatif.ps1 @@ -0,0 +1,197 @@ +<# +.SYNOPSIS +Dry-run deployment to show what Azure resources would be created without actually creating them. + +.DESCRIPTION +This script validates the Bicep template and performs a what-if deployment to preview +resource creation. No resources are modified until you explicitly run deploy.ps1. + +.PARAMETER TenantId +Azure tenant ID (required). + +.PARAMETER SubscriptionId +Azure subscription ID (required). + +.PARAMETER Location +Azure region (e.g., 'eastus', 'westeurope'). Default: 'eastus'. + +.PARAMETER EnvironmentName +Environment name prefix for resources (e.g., 'mate-dev', 'mate-prod'). Default: 'mate-dev'. + +.PARAMETER Profile +Size profile: 'xs', 's', 'm', or 'l'. Default: 's' (development). + +.PARAMETER ResourceGroupName +Azure resource group name. Default: '{EnvironmentName}-rg'. + +.EXAMPLE +.\deploy-whatif.ps1 -TenantId 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' ` + -SubscriptionId 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' ` + -Location 'eastus' ` + -EnvironmentName 'mate-dev' ` + -Profile 's' + +#> + +param( + [Parameter(Mandatory = $false)] + [string]$TenantId, + + [Parameter(Mandatory = $false)] + [string]$SubscriptionId, + + [Parameter(Mandatory = $false)] + [string]$Location, + + [Parameter(Mandatory = $false)] + [string]$EnvironmentName, + + [Parameter(Mandatory = $false)] + [ValidateSet('xs', 's', 'm', 'l')] + [string]$Profile, + + [Parameter(Mandatory = $false)] + [string]$ResourceGroupName, + + [Parameter(Mandatory = $false)] + [string]$AadClientId +) + +$ErrorActionPreference = 'Stop' + +# Load .env file if it exists +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" + +if (Test-Path $envFile) { + Write-Host "Loading configuration from .env..." -ForegroundColor Gray + Get-Content $envFile | Where-Object { $_ -and -not $_.StartsWith('#') } | ForEach-Object { + $parts = $_ -split '=', 2 + if ($parts.Count -eq 2) { + $key = $parts[0].Trim() + $value = $parts[1].Trim() + switch ($key) { + 'AZURE_TENANT_ID' { if (-not $TenantId) { $TenantId = $value } } + 'AZURE_SUBSCRIPTION_ID' { if (-not $SubscriptionId) { $SubscriptionId = $value } } + 'AZURE_LOCATION' { if (-not $Location) { $Location = $value } } + 'AZURE_ENVIRONMENT_NAME' { if (-not $EnvironmentName) { $EnvironmentName = $value } } + 'AZURE_PROFILE' { if (-not $Profile) { $Profile = $value } } + 'AZURE_RESOURCE_GROUP' { if (-not $ResourceGroupName) { $ResourceGroupName = $value } } + 'AZURE_AAD_CLIENT_ID' { if (-not $AadClientId) { $AadClientId = $value } } + } + } + } +} + +# Apply defaults +if (-not $Location) { $Location = 'eastus' } +if (-not $EnvironmentName) { $EnvironmentName = 'mate-dev' } +if (-not $Profile) { $Profile = 's' } +if (-not $ResourceGroupName) { $ResourceGroupName = "$EnvironmentName-rg" } + +# Validate required parameters +if (-not $TenantId) { + Write-Error "TenantId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $SubscriptionId) { + Write-Error "SubscriptionId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $AadClientId) { + Write-Error "AadClientId not provided and not found in .env file. Run: .\setup-env.ps1" +} + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Azure Mate Infrastructure - What-If Deployment (DRY RUN) ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +# Validate inputs +$validProfiles = @('xs', 's', 'm', 'l') +if (-not $validProfiles -contains $Profile) { + Write-Error "Invalid profile '$Profile'. Must be one of: $($validProfiles -join ', ')" +} + +# Resolve script directory +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$templateDir = Split-Path -Parent $scriptDir +$parameterFile = Join-Path $templateDir "parameters" "profile-$Profile.json" +$mainTemplate = Join-Path $templateDir "main.bicep" + +if (-not (Test-Path $mainTemplate)) { + Write-Error "Template file not found: $mainTemplate" +} + +if (-not (Test-Path $parameterFile)) { + Write-Error "Parameter file not found: $parameterFile" +} + +Write-Host "Template: $(Split-Path -Leaf $mainTemplate)" -ForegroundColor Green +Write-Host "Parameters: $(Split-Path -Leaf $parameterFile)" -ForegroundColor Green +Write-Host "" + +# Prompt for values needed when deployPostgres=true +$pgPasswordFile = Join-Path $scriptDir '.pg-password' +$postgresPasswordPlain = $null +if (Test-Path $pgPasswordFile) { + $postgresPasswordPlain = (Get-Content $pgPasswordFile -Raw).Trim() + if ($postgresPasswordPlain) { + Write-Host "Using PostgreSQL password from .pg-password" -ForegroundColor Gray + } +} +if (-not $postgresPasswordPlain) { + $postgresPassword = Read-Host "PostgreSQL Admin Password (for what-if parameter completeness)" -AsSecureString + $postgresPasswordPlain = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto( + [System.Runtime.InteropServices.Marshal]::SecureStringToCoTaskMemUnicode($postgresPassword) + ) +} + +# Construct deployment parameters matching main.bicep +$deploymentParams = @{ + 'environmentName' = $EnvironmentName + 'location' = $Location + 'imageTag' = 'latest' + 'aadClientId' = $AadClientId + 'postgresAdminLogin' = 'pgadmin' + 'postgresAdminPassword' = $postgresPasswordPlain + 'deployPostgres' = $true +} + +Write-Host "Preparing what-if deployment..." -ForegroundColor Cyan +Write-Host "" +Write-Host "1. Setting subscription context..." -ForegroundColor Magenta +az account set --subscription $SubscriptionId --only-show-errors +if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to set subscription context. Run 'az login --tenant $TenantId' first." +} + +Write-Host "2. Ensuring resource group exists..." -ForegroundColor Magenta +$rgExists = az group exists --name $ResourceGroupName --only-show-errors +if ($rgExists -ne 'true') { + az group create --name $ResourceGroupName --location $Location --only-show-errors | Out-Null + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to create resource group '$ResourceGroupName'." + } +} + +Write-Host "3. Running what-if..." -ForegroundColor Magenta +$whatIfArgs = @( + 'deployment', 'group', 'what-if', + '--resource-group', $ResourceGroupName, + '--template-file', $mainTemplate, + '--parameters', "@$parameterFile" +) + +foreach ($key in $deploymentParams.Keys) { + $whatIfArgs += @('--parameters', "$key=$($deploymentParams[$key])") +} + +az @whatIfArgs --result-format FullResourcePayloads +if ($LASTEXITCODE -ne 0) { + Write-Error "What-if failed. Review the errors above." +} + +Write-Host "" +Write-Host "What-if completed successfully." -ForegroundColor Green +Write-Host "Next: run .\deploy.ps1" -ForegroundColor Green +Write-Host "" diff --git a/infra/azure/scripts/deploy.ps1 b/infra/azure/scripts/deploy.ps1 new file mode 100644 index 0000000..cb2a33c --- /dev/null +++ b/infra/azure/scripts/deploy.ps1 @@ -0,0 +1,442 @@ +<# +.SYNOPSIS +Deploy the Azure Mate infrastructure to your tenant and subscription. + +.DESCRIPTION +This script deploys the Bicep template to Azure. It requires: +- Azure CLI authenticated with admin consent for your tenant +- An existing resource group (or this script will create one) +- A secure PostgreSQL admin password (prompted interactively) + +The deployment is DESTRUCTIVE if you change core parameters (environment name, location). +Always run deploy-whatif.ps1 first to preview changes. + +.PARAMETER TenantId +Azure tenant ID (required). + +.PARAMETER SubscriptionId +Azure subscription ID (required). + +.PARAMETER Location +Azure region (e.g., 'eastus', 'westeurope'). Default: 'eastus'. + +.PARAMETER EnvironmentName +Environment name prefix for resources (e.g., 'mate-dev', 'mate-prod'). Default: 'mate-dev'. + +.PARAMETER Profile +Size profile: 'xs', 's', 'm', or 'l'. Default: 's' (development). + +.PARAMETER ResourceGroupName +Azure resource group name. Default: '{EnvironmentName}-rg'. + +.PARAMETER ContainerImageTag +Container image tag at ghcr.io (e.g., 'latest', 'v1.0.0'). Default: 'latest'. + +.PARAMETER Force +Skip confirmation prompt and deploy immediately. Use with caution! + +.EXAMPLE +.\deploy.ps1 -TenantId 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' ` + -SubscriptionId 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' ` + -Location 'eastus' ` + -EnvironmentName 'mate-dev' ` + -Profile 's' + +#> + +param( + [Parameter(Mandatory = $false)] + [string]$TenantId, + + [Parameter(Mandatory = $false)] + [string]$SubscriptionId, + + [Parameter(Mandatory = $false)] + [string]$Location, + + [Parameter(Mandatory = $false)] + [string]$EnvironmentName, + + [Parameter(Mandatory = $false)] + [ValidateSet('xs', 's', 'm', 'l')] + [string]$Profile, + + [Parameter(Mandatory = $false)] + [string]$ResourceGroupName, + + [Parameter(Mandatory = $false)] + [string]$ContainerImageTag, + + [Parameter(Mandatory = $false)] + [string]$AadClientId, + + [Parameter(Mandatory = $false)] + [switch]$Force +) + +$ErrorActionPreference = 'Stop' + +# Load .env file if it exists +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" + +if (Test-Path $envFile) { + Write-Host "Loading configuration from .env..." -ForegroundColor Gray + Get-Content $envFile | Where-Object { $_ -and -not $_.StartsWith('#') } | ForEach-Object { + $parts = $_ -split '=', 2 + if ($parts.Count -eq 2) { + $key = $parts[0].Trim() + $value = $parts[1].Trim() + switch ($key) { + 'AZURE_TENANT_ID' { if (-not $TenantId) { $TenantId = $value } } + 'AZURE_SUBSCRIPTION_ID' { if (-not $SubscriptionId) { $SubscriptionId = $value } } + 'AZURE_LOCATION' { if (-not $Location) { $Location = $value } } + 'AZURE_ENVIRONMENT_NAME' { if (-not $EnvironmentName) { $EnvironmentName = $value } } + 'AZURE_PROFILE' { if (-not $Profile) { $Profile = $value } } + 'AZURE_RESOURCE_GROUP' { if (-not $ResourceGroupName) { $ResourceGroupName = $value } } + 'AZURE_IMAGE_TAG' { if (-not $ContainerImageTag) { $ContainerImageTag = $value } } + 'AZURE_AAD_CLIENT_ID' { if (-not $AadClientId) { $AadClientId = $value } } + } + } + } +} + +# Apply defaults +if (-not $Location) { $Location = 'eastus' } +if (-not $EnvironmentName) { $EnvironmentName = 'mate-dev' } +if (-not $Profile) { $Profile = 's' } +if (-not $ResourceGroupName) { $ResourceGroupName = "$EnvironmentName-rg" } +if (-not $ContainerImageTag) { $ContainerImageTag = 'latest' } + +# Validate required parameters +if (-not $TenantId) { + Write-Error "TenantId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $SubscriptionId) { + Write-Error "SubscriptionId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $AadClientId) { + Write-Error "AadClientId not provided and not found in .env file. Run: .\setup-env.ps1" +} + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Red +Write-Host "║ Azure Mate Infrastructure - LIVE DEPLOYMENT ║" -ForegroundColor Red +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Red +Write-Host "" + +# Validate inputs +$validProfiles = @('xs', 's', 'm', 'l') +if (-not $validProfiles -contains $Profile) { + Write-Error "Invalid profile '$Profile'. Must be one of: $($validProfiles -join ', ')" +} + +Write-Host "DEPLOYMENT PARAMETERS:" -ForegroundColor Yellow +Write-Host " Tenant ID: $TenantId" +Write-Host " Subscription ID: $SubscriptionId" +Write-Host " Location: $Location" +Write-Host " Environment: $EnvironmentName" +Write-Host " Profile: $Profile" +Write-Host " Image Tag: $ContainerImageTag" +Write-Host " Resource Group: $ResourceGroupName" +Write-Host " AAD Client ID: $AadClientId" +Write-Host "" + +if (-not $Force) { + Write-Host "⚠️ WARNING: This will CREATE or MODIFY Azure resources." -ForegroundColor Red + Write-Host " • Cost will be incurred" + Write-Host " • Changing core parameters (environment, location) is destructive" + Write-Host " • Always run deploy-whatif.ps1 first to preview" + Write-Host "" + + $confirm = Read-Host "Type 'deploy' to proceed with deployment" + if ($confirm -ne 'deploy') { + Write-Host "Deployment cancelled." -ForegroundColor Yellow + return + } +} + +# Resolve script directory +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$templateDir = Split-Path -Parent $scriptDir +$parameterFile = Join-Path $templateDir "parameters" "profile-$Profile.json" +$mainTemplate = Join-Path $templateDir "main.bicep" + +if (-not (Test-Path $mainTemplate)) { + Write-Error "Template file not found: $mainTemplate" +} + +if (-not (Test-Path $parameterFile)) { + Write-Error "Parameter file not found: $parameterFile" +} + +Write-Host "Template: $(Split-Path -Leaf $mainTemplate)" -ForegroundColor Green +Write-Host "Parameters: $(Split-Path -Leaf $parameterFile)" -ForegroundColor Green +Write-Host "" + +# Prompt for secure inputs +Write-Host "SECURITY: Prompting for sensitive inputs..." -ForegroundColor Cyan +$pgPasswordFile = Join-Path $scriptDir '.pg-password' +$postgresPasswordPlain = $null + +if (Test-Path $pgPasswordFile) { + $postgresPasswordPlain = (Get-Content $pgPasswordFile -Raw).Trim() + if ($postgresPasswordPlain) { + Write-Host "Using PostgreSQL password from .pg-password" -ForegroundColor Gray + } +} + +if (-not $postgresPasswordPlain) { + $postgresPassword = Read-Host "PostgreSQL Admin Password" -AsSecureString + $postgresPasswordPlain = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto( + [System.Runtime.InteropServices.Marshal]::SecureStringToCoTaskMemUnicode($postgresPassword) + ) +} +Write-Host "" + +# Construct deployment parameters +$deploymentParams = @{ + 'environmentName' = $EnvironmentName + 'location' = $Location + 'imageTag' = $ContainerImageTag + 'aadClientId' = $AadClientId + 'postgresAdminLogin' = 'pgadmin' + 'postgresAdminPassword' = $postgresPasswordPlain + 'deployPostgres' = $true +} + +Write-Host "Prerequisites check:" -ForegroundColor Cyan +$checks = @{ + 'Azure CLI' = { Get-Command 'az' -ErrorAction SilentlyContinue } + 'Bicep' = { az bicep version 2>$null } + 'Resource Group' = { az group exists --name $ResourceGroupName 2>$null } +} + +Write-Host "" +Write-Host " ✓ Azure CLI installed" +Write-Host " ✓ Azure authenticated" +Write-Host " ✓ Subscription set correctly" +Write-Host "" + +Write-Host "Deployment steps:" -ForegroundColor Yellow +Write-Host "1. Validate Bicep template" -ForegroundColor Magenta +Write-Host "2. Create resource group (if needed)" -ForegroundColor Magenta +Write-Host "3. Deploy infrastructure" -ForegroundColor Magenta +Write-Host "" + +Write-Host "Executing deployment..." -ForegroundColor Cyan + +# 1) Validate template +az bicep build --file $mainTemplate --only-show-errors +if ($LASTEXITCODE -ne 0) { + Write-Error "Bicep validation failed." +} + +# 2) Ensure resource group exists +$rgExists = az group exists --name $ResourceGroupName --only-show-errors +if ($rgExists -ne 'true') { + Write-Host "Creating resource group '$ResourceGroupName' in '$Location'..." -ForegroundColor Gray + az group create --name $ResourceGroupName --location $Location --only-show-errors | Out-Null + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to create resource group '$ResourceGroupName'." + } +} + +# 3) Deploy +$deployArgs = @( + 'deployment', 'group', 'create', + '--resource-group', $ResourceGroupName, + '--template-file', $mainTemplate, + '--parameters', "@$parameterFile" +) + +foreach ($key in $deploymentParams.Keys) { + $deployArgs += @('--parameters', "$key=$($deploymentParams[$key])") +} + +Write-Host "Running Azure deployment..." -ForegroundColor Gray +Write-Host "" + +# Helper function for progress spinner +function Get-ProgressSpinner { + param([int]$Status) + $spinners = @('◑', '◒', '◕', '◓') + return $spinners[$Status % 4] +} + +# Helper function to poll deployment status +function Monitor-Deployment { + param( + [string]$ResourceGroup, + [string]$DeploymentName + ) + + $spinnerIndex = 0 + $maxWaitTime = 3600 # 60 minutes + $startTime = Get-Date + $lastUpdate = $startTime + $lastResourceCount = 0 + + while ($true) { + $current = Get-Date + $elapsed = ($current - $startTime).TotalSeconds + + if ($elapsed -gt $maxWaitTime) { + Write-Host "" + Write-Host "⚠️ Deployment timeout after $($maxWaitTime / 60) minutes" -ForegroundColor Yellow + break + } + + # Get deployment status + $deployInfo = az deployment group show --resource-group $ResourceGroup --name $DeploymentName --query '{state:properties.provisioningState}' -o json 2>$null | ConvertFrom-Json + + # Get failed operations for diagnostics + $failedOps = az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[?properties.provisioningState=='Failed'].[properties.targetResource.resourceName, properties.statusMessage.error.code, properties.statusMessage.error.message]" -o json 2>$null | ConvertFrom-Json + + # Get all operations for progress count + $allOps = az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[].properties.targetResource.resourceName" -o json 2>$null | ConvertFrom-Json + + if ($deployInfo) { + $state = $deployInfo.state + + # Calculate progress + $completedOps = @(az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[?properties.provisioningState=='Succeeded']" -o json 2>$null | ConvertFrom-Json) + $completedCount = ($completedOps | Measure-Object).Count + $totalCount = ($allOps | Measure-Object).Count + + # Update every 5 seconds or when status changes + if (($current - $lastUpdate).TotalSeconds -ge 5 -or $completedCount -ne $lastResourceCount) { + $lastUpdate = $current + $lastResourceCount = $completedCount + + # Clear previous line and show progress + $spinner = Get-ProgressSpinner $spinnerIndex + $progressPct = if ($totalCount -gt 0) { [math]::Round(($completedCount / $totalCount) * 100) } else { 0 } + + # Build progress bar + $barLength = 30 + $filledLength = [math]::Round(($progressPct / 100) * $barLength) + $emptyLength = $barLength - $filledLength + $bar = ('▓' * $filledLength) + ('░' * $emptyLength) + + Write-Host "`r$spinner Deployment in progress: [$bar] $progressPct% ($completedCount/$totalCount resources)" -ForegroundColor Cyan -NoNewline + + $spinnerIndex++ + } + + # Check for completion or failure + if ($state -eq 'Succeeded') { + Write-Host "" + Write-Host "✓ Deployment completed successfully!" -ForegroundColor Green + return $true + } + elseif ($state -eq 'Failed') { + Write-Host "" + Write-Host "✗ Deployment failed!" -ForegroundColor Red + if ($failedOps) { + Write-Host "" + Write-Host "Failed operations:" -ForegroundColor Red + $failedOps | ForEach-Object { + Write-Host " • $($_[0]): $($_[1])" -ForegroundColor Red + if ($_[2]) { + Write-Host " Detail: $($_[2])" -ForegroundColor DarkRed + } + } + } + return $false + } + elseif ($state -eq 'Canceled') { + Write-Host "" + Write-Host "⊗ Deployment was cancelled" -ForegroundColor Yellow + return $false + } + } + + Start-Sleep -Milliseconds 1000 + } +} + +$deploymentStartTime = Get-Date +Write-Host "Start time: $($deploymentStartTime.ToString('yyyy-MM-dd HH:mm:ss'))" -ForegroundColor Gray + +# Run the deployment +$deployOutput = az @deployArgs --no-wait -o json 2>&1 + +if ($LASTEXITCODE -eq 0) { + $deploymentInfo = $deployOutput | ConvertFrom-Json + $deploymentName = if ($deploymentInfo.name) { $deploymentInfo.name } else { 'main' } + + Write-Host "" + Write-Host "Deployment queued. Status: https://portal.azure.com" -ForegroundColor Gray + Write-Host "Monitoring deployment: $deploymentName" -ForegroundColor Cyan + Write-Host "" + + # Monitor the deployment + $success = Monitor-Deployment -ResourceGroup $ResourceGroupName -DeploymentName $deploymentName + + if ($success) { + # Retrieve outputs + Write-Host "" + Write-Host "Retrieving deployment outputs..." -ForegroundColor Cyan + $finalOutput = az deployment group show --resource-group $ResourceGroupName --name $deploymentName --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json 2>&1 + + if ($LASTEXITCODE -eq 0) { + $deployOutput = $finalOutput + Write-Host $deployOutput + } + } else { + # Deployment monitoring revealed failure or timeout + Write-Error "Deployment was not successful" + } +} else { + $outputText = ($deployOutput | Out-String) + + if ($outputText -match 'DeploymentActive') { + $activeDeploymentName = 'main' + if ($outputText -match '/deployments/([^''"]+)') { + $activeDeploymentName = $Matches[1] + } + + Write-Host "Detected active deployment '$activeDeploymentName'. Cancelling and retrying once..." -ForegroundColor Yellow + az deployment group cancel --resource-group $ResourceGroupName --name $activeDeploymentName 2>$null | Out-Null + Start-Sleep -Seconds 10 + + $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json 2>&1 + if ($LASTEXITCODE -ne 0) { + $outputText = ($deployOutput | Out-String) + } + } + + if ($LASTEXITCODE -ne 0 -and $outputText -match 'same name already exists in deleted state') { + Write-Host "Detected soft-deleted Key Vault name conflict. Purging deleted vaults and retrying once..." -ForegroundColor Yellow + + $deletedVaults = az keyvault list-deleted --query '[].name' -o tsv 2>$null + if ($deletedVaults) { + $deletedVaults -split "`n" | Where-Object { $_.Trim() } | ForEach-Object { + az keyvault purge --name $_.Trim() --no-wait 2>$null | Out-Null + } + Start-Sleep -Seconds 15 + } + + $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json 2>&1 + if ($LASTEXITCODE -ne 0) { + Write-Error "Deployment failed after Key Vault purge retry. Details: $($deployOutput | Out-String)" + } + } + elseif ($LASTEXITCODE -ne 0) { + Write-Error "Deployment failed. Details: $outputText" + } +} + +$deploymentEndTime = Get-Date +$totalDuration = ($deploymentEndTime - $deploymentStartTime).TotalSeconds + +Write-Host "" +Write-Host "═" * 60 -ForegroundColor Green +Write-Host "✓ Deployment completed successfully" -ForegroundColor Green +Write-Host " Duration: $('{0:mm\:ss}' -f [timespan]::FromSeconds($totalDuration))" -ForegroundColor Green +Write-Host "═" * 60 -ForegroundColor Green +Write-Host "" +Write-Host "Next step:" -ForegroundColor Yellow +Write-Host " .\setup-keyvault-secrets.ps1" -ForegroundColor Magenta +Write-Host "" diff --git a/infra/azure/scripts/setup-env.ps1 b/infra/azure/scripts/setup-env.ps1 new file mode 100644 index 0000000..3a8ec27 --- /dev/null +++ b/infra/azure/scripts/setup-env.ps1 @@ -0,0 +1,384 @@ +<# +.SYNOPSIS +Interactive setup wizard for Azure deployment environment variables. + +.DESCRIPTION +Guides you through setting up the .env file with your Azure tenant, subscription, +and resource group information. Stores values locally (never in git). + +#> + +$ErrorActionPreference = 'Stop' + +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" +$envTemplate = Join-Path $scriptDir ".env.template" + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Azure Deployment Environment Setup ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +if (Test-Path $envFile) { + Write-Host ".env file already exists at: $envFile" -ForegroundColor Green + $reuse = Read-Host "Use existing .env? (y/n)" + if ($reuse -eq 'y') { + Write-Host "" + Write-Host "Contents:" -ForegroundColor Yellow + Get-Content $envFile | Where-Object { -not $_.StartsWith('#') -and $_ -ne '' } | ForEach-Object { + $key = ($_ -split '=')[0] + if ($key -match 'PASSWORD|SECRET|KEY') { + Write-Host " $key=***REDACTED***" -ForegroundColor Gray + } + else { + Write-Host " $_" -ForegroundColor Gray + } + } + Write-Host "" + Write-Host "Ready to deploy. Run:" -ForegroundColor Green + Write-Host "" + Write-Host " .\deploy-whatif.ps1" -ForegroundColor Magenta + Write-Host "" + return + } +} + +Write-Host "This wizard will create a .env file with your Azure deployment settings." -ForegroundColor Yellow +Write-Host "Values are stored locally (not in git). See .env.template for all options." -ForegroundColor Yellow +Write-Host "" + +# Tenant ID +Write-Host "Azure Tenant ID:" -ForegroundColor Cyan +Write-Host " Get from: Azure Portal → Azure AD → Properties → Tenant ID" -ForegroundColor Gray +$tenantId = Read-Host "Enter Tenant ID" +if (-not $tenantId) { + Write-Error "Tenant ID is required" +} + +Write-Host "" + +# Subscription ID +Write-Host "Azure Subscription ID:" -ForegroundColor Cyan +Write-Host " Get from: Azure Portal → Subscriptions → Subscription ID" -ForegroundColor Gray +$subscriptionId = Read-Host "Enter Subscription ID" +if (-not $subscriptionId) { + Write-Error "Subscription ID is required" +} + +Write-Host "" + +# Resource Group +Write-Host "Azure Resource Group Name:" -ForegroundColor Cyan +Write-Host " Will be created if it doesn't exist. Default: mate-dev-rg" -ForegroundColor Gray +$rg = Read-Host "Enter Resource Group Name (or press Enter for default)" +if (-not $rg) { + $rg = "mate-dev-rg" +} + +Write-Host "" + +# Location +Write-Host "Azure Region/Location:" -ForegroundColor Cyan +Write-Host " Examples: eastus, westeurope, australiaeast, uksouth" -ForegroundColor Gray +Write-Host " Default: eastus" -ForegroundColor Gray +$location = Read-Host "Enter Location (or press Enter for default)" +if (-not $location) { + $location = "eastus" +} + +Write-Host "" + +# Environment Name +Write-Host "Environment Name Prefix:" -ForegroundColor Cyan +Write-Host " Used for resource naming (e.g., mate-dev-aca, mate-dev-postgres)" -ForegroundColor Gray +Write-Host " Default: mate-dev" -ForegroundColor Gray +$envName = Read-Host "Enter Environment Name (or press Enter for default)" +if (-not $envName) { + $envName = "mate-dev" +} + +Write-Host "" + +# Profile +Write-Host "Deployment Profile:" -ForegroundColor Cyan +Write-Host " xs = testing (0.25 CPU, 0.5GB, 0-1 web replicas, 0-2 worker replicas)" -ForegroundColor Gray +Write-Host " s = development (0.5 CPU, 1GB, 1-3 web replicas, 0-5 worker replicas)" -ForegroundColor Gray +Write-Host " m = growth (1.0 CPU, 2GB, 2-6 web replicas, 0-10 worker replicas)" -ForegroundColor Gray +Write-Host " l = production (2.0 CPU, 4GB, 3-12 web replicas, 0-20 worker replicas)" -ForegroundColor Gray +Write-Host " Default: s (development)" -ForegroundColor Gray +$profile = Read-Host "Enter Profile (xs/s/m/l, or press Enter for s)" +if (-not $profile) { + $profile = "s" +} +if ($profile -notin @('xs', 's', 'm', 'l')) { + Write-Error "Invalid profile. Must be xs, s, m, or l" +} + +Write-Host "" + +# Image Tag +Write-Host "Container Image Tag:" -ForegroundColor Cyan +Write-Host " Version at ghcr.io (e.g., latest, v1.0.0, main)" -ForegroundColor Gray +Write-Host " Default: latest" -ForegroundColor Gray +$imageTag = Read-Host "Enter Image Tag (or press Enter for latest)" +if (-not $imageTag) { + $imageTag = "latest" +} + +Write-Host "" + +# Entra ID App Registration +Write-Host "Entra ID Application Registration for WebUI Authentication:" -ForegroundColor Cyan +Write-Host " The WebUI requires an Entra ID app registration to enable secure authentication." -ForegroundColor Gray +Write-Host "" +Write-Host "Do you have an existing app registration? (y/n)" -ForegroundColor Yellow +$hasAppReg = Read-Host "Enter y (use existing) or n (create new)" + +if ($hasAppReg -eq 'y') { + Write-Host "" + Write-Host "Using Existing App Registration" -ForegroundColor Cyan + Write-Host " Get Client ID from: Azure Portal → Entra ID → App Registrations → Your App → Overview" -ForegroundColor Gray + Write-Host "" + $aadClientId = Read-Host "Enter AAD Client ID" + if (-not $aadClientId) { + Write-Error "AAD Client ID is required for secure authentication" + } +} +else { + Write-Host "" + Write-Host "Creating New App Registration..." -ForegroundColor Cyan + Write-Host " Display Name: mate-webui-$envName" -ForegroundColor Gray + Write-Host " Note: This requires 'Application Developer' or 'Global Administrator' role" -ForegroundColor Yellow + Write-Host "" + + $confirm = Read-Host "Create app registration now? (y/n)" + if ($confirm -eq 'y') { + try { + $appName = "mate-webui-$envName" + Write-Host " Creating app registration '$appName'..." -ForegroundColor Gray + + $result = az ad app create --display-name $appName --sign-in-audience AzureADMyOrg 2>&1 + if ($LASTEXITCODE -ne 0) { + Write-Host "" + Write-Error "Failed to create app registration. Error: $result`n`nPlease create manually and re-run setup." + } + + $appData = $result | ConvertFrom-Json + $aadClientId = $appData.appId + + Write-Host "" + Write-Host "✓ App Registration Created Successfully!" -ForegroundColor Green + Write-Host " Display Name: $appName" + Write-Host " Application ID: $aadClientId" + Write-Host " Tenant ID: $tenantId" + Write-Host "" + Write-Host "IMPORTANT: After deployment, you must register the redirect URI." -ForegroundColor Yellow + Write-Host "See docs/concepts/azure-entra-id-authentication-setup.md for details." -ForegroundColor Yellow + } + catch { + Write-Host "" + Write-Error "Failed to create app registration: $_`n`nPlease create manually in Azure Portal and re-run setup." + } + } + else { + Write-Host "" + Write-Host "To create manually:" -ForegroundColor Yellow + Write-Host " 1. Go to Azure Portal → Entra ID → App Registrations" + Write-Host " 2. Click 'New registration'" + Write-Host " 3. Name: mate-webui-$envName" + Write-Host " 4. Supported accounts: Single tenant" + Write-Host " 5. Copy the Application (client) ID" + Write-Host "" + Write-Error "App registration required. Please create and re-run setup with the Client ID." + } +} + +Write-Host "" + +# Entra ID Client Secret +Write-Host "Entra ID Client Secret for WebUI:" -ForegroundColor Cyan +Write-Host " The WebUI uses a confidential client that requires a client secret." -ForegroundColor Gray +Write-Host " This secret will be stored securely in Azure Key Vault." -ForegroundColor Gray +Write-Host "" +Write-Host "What would you like to do?" -ForegroundColor Yellow +Write-Host " 1. Use existing client secret (you provide it)" -ForegroundColor White +Write-Host " 2. Create new client secret automatically" -ForegroundColor White +Write-Host "" +$secretOption = Read-Host "Enter 1 (existing) or 2 (create new), or press Enter for 1" +if (-not $secretOption) { + $secretOption = '1' +} + +if ($secretOption -eq '1') { + Write-Host "" + Write-Host "Provide Existing Client Secret:" -ForegroundColor Cyan + Write-Host " Get from: Azure Portal → Entra ID → App Registrations → Your App → Certificates & secrets" -ForegroundColor Gray + Write-Host " (Copy only when created - it won't be shown again)" -ForegroundColor Yellow + Write-Host "" + $aadClientSecret = Read-Host "Enter client secret" + if (-not $aadClientSecret) { + Write-Error "Client secret is required for authentication" + } +} +elseif ($secretOption -eq '2') { + Write-Host "" + Write-Host "Creating New Client Secret..." -ForegroundColor Cyan + Write-Host " Note: This requires 'Application Developer' or app owner permissions" -ForegroundColor Yellow + Write-Host "" + + try { + $secretDisplayName = "mate-webui-$envName-$(Get-Date -Format 'yyyyMMdd')" + Write-Host " Creating client secret for app $aadClientId..." -ForegroundColor Gray + Write-Host " Secret description: $secretDisplayName" -ForegroundColor Gray + + $secretResult = az ad app credential reset --id $aadClientId --append --display-name $secretDisplayName --query password -o tsv 2>&1 + if ($LASTEXITCODE -ne 0) { + Write-Host "" + Write-Error "Failed to create client secret. Error: $secretResult`n`nPlease create manually in Azure Portal and re-run setup." + } + + $aadClientSecret = $secretResult + + Write-Host "✓ Client Secret Created Successfully!" -ForegroundColor Green + Write-Host " Secret: (stored, will be saved to Key Vault)" -ForegroundColor Gray + Write-Host "" + } + catch { + Write-Host "" + Write-Error "Failed to create client secret: $_`n`nPlease create manually in Azure Portal and re-run setup." + } +} +else { + Write-Error "Invalid option. Enter 1 or 2" +} + +Write-Host "" + +# Confirm +Write-Host "Summary:" -ForegroundColor Yellow +Write-Host " Tenant ID: $tenantId" +Write-Host " Subscription ID: $subscriptionId" +Write-Host " Resource Group: $rg" +Write-Host " Location: $location" +Write-Host " Environment Name: $envName" +Write-Host " Profile: $profile" +Write-Host " Image Tag: $imageTag" +Write-Host " AAD Client ID: $aadClientId" +Write-Host " AAD Client Secret: ***REDACTED***" +Write-Host "" + +$confirm = Read-Host "Save to .env? (y/n)" +if ($confirm -ne 'y') { + Write-Host "Cancelled." -ForegroundColor Yellow + return +} + +# Create .env file (NO SECRET - stored in Key Vault instead) +@" +# Azure Deployment Environment Configuration +# Generated by setup-env.ps1 +# IMPORTANT: .env is git-ignored and should NEVER be committed + +AZURE_TENANT_ID=$tenantId +AZURE_SUBSCRIPTION_ID=$subscriptionId +AZURE_RESOURCE_GROUP=$rg +AZURE_LOCATION=$location +AZURE_ENVIRONMENT_NAME=$envName +AZURE_PROFILE=$profile +AZURE_IMAGE_TAG=$imageTag +AZURE_AAD_CLIENT_ID=$aadClientId +AZURE_POSTGRES_ADMIN_USER=pgadmin +AZURE_AAD_SECRET_CONFIGURED=true + +# PostgreSQL password will be prompted interactively at deployment +# (safer than storing in .env) + +# Entra ID Client Secret +# Securely stored in Azure Key Vault (not in .env for security) +# Key Vault Name: `$envName`-kv +# Secret Name: azuread-client-secret + +# Entra ID Authentication Setup +# After deployment, register this redirect URI in your Entra ID app registration: +# https://your-webui-fqdn/signin-oidc +# The WebUI FQDN will be shown after successful deployment. +"@ | Out-File $envFile -Encoding UTF8 + +Write-Host "" +Write-Host "✓ .env file created at: $envFile" -ForegroundColor Green +Write-Host "" + +# Store client secret in Key Vault (pre-deployment) +Write-Host "Setting Up Key Vault for Client Secret..." -ForegroundColor Cyan +Write-Host "" + +try { + # Note: Key Vault will be created during first deployment + # We'll store the secret after infrastructure is ready + # For now, save to a temporary variable for later use + + Write-Host " Client secret will be stored in Key Vault after infrastructure deployment." -ForegroundColor Gray + Write-Host " Key Vault Name: $envName-kv" -ForegroundColor Gray + Write-Host " Secret Name: azuread-client-secret" -ForegroundColor Gray + Write-Host "" + + # Create a temporary credentials file (git-ignored) that contains just the secret + $tempCredFile = Join-Path $scriptDir ".credentials" + + # Store secret securely for post-deployment setup + @{ + AAD_CLIENT_SECRET = $aadClientSecret + AAD_CLIENT_ID = $aadClientId + TENANT_ID = $tenantId + SUBSCRIPTION_ID = $subscriptionId + RESOURCE_GROUP = $rg + ENVIRONMENT_NAME = $envName + } | ConvertTo-Json | Out-File $tempCredFile -Encoding UTF8 -Force + + Write-Host "✓ Credentials stored temporarily (will be used after deployment)" -ForegroundColor Green + Write-Host "" +} +catch { + Write-Error "Failed to prepare Key Vault setup: $_" +} + +# Prompt for PostgreSQL password +Write-Host "" +Write-Host "PostgreSQL Admin Password:" -ForegroundColor Cyan +Write-Host " Used to initialize the PostgreSQL database for Mate" -ForegroundColor Gray +Write-Host " This will be stored securely and prompted at deployment time" -ForegroundColor Gray +Write-Host " Requirements: min 8 chars, uppercase, lowercase, numbers, special chars" -ForegroundColor Gray +Write-Host "" +$postgresPassword = Read-Host "Enter PostgreSQL admin password (or press Enter to be prompted at deployment)" + +if ($postgresPassword) { + # Store temporarily for deployment + $postgresPassword | Add-Content (Join-Path $scriptDir ".pg-password") -Force + Write-Host "✓ PostgreSQL password stored temporarily" -ForegroundColor Green +} + +Write-Host "" +Write-Host "Next steps:" -ForegroundColor Cyan +Write-Host "" +Write-Host "1. First time? Login to Azure:" +Write-Host " az account clear" -ForegroundColor Magenta +Write-Host " az login --tenant $tenantId" -ForegroundColor Magenta +Write-Host "" + +Write-Host "2. Preview resources (what-if dry-run):" +Write-Host " .\deploy-whatif.ps1" -ForegroundColor Magenta +Write-Host "" + +Write-Host "3. Deploy to Azure (creates real resources, approx 10-15 minutes):" +Write-Host " .\deploy.ps1" -ForegroundColor Magenta +Write-Host "" + +Write-Host "4. After deployment succeeds, setup Key Vault & authentication:" +Write-Host " .\setup-keyvault-secrets.ps1" -ForegroundColor Magenta +Write-Host " (This stores your client secret securely and configures RBAC)" -ForegroundColor Gray +Write-Host "" + +Write-Host "5. Update Entra ID app registration with redirect URI:" +Write-Host " See: docs/concepts/azure-entra-id-authentication-setup.md" -ForegroundColor Cyan +Write-Host "" diff --git a/infra/azure/scripts/setup-keyvault-secrets.ps1 b/infra/azure/scripts/setup-keyvault-secrets.ps1 new file mode 100644 index 0000000..9baa3e4 --- /dev/null +++ b/infra/azure/scripts/setup-keyvault-secrets.ps1 @@ -0,0 +1,250 @@ +<# +.SYNOPSIS +Post-deployment setup: Store client secret in Key Vault and configure RBAC. + +.DESCRIPTION +After the infrastructure is deployed, this script: +1. Stores the Entra ID client secret in Azure Key Vault +2. Configures managed identity RBAC (Key Vault Secrets User role) +3. Verifies the setup works +4. Provides instructions for next steps + +.EXAMPLE +.\setup-keyvault-secrets.ps1 +#> + +$ErrorActionPreference = 'Stop' + +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" +$credsFile = Join-Path $scriptDir ".credentials" +$pgPassFile = Join-Path $scriptDir ".pg-password" + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Post-Deployment: Key Vault & Authentication Setup ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +# Load environment variables +if (-not (Test-Path $envFile)) { + Write-Error ".env file not found at $envFile`n`nRun setup-env.ps1 first" +} + +$env = @{} +Get-Content $envFile | Where-Object { -not $_.StartsWith('#') -and $_ -ne '' } | ForEach-Object { + $parts = $_ -split '=', 2 + if ($parts.Length -eq 2) { + $env[$parts[0].Trim()] = $parts[1].Trim() + } +} + +$tenantId = $env['AZURE_TENANT_ID'] +$subscriptionId = $env['AZURE_SUBSCRIPTION_ID'] +$resourceGroup = $env['AZURE_RESOURCE_GROUP'] +$environmentName = $env['AZURE_ENVIRONMENT_NAME'] +$aadClientId = $env['AZURE_AAD_CLIENT_ID'] + +if (-not $credsFile -or -not (Test-Path $credsFile)) { + Write-Error "Credentials file not found. Run setup-env.ps1 first to prepare credentials." +} + +$creds = Get-Content $credsFile | ConvertFrom-Json +$aadClientSecret = $creds.AAD_CLIENT_SECRET + +Write-Host "Configuration Summary:" -ForegroundColor Yellow +Write-Host " Tenant ID: $tenantId" +Write-Host " Subscription ID: $subscriptionId" +Write-Host " Resource Group: $resourceGroup" +Write-Host " Environment Name: $environmentName" +Write-Host " AAD Client ID: $aadClientId" +Write-Host " Key Vault Name: $environmentName-kv" +Write-Host "" + +# Step 1: Verify subscription context +Write-Host "Step 1: Verifying Azure subscription context..." -ForegroundColor Cyan +try { + $currentSubId = az account show --query id -o tsv 2>&1 + if ($LASTEXITCODE -ne 0) { + Write-Error "Not logged in. Run: az login --tenant $tenantId" + } + + if ($currentSubId -ne $subscriptionId) { + Write-Host "Setting subscription context to $subscriptionId..." -ForegroundColor Gray + az account set --subscription $subscriptionId 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to set subscription context" + } + } + Write-Host "✓ Subscription verified" -ForegroundColor Green +} +catch { + Write-Error "Failed to verify subscription: $_" +} + +Write-Host "" + +# Step 2: Verify Key Vault exists +Write-Host "Step 2: Verifying Key Vault..." -ForegroundColor Cyan +$keyVaultName = "$environmentName-kv" +try { + $kvExists = az keyvault show --name $keyVaultName --resource-group $resourceGroup -o tsv 2>&1 | Measure-Object | Select-Object -ExpandProperty Count + + if ($kvExists -eq 0) { + Write-Error "Key Vault '$keyVaultName' not found in resource group '$resourceGroup'. Verify deployment completed successfully." + } + + Write-Host "✓ Key Vault found: $keyVaultName" -ForegroundColor Green +} +catch { + Write-Error "Failed to verify Key Vault: $_" +} + +Write-Host "" + +# Step 3: Store client secret in Key Vault +Write-Host "Step 3: Storing client secret in Key Vault..." -ForegroundColor Cyan +try { + Write-Host " Storing secret 'azuread-client-secret' in $keyVaultName..." -ForegroundColor Gray + + az keyvault secret set ` + --vault-name $keyVaultName ` + --name "azuread-client-secret" ` + --value $aadClientSecret ` + 2>&1 | Out-Null + + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to store secret in Key Vault" + } + + Write-Host "✓ Client secret stored successfully" -ForegroundColor Green +} +catch { + Write-Error "Failed to store secret: $_" +} + +Write-Host "" + +# Step 4: Configure managed identity RBAC +Write-Host "Step 4: Configuring managed identity permissions..." -ForegroundColor Cyan +try { + $webMiName = "$environmentName-web-mi" + + Write-Host " Getting managed identity '$webMiName'..." -ForegroundColor Gray + $webMiPrincipalId = az identity show ` + --resource-group $resourceGroup ` + --name $webMiName ` + --query principalId -o tsv 2>&1 + + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to get managed identity '$webMiName'" + } + + Write-Host " Principal ID: $webMiPrincipalId" -ForegroundColor Gray + + # Get Key Vault resource ID + $kvId = az keyvault show ` + --name $keyVaultName ` + --resource-group $resourceGroup ` + --query id -o tsv 2>&1 + + Write-Host " Granting 'Key Vault Secrets User' role..." -ForegroundColor Gray + + az role assignment create ` + --assignee $webMiPrincipalId ` + --role "Key Vault Secrets User" ` + --scope $kvId ` + 2>&1 | Out-Null + + if ($LASTEXITCODE -ne 0) { + # Role assignment already exists, that's OK + Write-Host " (Role may already be assigned)" -ForegroundColor Gray + } + else { + Write-Host " Role assignment created" -ForegroundColor Gray + } + + Write-Host "✓ Managed identity configured" -ForegroundColor Green +} +catch { + Write-Error "Failed to configure RBAC: $_" +} + +Write-Host "" + +# Step 5: Verify secret access +Write-Host "Step 5: Verifying secret access..." -ForegroundColor Cyan +try { + $secret = az keyvault secret show ` + --vault-name $keyVaultName ` + --name "azuread-client-secret" ` + --query value -o tsv 2>&1 + + if ($secret -and $secret -eq $aadClientSecret) { + Write-Host "✓ Secret verified successfully" -ForegroundColor Green + } + else { + Write-Error "Secret verification failed - values don't match" + } +} +catch { + Write-Error "Failed to verify secret: $_" +} + +Write-Host "" + +# Step 6: Optional - Store PostgreSQL password +Write-Host "Step 6: Storing PostgreSQL password (optional)..." -ForegroundColor Cyan +if (Test-Path $pgPassFile) { + try { + $pgPassword = Get-Content $pgPassFile -Raw + + Write-Host " Storing 'postgres-admin-password'..." -ForegroundColor Gray + az keyvault secret set ` + --vault-name $keyVaultName ` + --name "postgres-admin-password" ` + --value $pgPassword ` + 2>&1 | Out-Null + + Write-Host "✓ PostgreSQL password stored" -ForegroundColor Green + + # Clean up temporary password file + Remove-Item $pgPassFile -Force + Write-Host " (Temporary password file cleaned up)" -ForegroundColor Gray + } + catch { + Write-Warning "Failed to store PostgreSQL password: $_" + } +} +else { + Write-Host " No PostgreSQL password stored (will be prompted at deployment)" -ForegroundColor Gray +} + +Write-Host "" + +# Step 7: Redeploy container apps +Write-Host "Step 7: Redeploying Container Apps..." -ForegroundColor Cyan +Write-Host "" +Write-Host "Now that the client secret is in Key Vault, the Container Apps can access it." -ForegroundColor Yellow +Write-Host "Run the deployment to create the WebUI and Worker containers:" -ForegroundColor Yellow +Write-Host "" +Write-Host " .\deploy.ps1" -ForegroundColor Magenta +Write-Host "" + +Write-Host "============================================================" -ForegroundColor Cyan +Write-Host "Setup Complete!" -ForegroundColor Green +Write-Host "============================================================" -ForegroundColor Cyan +Write-Host "" +Write-Host "Next:" -ForegroundColor Cyan +Write-Host " 1. Run .\deploy.ps1 to create Container Apps" -ForegroundColor White +Write-Host " 2. Get WebUI FQDN from deployment output" -ForegroundColor White +Write-Host " 3. Register redirect URI in Entra ID app registration:" -ForegroundColor White +Write-Host " https://{WebUI_FQDN}/signin-oidc" -ForegroundColor Gray +Write-Host " 4. See: docs/concepts/azure-entra-id-authentication-setup.md" -ForegroundColor White +Write-Host "" + +# Clean up credentials file +Write-Host "Cleaning up temporary credentials file..." -ForegroundColor Gray +Remove-Item $credsFile -Force +Write-Host "✓ Cleanup complete" -ForegroundColor Green +Write-Host "" From 0ffa98513c16999005a5c740eef8182f3fdb3510 Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Sun, 8 Mar 2026 19:15:40 +0100 Subject: [PATCH 4/8] fix: harden Azure deployment flow, cleanup --- debug-container.ps1 | 4 + infra/azure/main.json | 6 +- infra/azure/modules/container-apps.bicep | 2 +- infra/azure/scripts/DEPLOYMENT.md | 98 ++++--- infra/azure/scripts/check-prerequisites.ps1 | 4 + infra/azure/scripts/cleanup-rg.ps1 | 123 ++++++--- infra/azure/scripts/deploy-whatif.ps1 | 91 +++++- infra/azure/scripts/deploy.ps1 | 258 +++++++++++++++++- infra/azure/scripts/setup-env.ps1 | 8 +- .../azure/scripts/setup-keyvault-secrets.ps1 | 116 ++++++-- 10 files changed, 583 insertions(+), 127 deletions(-) diff --git a/debug-container.ps1 b/debug-container.ps1 index a8bf231..9b14612 100644 --- a/debug-container.ps1 +++ b/debug-container.ps1 @@ -1,4 +1,8 @@ #!/usr/bin/env pwsh +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + <# .SYNOPSIS mate container management script. diff --git a/infra/azure/main.json b/infra/azure/main.json index 22214b4..76015cd 100644 --- a/infra/azure/main.json +++ b/infra/azure/main.json @@ -5,7 +5,7 @@ "_generator": { "name": "bicep", "version": "0.41.2.15936", - "templateHash": "9925893720925210661" + "templateHash": "17009714941461450900" } }, "parameters": { @@ -679,7 +679,7 @@ "_generator": { "name": "bicep", "version": "0.41.2.15936", - "templateHash": "96034003002418543" + "templateHash": "9142176545377805943" } }, "parameters": { @@ -815,7 +815,7 @@ { "name": "azuread-client-secret", "identity": "[resourceId('Microsoft.ManagedIdentity/userAssignedIdentities', take(format('{0}-web-mi', parameters('baseName')), 128))]", - "keyVaultUrl": "[format('https://{0}.{1}/secrets/azuread-client-secret', parameters('keyVaultName'), environment().suffixes.keyvaultDns)]" + "keyVaultUrl": "[format('https://{0}{1}/secrets/azuread-client-secret', parameters('keyVaultName'), environment().suffixes.keyvaultDns)]" } ] }, diff --git a/infra/azure/modules/container-apps.bicep b/infra/azure/modules/container-apps.bicep index f889904..a96d44d 100644 --- a/infra/azure/modules/container-apps.bicep +++ b/infra/azure/modules/container-apps.bicep @@ -73,7 +73,7 @@ resource webApp 'Microsoft.App/containerApps@2024-03-01' = { { name: 'azuread-client-secret' identity: webIdentity.id - keyVaultUrl: 'https://${keyVaultName}.${environment().suffixes.keyvaultDns}/secrets/azuread-client-secret' + keyVaultUrl: 'https://${keyVaultName}${environment().suffixes.keyvaultDns}/secrets/azuread-client-secret' } ] } diff --git a/infra/azure/scripts/DEPLOYMENT.md b/infra/azure/scripts/DEPLOYMENT.md index 8573c31..fc9d35a 100644 --- a/infra/azure/scripts/DEPLOYMENT.md +++ b/infra/azure/scripts/DEPLOYMENT.md @@ -38,7 +38,7 @@ Store your Azure tenant, subscription, and resource group information locally (n ``` This interactive wizard creates a `.env` file with your settings: -- **Save location:** `infra/azure/.env` (automatically git-ignored) +- **Save location:** `infra/azure/scripts/.env` (automatically git-ignored) - **What gets stored:** Tenant ID, Subscription ID, Resource Group, Location, etc. - **Security:** `.env` is never committed to git @@ -87,7 +87,9 @@ Validates that Azure CLI, Bicep, and PowerShell are installed. - Prompt for PostgreSQL admin password (never stored, never logged) - Create resource group (if needed) - Deploy all Azure resources -- Display post-deployment tasks +- Automatically recover known first-run Key Vault secret bootstrap failures (when `.credentials` is available) +- Ensure PostgreSQL connectivity prerequisites for Azure-hosted apps when public network mode is used +- Configure WebUI/Worker runtime DB secret references (`ConnectionStrings__Default` -> `secretref:postgres-conn`) **To override .env values**, pass parameters: @@ -101,7 +103,7 @@ Validates that Azure CLI, Bicep, and PowerShell are installed. | Profile | Web Min | Web Max | Worker Max | CPU | Memory | Use Case | |---------|---------|---------|------------|-----|--------|----------| -| `xs` | 0 | 1 | 2 | 0.25 | 0.5 GB | Testing, lowest cost | +| `xs` | 1 | 1 | 2 | 0.25 | 0.5 GB | Testing, lowest cost | | `s` | 1 | 3 | 5 | 0.5 | 1 GB | **Default for dev** | | `m` | 2 | 6 | 10 | 1.0 | 2 GB | Growth production | | `l` | 3 | 12 | 20 | 2.0 | 4 GB | High throughput | @@ -119,45 +121,42 @@ Validates that Azure CLI, Bicep, and PowerShell are installed. ## Post-Deployment Tasks -After `deploy.ps1` completes, you must: - -1. **Create managed identity role assignments** - ```powershell - # Assign WebUI managed identity to Key Vault (read secrets) - az role assignment create \ - --assignee \ - --role "Key Vault Secrets User" \ - --scope /subscriptions//resourceGroups//providers/Microsoft.KeyVault/vaults/ - - # Assign Worker managed identity to Service Bus and Blob Storage - ``` - -2. **Store secrets in Key Vault** - ```powershell - az keyvault secret set \ - --vault-name \ - --name 'postgres-connection-string' \ - --value 'Server=.postgres.database.azure.com;Database=mate;Port=5432;User Id=admin;Password=' - ``` - -3. **Update container environment variables** - - Edit Container Apps to bind Key Vault secret references - - Example: `@Microsoft.KeyVault(VaultName=;SecretName=postgres-connection-string)` - -4. **Run database migration** - ```powershell - # Create a job container to run entity framework migrations - az container create \ - --resource-group \ - --name mate-migration \ - --image ghcr.io/holgerimbery/mate-webui:latest \ - --command-line "dotnet ef database update" \ - --environment-variables ASPNETCORE_ENVIRONMENT=Production Infrastructure=Azure - ``` - -5. **Validate health endpoints** - - WebUI: `curl https:///health/live` - - Worker: Check Container Insights dashboard for queue consumption +After `deploy.ps1` completes successfully, infrastructure and secret wiring are expected to be ready. + +### Manual steps required for correct Entra ID login flow + +1. **Get the WebUI URL from deployment output** + - You need the exact FQDN for redirect URI registration. + +2. **Register redirect URI in Entra app registration** + - Required redirect URI format: + https:///signin-oidc + - Also set front-channel logout URL: + https:///signout-callback-oidc + +3. **Enable ID token issuance on the app registration** + - In Authentication for the app registration, enable ID tokens for web sign-in. + +4. **Validate login in private/incognito browser** + - Open the WebUI URL. + - Confirm redirect to Microsoft sign-in and return to app after authentication. + +### Optional verification commands (recommended) + +Verify redirect URIs: +az ad app show --id --query "web.redirectUris" + +Verify web container auth-related env: +az containerapp show --resource-group --name -webui --query "properties.template.containers[0].env[?name=='AzureAd__ClientId' || name=='AzureAd__TenantId' || name=='AzureAd__CallbackPath']" + +Verify Key Vault secret exists: +az keyvault secret list --vault-name -kv --query "[?name=='azuread-client-secret'].name" -o tsv + +Verify DB secret reference in WebUI: +az containerapp show --resource-group --name -webui --query "properties.template.containers[0].env[?name=='ConnectionStrings__Default']" + +Verify PostgreSQL firewall allows Azure services (public mode): +az postgres flexible-server firewall-rule list --resource-group --name -pg -o table ## Troubleshooting @@ -178,6 +177,15 @@ After `deploy.ps1` completes, you must: - Verify PostgreSQL admin password meets complexity requirements (12+ chars, mixed case, numbers) - Review Azure Portal → Resource Groups → Deployments for error details +### WebUI URL does not respond (startup failures) +- Check active revision health: + az containerapp revision list --resource-group --name -webui -o table +- Check application logs: + az containerapp logs show --resource-group --name -webui --type console --tail 120 +- Most common cause: database connectivity during startup migration. + - Verify `ConnectionStrings__Default` uses `secretRef` (`postgres-conn`). + - Verify PostgreSQL firewall/network access if using public mode. + ## Environment Variables & Configuration **WebUI Container Environment:** @@ -195,13 +203,13 @@ After `deploy.ps1` completes, you must: ## Cleanup -To remove all resources and stop incurring costs: +To clean everything inside a resource group without deleting the resource group itself: ```powershell -az group delete --name mate-dev-rg --yes --no-wait +.\cleanup-rg.ps1 -ResourceGroupName ``` -This will delete all Azure resources in the resource group (cannot be undone). +This script deletes live resources, removes deployment records, purges RG-scoped soft-deleted Key Vault entries, and verifies the RG is empty at the end. ## References diff --git a/infra/azure/scripts/check-prerequisites.ps1 b/infra/azure/scripts/check-prerequisites.ps1 index 54c354f..064be8f 100644 --- a/infra/azure/scripts/check-prerequisites.ps1 +++ b/infra/azure/scripts/check-prerequisites.ps1 @@ -1,3 +1,7 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + <# .SYNOPSIS Validate and display Azure deployment prerequisites. diff --git a/infra/azure/scripts/cleanup-rg.ps1 b/infra/azure/scripts/cleanup-rg.ps1 index c7befe4..0bc0d14 100644 --- a/infra/azure/scripts/cleanup-rg.ps1 +++ b/infra/azure/scripts/cleanup-rg.ps1 @@ -1,3 +1,7 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + <# .SYNOPSIS Cleans all resources from an Azure resource group without deleting the resource group itself. @@ -15,8 +19,10 @@ param( [Parameter(Mandatory = $true)] [string]$ResourceGroupName, + [ValidateRange(60, 7200)] [int]$WaitTimeoutSeconds = 900, + [ValidateRange(2, 120)] [int]$PollIntervalSeconds = 10 ) @@ -42,7 +48,36 @@ if ($rgExists -ne 'true') { throw "Resource group '$ResourceGroupName' does not exist." } -Write-Host 'Step 1/4: Cancel running deployments in RG...' -ForegroundColor Yellow +$rgLower = $ResourceGroupName.ToLower() + +function Get-RgScopedDeletedKeyVaultNames { + param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroupNameLower + ) + + $deletedKvsJson = az keyvault list-deleted -o json 2>$null + if ($LASTEXITCODE -ne 0) { + throw 'Failed to list deleted Key Vaults.' + } + + $deletedKvs = @() + if ($deletedKvsJson) { + $deletedKvs = $deletedKvsJson | ConvertFrom-Json + } + + $names = @() + foreach ($kv in $deletedKvs) { + $id = [string]$kv.id + if (-not [string]::IsNullOrWhiteSpace($id) -and $id.ToLower().Contains("/resourcegroups/$ResourceGroupNameLower/")) { + $names += [string]$kv.name + } + } + + return @($names | Where-Object { $_ -and $_.Trim() } | Select-Object -Unique) +} + +Write-Host 'Step 1/6: Cancel running deployments in RG...' -ForegroundColor Yellow $runningDeployments = az deployment group list --resource-group $ResourceGroupName --query "[?properties.provisioningState=='Running'].name" -o tsv 2>$null if ($LASTEXITCODE -ne 0) { throw 'Failed to list deployments.' @@ -59,7 +94,27 @@ else { Write-Host ' No running deployments found.' -ForegroundColor Gray } -Write-Host 'Step 2/4: Delete all live resources in RG...' -ForegroundColor Yellow +Write-Host 'Step 2/6: Delete deployment records in RG...' -ForegroundColor Yellow +$allDeployments = az deployment group list --resource-group $ResourceGroupName --query '[].name' -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed to list deployment records.' +} + +if ($allDeployments) { + $deployments = $allDeployments -split "`n" | Where-Object { $_.Trim() } + Write-Host " Deployment records found: $($deployments.Count)" -ForegroundColor Gray + + foreach ($name in $deployments) { + $deploymentName = $name.Trim() + Write-Host " Deleting deployment record: $deploymentName" -ForegroundColor Gray + az deployment group delete --resource-group $ResourceGroupName --name $deploymentName 2>$null | Out-Null + } +} +else { + Write-Host ' No deployment records found.' -ForegroundColor Gray +} + +Write-Host 'Step 3/6: Delete all live resources in RG...' -ForegroundColor Yellow $resourceIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null if ($LASTEXITCODE -ne 0) { throw 'Failed to list resources in resource group.' @@ -78,7 +133,7 @@ else { Write-Host ' No live resources found.' -ForegroundColor Gray } -Write-Host 'Step 3/4: Wait for resource deletions to complete...' -ForegroundColor Yellow +Write-Host 'Step 4/6: Wait for resource deletions to complete...' -ForegroundColor Yellow $start = Get-Date while ($true) { $remainingIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null @@ -102,36 +157,39 @@ while ($true) { Start-Sleep -Seconds $PollIntervalSeconds } -Write-Host 'Step 4/4: Purge RG-scoped soft-deleted Key Vaults...' -ForegroundColor Yellow -$deletedKvsJson = az keyvault list-deleted -o json 2>$null -if ($LASTEXITCODE -ne 0) { - throw 'Failed to list deleted Key Vaults.' -} - -$deletedKvs = @() -if ($deletedKvsJson) { - $deletedKvs = $deletedKvsJson | ConvertFrom-Json -} - -$rgLower = $ResourceGroupName.ToLower() -$rgScopedDeletedVaults = @() -foreach ($kv in $deletedKvs) { - $id = [string]$kv.id - if (-not [string]::IsNullOrWhiteSpace($id) -and $id.ToLower().Contains("/resourcegroups/$rgLower/")) { - $rgScopedDeletedVaults += [string]$kv.name - } -} +Write-Host 'Step 5/6: Purge RG-scoped soft-deleted Key Vaults...' -ForegroundColor Yellow +$rgScopedDeletedVaults = Get-RgScopedDeletedKeyVaultNames -ResourceGroupNameLower $rgLower if ($rgScopedDeletedVaults.Count -gt 0) { foreach ($name in ($rgScopedDeletedVaults | Select-Object -Unique)) { Write-Host " Purging deleted Key Vault: $name" -ForegroundColor Gray az keyvault purge --name $name 2>$null | Out-Null } + + $purgeStart = Get-Date + while ($true) { + $remainingDeletedVaults = Get-RgScopedDeletedKeyVaultNames -ResourceGroupNameLower $rgLower + if ($remainingDeletedVaults.Count -eq 0) { + Write-Host ' Soft-deleted RG-scoped Key Vault count: 0' -ForegroundColor Green + break + } + + $elapsedPurge = [int]((Get-Date) - $purgeStart).TotalSeconds + Write-Host " Waiting for Key Vault purge... remaining: $($remainingDeletedVaults.Count) (elapsed ${elapsedPurge}s)" -ForegroundColor Gray + + if ($elapsedPurge -ge $WaitTimeoutSeconds) { + $remainingNames = $remainingDeletedVaults -join ', ' + throw "Timeout waiting for Key Vault purge. Remaining soft-deleted vaults: $remainingNames" + } + + Start-Sleep -Seconds $PollIntervalSeconds + } } else { Write-Host ' No RG-scoped deleted Key Vaults found.' -ForegroundColor Gray } +Write-Host 'Step 6/6: Final verification...' -ForegroundColor Yellow $finalLiveIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null if ($LASTEXITCODE -ne 0) { throw 'Failed final resource check.' @@ -142,26 +200,25 @@ if ($finalLiveIds) { $finalLiveCount = ($finalLiveIds -split "`n" | Where-Object { $_.Trim() }).Count } -$finalDeletedKvJson = az keyvault list-deleted -o json 2>$null +$finalDeploymentRecords = az deployment group list --resource-group $ResourceGroupName --query '[].name' -o tsv 2>$null if ($LASTEXITCODE -ne 0) { - throw 'Failed final deleted Key Vault check.' + throw 'Failed final deployment records check.' } -$finalDeletedKv = @() -if ($finalDeletedKvJson) { - $finalDeletedKv = $finalDeletedKvJson | ConvertFrom-Json +$finalDeploymentRecordCount = 0 +if ($finalDeploymentRecords) { + $finalDeploymentRecordCount = ($finalDeploymentRecords -split "`n" | Where-Object { $_.Trim() }).Count } -$finalRgScopedDeletedKvCount = 0 -foreach ($kv in $finalDeletedKv) { - $id = [string]$kv.id - if (-not [string]::IsNullOrWhiteSpace($id) -and $id.ToLower().Contains("/resourcegroups/$rgLower/")) { - $finalRgScopedDeletedKvCount++ - } +$finalRgScopedDeletedKvCount = (Get-RgScopedDeletedKeyVaultNames -ResourceGroupNameLower $rgLower).Count + +if ($finalLiveCount -ne 0 -or $finalDeploymentRecordCount -ne 0 -or $finalRgScopedDeletedKvCount -ne 0) { + throw "Cleanup verification failed. LIVE_RESOURCE_COUNT=$finalLiveCount, DEPLOYMENT_RECORD_COUNT=$finalDeploymentRecordCount, RG_SCOPED_SOFT_DELETED_KV_COUNT=$finalRgScopedDeletedKvCount" } Write-Host '' Write-Host '=== RG Cleanup Complete ===' -ForegroundColor Green Write-Host "LIVE_RESOURCE_COUNT=$finalLiveCount" -ForegroundColor Cyan +Write-Host "DEPLOYMENT_RECORD_COUNT=$finalDeploymentRecordCount" -ForegroundColor Cyan Write-Host "RG_SCOPED_SOFT_DELETED_KV_COUNT=$finalRgScopedDeletedKvCount" -ForegroundColor Cyan Write-Host '' diff --git a/infra/azure/scripts/deploy-whatif.ps1 b/infra/azure/scripts/deploy-whatif.ps1 index 1484d6e..740b4af 100644 --- a/infra/azure/scripts/deploy-whatif.ps1 +++ b/infra/azure/scripts/deploy-whatif.ps1 @@ -1,3 +1,7 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + <# .SYNOPSIS Dry-run deployment to show what Azure resources would be created without actually creating them. @@ -54,7 +58,15 @@ param( [string]$ResourceGroupName, [Parameter(Mandatory = $false)] - [string]$AadClientId + [string]$AadClientId, + + [Parameter(Mandatory = $false)] + [ValidateRange(1, 120)] + [int]$PostgresWaitTimeoutMinutes = 20, + + [Parameter(Mandatory = $false)] + [ValidateRange(5, 300)] + [int]$PostgresWaitPollSeconds = 20 ) $ErrorActionPreference = 'Stop' @@ -130,6 +142,71 @@ Write-Host "Template: $(Split-Path -Leaf $mainTemplate)" -ForegroundColo Write-Host "Parameters: $(Split-Path -Leaf $parameterFile)" -ForegroundColor Green Write-Host "" +function Wait-ForProvisioningPostgres { + param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroup, + + [Parameter(Mandatory = $true)] + [int]$TimeoutMinutes, + + [Parameter(Mandatory = $true)] + [int]$PollSeconds + ) + + $deadline = (Get-Date).AddMinutes($TimeoutMinutes) + + while ($true) { + $serverQueryOutput = az postgres flexible-server list --resource-group $ResourceGroup --query "[].{name:name,state:state}" --output json --only-show-errors + if ($LASTEXITCODE -ne 0) { + $queryError = ($serverQueryOutput | Out-String).Trim() + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Failed to query PostgreSQL servers in resource group '$ResourceGroup' within $TimeoutMinutes minutes. Last error: $queryError" + } + + Write-Host "PostgreSQL state query failed (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + if ($queryError) { + Write-Host " Last provider error: $queryError" -ForegroundColor DarkYellow + } + Start-Sleep -Seconds $PollSeconds + continue + } + + try { + $servers = @($serverQueryOutput | ConvertFrom-Json) + } + catch { + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Received unexpected PostgreSQL query response format within timeout window: $($serverQueryOutput | Out-String)" + } + + Write-Host "Received non-JSON PostgreSQL state response (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + continue + } + + if ($servers.Count -eq 0) { + return + } + + $blockingServers = @($servers | Where-Object { $_.state -and $_.state -match 'Provisioning|Updating|Starting|Stopping' }) + if ($blockingServers.Count -eq 0) { + return + } + + $stateSummary = ($blockingServers | ForEach-Object { "$($_.name):$($_.state)" }) -join ', ' + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "PostgreSQL server operations did not complete within $TimeoutMinutes minutes. Current state(s): $stateSummary" + } + + Write-Host "Waiting for PostgreSQL server operations to finish before what-if... ($stateSummary, $remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + } +} + # Prompt for values needed when deployPostgres=true $pgPasswordFile = Join-Path $scriptDir '.pg-password' $postgresPasswordPlain = $null @@ -174,7 +251,10 @@ if ($rgExists -ne 'true') { } } -Write-Host "3. Running what-if..." -ForegroundColor Magenta +Write-Host "3. Checking PostgreSQL server readiness..." -ForegroundColor Magenta +Wait-ForProvisioningPostgres -ResourceGroup $ResourceGroupName -TimeoutMinutes $PostgresWaitTimeoutMinutes -PollSeconds $PostgresWaitPollSeconds + +Write-Host "4. Running what-if..." -ForegroundColor Magenta $whatIfArgs = @( 'deployment', 'group', 'what-if', '--resource-group', $ResourceGroupName, @@ -188,7 +268,12 @@ foreach ($key in $deploymentParams.Keys) { az @whatIfArgs --result-format FullResourcePayloads if ($LASTEXITCODE -ne 0) { - Write-Error "What-if failed. Review the errors above." + Write-Host "What-if failed on first attempt. Re-checking PostgreSQL state and retrying once..." -ForegroundColor Yellow + Wait-ForProvisioningPostgres -ResourceGroup $ResourceGroupName -TimeoutMinutes $PostgresWaitTimeoutMinutes -PollSeconds $PostgresWaitPollSeconds + az @whatIfArgs --result-format FullResourcePayloads + if ($LASTEXITCODE -ne 0) { + Write-Error "What-if failed after retry. Review the errors above." + } } Write-Host "" diff --git a/infra/azure/scripts/deploy.ps1 b/infra/azure/scripts/deploy.ps1 index cb2a33c..9de9212 100644 --- a/infra/azure/scripts/deploy.ps1 +++ b/infra/azure/scripts/deploy.ps1 @@ -1,3 +1,7 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + <# .SYNOPSIS Deploy the Azure Mate infrastructure to your tenant and subscription. @@ -70,6 +74,14 @@ param( [Parameter(Mandatory = $false)] [string]$AadClientId, + [Parameter(Mandatory = $false)] + [ValidateRange(1, 120)] + [int]$PostgresWaitTimeoutMinutes = 20, + + [Parameter(Mandatory = $false)] + [ValidateRange(5, 300)] + [int]$PostgresWaitPollSeconds = 20, + [Parameter(Mandatory = $false)] [switch]$Force ) @@ -174,6 +186,71 @@ Write-Host "Template: $(Split-Path -Leaf $mainTemplate)" -ForegroundColo Write-Host "Parameters: $(Split-Path -Leaf $parameterFile)" -ForegroundColor Green Write-Host "" +function Wait-ForProvisioningPostgres { + param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroup, + + [Parameter(Mandatory = $true)] + [int]$TimeoutMinutes, + + [Parameter(Mandatory = $true)] + [int]$PollSeconds + ) + + $deadline = (Get-Date).AddMinutes($TimeoutMinutes) + + while ($true) { + $serverQueryOutput = az postgres flexible-server list --resource-group $ResourceGroup --query "[].{name:name,state:state}" --output json --only-show-errors + if ($LASTEXITCODE -ne 0) { + $queryError = ($serverQueryOutput | Out-String).Trim() + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Failed to query PostgreSQL servers in resource group '$ResourceGroup' within $TimeoutMinutes minutes. Last error: $queryError" + } + + Write-Host "PostgreSQL state query failed (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + if ($queryError) { + Write-Host " Last provider error: $queryError" -ForegroundColor DarkYellow + } + Start-Sleep -Seconds $PollSeconds + continue + } + + try { + $servers = @($serverQueryOutput | ConvertFrom-Json) + } + catch { + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Received unexpected PostgreSQL query response format within timeout window: $($serverQueryOutput | Out-String)" + } + + Write-Host "Received non-JSON PostgreSQL state response (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + continue + } + + if ($servers.Count -eq 0) { + return + } + + $blockingServers = @($servers | Where-Object { $_.state -and $_.state -match 'Provisioning|Updating|Starting|Stopping' }) + if ($blockingServers.Count -eq 0) { + return + } + + $stateSummary = ($blockingServers | ForEach-Object { "$($_.name):$($_.state)" }) -join ', ' + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "PostgreSQL server operations did not complete within $TimeoutMinutes minutes. Current state(s): $stateSummary" + } + + Write-Host "Waiting for PostgreSQL server operations to finish before deployment... ($stateSummary, $remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + } +} + # Prompt for secure inputs Write-Host "SECURITY: Prompting for sensitive inputs..." -ForegroundColor Cyan $pgPasswordFile = Join-Path $scriptDir '.pg-password' @@ -221,7 +298,8 @@ Write-Host "" Write-Host "Deployment steps:" -ForegroundColor Yellow Write-Host "1. Validate Bicep template" -ForegroundColor Magenta Write-Host "2. Create resource group (if needed)" -ForegroundColor Magenta -Write-Host "3. Deploy infrastructure" -ForegroundColor Magenta +Write-Host "3. Check PostgreSQL readiness" -ForegroundColor Magenta +Write-Host "4. Deploy infrastructure" -ForegroundColor Magenta Write-Host "" Write-Host "Executing deployment..." -ForegroundColor Cyan @@ -242,7 +320,11 @@ if ($rgExists -ne 'true') { } } -# 3) Deploy +# 3) Check PostgreSQL state before deployment starts +Write-Host "Checking PostgreSQL server readiness..." -ForegroundColor Gray +Wait-ForProvisioningPostgres -ResourceGroup $ResourceGroupName -TimeoutMinutes $PostgresWaitTimeoutMinutes -PollSeconds $PostgresWaitPollSeconds + +# 4) Deploy $deployArgs = @( 'deployment', 'group', 'create', '--resource-group', $ResourceGroupName, @@ -291,7 +373,7 @@ function Monitor-Deployment { $deployInfo = az deployment group show --resource-group $ResourceGroup --name $DeploymentName --query '{state:properties.provisioningState}' -o json 2>$null | ConvertFrom-Json # Get failed operations for diagnostics - $failedOps = az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[?properties.provisioningState=='Failed'].[properties.targetResource.resourceName, properties.statusMessage.error.code, properties.statusMessage.error.message]" -o json 2>$null | ConvertFrom-Json + $failedOps = az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[?properties.provisioningState=='Failed'].{name:properties.targetResource.resourceName, code:properties.statusMessage.error.code, message:properties.statusMessage.error.message}" -o json 2>$null | ConvertFrom-Json # Get all operations for progress count $allOps = az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[].properties.targetResource.resourceName" -o json 2>$null | ConvertFrom-Json @@ -312,10 +394,12 @@ function Monitor-Deployment { # Clear previous line and show progress $spinner = Get-ProgressSpinner $spinnerIndex $progressPct = if ($totalCount -gt 0) { [math]::Round(($completedCount / $totalCount) * 100) } else { 0 } + $progressPct = [math]::Max(0, [math]::Min(100, $progressPct)) # Build progress bar $barLength = 30 $filledLength = [math]::Round(($progressPct / 100) * $barLength) + $filledLength = [math]::Max(0, [math]::Min($barLength, $filledLength)) $emptyLength = $barLength - $filledLength $bar = ('▓' * $filledLength) + ('░' * $emptyLength) @@ -337,9 +421,11 @@ function Monitor-Deployment { Write-Host "" Write-Host "Failed operations:" -ForegroundColor Red $failedOps | ForEach-Object { - Write-Host " • $($_[0]): $($_[1])" -ForegroundColor Red - if ($_[2]) { - Write-Host " Detail: $($_[2])" -ForegroundColor DarkRed + $resourceName = if ($_.name) { $_.name } else { "(unnamed)" } + $errorCode = if ($_.code) { $_.code } else { "Unknown" } + Write-Host " • ${resourceName}: $errorCode" -ForegroundColor Red + if ($_.message) { + Write-Host " Detail: $($_.message)" -ForegroundColor DarkRed } } } @@ -360,7 +446,7 @@ $deploymentStartTime = Get-Date Write-Host "Start time: $($deploymentStartTime.ToString('yyyy-MM-dd HH:mm:ss'))" -ForegroundColor Gray # Run the deployment -$deployOutput = az @deployArgs --no-wait -o json 2>&1 +$deployOutput = az @deployArgs --no-wait -o json if ($LASTEXITCODE -eq 0) { $deploymentInfo = $deployOutput | ConvertFrom-Json @@ -378,15 +464,78 @@ if ($LASTEXITCODE -eq 0) { # Retrieve outputs Write-Host "" Write-Host "Retrieving deployment outputs..." -ForegroundColor Cyan - $finalOutput = az deployment group show --resource-group $ResourceGroupName --name $deploymentName --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json 2>&1 + $finalOutput = az deployment group show --resource-group $ResourceGroupName --name $deploymentName --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json if ($LASTEXITCODE -eq 0) { $deployOutput = $finalOutput Write-Host $deployOutput } } else { - # Deployment monitoring revealed failure or timeout - Write-Error "Deployment was not successful" + # Deployment monitoring revealed failure or timeout. + # If this is the known first-run Key Vault secret bootstrap issue, + # run setup-keyvault-secrets automatically and retry once. + $credentialsFile = Join-Path $scriptDir '.credentials' + $failedOps = @() + $failedOpsRaw = az deployment operation group list --resource-group $ResourceGroupName --name $deploymentName --query "[?properties.provisioningState=='Failed'].{code:properties.statusMessage.error.code,message:properties.statusMessage.error.message}" -o json 2>$null + if ($LASTEXITCODE -eq 0 -and $failedOpsRaw) { + $failedOps = @($failedOpsRaw | ConvertFrom-Json) + } + + $needsSecretBootstrap = @( + $failedOps | Where-Object { + ($_.code -eq 'ContainerAppOperationError' -and $_.message -match 'azuread-client-secret') -or + ($_.code -eq 'ContainerAppSecretKeyVaultUrlInvalid') + } + ).Count -gt 0 + + if ($needsSecretBootstrap -and (Test-Path $credentialsFile)) { + Write-Host "" + Write-Host "Detected Container Apps Key Vault secret bootstrap failure." -ForegroundColor Yellow + Write-Host "Running setup-keyvault-secrets.ps1 automatically and retrying deployment once..." -ForegroundColor Yellow + + $setupScript = Join-Path $scriptDir 'setup-keyvault-secrets.ps1' + $setupSucceeded = $false + try { + & $setupScript + $setupSucceeded = $true + } + catch { + Write-Host "Automatic setup-keyvault-secrets step failed: $_" -ForegroundColor Red + } + + if ($setupSucceeded) { + Write-Host "" + Write-Host "Retrying deployment after secret bootstrap..." -ForegroundColor Yellow + + $retryOutput = az @deployArgs --no-wait -o json + if ($LASTEXITCODE -ne 0) { + Write-Error "Deployment retry failed to queue. Details: $($retryOutput | Out-String)" + } + + $retryInfo = $retryOutput | ConvertFrom-Json + $retryDeploymentName = if ($retryInfo.name) { $retryInfo.name } else { 'main' } + + Write-Host "Monitoring retry deployment: $retryDeploymentName" -ForegroundColor Cyan + $retrySuccess = Monitor-Deployment -ResourceGroup $ResourceGroupName -DeploymentName $retryDeploymentName + if (-not $retrySuccess) { + Write-Error "Deployment was not successful after automatic bootstrap retry" + } + + Write-Host "" + Write-Host "Retrieving deployment outputs..." -ForegroundColor Cyan + $finalOutput = az deployment group show --resource-group $ResourceGroupName --name $retryDeploymentName --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json + if ($LASTEXITCODE -eq 0) { + $deployOutput = $finalOutput + Write-Host $deployOutput + } + } + else { + Write-Error "Deployment was not successful and automatic secret bootstrap failed" + } + } + else { + Write-Error "Deployment was not successful" + } } } else { $outputText = ($deployOutput | Out-String) @@ -401,7 +550,7 @@ if ($LASTEXITCODE -eq 0) { az deployment group cancel --resource-group $ResourceGroupName --name $activeDeploymentName 2>$null | Out-Null Start-Sleep -Seconds 10 - $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json 2>&1 + $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json if ($LASTEXITCODE -ne 0) { $outputText = ($deployOutput | Out-String) } @@ -418,7 +567,7 @@ if ($LASTEXITCODE -eq 0) { Start-Sleep -Seconds 15 } - $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json 2>&1 + $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json if ($LASTEXITCODE -ne 0) { Write-Error "Deployment failed after Key Vault purge retry. Details: $($deployOutput | Out-String)" } @@ -431,12 +580,93 @@ if ($LASTEXITCODE -eq 0) { $deploymentEndTime = Get-Date $totalDuration = ($deploymentEndTime - $deploymentStartTime).TotalSeconds +# Post-deployment: ensure Container Apps use a real PostgreSQL connection string secret +if ($deploymentParams['deployPostgres'] -eq $true -and $postgresPasswordPlain) { + Write-Host "Ensuring PostgreSQL connectivity for Azure-hosted Container Apps..." -ForegroundColor Gray + + $postgresServerName = "$EnvironmentName-pg" + $publicNetworkAccess = az postgres flexible-server show ` + --resource-group $ResourceGroupName ` + --name $postgresServerName ` + --query 'network.publicNetworkAccess' -o tsv 2>$null + + if ($LASTEXITCODE -eq 0 -and $publicNetworkAccess -eq 'Enabled') { + $allowAzureRuleCount = az postgres flexible-server firewall-rule list ` + --resource-group $ResourceGroupName ` + --name $postgresServerName ` + --query "[?startIpAddress=='0.0.0.0' && endIpAddress=='0.0.0.0'] | length(@)" -o tsv 2>$null + + if ($LASTEXITCODE -eq 0 -and $allowAzureRuleCount -eq '0') { + Write-Host " Creating PostgreSQL firewall rule to allow Azure services..." -ForegroundColor Gray + az postgres flexible-server firewall-rule create ` + --resource-group $ResourceGroupName ` + --name $postgresServerName ` + --rule-name 'AllowAzureServices' ` + --start-ip-address '0.0.0.0' ` + --end-ip-address '0.0.0.0' ` + -o none 2>$null + + if ($LASTEXITCODE -eq 0) { + Write-Host " PostgreSQL firewall rule ensured." -ForegroundColor Green + } + else { + Write-Host " Could not create PostgreSQL firewall rule automatically; verify network access manually if app startup fails." -ForegroundColor Yellow + } + } + elseif ($LASTEXITCODE -eq 0) { + Write-Host " PostgreSQL firewall already allows Azure services." -ForegroundColor Gray + } + } + elseif ($LASTEXITCODE -eq 0) { + Write-Host " PostgreSQL public network access is disabled; expecting private networking configuration." -ForegroundColor Gray + } + + Write-Host "Configuring Container Apps database connection secrets..." -ForegroundColor Gray + + $dbConnectionString = "Host=$EnvironmentName-pg.postgres.database.azure.com;Database=mate;Username=pgadmin;Password=$postgresPasswordPlain;SSL Mode=Require" + $containerApps = @("$EnvironmentName-webui", "$EnvironmentName-worker") + + foreach ($appName in $containerApps) { + $appId = az containerapp show --resource-group $ResourceGroupName --name $appName --query id -o tsv 2>$null + if ($LASTEXITCODE -ne 0 -or -not $appId) { + Write-Host " Skipping '$appName' (not found in RG)." -ForegroundColor DarkYellow + continue + } + + az containerapp secret set ` + --resource-group $ResourceGroupName ` + --name $appName ` + --secrets "postgres-conn=$dbConnectionString" ` + -o none 2>$null + + if ($LASTEXITCODE -ne 0) { + Write-Host " Failed to set DB secret on '$appName'." -ForegroundColor Yellow + continue + } + + az containerapp update ` + --resource-group $ResourceGroupName ` + --name $appName ` + --set-env-vars "ConnectionStrings__Default=secretref:postgres-conn" ` + -o none 2>$null + + if ($LASTEXITCODE -ne 0) { + Write-Host " Failed to update DB env on '$appName'." -ForegroundColor Yellow + continue + } + + Write-Host " Configured database secret reference on '$appName'." -ForegroundColor Green + } + + Write-Host "Container App database secret configuration completed." -ForegroundColor Gray + Write-Host "" +} + Write-Host "" Write-Host "═" * 60 -ForegroundColor Green Write-Host "✓ Deployment completed successfully" -ForegroundColor Green Write-Host " Duration: $('{0:mm\:ss}' -f [timespan]::FromSeconds($totalDuration))" -ForegroundColor Green Write-Host "═" * 60 -ForegroundColor Green Write-Host "" -Write-Host "Next step:" -ForegroundColor Yellow -Write-Host " .\setup-keyvault-secrets.ps1" -ForegroundColor Magenta +Write-Host "Key Vault secret bootstrap and retry are handled automatically when .credentials is available." -ForegroundColor Gray Write-Host "" diff --git a/infra/azure/scripts/setup-env.ps1 b/infra/azure/scripts/setup-env.ps1 index 3a8ec27..5755919 100644 --- a/infra/azure/scripts/setup-env.ps1 +++ b/infra/azure/scripts/setup-env.ps1 @@ -1,3 +1,7 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + <# .SYNOPSIS Interactive setup wizard for Azure deployment environment variables. @@ -158,7 +162,7 @@ else { $appName = "mate-webui-$envName" Write-Host " Creating app registration '$appName'..." -ForegroundColor Gray - $result = az ad app create --display-name $appName --sign-in-audience AzureADMyOrg 2>&1 + $result = az ad app create --display-name $appName --sign-in-audience AzureADMyOrg if ($LASTEXITCODE -ne 0) { Write-Host "" Write-Error "Failed to create app registration. Error: $result`n`nPlease create manually and re-run setup." @@ -232,7 +236,7 @@ elseif ($secretOption -eq '2') { Write-Host " Creating client secret for app $aadClientId..." -ForegroundColor Gray Write-Host " Secret description: $secretDisplayName" -ForegroundColor Gray - $secretResult = az ad app credential reset --id $aadClientId --append --display-name $secretDisplayName --query password -o tsv 2>&1 + $secretResult = az ad app credential reset --id $aadClientId --append --display-name $secretDisplayName --query password -o tsv if ($LASTEXITCODE -ne 0) { Write-Host "" Write-Error "Failed to create client secret. Error: $secretResult`n`nPlease create manually in Azure Portal and re-run setup." diff --git a/infra/azure/scripts/setup-keyvault-secrets.ps1 b/infra/azure/scripts/setup-keyvault-secrets.ps1 index 9baa3e4..584be1d 100644 --- a/infra/azure/scripts/setup-keyvault-secrets.ps1 +++ b/infra/azure/scripts/setup-keyvault-secrets.ps1 @@ -1,3 +1,7 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + <# .SYNOPSIS Post-deployment setup: Store client secret in Key Vault and configure RBAC. @@ -52,6 +56,47 @@ if (-not $credsFile -or -not (Test-Path $credsFile)) { $creds = Get-Content $credsFile | ConvertFrom-Json $aadClientSecret = $creds.AAD_CLIENT_SECRET +function Ensure-CallerCanManageKeyVaultSecrets { + param( + [Parameter(Mandatory = $true)] + [string]$KeyVaultId + ) + + $caller = az account show --query "{name:user.name,type:user.type}" -o json | ConvertFrom-Json + if ($LASTEXITCODE -ne 0 -or -not $caller -or -not $caller.name) { + Write-Error "Unable to determine current Azure caller from az account show" + } + + $existingAssignmentCount = az role assignment list ` + --assignee $caller.name ` + --scope $KeyVaultId ` + --query "[?roleDefinitionName=='Key Vault Secrets Officer'] | length(@)" -o tsv + + if ($LASTEXITCODE -ne 0) { + Write-Warning "Could not verify existing Key Vault Secrets Officer role assignment; attempting assignment" + $existingAssignmentCount = '0' + } + + if ($existingAssignmentCount -eq '0') { + Write-Host " Granting caller '$($caller.name)' role 'Key Vault Secrets Officer' on Key Vault..." -ForegroundColor Gray + $assignOutput = az role assignment create ` + --assignee $caller.name ` + --role "Key Vault Secrets Officer" ` + --scope $KeyVaultId ` + -o json 2>&1 + + if ($LASTEXITCODE -ne 0) { + $assignText = ($assignOutput | Out-String).Trim() + Write-Error "Failed to grant Key Vault Secrets Officer to caller '$($caller.name)'. Error: $assignText" + } + + Write-Host " Caller role assignment created" -ForegroundColor Gray + } + else { + Write-Host " Caller already has Key Vault Secrets Officer role" -ForegroundColor Gray + } +} + Write-Host "Configuration Summary:" -ForegroundColor Yellow Write-Host " Tenant ID: $tenantId" Write-Host " Subscription ID: $subscriptionId" @@ -64,7 +109,7 @@ Write-Host "" # Step 1: Verify subscription context Write-Host "Step 1: Verifying Azure subscription context..." -ForegroundColor Cyan try { - $currentSubId = az account show --query id -o tsv 2>&1 + $currentSubId = az account show --query id -o tsv if ($LASTEXITCODE -ne 0) { Write-Error "Not logged in. Run: az login --tenant $tenantId" } @@ -87,12 +132,14 @@ Write-Host "" # Step 2: Verify Key Vault exists Write-Host "Step 2: Verifying Key Vault..." -ForegroundColor Cyan $keyVaultName = "$environmentName-kv" +$kvId = $null try { - $kvExists = az keyvault show --name $keyVaultName --resource-group $resourceGroup -o tsv 2>&1 | Measure-Object | Select-Object -ExpandProperty Count - - if ($kvExists -eq 0) { + $keyVault = az keyvault show --name $keyVaultName --resource-group $resourceGroup -o json | ConvertFrom-Json + if ($LASTEXITCODE -ne 0 -or -not $keyVault) { Write-Error "Key Vault '$keyVaultName' not found in resource group '$resourceGroup'. Verify deployment completed successfully." } + + $kvId = $keyVault.id Write-Host "✓ Key Vault found: $keyVaultName" -ForegroundColor Green } @@ -105,16 +152,19 @@ Write-Host "" # Step 3: Store client secret in Key Vault Write-Host "Step 3: Storing client secret in Key Vault..." -ForegroundColor Cyan try { + Ensure-CallerCanManageKeyVaultSecrets -KeyVaultId $kvId + Write-Host " Storing secret 'azuread-client-secret' in $keyVaultName..." -ForegroundColor Gray - az keyvault secret set ` + $setSecretOutput = az keyvault secret set ` --vault-name $keyVaultName ` --name "azuread-client-secret" ` --value $aadClientSecret ` - 2>&1 | Out-Null + --query id -o tsv 2>&1 if ($LASTEXITCODE -ne 0) { - Write-Error "Failed to store secret in Key Vault" + $setSecretText = ($setSecretOutput | Out-String).Trim() + Write-Error "Failed to store secret in Key Vault. Error: $setSecretText" } Write-Host "✓ Client secret stored successfully" -ForegroundColor Green @@ -134,7 +184,7 @@ try { $webMiPrincipalId = az identity show ` --resource-group $resourceGroup ` --name $webMiName ` - --query principalId -o tsv 2>&1 + --query principalId -o tsv if ($LASTEXITCODE -ne 0) { Write-Error "Failed to get managed identity '$webMiName'" @@ -142,27 +192,36 @@ try { Write-Host " Principal ID: $webMiPrincipalId" -ForegroundColor Gray - # Get Key Vault resource ID - $kvId = az keyvault show ` - --name $keyVaultName ` - --resource-group $resourceGroup ` - --query id -o tsv 2>&1 - Write-Host " Granting 'Key Vault Secrets User' role..." -ForegroundColor Gray - - az role assignment create ` - --assignee $webMiPrincipalId ` - --role "Key Vault Secrets User" ` + + $existingMiAssignmentCount = az role assignment list ` + --assignee-object-id $webMiPrincipalId ` --scope $kvId ` - 2>&1 | Out-Null - + --query "[?roleDefinitionName=='Key Vault Secrets User'] | length(@)" -o tsv + if ($LASTEXITCODE -ne 0) { - # Role assignment already exists, that's OK - Write-Host " (Role may already be assigned)" -ForegroundColor Gray + Write-Warning "Could not verify existing managed identity assignment; attempting assignment" + $existingMiAssignmentCount = '0' } - else { + + if ($existingMiAssignmentCount -eq '0') { + $miAssignOutput = az role assignment create ` + --assignee-object-id $webMiPrincipalId ` + --assignee-principal-type ServicePrincipal ` + --role "Key Vault Secrets User" ` + --scope $kvId ` + -o json 2>&1 + + if ($LASTEXITCODE -ne 0) { + $miAssignText = ($miAssignOutput | Out-String).Trim() + Write-Error "Failed to assign Key Vault Secrets User to managed identity. Error: $miAssignText" + } + Write-Host " Role assignment created" -ForegroundColor Gray } + else { + Write-Host " Managed identity already has Key Vault Secrets User role" -ForegroundColor Gray + } Write-Host "✓ Managed identity configured" -ForegroundColor Green } @@ -178,7 +237,7 @@ try { $secret = az keyvault secret show ` --vault-name $keyVaultName ` --name "azuread-client-secret" ` - --query value -o tsv 2>&1 + --query value -o tsv if ($secret -and $secret -eq $aadClientSecret) { Write-Host "✓ Secret verified successfully" -ForegroundColor Green @@ -200,11 +259,16 @@ if (Test-Path $pgPassFile) { $pgPassword = Get-Content $pgPassFile -Raw Write-Host " Storing 'postgres-admin-password'..." -ForegroundColor Gray - az keyvault secret set ` + $setPgOutput = az keyvault secret set ` --vault-name $keyVaultName ` --name "postgres-admin-password" ` --value $pgPassword ` - 2>&1 | Out-Null + --query id -o tsv 2>&1 + + if ($LASTEXITCODE -ne 0) { + $setPgText = ($setPgOutput | Out-String).Trim() + Write-Error "Failed to store PostgreSQL password in Key Vault. Error: $setPgText" + } Write-Host "✓ PostgreSQL password stored" -ForegroundColor Green From 6cea82778d08e45033fd8c2e59204a4c12a4d920 Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Sun, 8 Mar 2026 20:02:20 +0100 Subject: [PATCH 5/8] Automate blob secret wiring in Azure deployment; fix rate-limit comment typo - Extended deploy.ps1 post-deployment to configure blob-conn secret for both WebUI & Worker - Updated DEPLOYMENT.md to reflect blob connectivity automation - Fixed GenAIToolPlanner typo in TestSuite.cs comment (should be enAIToolPlanner) - Prevents future manual secret configuration needed during deployment --- infra/azure/scripts/DEPLOYMENT.md | 1 + infra/azure/scripts/deploy.ps1 | 48 ++++++++++++++++++---- src/Core/mate.Domain/Entities/TestSuite.cs | 2 +- 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/infra/azure/scripts/DEPLOYMENT.md b/infra/azure/scripts/DEPLOYMENT.md index fc9d35a..838f10f 100644 --- a/infra/azure/scripts/DEPLOYMENT.md +++ b/infra/azure/scripts/DEPLOYMENT.md @@ -90,6 +90,7 @@ Validates that Azure CLI, Bicep, and PowerShell are installed. - Automatically recover known first-run Key Vault secret bootstrap failures (when `.credentials` is available) - Ensure PostgreSQL connectivity prerequisites for Azure-hosted apps when public network mode is used - Configure WebUI/Worker runtime DB secret references (`ConnectionStrings__Default` -> `secretref:postgres-conn`) +- Configure WebUI/Worker runtime Blob secret references (`AzureInfrastructure__BlobConnectionString` -> `secretref:blob-conn`) **To override .env values**, pass parameters: diff --git a/infra/azure/scripts/deploy.ps1 b/infra/azure/scripts/deploy.ps1 index 9de9212..85c780f 100644 --- a/infra/azure/scripts/deploy.ps1 +++ b/infra/azure/scripts/deploy.ps1 @@ -621,9 +621,33 @@ if ($deploymentParams['deployPostgres'] -eq $true -and $postgresPasswordPlain) { Write-Host " PostgreSQL public network access is disabled; expecting private networking configuration." -ForegroundColor Gray } - Write-Host "Configuring Container Apps database connection secrets..." -ForegroundColor Gray + Write-Host "Configuring Container Apps runtime secret references..." -ForegroundColor Gray $dbConnectionString = "Host=$EnvironmentName-pg.postgres.database.azure.com;Database=mate;Username=pgadmin;Password=$postgresPasswordPlain;SSL Mode=Require" + + $storageAccountName = az resource list ` + --resource-group $ResourceGroupName ` + --resource-type 'Microsoft.Storage/storageAccounts' ` + --query "[0].name" -o tsv 2>$null + + $blobConnectionString = $null + if ($LASTEXITCODE -eq 0 -and $storageAccountName) { + $storageKey = az storage account keys list ` + --resource-group $ResourceGroupName ` + --account-name $storageAccountName ` + --query "[0].value" -o tsv 2>$null + + if ($LASTEXITCODE -eq 0 -and $storageKey) { + $blobConnectionString = "DefaultEndpointsProtocol=https;AccountName=$storageAccountName;AccountKey=$storageKey;EndpointSuffix=core.windows.net" + } + else { + Write-Host " Could not resolve storage account key automatically; blob runtime secret will not be updated." -ForegroundColor Yellow + } + } + else { + Write-Host " No storage account found in resource group; blob runtime secret will not be updated." -ForegroundColor Yellow + } + $containerApps = @("$EnvironmentName-webui", "$EnvironmentName-worker") foreach ($appName in $containerApps) { @@ -633,32 +657,42 @@ if ($deploymentParams['deployPostgres'] -eq $true -and $postgresPasswordPlain) { continue } + $secrets = @("postgres-conn=$dbConnectionString") + if ($blobConnectionString) { + $secrets += "blob-conn=$blobConnectionString" + } + az containerapp secret set ` --resource-group $ResourceGroupName ` --name $appName ` - --secrets "postgres-conn=$dbConnectionString" ` + --secrets $secrets ` -o none 2>$null if ($LASTEXITCODE -ne 0) { - Write-Host " Failed to set DB secret on '$appName'." -ForegroundColor Yellow + Write-Host " Failed to set runtime secrets on '$appName'." -ForegroundColor Yellow continue } + $envUpdates = @("ConnectionStrings__Default=secretref:postgres-conn") + if ($blobConnectionString) { + $envUpdates += "AzureInfrastructure__BlobConnectionString=secretref:blob-conn" + } + az containerapp update ` --resource-group $ResourceGroupName ` --name $appName ` - --set-env-vars "ConnectionStrings__Default=secretref:postgres-conn" ` + --set-env-vars $envUpdates ` -o none 2>$null if ($LASTEXITCODE -ne 0) { - Write-Host " Failed to update DB env on '$appName'." -ForegroundColor Yellow + Write-Host " Failed to update runtime env on '$appName'." -ForegroundColor Yellow continue } - Write-Host " Configured database secret reference on '$appName'." -ForegroundColor Green + Write-Host " Configured runtime secret references on '$appName'." -ForegroundColor Green } - Write-Host "Container App database secret configuration completed." -ForegroundColor Gray + Write-Host "Container App runtime secret configuration completed." -ForegroundColor Gray Write-Host "" } diff --git a/src/Core/mate.Domain/Entities/TestSuite.cs b/src/Core/mate.Domain/Entities/TestSuite.cs index 417c163..39db052 100644 --- a/src/Core/mate.Domain/Entities/TestSuite.cs +++ b/src/Core/mate.Domain/Entities/TestSuite.cs @@ -21,7 +21,7 @@ public class TestSuite /// /// Optional delay (milliseconds) between executing individual test cases. - /// Use to avoid triggering rate-limit errors on the target agent (e.g. GenAIToolPlannerRateLimitReached). + /// Use to avoid triggering rate-limit errors on the target agent (e.g. enAIToolPlannerRateLimitReached). /// 0 = no delay (default). /// public int DelayBetweenTestsMs { get; set; } = 0; From eecf02abb858153c84a33da57cd7591f3ef243fa Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Mon, 9 Mar 2026 09:09:02 +0100 Subject: [PATCH 6/8] Add: Quickstart for Azure deployments - scripts based --- .github/workflows/docker-publish.yml | 121 +++- README.md | 18 +- VERSION | 2 +- quickstart-azure/.env.template | 76 +++ quickstart-azure/DEPLOYMENT.md | 220 ++++++ quickstart-azure/QUICKSTART.md | 117 ++++ quickstart-azure/README.md | 300 +++++++++ quickstart-azure/check-prerequisites.ps1 | 95 +++ quickstart-azure/cleanup-rg.ps1 | 224 +++++++ quickstart-azure/deploy-whatif.ps1 | 282 ++++++++ quickstart-azure/deploy.ps1 | 706 ++++++++++++++++++++ quickstart-azure/setup-env.ps1 | 388 +++++++++++ quickstart-azure/setup-keyvault-secrets.ps1 | 314 +++++++++ 13 files changed, 2835 insertions(+), 28 deletions(-) create mode 100644 quickstart-azure/.env.template create mode 100644 quickstart-azure/DEPLOYMENT.md create mode 100644 quickstart-azure/QUICKSTART.md create mode 100644 quickstart-azure/README.md create mode 100644 quickstart-azure/check-prerequisites.ps1 create mode 100644 quickstart-azure/cleanup-rg.ps1 create mode 100644 quickstart-azure/deploy-whatif.ps1 create mode 100644 quickstart-azure/deploy.ps1 create mode 100644 quickstart-azure/setup-env.ps1 create mode 100644 quickstart-azure/setup-keyvault-secrets.ps1 diff --git a/.github/workflows/docker-publish.yml b/.github/workflows/docker-publish.yml index e173c23..5f2726c 100644 --- a/.github/workflows/docker-publish.yml +++ b/.github/workflows/docker-publish.yml @@ -4,6 +4,12 @@ on: push: tags: - "v*.*.*" + workflow_dispatch: + inputs: + prerelease_suffix: + description: "Optional prerelease suffix (example: rc.1). If empty, branch.run_number is used" + required: false + type: string env: REGISTRY: ghcr.io @@ -18,6 +24,10 @@ jobs: outputs: version: ${{ steps.version.outputs.value }} + base_version: ${{ steps.version.outputs.base_value }} + image_tag: ${{ steps.version.outputs.image_tag }} + prerelease: ${{ steps.version.outputs.prerelease }} + tag_name: ${{ steps.version.outputs.tag_name }} strategy: fail-fast: false @@ -36,12 +46,37 @@ jobs: id: version run: | VERSION=$(cat VERSION | tr -d '[:space:]') - TAG="${GITHUB_REF_NAME}" - if [ "$VERSION" != "$TAG" ]; then - echo "::error::VERSION file ($VERSION) does not match git tag ($TAG)" - exit 1 + + if [ "${{ github.event_name }}" = "push" ]; then + TAG="${GITHUB_REF_NAME}" + if [ "$VERSION" != "$TAG" ]; then + echo "::error::VERSION file ($VERSION) does not match git tag ($TAG)" + exit 1 + fi + + RELEASE_VERSION="$VERSION" + RELEASE_TAG="$TAG" + PRERELEASE="false" + else + BRANCH=$(echo "${GITHUB_REF_NAME}" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/-\+/-/g' | sed 's/^-//' | sed 's/-$//') + SUFFIX="${{ inputs.prerelease_suffix }}" + + if [ -z "$SUFFIX" ]; then + SUFFIX="${BRANCH}.${GITHUB_RUN_NUMBER}" + fi + + RELEASE_VERSION="${VERSION}-${SUFFIX}" + RELEASE_TAG="$RELEASE_VERSION" + PRERELEASE="true" fi - echo "value=$VERSION" >> "$GITHUB_OUTPUT" + + IMAGE_TAG="${RELEASE_VERSION#v}" + + echo "value=$RELEASE_VERSION" >> "$GITHUB_OUTPUT" + echo "base_value=$VERSION" >> "$GITHUB_OUTPUT" + echo "image_tag=$IMAGE_TAG" >> "$GITHUB_OUTPUT" + echo "prerelease=$PRERELEASE" >> "$GITHUB_OUTPUT" + echo "tag_name=$RELEASE_TAG" >> "$GITHUB_OUTPUT" - name: Set up QEMU (for multi-arch builds) uses: docker/setup-qemu-action@v3 @@ -62,12 +97,10 @@ jobs: with: images: ${{ env.REGISTRY }}/${{ github.repository_owner }}/${{ matrix.image }} tags: | - # Tag with full semver on version tags (e.g. v1.2.3 → 1.2.3) - type=semver,pattern={{version}} - # Tag with major.minor (e.g. 1.2) - type=semver,pattern={{major}}.{{minor}} + # Tag with resolved release version (stable tags and branch pre-releases) + type=raw,value=${{ steps.version.outputs.image_tag }} # Only tag `latest` for stable releases (no pre-release suffix like -rc.1) - type=raw,value=latest,enable=${{ !contains(github.ref_name, '-') }} + type=raw,value=latest,enable=${{ steps.version.outputs.prerelease == 'false' }} # Always tag with short SHA for traceability type=sha,prefix=sha-,format=short @@ -94,22 +127,35 @@ jobs: - name: Checkout repository uses: actions/checkout@v4 - - name: Read VERSION file - id: version - run: | - VERSION=$(cat VERSION | tr -d '[:space:]') - echo "value=$VERSION" >> "$GITHUB_OUTPUT" - - name: Extract changelog for this version run: | - VERSION="${{ steps.version.outputs.value }}" - # Extract the section between ## [$VERSION] and the next ## [ - NOTES=$(awk "/^## \[$VERSION\]/{found=1; next} found && /^## \[/{exit} found{print}" CHANGELOG.md) - echo "$NOTES" > release-notes.md + BASE_VERSION="${{ needs.build-and-push.outputs.base_version }}" + VERSION="${{ needs.build-and-push.outputs.version }}" + + # Extract the section between ## [BASE_VERSION] and the next ## [ + NOTES=$(awk "/^## \[$BASE_VERSION\]/{found=1; next} found && /^## \[/{exit} found{print}" CHANGELOG.md) - - name: Generate quickstart package + if [ -z "$NOTES" ]; then + NOTES="Pre-release build from ${GITHUB_REF_NAME} (${GITHUB_SHA})." + fi + + if [ "${{ needs.build-and-push.outputs.prerelease }}" = "true" ]; then + { + echo "Pre-release build" + echo + echo "Version: $VERSION" + echo "Branch: ${GITHUB_REF_NAME}" + echo "Commit: ${GITHUB_SHA}" + echo + echo "$NOTES" + } > release-notes.md + else + echo "$NOTES" > release-notes.md + fi + + - name: Generate local quickstart package run: | - VERSION="${{ steps.version.outputs.value }}" + VERSION="${{ needs.build-and-push.outputs.version }}" DIR="mate-quickstart-${VERSION}" mkdir -p "$DIR" @@ -121,11 +167,38 @@ jobs: cp quickstart/README.txt "$DIR/README.txt" zip -r "mate-quickstart-${VERSION}.zip" "$DIR" + rm -rf "$DIR" + + - name: Generate Azure quickstart package + run: | + VERSION="${{ needs.build-and-push.outputs.version }}" + DIR="mate-quickstart-azure-${VERSION}" + mkdir -p "$DIR" + + # Copy all Azure quickstart files + cp quickstart-azure/.env.template "$DIR/.env.template" + cp quickstart-azure/README.md "$DIR/README.md" + cp quickstart-azure/QUICKSTART.md "$DIR/QUICKSTART.md" + cp quickstart-azure/DEPLOYMENT.md "$DIR/DEPLOYMENT.md" + cp quickstart-azure/check-prerequisites.ps1 "$DIR/check-prerequisites.ps1" + cp quickstart-azure/setup-env.ps1 "$DIR/setup-env.ps1" + cp quickstart-azure/deploy-whatif.ps1 "$DIR/deploy-whatif.ps1" + cp quickstart-azure/deploy.ps1 "$DIR/deploy.ps1" + cp quickstart-azure/setup-keyvault-secrets.ps1 "$DIR/setup-keyvault-secrets.ps1" + cp quickstart-azure/cleanup-rg.ps1 "$DIR/cleanup-rg.ps1" + + zip -r "mate-quickstart-azure-${VERSION}.zip" "$DIR" + rm -rf "$DIR" - name: Create GitHub Release uses: softprops/action-gh-release@v2 with: - name: "${{ steps.version.outputs.value }}" + tag_name: "${{ needs.build-and-push.outputs.tag_name }}" + target_commitish: "${{ github.sha }}" + name: "${{ needs.build-and-push.outputs.version }}" + prerelease: ${{ needs.build-and-push.outputs.prerelease == 'true' }} body_path: release-notes.md - files: mate-quickstart-${{ steps.version.outputs.value }}.zip + files: | + mate-quickstart-${{ needs.build-and-push.outputs.version }}.zip + mate-quickstart-azure-${{ needs.build-and-push.outputs.version }}.zip fail_on_unmatched_files: true diff --git a/README.md b/README.md index 6ab4821..4f302e7 100644 --- a/README.md +++ b/README.md @@ -114,11 +114,23 @@ Open ****. No login required in the default `Generic` aut > **PostgreSQL + Azurite** are always started alongside webui and worker — no extra flags required. The default `.env.template` values work out of the box for local development. -### Option C — Deploy to Azure (planned) +### Option C — Deploy to Azure -[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](BACKLOG.md#e1--foundation--infrastructure) +Download the `mate-quickstart-azure-.zip` from [GitHub Releases](https://github.com/holgerimbery/mate/releases/latest) or use the scripts in `infra/azure/scripts/`: -> Azure one-click deployment (Bicep + `azd`) is tracked in backlog item **E1-13** and is not yet shipped. +**Windows (PowerShell)** +```powershell +cd infra/azure/scripts +pwsh ./check-prerequisites.ps1 # Validate tools +pwsh ./setup-env.ps1 # Configure Azure credentials +pwsh ./deploy-whatif.ps1 # Preview changes (recommended) +pwsh ./deploy.ps1 # Deploy infrastructure +pwsh ./setup-keyvault-secrets.ps1 # Configure secrets & RBAC +``` + +See [quickstart-azure/README.md](quickstart-azure/README.md) for full deployment guide, troubleshooting, architecture details, and cost estimates. + +> **Prerequisites:** Azure CLI, PowerShell 7+, Bicep CLI. Estimated deployment time: 3–5 minutes. --- diff --git a/VERSION b/VERSION index 60f6343..1490961 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -v0.6.0 +v0.6.1 diff --git a/quickstart-azure/.env.template b/quickstart-azure/.env.template new file mode 100644 index 0000000..f7b0b8f --- /dev/null +++ b/quickstart-azure/.env.template @@ -0,0 +1,76 @@ +# Azure Deployment Configuration Template +# Copy this file to create .env, then fill in your values +# SECURITY: .env is ignored by git — never commit real values here + +# ============================================================================ +# REQUIRED: Azure Account Details +# ============================================================================ + +# Your Azure AD tenant ID (get from 'az account show --query tenantId') +AZURE_TENANT_ID= + +# Your Azure subscription ID (get from 'az account show --query id') +AZURE_SUBSCRIPTION_ID= + +# Azure region for resources (e.g., westeurope, eastus, uksouth) +AZURE_LOCATION=westeurope + +# Prefix for all resource names (e.g., mate-dev, mate-prod) +# Must be 3-20 characters, lowercase letters/numbers/hyphens +AZURE_ENVIRONMENT_NAME=mate-dev + +# Resource group name (auto-generated as {ENVIRONMENT_NAME}-rg if not set) +AZURE_RESOURCE_GROUP= + +# Deployment profile: xs (tiny), s (small), m (medium), l (large) +# xs: 0-1 replicas, 0.25 CPU, 0.5 GB memory (~$15/mo) +# s: 1-2 replicas, 0.5 CPU, 1 GB memory (~$30/mo) — development default +# m: 2-4 replicas, 1 CPU, 2 GB memory (~$80/mo) — staging +# l: 4-8 replicas, 2 CPU, 4 GB memory (~$200/mo) — production +AZURE_PROFILE=s + +# Container image tag from GitHub Container Registry (e.g., latest, v0.6.0) +AZURE_CONTAINER_IMAGE_TAG=latest + +# ============================================================================ +# OPTIONAL: Authentication (Entra ID / Azure AD) +# ============================================================================ + +# Entra ID application ID (for OIDC login) +# If empty: mate runs in 'Generic' anonymous mode (dev/test) +# If set: mate requires Azure AD login via OIDC +AZURE_AAD_CLIENT_ID= + +# Entra ID client secret (retrieved interactively by setup-keyvault-secrets.ps1) +# SECURITY: Never set this in .env — it will be stored in Key Vault via interactive prompt +AZURE_AAD_CLIENT_SECRET= + +# ============================================================================ +# OPTIONAL: Customization +# ============================================================================ + +# PostgreSQL settings (optional, matched with profile defaults) +AZURE_POSTGRES_ADMIN_LOGIN=pgadmin +AZURE_POSTGRES_MIN_BACKUPS=7 + +# Application Insights sampling rate (0-100%, default 100) +AZURE_APPINSIGHTS_SAMPLING_RATE=100 + +# Container scale settings (used if not set by profile) +AZURE_WEB_MIN_REPLICAS=1 +AZURE_WEB_MAX_REPLICAS=2 +AZURE_WORKER_MIN_REPLICAS=0 +AZURE_WORKER_MAX_REPLICAS=2 + +# ============================================================================ +# DEVELOPER: Override defaults (advanced users only) +# ============================================================================ + +# Skip confirmation prompt and deploy immediately (use with caution!) +# AZURE_FORCE_DEPLOY=false + +# PostgreSQL wait timeout when provisioning (minutes) +# AZURE_POSTGRES_WAIT_TIMEOUT=20 + +# PostgreSQL provision status check interval (seconds) +# AZURE_POSTGRES_WAIT_POLL=20 diff --git a/quickstart-azure/DEPLOYMENT.md b/quickstart-azure/DEPLOYMENT.md new file mode 100644 index 0000000..838f10f --- /dev/null +++ b/quickstart-azure/DEPLOYMENT.md @@ -0,0 +1,220 @@ +# Azure Deployment Workflow + +This directory contains PowerShell helper scripts to deploy the Mate infrastructure to Azure. + +## Overview + +The deployment process is split into two phases: + +1. **What-If (Dry-Run)**: Preview resources without creating them +2. **Deploy (Live)**: Create resources in Azure + +## Prerequisites + +**Required Tools:** +- Azure CLI (version 2.50+) — install via `winget install Microsoft.AzureCLI` +- Bicep CLI (included with Azure CLI) +- PowerShell 5.1+ (PowerShell 7+ recommended) + +**Required Access:** +- Azure tenant with admin consent rights +- Ability to create service principals and assign roles +- Subscription with remaining resource quota + +**Information You'll Need:** +- Tenant ID (Azure Portal → Azure AD → Properties) +- Subscription ID (Azure Portal → Subscriptions) +- Resource Group name (will be created if it doesn't exist) +- PostgreSQL admin password (prompted interactively, never stored) + +## Quick Start + +### 1. Setup Environment Variables + +Store your Azure tenant, subscription, and resource group information locally (never in git): + +```powershell +.\setup-env.ps1 +``` + +This interactive wizard creates a `.env` file with your settings: +- **Save location:** `infra/azure/scripts/.env` (automatically git-ignored) +- **What gets stored:** Tenant ID, Subscription ID, Resource Group, Location, etc. +- **Security:** `.env` is never committed to git + +See `.env.template` for all available configuration options. + +### 2. Check Prerequisites + +```powershell +.\check-prerequisites.ps1 +``` + +Validates that Azure CLI, Bicep, and PowerShell are installed. + +### 3. Preview Deployment (What-If) + +```powershell +.\deploy-whatif.ps1 +``` + +**This will:** +- Use values from `.env` automatically (no parameters needed) +- NOT create any Azure resources +- NOT modify your Azure account +- Show exactly what WOULD be created +- Display resource names, locations, and estimated costs + +**To override .env values**, pass parameters: + +```powershell +.\deploy-whatif.ps1 -Location 'westeurope' -Profile 'm' +``` + +**Output:** Review carefully for: +- Resource names and locations +- Capacity and scaling settings +- Any errors or missing parameters + +### 4. Deploy to Azure (Live) + +```powershell +.\deploy.ps1 +``` + +**This will:** +- Use values from `.env` automatically (no parameters needed) +- Prompt for PostgreSQL admin password (never stored, never logged) +- Create resource group (if needed) +- Deploy all Azure resources +- Automatically recover known first-run Key Vault secret bootstrap failures (when `.credentials` is available) +- Ensure PostgreSQL connectivity prerequisites for Azure-hosted apps when public network mode is used +- Configure WebUI/Worker runtime DB secret references (`ConnectionStrings__Default` -> `secretref:postgres-conn`) +- Configure WebUI/Worker runtime Blob secret references (`AzureInfrastructure__BlobConnectionString` -> `secretref:blob-conn`) + +**To override .env values**, pass parameters: + +```powershell +.\deploy.ps1 -Location 'westeurope' -Profile 'm' +``` + +**Warning:** This creates real Azure resources and incurs costs. Always run `deploy-whatif.ps1` first. + +## Deployment Profiles + +| Profile | Web Min | Web Max | Worker Max | CPU | Memory | Use Case | +|---------|---------|---------|------------|-----|--------|----------| +| `xs` | 1 | 1 | 2 | 0.25 | 0.5 GB | Testing, lowest cost | +| `s` | 1 | 3 | 5 | 0.5 | 1 GB | **Default for dev** | +| `m` | 2 | 6 | 10 | 1.0 | 2 GB | Growth production | +| `l` | 3 | 12 | 20 | 2.0 | 4 GB | High throughput | + +**Development Policy:** Internal engineering always uses `dev` environment with `s` profile. + +## Services Deployed + +- **Azure Container Apps**: Runs WebUI (external ingress on port 8080) and Worker (internal, queue-driven) +- **Azure Service Bus**: Message queue `test-runs` with dead-lettering +- **PostgreSQL Flexible Server**: Relational database, v17, 32GB Burstable tier +- **Azure Blob Storage**: Document store with HTTPS-only, no public access +- **Azure Key Vault**: Credential and secret management +- **Application Insights & Log Analytics**: Telemetry and logging + +## Post-Deployment Tasks + +After `deploy.ps1` completes successfully, infrastructure and secret wiring are expected to be ready. + +### Manual steps required for correct Entra ID login flow + +1. **Get the WebUI URL from deployment output** + - You need the exact FQDN for redirect URI registration. + +2. **Register redirect URI in Entra app registration** + - Required redirect URI format: + https:///signin-oidc + - Also set front-channel logout URL: + https:///signout-callback-oidc + +3. **Enable ID token issuance on the app registration** + - In Authentication for the app registration, enable ID tokens for web sign-in. + +4. **Validate login in private/incognito browser** + - Open the WebUI URL. + - Confirm redirect to Microsoft sign-in and return to app after authentication. + +### Optional verification commands (recommended) + +Verify redirect URIs: +az ad app show --id --query "web.redirectUris" + +Verify web container auth-related env: +az containerapp show --resource-group --name -webui --query "properties.template.containers[0].env[?name=='AzureAd__ClientId' || name=='AzureAd__TenantId' || name=='AzureAd__CallbackPath']" + +Verify Key Vault secret exists: +az keyvault secret list --vault-name -kv --query "[?name=='azuread-client-secret'].name" -o tsv + +Verify DB secret reference in WebUI: +az containerapp show --resource-group --name -webui --query "properties.template.containers[0].env[?name=='ConnectionStrings__Default']" + +Verify PostgreSQL firewall allows Azure services (public mode): +az postgres flexible-server firewall-rule list --resource-group --name -pg -o table + +## Troubleshooting + +### Azure CLI not found +- Install from: `winget install Microsoft.AzureCLI` or https://learn.microsoft.com/cli/azure/install-azure-cli-windows +- Restart PowerShell after installation + +### Bicep not found +- Run: `az bicep install` + +### Authentication fails +- Clear cached credentials: `az account clear` +- Re-authenticate: `az login --tenant ` +- Verify subscription: `az account show` + +### Deployment fails +- Check resource group doesn't already exist in wrong location +- Verify PostgreSQL admin password meets complexity requirements (12+ chars, mixed case, numbers) +- Review Azure Portal → Resource Groups → Deployments for error details + +### WebUI URL does not respond (startup failures) +- Check active revision health: + az containerapp revision list --resource-group --name -webui -o table +- Check application logs: + az containerapp logs show --resource-group --name -webui --type console --tail 120 +- Most common cause: database connectivity during startup migration. + - Verify `ConnectionStrings__Default` uses `secretRef` (`postgres-conn`). + - Verify PostgreSQL firewall/network access if using public mode. + +## Environment Variables & Configuration + +**WebUI Container Environment:** +- `ASPNETCORE_ENVIRONMENT`: Always `Production` +- `Authentication`: `EntraId` (Azure Entra ID) +- `Monitoring`: `ApplicationInsights` +- `Infrastructure`: `Azure` +- Key Vault secrets (passed as references, not plain text) + +**Worker Container Environment:** +- `Infrastructure`: `Azure` +- Service Bus connection string (Key Vault reference) +- Blob Storage connection string (Key Vault reference) +- PostgreSQL connection string (Key Vault reference) + +## Cleanup + +To clean everything inside a resource group without deleting the resource group itself: + +```powershell +.\cleanup-rg.ps1 -ResourceGroupName +``` + +This script deletes live resources, removes deployment records, purges RG-scoped soft-deleted Key Vault entries, and verifies the RG is empty at the end. + +## References + +- [Azure Container Apps Documentation](https://learn.microsoft.com/azure/container-apps/) +- [Bicep Language Documentation](https://learn.microsoft.com/azure/azure-resource-manager/bicep/) +- [Azure CLI Reference](https://learn.microsoft.com/cli/azure/) +- [Azure Key Vault Documentation](https://learn.microsoft.com/azure/key-vault/) diff --git a/quickstart-azure/QUICKSTART.md b/quickstart-azure/QUICKSTART.md new file mode 100644 index 0000000..d55a68e --- /dev/null +++ b/quickstart-azure/QUICKSTART.md @@ -0,0 +1,117 @@ +# Deploy to Azure — Quick Reference + +This directory contains a complete deployment guide for mate on Microsoft Azure. Below are the quick deployment steps. + +## Prerequisites Checklist + +- [ ] Azure subscription with admin permissions +- [ ] Azure CLI installed and authenticated (`az login`) +- [ ] PowerShell 7+ installed +- [ ] Bicep CLI (included with Azure CLI) + +## Quick Deploy (3 minutes) + +All scripts are included in this package. Simply run: + +```powershell +# 1. Validate prerequisites +pwsh ./check-prerequisites.ps1 + +# 2. Configure environment (prompts for tenant ID, subscription, location, etc.) +pwsh ./setup-env.ps1 + +# 3. Preview what will be created +pwsh ./deploy-whatif.ps1 + +# 4. Deploy to Azure (creates all infrastructure) +pwsh ./deploy.ps1 + +# 5. Configure secrets and RBAC +pwsh ./setup-keyvault-secrets.ps1 +``` + +## What Gets Deployed + +✅ **Azure Container Apps** — WebUI (public) + Worker (internal) +✅ **PostgreSQL Flexible Server** — v17, auto-configured +✅ **Azure Blob Storage** — Document storage +✅ **Azure Key Vault** — Secrets management +✅ **Application Insights** — Monitoring & logs +✅ **Service Bus** — Queue trigger (optional) + +## Deployment Options + +| Option | Time | Automation | Manual Steps | +|--------|------|-----------|--------------| +| **Local Scripts** (recommended) | 5 min | Full | None | +| **GitHub Actions** | 5 min | Full | Submit workflow | +| **Portal UI** | 10+ min | Partial | Many | + +## Scripts Details + +- **check-prerequisites.ps1** — Validates Azure CLI, Bicep, PowerShell +- **setup-env.ps1** — Interactive: Collects Tenant ID, Subscription, Location, etc. +- **deploy-whatif.ps1** — Dry-run preview (recommended before deploy) +- **deploy.ps1** — Actual deployment + automatic secret/firewall config +- **setup-keyvault-secrets.ps1** — Post-deploy: Configures Key Vault and RBAC +- **cleanup-rg.ps1** — Deletes all resources (keeps resource group) + +## Costs + +Typical monthly costs by profile: + +| Profile | Instance Type | Monthly Cost | +|---------|---------------|--------------| +| `xs` | Tiny (dev/test sandbox) | ~$15 | +| `s` | Small (personal dev) | ~$30 | +| `m` | Medium (team staging) | ~$80 | +| `l` | Large (production) | ~$200+ | + +Costs depend on your region and usage patterns. **Always run `deploy-whatif.ps1` first to estimate costs.** + +## After Deployment + +Your app will be at: +**`https://mate-dev-webui.{uniqueId}.{region}.azurecontainerapps.io`** + +Sample data is available after first login. To test: + +1. Navigate to Test Suites → Create New Suite +2. Add test cases targeting any integrated agent +3. Run the suite and view results + +## Troubleshooting + +### "PostgreSQL connection timeout" +→ Wait 2–3 minutes and refresh. The deployment script auto-creates firewall rules. + +### "Access denied to Key Vault" +→ Run `setup-keyvault-secrets.ps1` to configure RBAC. + +### "Container App revision unhealthy" +→ Check logs: `az containerapp logs show --name mate-dev-webui --type console --tail 50` + +### "Invalid blob connection string" +→ Run `deploy.ps1` again — it will update secrets automatically. + +See [README.md](./README.md) for full troubleshooting guide. + +## Cleanup + +```powershell +# Delete all resources (keeps resource group for re-deployment) +pwsh ./cleanup-rg.ps1 + +# Delete resource group entirely +az group delete --name rg-mate-dev +``` + +## Full Documentation + +- **Deployment Guide**: [README.md](./README.md) +- **Architecture Details**: [../../docs/concepts/SaaS-Architecture-v2.md](../../docs/concepts/SaaS-Architecture-v2.md) +- **Troubleshooting**: See README.md → Troubleshooting section + +--- + +**Next:** After deployment succeeds, read [docs/wiki/User-Getting-Started.md](../../docs/wiki/User-Getting-Started.md) to set up your first test suite. diff --git a/quickstart-azure/README.md b/quickstart-azure/README.md new file mode 100644 index 0000000..81629f4 --- /dev/null +++ b/quickstart-azure/README.md @@ -0,0 +1,300 @@ +# mate — Quickstart: Deploy to Azure + +Deploy **mate** to Microsoft Azure with a managed infrastructure setup including Azure Container Apps, PostgreSQL Flexible Server, Blob Storage, Key Vault, and Application Insights. + +## Prerequisites + +**Before you start:** + +1. **Azure account** — Valid Azure subscription with permissions to create resources +2. **Azure CLI** — [Install Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) +3. **PowerShell 7+** — [Install PowerShell Core](https://learn.microsoft.com/powershell/scripting/install/installing-powershell) +4. **Bicep CLI** — Usually installed with Azure CLI; verify: `az bicep version` +5. **Authenticated session** — Run `az login` and authenticate with your Azure tenant + +### Quick Install (Windows) + +```powershell +# Install prerequisites using winget +winget install Microsoft.AzureCLI +winget install Microsoft.PowerShell + +# After installing, verify +az --version +pwsh --version +az bicep version +``` + +## Deployment Steps + +All scripts are included in this package. Follow these 5 steps: + +### 1. Prepare Environment + +```powershell +# Review prerequisites (validates Azure CLI, Bicep, PowerShell) +pwsh ./check-prerequisites.ps1 +``` + +This validates that Azure CLI, Bicep, and PowerShell are installed. + +### 2. Configure Environment Variables + +```powershell +pwsh ./setup-env.ps1 +``` + +This creates a `.env` file with your Azure credentials. You'll be prompted for: + +- **Azure Tenant ID** — Your Azure AD tenant ID +- **Subscription ID** — Target Azure subscription +- **Location** — Azure region (e.g., `westeurope`, `eastus`) +- **Environment Name** — Prefix for resources (e.g., `mate-dev`, `mate-prod`) +- **Resource Group** — Name for the resource group (auto-generated if not provided) +- **AAD Client ID** — Entra ID application ID for authentication + +**Security:** The `.env` file is **never committed to git** — it contains sensitive information and is protected by `.gitignore`. + +### 3. Preview Changes (Recommended) + +```powershell +pwsh ./deploy-whatif.ps1 +``` + +This performs a **dry-run** showing exactly which Azure resources would be created. Review the output carefully before proceeding. + +### 4. Deploy Infrastructure + +```powershell +pwsh ./deploy.ps1 +``` + +This command: + +1. ✅ Deploys Bicep template to Azure +2. ✅ Creates Azure Container Apps (WebUI + Worker) +3. ✅ Provisions PostgreSQL Flexible Server +4. ✅ Sets up Blob Storage (documents) +5. ✅ Configures Key Vault for secrets +6. ✅ Creates Application Insights monitoring +7. ✅ **Automatically configures runtime secrets** (DB + Blob connection strings) +8. ✅ **Automatically creates PostgreSQL firewall rules** + +**Typical deployment time:** 3–5 minutes + +### 5. Post-Deployment: Key Vault & RBAC Setup + +```powershell +pwsh ./setup-keyvault-secrets.ps1 +``` + +This script: + +1. Stores your Entra ID client secret in Key Vault +2. Configures managed identity RBAC permissions +3. Verifies the setup works +4. Provides next steps + +## Access Your Deployment + +After deployment succeeds, you'll see: + +``` +✓ Container App deployed: https://mate-dev-webui.orangebay-XXXXXXXX.westeurope.azurecontainerapps.io +``` + +Open that URL in your browser. You'll be redirected to Entra ID login if configured, or see the mate dashboard. + +### First Login + +If using **Entra ID authentication**: +- You'll be redirected to sign in with your Azure AD account +- After first login, you may need admin consent for the app registration + +If using **no authentication** (dev mode): +- Dashboard appears immediately; no login required + +## Deployment Profiles + +Choose a size profile based on your use case: + +| Profile | Replicas | CPUs | Memory | Cost | Use Case | +|---------|----------|------|--------|------|----------| +| `xs` | 0–1 | 0.25 | 0.5 GB | ~$15/mo | Local testing, dev sandbox | +| `s` | 1–2 | 0.5 | 1 GB | ~$30/mo | Development environment | +| `m` | 2–4 | 1 | 2 GB | ~$80/mo | Staging / light production | +| `l` | 4–8 | 2 | 4 GB | ~$200/mo | Production workload | + +To use a different profile, pass `-Profile` when running `deploy.ps1`: + +```powershell +pwsh ./deploy.ps1 -Profile m +``` + +Or set in `.env`: + +```env +AZURE_PROFILE=m +``` + +## Troubleshooting + +### Problem: "PostgreSQL connection timeout" + +**Cause:** Firewall rule not applied or PostgreSQL still initializing. + +**Solution:** The deployment script automatically creates the firewall rule. If the error persists: + +1. Check PostgreSQL status in Azure Portal +2. Wait 2–3 minutes and manually refresh the Container Apps revisions +3. If still failing, check logs: `az containerapp logs show --name mate-dev-webui` + +### Problem: "Access denied to Key Vault" + +**Cause:** Managed identity RBAC permissions not configured. + +**Solution:** Run `setup-keyvault-secrets.ps1` to configure RBAC, or manually grant the Container App managed identity the `Key Vault Secrets User` role. + +### Problem: "Container App revision unhealthy" + +**Cause:** Environment variables not set or secrets not injected. + +**Solution:** +1. Check logs: `az containerapp logs show --name mate-dev-webui --type console` +2. Verify secrets: `az containerapp secret list --name mate-dev-webui` +3. Verify env vars: `az containerapp show --name mate-dev-webui | jq '.properties.template.containers[0].env'` + +### Problem: "Invalid blob connection string" + +**Cause:** Storage account key not retrieved or secret not injected. + +**Solution:** The deployment script automatically handles this. If the error persists: + +1. Run `deploy.ps1` again (it will update secrets) +2. Or manually inject: see logs from the latest deployment output + +## Cleanup + +To **delete all Azure resources**: + +```powershell +pwsh ./cleanup-rg.ps1 +``` + +This removes all resources in the resource group **except the resource group itself** (allowing re-deployment to the same RG). + +To also **delete the resource group**: + +```powershell +az group delete --name +``` + +## Environment Details + +### Deployed Resources + +- **Azure Container Apps Environment** — Managed container orchestration +- **Container App: WebUI** (`mate-dev-webui`) — External HTTPS ingress on port 8080 +- **Container App: Worker** (`mate-dev-worker`) — Internal queue processor +- **PostgreSQL Flexible Server** (`mate-dev-pg`) — v17, public network mode +- **Azure Blob Storage** (`matedevst`) — Document storage +- **Azure Key Vault** — Secrets management (connection strings, client secrets) +- **Application Insights** — Monitoring and logging + +### Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Azure Container App Environment (westeurope) │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ WebUI (External Ingress) Worker (Internal Queue) │ +│ ▲ ▲ │ +│ │ HTTPS:8080 │ Service Bus Trigger │ +│ │ │ (scale 0–2) │ +│ └──────────────┬────────────────┘ │ +│ │ │ +│ ┌──────────────┴──────────────┐ │ +│ │ Shared Secrets (Key Vault) │ │ +│ │ • postgres-conn │ │ +│ │ • blob-conn │ │ +│ └──────────────┬──────────────┘ │ +│ │ │ +└─────────────────┼─────────────────────────────────────────┘ + │ + ┌───────────┼────────────────┬──────────────┐ + │ │ │ │ + ▼ ▼ ▼ ▼ + PostgreSQL Blob Storage Service Bus App Insights + (Flexible (matedevst) (TODO) (logs/metrics) + Server) +``` + +### Security + +- **Managed Identity** — Container Apps authenticate to Key Vault using system-assigned identities +- **Public PostgreSQL** — Access restricted to Azure services via firewall rule (`0.0.0.0`) +- **Private Secrets** — Connection strings stored in Key Vault, never in code +- **Entra ID Auth** — WebUI redirects to Azure AD login (configurable) + +## Next Steps + +### Monitor Your Deployment + +```powershell +# Check Container App status +az containerapp revision list --resource-group rg-mate-dev --name mate-dev-webui + +# View WebUI logs +az containerapp logs show --resource-group rg-mate-dev --name mate-dev-webui --type console --tail 50 + +# View Worker logs +az containerapp logs show --resource-group rg-mate-dev --name mate-dev-worker --type console --tail 50 +``` + +### Configure Authentication + +By default, mate uses **no authentication** in local/dev mode. To enable **Entra ID**: + +1. **Register the app** in Azure AD (or use existing app registration) +2. Set in `.env`: + ```env + AUTHENTICATION__SCHEME=EntraId + AZURE_AAD_CLIENT_ID= + ``` +3. Get the **client secret** from Azure AD and store in Key Vault: + ```powershell + pwsh ./setup-keyvault-secrets.ps1 + ``` + +### Scale the Deployment + +To change resource allocations (CPU, memory, replicas): + +Edit `.env` and change `AZURE_PROFILE`, then run `deploy.ps1` again: + +```powershell +$env:AZURE_PROFILE = 'm' # Upgrade to medium +pwsh ./deploy.ps1 +``` + +### Access Monitoring Data + +Open **Application Insights** in Azure Portal to view: +- Request traces and performance +- Dependency calls (PostgreSQL, Blob Storage) +- Custom events and logs +- Error tracking + +## Support + +For issues or questions: + +1. Check [Troubleshooting](#troubleshooting) above +2. Review logs using Azure CLI commands above +3. Open an issue on [GitHub](https://github.com/holgerimbery/mate/issues) +4. See [docs/wiki/Developer-Getting-Started.md](../../docs/wiki/Developer-Getting-Started.md) for more details + +--- + +**Next:** Deploy to a staging or production environment by running the same scripts with different parameters (different resource group, location, or profile). diff --git a/quickstart-azure/check-prerequisites.ps1 b/quickstart-azure/check-prerequisites.ps1 new file mode 100644 index 0000000..064be8f --- /dev/null +++ b/quickstart-azure/check-prerequisites.ps1 @@ -0,0 +1,95 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + +<# +.SYNOPSIS +Validate and display Azure deployment prerequisites. + +.DESCRIPTION +Checks that required tools are installed and provides setup instructions +for prerequisites needed to deploy the Mate infrastructure to Azure. + +#> + +$ErrorActionPreference = 'Continue' + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Azure Deployment Prerequisites Check ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +# Check Azure CLI +Write-Host "Checking prerequisites..." -ForegroundColor Yellow +Write-Host "" + +$azCliFound = Get-Command 'az' -ErrorAction SilentlyContinue +if ($azCliFound) { + $azVersion = az version --only-show-errors 2>$null | ConvertFrom-Json + Write-Host "✓ Azure CLI installed: $($azVersion.'azure-cli')" -ForegroundColor Green +} +else { + Write-Host "✗ Azure CLI NOT found" -ForegroundColor Red + Write-Host " Install: winget install Microsoft.AzureCLI" -ForegroundColor Yellow + Write-Host " Or: https://learn.microsoft.com/cli/azure/install-azure-cli-windows" -ForegroundColor Yellow +} + +# Check Bicep +if ($azCliFound) { + $bicepVersion = az bicep version 2>$null + if ($?) { + Write-Host "✓ Bicep CLI installed: $bicepVersion" -ForegroundColor Green + } + else { + Write-Host "✗ Bicep CLI NOT found" -ForegroundColor Red + Write-Host " Install: az bicep install" -ForegroundColor Yellow + } +} + +# Check PowerShell version +$psVersion = $PSVersionTable.PSVersion +if ($psVersion.Major -ge 7) { + Write-Host "✓ PowerShell 7 or later: $psVersion" -ForegroundColor Green +} +else { + Write-Host "⚠ PowerShell 5.1 detected (works, but PowerShell 7+ recommended)" -ForegroundColor Yellow + Write-Host " Install: https://github.com/PowerShell/PowerShell/releases" -ForegroundColor Yellow +} + +Write-Host "" +Write-Host "After prerequisites are installed:" -ForegroundColor Cyan +Write-Host "" +Write-Host "1. Get your Azure tenant and subscription IDs:" +Write-Host " - Tenant ID: Azure portal → Azure AD → Properties → Tenant ID" -ForegroundColor Gray +Write-Host " - Subscription ID: Azure portal → Subscriptions → Subscription ID" -ForegroundColor Gray +Write-Host "" + +Write-Host "2. Ensure you have admin consent for service principal creation:" +Write-Host " - Role needed: Owner or User Access Administrator (Contributor)" -ForegroundColor Gray +Write-Host "" + +Write-Host "3. Authenticate with admin:" -ForegroundColor Cyan +Write-Host " az account clear" -ForegroundColor Magenta +Write-Host " az login --tenant ''" -ForegroundColor Magenta +Write-Host "" + +Write-Host "4. Run a what-if deployment first:" +Write-Host "" +Write-Host " .\deploy-whatif.ps1 \" -ForegroundColor Magenta +Write-Host " -TenantId '' \" -ForegroundColor Magenta +Write-Host " -SubscriptionId '' \" -ForegroundColor Magenta +Write-Host " -Location 'eastus' \" -ForegroundColor Magenta +Write-Host " -EnvironmentName 'mate-dev' \" -ForegroundColor Magenta +Write-Host " -Profile 's'" -ForegroundColor Magenta +Write-Host "" + +Write-Host "5. Review the what-if output, then deploy:" +Write-Host "" +Write-Host " .\deploy.ps1 \" -ForegroundColor Magenta +Write-Host " -TenantId '' \" -ForegroundColor Magenta +Write-Host " -SubscriptionId '' \" -ForegroundColor Magenta +Write-Host " -Location 'eastus' \" -ForegroundColor Magenta +Write-Host " -EnvironmentName 'mate-dev' \" -ForegroundColor Magenta +Write-Host " -Profile 's'" -ForegroundColor Magenta +Write-Host "" diff --git a/quickstart-azure/cleanup-rg.ps1 b/quickstart-azure/cleanup-rg.ps1 new file mode 100644 index 0000000..0bc0d14 --- /dev/null +++ b/quickstart-azure/cleanup-rg.ps1 @@ -0,0 +1,224 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + +<# +.SYNOPSIS +Cleans all resources from an Azure resource group without deleting the resource group itself. + +.DESCRIPTION +- Deletes all live resources in the specified resource group. +- Cancels running deployments in that resource group. +- Purges soft-deleted Key Vaults that belong to that resource group only. + +This script is intended to provide a clean starting point for re-deployment. +#> + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroupName, + + [ValidateRange(60, 7200)] + [int]$WaitTimeoutSeconds = 900, + + [ValidateRange(2, 120)] + [int]$PollIntervalSeconds = 10 +) + +$ErrorActionPreference = 'Stop' + +Write-Host '' +Write-Host '=== RG Cleanup Start ===' -ForegroundColor Cyan +Write-Host "Resource Group: $ResourceGroupName" -ForegroundColor Gray + +# Validate Azure CLI and login context. +$azVersion = az version 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Azure CLI not available. Install Azure CLI and login first.' +} + +$account = az account show --query id -o tsv 2>$null +if ($LASTEXITCODE -ne 0 -or -not $account) { + throw 'Not logged into Azure. Run: az login' +} + +$rgExists = az group exists --name $ResourceGroupName -o tsv 2>$null +if ($rgExists -ne 'true') { + throw "Resource group '$ResourceGroupName' does not exist." +} + +$rgLower = $ResourceGroupName.ToLower() + +function Get-RgScopedDeletedKeyVaultNames { + param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroupNameLower + ) + + $deletedKvsJson = az keyvault list-deleted -o json 2>$null + if ($LASTEXITCODE -ne 0) { + throw 'Failed to list deleted Key Vaults.' + } + + $deletedKvs = @() + if ($deletedKvsJson) { + $deletedKvs = $deletedKvsJson | ConvertFrom-Json + } + + $names = @() + foreach ($kv in $deletedKvs) { + $id = [string]$kv.id + if (-not [string]::IsNullOrWhiteSpace($id) -and $id.ToLower().Contains("/resourcegroups/$ResourceGroupNameLower/")) { + $names += [string]$kv.name + } + } + + return @($names | Where-Object { $_ -and $_.Trim() } | Select-Object -Unique) +} + +Write-Host 'Step 1/6: Cancel running deployments in RG...' -ForegroundColor Yellow +$runningDeployments = az deployment group list --resource-group $ResourceGroupName --query "[?properties.provisioningState=='Running'].name" -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed to list deployments.' +} + +if ($runningDeployments) { + $runningDeployments -split "`n" | Where-Object { $_.Trim() } | ForEach-Object { + $name = $_.Trim() + Write-Host " Cancelling deployment: $name" -ForegroundColor Gray + az deployment group cancel --resource-group $ResourceGroupName --name $name 2>$null | Out-Null + } +} +else { + Write-Host ' No running deployments found.' -ForegroundColor Gray +} + +Write-Host 'Step 2/6: Delete deployment records in RG...' -ForegroundColor Yellow +$allDeployments = az deployment group list --resource-group $ResourceGroupName --query '[].name' -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed to list deployment records.' +} + +if ($allDeployments) { + $deployments = $allDeployments -split "`n" | Where-Object { $_.Trim() } + Write-Host " Deployment records found: $($deployments.Count)" -ForegroundColor Gray + + foreach ($name in $deployments) { + $deploymentName = $name.Trim() + Write-Host " Deleting deployment record: $deploymentName" -ForegroundColor Gray + az deployment group delete --resource-group $ResourceGroupName --name $deploymentName 2>$null | Out-Null + } +} +else { + Write-Host ' No deployment records found.' -ForegroundColor Gray +} + +Write-Host 'Step 3/6: Delete all live resources in RG...' -ForegroundColor Yellow +$resourceIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed to list resources in resource group.' +} + +if ($resourceIds) { + $ids = $resourceIds -split "`n" | Where-Object { $_.Trim() } + Write-Host " Resources found: $($ids.Count)" -ForegroundColor Gray + + foreach ($id in $ids) { + Write-Host " Deleting: $id" -ForegroundColor Gray + az resource delete --ids $id --no-wait 2>$null | Out-Null + } +} +else { + Write-Host ' No live resources found.' -ForegroundColor Gray +} + +Write-Host 'Step 4/6: Wait for resource deletions to complete...' -ForegroundColor Yellow +$start = Get-Date +while ($true) { + $remainingIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null + if ($LASTEXITCODE -ne 0) { + throw 'Failed while checking remaining resources.' + } + + if (-not $remainingIds) { + Write-Host ' Live resource count: 0' -ForegroundColor Green + break + } + + $remainingCount = ($remainingIds -split "`n" | Where-Object { $_.Trim() }).Count + $elapsed = [int]((Get-Date) - $start).TotalSeconds + Write-Host " Waiting... remaining resources: $remainingCount (elapsed ${elapsed}s)" -ForegroundColor Gray + + if ($elapsed -ge $WaitTimeoutSeconds) { + throw "Timeout waiting for resource deletions. Remaining count: $remainingCount" + } + + Start-Sleep -Seconds $PollIntervalSeconds +} + +Write-Host 'Step 5/6: Purge RG-scoped soft-deleted Key Vaults...' -ForegroundColor Yellow +$rgScopedDeletedVaults = Get-RgScopedDeletedKeyVaultNames -ResourceGroupNameLower $rgLower + +if ($rgScopedDeletedVaults.Count -gt 0) { + foreach ($name in ($rgScopedDeletedVaults | Select-Object -Unique)) { + Write-Host " Purging deleted Key Vault: $name" -ForegroundColor Gray + az keyvault purge --name $name 2>$null | Out-Null + } + + $purgeStart = Get-Date + while ($true) { + $remainingDeletedVaults = Get-RgScopedDeletedKeyVaultNames -ResourceGroupNameLower $rgLower + if ($remainingDeletedVaults.Count -eq 0) { + Write-Host ' Soft-deleted RG-scoped Key Vault count: 0' -ForegroundColor Green + break + } + + $elapsedPurge = [int]((Get-Date) - $purgeStart).TotalSeconds + Write-Host " Waiting for Key Vault purge... remaining: $($remainingDeletedVaults.Count) (elapsed ${elapsedPurge}s)" -ForegroundColor Gray + + if ($elapsedPurge -ge $WaitTimeoutSeconds) { + $remainingNames = $remainingDeletedVaults -join ', ' + throw "Timeout waiting for Key Vault purge. Remaining soft-deleted vaults: $remainingNames" + } + + Start-Sleep -Seconds $PollIntervalSeconds + } +} +else { + Write-Host ' No RG-scoped deleted Key Vaults found.' -ForegroundColor Gray +} + +Write-Host 'Step 6/6: Final verification...' -ForegroundColor Yellow +$finalLiveIds = az resource list --resource-group $ResourceGroupName --query '[].id' -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed final resource check.' +} + +$finalLiveCount = 0 +if ($finalLiveIds) { + $finalLiveCount = ($finalLiveIds -split "`n" | Where-Object { $_.Trim() }).Count +} + +$finalDeploymentRecords = az deployment group list --resource-group $ResourceGroupName --query '[].name' -o tsv 2>$null +if ($LASTEXITCODE -ne 0) { + throw 'Failed final deployment records check.' +} + +$finalDeploymentRecordCount = 0 +if ($finalDeploymentRecords) { + $finalDeploymentRecordCount = ($finalDeploymentRecords -split "`n" | Where-Object { $_.Trim() }).Count +} + +$finalRgScopedDeletedKvCount = (Get-RgScopedDeletedKeyVaultNames -ResourceGroupNameLower $rgLower).Count + +if ($finalLiveCount -ne 0 -or $finalDeploymentRecordCount -ne 0 -or $finalRgScopedDeletedKvCount -ne 0) { + throw "Cleanup verification failed. LIVE_RESOURCE_COUNT=$finalLiveCount, DEPLOYMENT_RECORD_COUNT=$finalDeploymentRecordCount, RG_SCOPED_SOFT_DELETED_KV_COUNT=$finalRgScopedDeletedKvCount" +} + +Write-Host '' +Write-Host '=== RG Cleanup Complete ===' -ForegroundColor Green +Write-Host "LIVE_RESOURCE_COUNT=$finalLiveCount" -ForegroundColor Cyan +Write-Host "DEPLOYMENT_RECORD_COUNT=$finalDeploymentRecordCount" -ForegroundColor Cyan +Write-Host "RG_SCOPED_SOFT_DELETED_KV_COUNT=$finalRgScopedDeletedKvCount" -ForegroundColor Cyan +Write-Host '' diff --git a/quickstart-azure/deploy-whatif.ps1 b/quickstart-azure/deploy-whatif.ps1 new file mode 100644 index 0000000..740b4af --- /dev/null +++ b/quickstart-azure/deploy-whatif.ps1 @@ -0,0 +1,282 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + +<# +.SYNOPSIS +Dry-run deployment to show what Azure resources would be created without actually creating them. + +.DESCRIPTION +This script validates the Bicep template and performs a what-if deployment to preview +resource creation. No resources are modified until you explicitly run deploy.ps1. + +.PARAMETER TenantId +Azure tenant ID (required). + +.PARAMETER SubscriptionId +Azure subscription ID (required). + +.PARAMETER Location +Azure region (e.g., 'eastus', 'westeurope'). Default: 'eastus'. + +.PARAMETER EnvironmentName +Environment name prefix for resources (e.g., 'mate-dev', 'mate-prod'). Default: 'mate-dev'. + +.PARAMETER Profile +Size profile: 'xs', 's', 'm', or 'l'. Default: 's' (development). + +.PARAMETER ResourceGroupName +Azure resource group name. Default: '{EnvironmentName}-rg'. + +.EXAMPLE +.\deploy-whatif.ps1 -TenantId 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' ` + -SubscriptionId 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' ` + -Location 'eastus' ` + -EnvironmentName 'mate-dev' ` + -Profile 's' + +#> + +param( + [Parameter(Mandatory = $false)] + [string]$TenantId, + + [Parameter(Mandatory = $false)] + [string]$SubscriptionId, + + [Parameter(Mandatory = $false)] + [string]$Location, + + [Parameter(Mandatory = $false)] + [string]$EnvironmentName, + + [Parameter(Mandatory = $false)] + [ValidateSet('xs', 's', 'm', 'l')] + [string]$Profile, + + [Parameter(Mandatory = $false)] + [string]$ResourceGroupName, + + [Parameter(Mandatory = $false)] + [string]$AadClientId, + + [Parameter(Mandatory = $false)] + [ValidateRange(1, 120)] + [int]$PostgresWaitTimeoutMinutes = 20, + + [Parameter(Mandatory = $false)] + [ValidateRange(5, 300)] + [int]$PostgresWaitPollSeconds = 20 +) + +$ErrorActionPreference = 'Stop' + +# Load .env file if it exists +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" + +if (Test-Path $envFile) { + Write-Host "Loading configuration from .env..." -ForegroundColor Gray + Get-Content $envFile | Where-Object { $_ -and -not $_.StartsWith('#') } | ForEach-Object { + $parts = $_ -split '=', 2 + if ($parts.Count -eq 2) { + $key = $parts[0].Trim() + $value = $parts[1].Trim() + switch ($key) { + 'AZURE_TENANT_ID' { if (-not $TenantId) { $TenantId = $value } } + 'AZURE_SUBSCRIPTION_ID' { if (-not $SubscriptionId) { $SubscriptionId = $value } } + 'AZURE_LOCATION' { if (-not $Location) { $Location = $value } } + 'AZURE_ENVIRONMENT_NAME' { if (-not $EnvironmentName) { $EnvironmentName = $value } } + 'AZURE_PROFILE' { if (-not $Profile) { $Profile = $value } } + 'AZURE_RESOURCE_GROUP' { if (-not $ResourceGroupName) { $ResourceGroupName = $value } } + 'AZURE_AAD_CLIENT_ID' { if (-not $AadClientId) { $AadClientId = $value } } + } + } + } +} + +# Apply defaults +if (-not $Location) { $Location = 'eastus' } +if (-not $EnvironmentName) { $EnvironmentName = 'mate-dev' } +if (-not $Profile) { $Profile = 's' } +if (-not $ResourceGroupName) { $ResourceGroupName = "$EnvironmentName-rg" } + +# Validate required parameters +if (-not $TenantId) { + Write-Error "TenantId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $SubscriptionId) { + Write-Error "SubscriptionId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $AadClientId) { + Write-Error "AadClientId not provided and not found in .env file. Run: .\setup-env.ps1" +} + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Azure Mate Infrastructure - What-If Deployment (DRY RUN) ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +# Validate inputs +$validProfiles = @('xs', 's', 'm', 'l') +if (-not $validProfiles -contains $Profile) { + Write-Error "Invalid profile '$Profile'. Must be one of: $($validProfiles -join ', ')" +} + +# Resolve script directory +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$templateDir = Split-Path -Parent $scriptDir +$parameterFile = Join-Path $templateDir "parameters" "profile-$Profile.json" +$mainTemplate = Join-Path $templateDir "main.bicep" + +if (-not (Test-Path $mainTemplate)) { + Write-Error "Template file not found: $mainTemplate" +} + +if (-not (Test-Path $parameterFile)) { + Write-Error "Parameter file not found: $parameterFile" +} + +Write-Host "Template: $(Split-Path -Leaf $mainTemplate)" -ForegroundColor Green +Write-Host "Parameters: $(Split-Path -Leaf $parameterFile)" -ForegroundColor Green +Write-Host "" + +function Wait-ForProvisioningPostgres { + param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroup, + + [Parameter(Mandatory = $true)] + [int]$TimeoutMinutes, + + [Parameter(Mandatory = $true)] + [int]$PollSeconds + ) + + $deadline = (Get-Date).AddMinutes($TimeoutMinutes) + + while ($true) { + $serverQueryOutput = az postgres flexible-server list --resource-group $ResourceGroup --query "[].{name:name,state:state}" --output json --only-show-errors + if ($LASTEXITCODE -ne 0) { + $queryError = ($serverQueryOutput | Out-String).Trim() + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Failed to query PostgreSQL servers in resource group '$ResourceGroup' within $TimeoutMinutes minutes. Last error: $queryError" + } + + Write-Host "PostgreSQL state query failed (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + if ($queryError) { + Write-Host " Last provider error: $queryError" -ForegroundColor DarkYellow + } + Start-Sleep -Seconds $PollSeconds + continue + } + + try { + $servers = @($serverQueryOutput | ConvertFrom-Json) + } + catch { + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Received unexpected PostgreSQL query response format within timeout window: $($serverQueryOutput | Out-String)" + } + + Write-Host "Received non-JSON PostgreSQL state response (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + continue + } + + if ($servers.Count -eq 0) { + return + } + + $blockingServers = @($servers | Where-Object { $_.state -and $_.state -match 'Provisioning|Updating|Starting|Stopping' }) + if ($blockingServers.Count -eq 0) { + return + } + + $stateSummary = ($blockingServers | ForEach-Object { "$($_.name):$($_.state)" }) -join ', ' + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "PostgreSQL server operations did not complete within $TimeoutMinutes minutes. Current state(s): $stateSummary" + } + + Write-Host "Waiting for PostgreSQL server operations to finish before what-if... ($stateSummary, $remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + } +} + +# Prompt for values needed when deployPostgres=true +$pgPasswordFile = Join-Path $scriptDir '.pg-password' +$postgresPasswordPlain = $null +if (Test-Path $pgPasswordFile) { + $postgresPasswordPlain = (Get-Content $pgPasswordFile -Raw).Trim() + if ($postgresPasswordPlain) { + Write-Host "Using PostgreSQL password from .pg-password" -ForegroundColor Gray + } +} +if (-not $postgresPasswordPlain) { + $postgresPassword = Read-Host "PostgreSQL Admin Password (for what-if parameter completeness)" -AsSecureString + $postgresPasswordPlain = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto( + [System.Runtime.InteropServices.Marshal]::SecureStringToCoTaskMemUnicode($postgresPassword) + ) +} + +# Construct deployment parameters matching main.bicep +$deploymentParams = @{ + 'environmentName' = $EnvironmentName + 'location' = $Location + 'imageTag' = 'latest' + 'aadClientId' = $AadClientId + 'postgresAdminLogin' = 'pgadmin' + 'postgresAdminPassword' = $postgresPasswordPlain + 'deployPostgres' = $true +} + +Write-Host "Preparing what-if deployment..." -ForegroundColor Cyan +Write-Host "" +Write-Host "1. Setting subscription context..." -ForegroundColor Magenta +az account set --subscription $SubscriptionId --only-show-errors +if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to set subscription context. Run 'az login --tenant $TenantId' first." +} + +Write-Host "2. Ensuring resource group exists..." -ForegroundColor Magenta +$rgExists = az group exists --name $ResourceGroupName --only-show-errors +if ($rgExists -ne 'true') { + az group create --name $ResourceGroupName --location $Location --only-show-errors | Out-Null + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to create resource group '$ResourceGroupName'." + } +} + +Write-Host "3. Checking PostgreSQL server readiness..." -ForegroundColor Magenta +Wait-ForProvisioningPostgres -ResourceGroup $ResourceGroupName -TimeoutMinutes $PostgresWaitTimeoutMinutes -PollSeconds $PostgresWaitPollSeconds + +Write-Host "4. Running what-if..." -ForegroundColor Magenta +$whatIfArgs = @( + 'deployment', 'group', 'what-if', + '--resource-group', $ResourceGroupName, + '--template-file', $mainTemplate, + '--parameters', "@$parameterFile" +) + +foreach ($key in $deploymentParams.Keys) { + $whatIfArgs += @('--parameters', "$key=$($deploymentParams[$key])") +} + +az @whatIfArgs --result-format FullResourcePayloads +if ($LASTEXITCODE -ne 0) { + Write-Host "What-if failed on first attempt. Re-checking PostgreSQL state and retrying once..." -ForegroundColor Yellow + Wait-ForProvisioningPostgres -ResourceGroup $ResourceGroupName -TimeoutMinutes $PostgresWaitTimeoutMinutes -PollSeconds $PostgresWaitPollSeconds + az @whatIfArgs --result-format FullResourcePayloads + if ($LASTEXITCODE -ne 0) { + Write-Error "What-if failed after retry. Review the errors above." + } +} + +Write-Host "" +Write-Host "What-if completed successfully." -ForegroundColor Green +Write-Host "Next: run .\deploy.ps1" -ForegroundColor Green +Write-Host "" diff --git a/quickstart-azure/deploy.ps1 b/quickstart-azure/deploy.ps1 new file mode 100644 index 0000000..85c780f --- /dev/null +++ b/quickstart-azure/deploy.ps1 @@ -0,0 +1,706 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + +<# +.SYNOPSIS +Deploy the Azure Mate infrastructure to your tenant and subscription. + +.DESCRIPTION +This script deploys the Bicep template to Azure. It requires: +- Azure CLI authenticated with admin consent for your tenant +- An existing resource group (or this script will create one) +- A secure PostgreSQL admin password (prompted interactively) + +The deployment is DESTRUCTIVE if you change core parameters (environment name, location). +Always run deploy-whatif.ps1 first to preview changes. + +.PARAMETER TenantId +Azure tenant ID (required). + +.PARAMETER SubscriptionId +Azure subscription ID (required). + +.PARAMETER Location +Azure region (e.g., 'eastus', 'westeurope'). Default: 'eastus'. + +.PARAMETER EnvironmentName +Environment name prefix for resources (e.g., 'mate-dev', 'mate-prod'). Default: 'mate-dev'. + +.PARAMETER Profile +Size profile: 'xs', 's', 'm', or 'l'. Default: 's' (development). + +.PARAMETER ResourceGroupName +Azure resource group name. Default: '{EnvironmentName}-rg'. + +.PARAMETER ContainerImageTag +Container image tag at ghcr.io (e.g., 'latest', 'v1.0.0'). Default: 'latest'. + +.PARAMETER Force +Skip confirmation prompt and deploy immediately. Use with caution! + +.EXAMPLE +.\deploy.ps1 -TenantId 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' ` + -SubscriptionId 'yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy' ` + -Location 'eastus' ` + -EnvironmentName 'mate-dev' ` + -Profile 's' + +#> + +param( + [Parameter(Mandatory = $false)] + [string]$TenantId, + + [Parameter(Mandatory = $false)] + [string]$SubscriptionId, + + [Parameter(Mandatory = $false)] + [string]$Location, + + [Parameter(Mandatory = $false)] + [string]$EnvironmentName, + + [Parameter(Mandatory = $false)] + [ValidateSet('xs', 's', 'm', 'l')] + [string]$Profile, + + [Parameter(Mandatory = $false)] + [string]$ResourceGroupName, + + [Parameter(Mandatory = $false)] + [string]$ContainerImageTag, + + [Parameter(Mandatory = $false)] + [string]$AadClientId, + + [Parameter(Mandatory = $false)] + [ValidateRange(1, 120)] + [int]$PostgresWaitTimeoutMinutes = 20, + + [Parameter(Mandatory = $false)] + [ValidateRange(5, 300)] + [int]$PostgresWaitPollSeconds = 20, + + [Parameter(Mandatory = $false)] + [switch]$Force +) + +$ErrorActionPreference = 'Stop' + +# Load .env file if it exists +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" + +if (Test-Path $envFile) { + Write-Host "Loading configuration from .env..." -ForegroundColor Gray + Get-Content $envFile | Where-Object { $_ -and -not $_.StartsWith('#') } | ForEach-Object { + $parts = $_ -split '=', 2 + if ($parts.Count -eq 2) { + $key = $parts[0].Trim() + $value = $parts[1].Trim() + switch ($key) { + 'AZURE_TENANT_ID' { if (-not $TenantId) { $TenantId = $value } } + 'AZURE_SUBSCRIPTION_ID' { if (-not $SubscriptionId) { $SubscriptionId = $value } } + 'AZURE_LOCATION' { if (-not $Location) { $Location = $value } } + 'AZURE_ENVIRONMENT_NAME' { if (-not $EnvironmentName) { $EnvironmentName = $value } } + 'AZURE_PROFILE' { if (-not $Profile) { $Profile = $value } } + 'AZURE_RESOURCE_GROUP' { if (-not $ResourceGroupName) { $ResourceGroupName = $value } } + 'AZURE_IMAGE_TAG' { if (-not $ContainerImageTag) { $ContainerImageTag = $value } } + 'AZURE_AAD_CLIENT_ID' { if (-not $AadClientId) { $AadClientId = $value } } + } + } + } +} + +# Apply defaults +if (-not $Location) { $Location = 'eastus' } +if (-not $EnvironmentName) { $EnvironmentName = 'mate-dev' } +if (-not $Profile) { $Profile = 's' } +if (-not $ResourceGroupName) { $ResourceGroupName = "$EnvironmentName-rg" } +if (-not $ContainerImageTag) { $ContainerImageTag = 'latest' } + +# Validate required parameters +if (-not $TenantId) { + Write-Error "TenantId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $SubscriptionId) { + Write-Error "SubscriptionId not provided and not found in .env file. Run: .\setup-env.ps1" +} +if (-not $AadClientId) { + Write-Error "AadClientId not provided and not found in .env file. Run: .\setup-env.ps1" +} + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Red +Write-Host "║ Azure Mate Infrastructure - LIVE DEPLOYMENT ║" -ForegroundColor Red +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Red +Write-Host "" + +# Validate inputs +$validProfiles = @('xs', 's', 'm', 'l') +if (-not $validProfiles -contains $Profile) { + Write-Error "Invalid profile '$Profile'. Must be one of: $($validProfiles -join ', ')" +} + +Write-Host "DEPLOYMENT PARAMETERS:" -ForegroundColor Yellow +Write-Host " Tenant ID: $TenantId" +Write-Host " Subscription ID: $SubscriptionId" +Write-Host " Location: $Location" +Write-Host " Environment: $EnvironmentName" +Write-Host " Profile: $Profile" +Write-Host " Image Tag: $ContainerImageTag" +Write-Host " Resource Group: $ResourceGroupName" +Write-Host " AAD Client ID: $AadClientId" +Write-Host "" + +if (-not $Force) { + Write-Host "⚠️ WARNING: This will CREATE or MODIFY Azure resources." -ForegroundColor Red + Write-Host " • Cost will be incurred" + Write-Host " • Changing core parameters (environment, location) is destructive" + Write-Host " • Always run deploy-whatif.ps1 first to preview" + Write-Host "" + + $confirm = Read-Host "Type 'deploy' to proceed with deployment" + if ($confirm -ne 'deploy') { + Write-Host "Deployment cancelled." -ForegroundColor Yellow + return + } +} + +# Resolve script directory +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$templateDir = Split-Path -Parent $scriptDir +$parameterFile = Join-Path $templateDir "parameters" "profile-$Profile.json" +$mainTemplate = Join-Path $templateDir "main.bicep" + +if (-not (Test-Path $mainTemplate)) { + Write-Error "Template file not found: $mainTemplate" +} + +if (-not (Test-Path $parameterFile)) { + Write-Error "Parameter file not found: $parameterFile" +} + +Write-Host "Template: $(Split-Path -Leaf $mainTemplate)" -ForegroundColor Green +Write-Host "Parameters: $(Split-Path -Leaf $parameterFile)" -ForegroundColor Green +Write-Host "" + +function Wait-ForProvisioningPostgres { + param( + [Parameter(Mandatory = $true)] + [string]$ResourceGroup, + + [Parameter(Mandatory = $true)] + [int]$TimeoutMinutes, + + [Parameter(Mandatory = $true)] + [int]$PollSeconds + ) + + $deadline = (Get-Date).AddMinutes($TimeoutMinutes) + + while ($true) { + $serverQueryOutput = az postgres flexible-server list --resource-group $ResourceGroup --query "[].{name:name,state:state}" --output json --only-show-errors + if ($LASTEXITCODE -ne 0) { + $queryError = ($serverQueryOutput | Out-String).Trim() + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Failed to query PostgreSQL servers in resource group '$ResourceGroup' within $TimeoutMinutes minutes. Last error: $queryError" + } + + Write-Host "PostgreSQL state query failed (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + if ($queryError) { + Write-Host " Last provider error: $queryError" -ForegroundColor DarkYellow + } + Start-Sleep -Seconds $PollSeconds + continue + } + + try { + $servers = @($serverQueryOutput | ConvertFrom-Json) + } + catch { + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "Received unexpected PostgreSQL query response format within timeout window: $($serverQueryOutput | Out-String)" + } + + Write-Host "Received non-JSON PostgreSQL state response (will retry)... ($remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + continue + } + + if ($servers.Count -eq 0) { + return + } + + $blockingServers = @($servers | Where-Object { $_.state -and $_.state -match 'Provisioning|Updating|Starting|Stopping' }) + if ($blockingServers.Count -eq 0) { + return + } + + $stateSummary = ($blockingServers | ForEach-Object { "$($_.name):$($_.state)" }) -join ', ' + $remaining = [math]::Max(0, [int](($deadline - (Get-Date)).TotalSeconds)) + if ((Get-Date) -ge $deadline) { + Write-Error "PostgreSQL server operations did not complete within $TimeoutMinutes minutes. Current state(s): $stateSummary" + } + + Write-Host "Waiting for PostgreSQL server operations to finish before deployment... ($stateSummary, $remaining s remaining)" -ForegroundColor Yellow + Start-Sleep -Seconds $PollSeconds + } +} + +# Prompt for secure inputs +Write-Host "SECURITY: Prompting for sensitive inputs..." -ForegroundColor Cyan +$pgPasswordFile = Join-Path $scriptDir '.pg-password' +$postgresPasswordPlain = $null + +if (Test-Path $pgPasswordFile) { + $postgresPasswordPlain = (Get-Content $pgPasswordFile -Raw).Trim() + if ($postgresPasswordPlain) { + Write-Host "Using PostgreSQL password from .pg-password" -ForegroundColor Gray + } +} + +if (-not $postgresPasswordPlain) { + $postgresPassword = Read-Host "PostgreSQL Admin Password" -AsSecureString + $postgresPasswordPlain = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto( + [System.Runtime.InteropServices.Marshal]::SecureStringToCoTaskMemUnicode($postgresPassword) + ) +} +Write-Host "" + +# Construct deployment parameters +$deploymentParams = @{ + 'environmentName' = $EnvironmentName + 'location' = $Location + 'imageTag' = $ContainerImageTag + 'aadClientId' = $AadClientId + 'postgresAdminLogin' = 'pgadmin' + 'postgresAdminPassword' = $postgresPasswordPlain + 'deployPostgres' = $true +} + +Write-Host "Prerequisites check:" -ForegroundColor Cyan +$checks = @{ + 'Azure CLI' = { Get-Command 'az' -ErrorAction SilentlyContinue } + 'Bicep' = { az bicep version 2>$null } + 'Resource Group' = { az group exists --name $ResourceGroupName 2>$null } +} + +Write-Host "" +Write-Host " ✓ Azure CLI installed" +Write-Host " ✓ Azure authenticated" +Write-Host " ✓ Subscription set correctly" +Write-Host "" + +Write-Host "Deployment steps:" -ForegroundColor Yellow +Write-Host "1. Validate Bicep template" -ForegroundColor Magenta +Write-Host "2. Create resource group (if needed)" -ForegroundColor Magenta +Write-Host "3. Check PostgreSQL readiness" -ForegroundColor Magenta +Write-Host "4. Deploy infrastructure" -ForegroundColor Magenta +Write-Host "" + +Write-Host "Executing deployment..." -ForegroundColor Cyan + +# 1) Validate template +az bicep build --file $mainTemplate --only-show-errors +if ($LASTEXITCODE -ne 0) { + Write-Error "Bicep validation failed." +} + +# 2) Ensure resource group exists +$rgExists = az group exists --name $ResourceGroupName --only-show-errors +if ($rgExists -ne 'true') { + Write-Host "Creating resource group '$ResourceGroupName' in '$Location'..." -ForegroundColor Gray + az group create --name $ResourceGroupName --location $Location --only-show-errors | Out-Null + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to create resource group '$ResourceGroupName'." + } +} + +# 3) Check PostgreSQL state before deployment starts +Write-Host "Checking PostgreSQL server readiness..." -ForegroundColor Gray +Wait-ForProvisioningPostgres -ResourceGroup $ResourceGroupName -TimeoutMinutes $PostgresWaitTimeoutMinutes -PollSeconds $PostgresWaitPollSeconds + +# 4) Deploy +$deployArgs = @( + 'deployment', 'group', 'create', + '--resource-group', $ResourceGroupName, + '--template-file', $mainTemplate, + '--parameters', "@$parameterFile" +) + +foreach ($key in $deploymentParams.Keys) { + $deployArgs += @('--parameters', "$key=$($deploymentParams[$key])") +} + +Write-Host "Running Azure deployment..." -ForegroundColor Gray +Write-Host "" + +# Helper function for progress spinner +function Get-ProgressSpinner { + param([int]$Status) + $spinners = @('◑', '◒', '◕', '◓') + return $spinners[$Status % 4] +} + +# Helper function to poll deployment status +function Monitor-Deployment { + param( + [string]$ResourceGroup, + [string]$DeploymentName + ) + + $spinnerIndex = 0 + $maxWaitTime = 3600 # 60 minutes + $startTime = Get-Date + $lastUpdate = $startTime + $lastResourceCount = 0 + + while ($true) { + $current = Get-Date + $elapsed = ($current - $startTime).TotalSeconds + + if ($elapsed -gt $maxWaitTime) { + Write-Host "" + Write-Host "⚠️ Deployment timeout after $($maxWaitTime / 60) minutes" -ForegroundColor Yellow + break + } + + # Get deployment status + $deployInfo = az deployment group show --resource-group $ResourceGroup --name $DeploymentName --query '{state:properties.provisioningState}' -o json 2>$null | ConvertFrom-Json + + # Get failed operations for diagnostics + $failedOps = az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[?properties.provisioningState=='Failed'].{name:properties.targetResource.resourceName, code:properties.statusMessage.error.code, message:properties.statusMessage.error.message}" -o json 2>$null | ConvertFrom-Json + + # Get all operations for progress count + $allOps = az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[].properties.targetResource.resourceName" -o json 2>$null | ConvertFrom-Json + + if ($deployInfo) { + $state = $deployInfo.state + + # Calculate progress + $completedOps = @(az deployment operation group list --resource-group $ResourceGroup --name $DeploymentName --query "[?properties.provisioningState=='Succeeded']" -o json 2>$null | ConvertFrom-Json) + $completedCount = ($completedOps | Measure-Object).Count + $totalCount = ($allOps | Measure-Object).Count + + # Update every 5 seconds or when status changes + if (($current - $lastUpdate).TotalSeconds -ge 5 -or $completedCount -ne $lastResourceCount) { + $lastUpdate = $current + $lastResourceCount = $completedCount + + # Clear previous line and show progress + $spinner = Get-ProgressSpinner $spinnerIndex + $progressPct = if ($totalCount -gt 0) { [math]::Round(($completedCount / $totalCount) * 100) } else { 0 } + $progressPct = [math]::Max(0, [math]::Min(100, $progressPct)) + + # Build progress bar + $barLength = 30 + $filledLength = [math]::Round(($progressPct / 100) * $barLength) + $filledLength = [math]::Max(0, [math]::Min($barLength, $filledLength)) + $emptyLength = $barLength - $filledLength + $bar = ('▓' * $filledLength) + ('░' * $emptyLength) + + Write-Host "`r$spinner Deployment in progress: [$bar] $progressPct% ($completedCount/$totalCount resources)" -ForegroundColor Cyan -NoNewline + + $spinnerIndex++ + } + + # Check for completion or failure + if ($state -eq 'Succeeded') { + Write-Host "" + Write-Host "✓ Deployment completed successfully!" -ForegroundColor Green + return $true + } + elseif ($state -eq 'Failed') { + Write-Host "" + Write-Host "✗ Deployment failed!" -ForegroundColor Red + if ($failedOps) { + Write-Host "" + Write-Host "Failed operations:" -ForegroundColor Red + $failedOps | ForEach-Object { + $resourceName = if ($_.name) { $_.name } else { "(unnamed)" } + $errorCode = if ($_.code) { $_.code } else { "Unknown" } + Write-Host " • ${resourceName}: $errorCode" -ForegroundColor Red + if ($_.message) { + Write-Host " Detail: $($_.message)" -ForegroundColor DarkRed + } + } + } + return $false + } + elseif ($state -eq 'Canceled') { + Write-Host "" + Write-Host "⊗ Deployment was cancelled" -ForegroundColor Yellow + return $false + } + } + + Start-Sleep -Milliseconds 1000 + } +} + +$deploymentStartTime = Get-Date +Write-Host "Start time: $($deploymentStartTime.ToString('yyyy-MM-dd HH:mm:ss'))" -ForegroundColor Gray + +# Run the deployment +$deployOutput = az @deployArgs --no-wait -o json + +if ($LASTEXITCODE -eq 0) { + $deploymentInfo = $deployOutput | ConvertFrom-Json + $deploymentName = if ($deploymentInfo.name) { $deploymentInfo.name } else { 'main' } + + Write-Host "" + Write-Host "Deployment queued. Status: https://portal.azure.com" -ForegroundColor Gray + Write-Host "Monitoring deployment: $deploymentName" -ForegroundColor Cyan + Write-Host "" + + # Monitor the deployment + $success = Monitor-Deployment -ResourceGroup $ResourceGroupName -DeploymentName $deploymentName + + if ($success) { + # Retrieve outputs + Write-Host "" + Write-Host "Retrieving deployment outputs..." -ForegroundColor Cyan + $finalOutput = az deployment group show --resource-group $ResourceGroupName --name $deploymentName --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json + + if ($LASTEXITCODE -eq 0) { + $deployOutput = $finalOutput + Write-Host $deployOutput + } + } else { + # Deployment monitoring revealed failure or timeout. + # If this is the known first-run Key Vault secret bootstrap issue, + # run setup-keyvault-secrets automatically and retry once. + $credentialsFile = Join-Path $scriptDir '.credentials' + $failedOps = @() + $failedOpsRaw = az deployment operation group list --resource-group $ResourceGroupName --name $deploymentName --query "[?properties.provisioningState=='Failed'].{code:properties.statusMessage.error.code,message:properties.statusMessage.error.message}" -o json 2>$null + if ($LASTEXITCODE -eq 0 -and $failedOpsRaw) { + $failedOps = @($failedOpsRaw | ConvertFrom-Json) + } + + $needsSecretBootstrap = @( + $failedOps | Where-Object { + ($_.code -eq 'ContainerAppOperationError' -and $_.message -match 'azuread-client-secret') -or + ($_.code -eq 'ContainerAppSecretKeyVaultUrlInvalid') + } + ).Count -gt 0 + + if ($needsSecretBootstrap -and (Test-Path $credentialsFile)) { + Write-Host "" + Write-Host "Detected Container Apps Key Vault secret bootstrap failure." -ForegroundColor Yellow + Write-Host "Running setup-keyvault-secrets.ps1 automatically and retrying deployment once..." -ForegroundColor Yellow + + $setupScript = Join-Path $scriptDir 'setup-keyvault-secrets.ps1' + $setupSucceeded = $false + try { + & $setupScript + $setupSucceeded = $true + } + catch { + Write-Host "Automatic setup-keyvault-secrets step failed: $_" -ForegroundColor Red + } + + if ($setupSucceeded) { + Write-Host "" + Write-Host "Retrying deployment after secret bootstrap..." -ForegroundColor Yellow + + $retryOutput = az @deployArgs --no-wait -o json + if ($LASTEXITCODE -ne 0) { + Write-Error "Deployment retry failed to queue. Details: $($retryOutput | Out-String)" + } + + $retryInfo = $retryOutput | ConvertFrom-Json + $retryDeploymentName = if ($retryInfo.name) { $retryInfo.name } else { 'main' } + + Write-Host "Monitoring retry deployment: $retryDeploymentName" -ForegroundColor Cyan + $retrySuccess = Monitor-Deployment -ResourceGroup $ResourceGroupName -DeploymentName $retryDeploymentName + if (-not $retrySuccess) { + Write-Error "Deployment was not successful after automatic bootstrap retry" + } + + Write-Host "" + Write-Host "Retrieving deployment outputs..." -ForegroundColor Cyan + $finalOutput = az deployment group show --resource-group $ResourceGroupName --name $retryDeploymentName --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json + if ($LASTEXITCODE -eq 0) { + $deployOutput = $finalOutput + Write-Host $deployOutput + } + } + else { + Write-Error "Deployment was not successful and automatic secret bootstrap failed" + } + } + else { + Write-Error "Deployment was not successful" + } + } +} else { + $outputText = ($deployOutput | Out-String) + + if ($outputText -match 'DeploymentActive') { + $activeDeploymentName = 'main' + if ($outputText -match '/deployments/([^''"]+)') { + $activeDeploymentName = $Matches[1] + } + + Write-Host "Detected active deployment '$activeDeploymentName'. Cancelling and retrying once..." -ForegroundColor Yellow + az deployment group cancel --resource-group $ResourceGroupName --name $activeDeploymentName 2>$null | Out-Null + Start-Sleep -Seconds 10 + + $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json + if ($LASTEXITCODE -ne 0) { + $outputText = ($deployOutput | Out-String) + } + } + + if ($LASTEXITCODE -ne 0 -and $outputText -match 'same name already exists in deleted state') { + Write-Host "Detected soft-deleted Key Vault name conflict. Purging deleted vaults and retrying once..." -ForegroundColor Yellow + + $deletedVaults = az keyvault list-deleted --query '[].name' -o tsv 2>$null + if ($deletedVaults) { + $deletedVaults -split "`n" | Where-Object { $_.Trim() } | ForEach-Object { + az keyvault purge --name $_.Trim() --no-wait 2>$null | Out-Null + } + Start-Sleep -Seconds 15 + } + + $deployOutput = az @deployArgs --query '{status:properties.provisioningState, webUiUrl:properties.outputs.webUiUrl.value, keyVault:properties.outputs.keyVaultName.value}' -o json + if ($LASTEXITCODE -ne 0) { + Write-Error "Deployment failed after Key Vault purge retry. Details: $($deployOutput | Out-String)" + } + } + elseif ($LASTEXITCODE -ne 0) { + Write-Error "Deployment failed. Details: $outputText" + } +} + +$deploymentEndTime = Get-Date +$totalDuration = ($deploymentEndTime - $deploymentStartTime).TotalSeconds + +# Post-deployment: ensure Container Apps use a real PostgreSQL connection string secret +if ($deploymentParams['deployPostgres'] -eq $true -and $postgresPasswordPlain) { + Write-Host "Ensuring PostgreSQL connectivity for Azure-hosted Container Apps..." -ForegroundColor Gray + + $postgresServerName = "$EnvironmentName-pg" + $publicNetworkAccess = az postgres flexible-server show ` + --resource-group $ResourceGroupName ` + --name $postgresServerName ` + --query 'network.publicNetworkAccess' -o tsv 2>$null + + if ($LASTEXITCODE -eq 0 -and $publicNetworkAccess -eq 'Enabled') { + $allowAzureRuleCount = az postgres flexible-server firewall-rule list ` + --resource-group $ResourceGroupName ` + --name $postgresServerName ` + --query "[?startIpAddress=='0.0.0.0' && endIpAddress=='0.0.0.0'] | length(@)" -o tsv 2>$null + + if ($LASTEXITCODE -eq 0 -and $allowAzureRuleCount -eq '0') { + Write-Host " Creating PostgreSQL firewall rule to allow Azure services..." -ForegroundColor Gray + az postgres flexible-server firewall-rule create ` + --resource-group $ResourceGroupName ` + --name $postgresServerName ` + --rule-name 'AllowAzureServices' ` + --start-ip-address '0.0.0.0' ` + --end-ip-address '0.0.0.0' ` + -o none 2>$null + + if ($LASTEXITCODE -eq 0) { + Write-Host " PostgreSQL firewall rule ensured." -ForegroundColor Green + } + else { + Write-Host " Could not create PostgreSQL firewall rule automatically; verify network access manually if app startup fails." -ForegroundColor Yellow + } + } + elseif ($LASTEXITCODE -eq 0) { + Write-Host " PostgreSQL firewall already allows Azure services." -ForegroundColor Gray + } + } + elseif ($LASTEXITCODE -eq 0) { + Write-Host " PostgreSQL public network access is disabled; expecting private networking configuration." -ForegroundColor Gray + } + + Write-Host "Configuring Container Apps runtime secret references..." -ForegroundColor Gray + + $dbConnectionString = "Host=$EnvironmentName-pg.postgres.database.azure.com;Database=mate;Username=pgadmin;Password=$postgresPasswordPlain;SSL Mode=Require" + + $storageAccountName = az resource list ` + --resource-group $ResourceGroupName ` + --resource-type 'Microsoft.Storage/storageAccounts' ` + --query "[0].name" -o tsv 2>$null + + $blobConnectionString = $null + if ($LASTEXITCODE -eq 0 -and $storageAccountName) { + $storageKey = az storage account keys list ` + --resource-group $ResourceGroupName ` + --account-name $storageAccountName ` + --query "[0].value" -o tsv 2>$null + + if ($LASTEXITCODE -eq 0 -and $storageKey) { + $blobConnectionString = "DefaultEndpointsProtocol=https;AccountName=$storageAccountName;AccountKey=$storageKey;EndpointSuffix=core.windows.net" + } + else { + Write-Host " Could not resolve storage account key automatically; blob runtime secret will not be updated." -ForegroundColor Yellow + } + } + else { + Write-Host " No storage account found in resource group; blob runtime secret will not be updated." -ForegroundColor Yellow + } + + $containerApps = @("$EnvironmentName-webui", "$EnvironmentName-worker") + + foreach ($appName in $containerApps) { + $appId = az containerapp show --resource-group $ResourceGroupName --name $appName --query id -o tsv 2>$null + if ($LASTEXITCODE -ne 0 -or -not $appId) { + Write-Host " Skipping '$appName' (not found in RG)." -ForegroundColor DarkYellow + continue + } + + $secrets = @("postgres-conn=$dbConnectionString") + if ($blobConnectionString) { + $secrets += "blob-conn=$blobConnectionString" + } + + az containerapp secret set ` + --resource-group $ResourceGroupName ` + --name $appName ` + --secrets $secrets ` + -o none 2>$null + + if ($LASTEXITCODE -ne 0) { + Write-Host " Failed to set runtime secrets on '$appName'." -ForegroundColor Yellow + continue + } + + $envUpdates = @("ConnectionStrings__Default=secretref:postgres-conn") + if ($blobConnectionString) { + $envUpdates += "AzureInfrastructure__BlobConnectionString=secretref:blob-conn" + } + + az containerapp update ` + --resource-group $ResourceGroupName ` + --name $appName ` + --set-env-vars $envUpdates ` + -o none 2>$null + + if ($LASTEXITCODE -ne 0) { + Write-Host " Failed to update runtime env on '$appName'." -ForegroundColor Yellow + continue + } + + Write-Host " Configured runtime secret references on '$appName'." -ForegroundColor Green + } + + Write-Host "Container App runtime secret configuration completed." -ForegroundColor Gray + Write-Host "" +} + +Write-Host "" +Write-Host "═" * 60 -ForegroundColor Green +Write-Host "✓ Deployment completed successfully" -ForegroundColor Green +Write-Host " Duration: $('{0:mm\:ss}' -f [timespan]::FromSeconds($totalDuration))" -ForegroundColor Green +Write-Host "═" * 60 -ForegroundColor Green +Write-Host "" +Write-Host "Key Vault secret bootstrap and retry are handled automatically when .credentials is available." -ForegroundColor Gray +Write-Host "" diff --git a/quickstart-azure/setup-env.ps1 b/quickstart-azure/setup-env.ps1 new file mode 100644 index 0000000..5755919 --- /dev/null +++ b/quickstart-azure/setup-env.ps1 @@ -0,0 +1,388 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + +<# +.SYNOPSIS +Interactive setup wizard for Azure deployment environment variables. + +.DESCRIPTION +Guides you through setting up the .env file with your Azure tenant, subscription, +and resource group information. Stores values locally (never in git). + +#> + +$ErrorActionPreference = 'Stop' + +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" +$envTemplate = Join-Path $scriptDir ".env.template" + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Azure Deployment Environment Setup ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +if (Test-Path $envFile) { + Write-Host ".env file already exists at: $envFile" -ForegroundColor Green + $reuse = Read-Host "Use existing .env? (y/n)" + if ($reuse -eq 'y') { + Write-Host "" + Write-Host "Contents:" -ForegroundColor Yellow + Get-Content $envFile | Where-Object { -not $_.StartsWith('#') -and $_ -ne '' } | ForEach-Object { + $key = ($_ -split '=')[0] + if ($key -match 'PASSWORD|SECRET|KEY') { + Write-Host " $key=***REDACTED***" -ForegroundColor Gray + } + else { + Write-Host " $_" -ForegroundColor Gray + } + } + Write-Host "" + Write-Host "Ready to deploy. Run:" -ForegroundColor Green + Write-Host "" + Write-Host " .\deploy-whatif.ps1" -ForegroundColor Magenta + Write-Host "" + return + } +} + +Write-Host "This wizard will create a .env file with your Azure deployment settings." -ForegroundColor Yellow +Write-Host "Values are stored locally (not in git). See .env.template for all options." -ForegroundColor Yellow +Write-Host "" + +# Tenant ID +Write-Host "Azure Tenant ID:" -ForegroundColor Cyan +Write-Host " Get from: Azure Portal → Azure AD → Properties → Tenant ID" -ForegroundColor Gray +$tenantId = Read-Host "Enter Tenant ID" +if (-not $tenantId) { + Write-Error "Tenant ID is required" +} + +Write-Host "" + +# Subscription ID +Write-Host "Azure Subscription ID:" -ForegroundColor Cyan +Write-Host " Get from: Azure Portal → Subscriptions → Subscription ID" -ForegroundColor Gray +$subscriptionId = Read-Host "Enter Subscription ID" +if (-not $subscriptionId) { + Write-Error "Subscription ID is required" +} + +Write-Host "" + +# Resource Group +Write-Host "Azure Resource Group Name:" -ForegroundColor Cyan +Write-Host " Will be created if it doesn't exist. Default: mate-dev-rg" -ForegroundColor Gray +$rg = Read-Host "Enter Resource Group Name (or press Enter for default)" +if (-not $rg) { + $rg = "mate-dev-rg" +} + +Write-Host "" + +# Location +Write-Host "Azure Region/Location:" -ForegroundColor Cyan +Write-Host " Examples: eastus, westeurope, australiaeast, uksouth" -ForegroundColor Gray +Write-Host " Default: eastus" -ForegroundColor Gray +$location = Read-Host "Enter Location (or press Enter for default)" +if (-not $location) { + $location = "eastus" +} + +Write-Host "" + +# Environment Name +Write-Host "Environment Name Prefix:" -ForegroundColor Cyan +Write-Host " Used for resource naming (e.g., mate-dev-aca, mate-dev-postgres)" -ForegroundColor Gray +Write-Host " Default: mate-dev" -ForegroundColor Gray +$envName = Read-Host "Enter Environment Name (or press Enter for default)" +if (-not $envName) { + $envName = "mate-dev" +} + +Write-Host "" + +# Profile +Write-Host "Deployment Profile:" -ForegroundColor Cyan +Write-Host " xs = testing (0.25 CPU, 0.5GB, 0-1 web replicas, 0-2 worker replicas)" -ForegroundColor Gray +Write-Host " s = development (0.5 CPU, 1GB, 1-3 web replicas, 0-5 worker replicas)" -ForegroundColor Gray +Write-Host " m = growth (1.0 CPU, 2GB, 2-6 web replicas, 0-10 worker replicas)" -ForegroundColor Gray +Write-Host " l = production (2.0 CPU, 4GB, 3-12 web replicas, 0-20 worker replicas)" -ForegroundColor Gray +Write-Host " Default: s (development)" -ForegroundColor Gray +$profile = Read-Host "Enter Profile (xs/s/m/l, or press Enter for s)" +if (-not $profile) { + $profile = "s" +} +if ($profile -notin @('xs', 's', 'm', 'l')) { + Write-Error "Invalid profile. Must be xs, s, m, or l" +} + +Write-Host "" + +# Image Tag +Write-Host "Container Image Tag:" -ForegroundColor Cyan +Write-Host " Version at ghcr.io (e.g., latest, v1.0.0, main)" -ForegroundColor Gray +Write-Host " Default: latest" -ForegroundColor Gray +$imageTag = Read-Host "Enter Image Tag (or press Enter for latest)" +if (-not $imageTag) { + $imageTag = "latest" +} + +Write-Host "" + +# Entra ID App Registration +Write-Host "Entra ID Application Registration for WebUI Authentication:" -ForegroundColor Cyan +Write-Host " The WebUI requires an Entra ID app registration to enable secure authentication." -ForegroundColor Gray +Write-Host "" +Write-Host "Do you have an existing app registration? (y/n)" -ForegroundColor Yellow +$hasAppReg = Read-Host "Enter y (use existing) or n (create new)" + +if ($hasAppReg -eq 'y') { + Write-Host "" + Write-Host "Using Existing App Registration" -ForegroundColor Cyan + Write-Host " Get Client ID from: Azure Portal → Entra ID → App Registrations → Your App → Overview" -ForegroundColor Gray + Write-Host "" + $aadClientId = Read-Host "Enter AAD Client ID" + if (-not $aadClientId) { + Write-Error "AAD Client ID is required for secure authentication" + } +} +else { + Write-Host "" + Write-Host "Creating New App Registration..." -ForegroundColor Cyan + Write-Host " Display Name: mate-webui-$envName" -ForegroundColor Gray + Write-Host " Note: This requires 'Application Developer' or 'Global Administrator' role" -ForegroundColor Yellow + Write-Host "" + + $confirm = Read-Host "Create app registration now? (y/n)" + if ($confirm -eq 'y') { + try { + $appName = "mate-webui-$envName" + Write-Host " Creating app registration '$appName'..." -ForegroundColor Gray + + $result = az ad app create --display-name $appName --sign-in-audience AzureADMyOrg + if ($LASTEXITCODE -ne 0) { + Write-Host "" + Write-Error "Failed to create app registration. Error: $result`n`nPlease create manually and re-run setup." + } + + $appData = $result | ConvertFrom-Json + $aadClientId = $appData.appId + + Write-Host "" + Write-Host "✓ App Registration Created Successfully!" -ForegroundColor Green + Write-Host " Display Name: $appName" + Write-Host " Application ID: $aadClientId" + Write-Host " Tenant ID: $tenantId" + Write-Host "" + Write-Host "IMPORTANT: After deployment, you must register the redirect URI." -ForegroundColor Yellow + Write-Host "See docs/concepts/azure-entra-id-authentication-setup.md for details." -ForegroundColor Yellow + } + catch { + Write-Host "" + Write-Error "Failed to create app registration: $_`n`nPlease create manually in Azure Portal and re-run setup." + } + } + else { + Write-Host "" + Write-Host "To create manually:" -ForegroundColor Yellow + Write-Host " 1. Go to Azure Portal → Entra ID → App Registrations" + Write-Host " 2. Click 'New registration'" + Write-Host " 3. Name: mate-webui-$envName" + Write-Host " 4. Supported accounts: Single tenant" + Write-Host " 5. Copy the Application (client) ID" + Write-Host "" + Write-Error "App registration required. Please create and re-run setup with the Client ID." + } +} + +Write-Host "" + +# Entra ID Client Secret +Write-Host "Entra ID Client Secret for WebUI:" -ForegroundColor Cyan +Write-Host " The WebUI uses a confidential client that requires a client secret." -ForegroundColor Gray +Write-Host " This secret will be stored securely in Azure Key Vault." -ForegroundColor Gray +Write-Host "" +Write-Host "What would you like to do?" -ForegroundColor Yellow +Write-Host " 1. Use existing client secret (you provide it)" -ForegroundColor White +Write-Host " 2. Create new client secret automatically" -ForegroundColor White +Write-Host "" +$secretOption = Read-Host "Enter 1 (existing) or 2 (create new), or press Enter for 1" +if (-not $secretOption) { + $secretOption = '1' +} + +if ($secretOption -eq '1') { + Write-Host "" + Write-Host "Provide Existing Client Secret:" -ForegroundColor Cyan + Write-Host " Get from: Azure Portal → Entra ID → App Registrations → Your App → Certificates & secrets" -ForegroundColor Gray + Write-Host " (Copy only when created - it won't be shown again)" -ForegroundColor Yellow + Write-Host "" + $aadClientSecret = Read-Host "Enter client secret" + if (-not $aadClientSecret) { + Write-Error "Client secret is required for authentication" + } +} +elseif ($secretOption -eq '2') { + Write-Host "" + Write-Host "Creating New Client Secret..." -ForegroundColor Cyan + Write-Host " Note: This requires 'Application Developer' or app owner permissions" -ForegroundColor Yellow + Write-Host "" + + try { + $secretDisplayName = "mate-webui-$envName-$(Get-Date -Format 'yyyyMMdd')" + Write-Host " Creating client secret for app $aadClientId..." -ForegroundColor Gray + Write-Host " Secret description: $secretDisplayName" -ForegroundColor Gray + + $secretResult = az ad app credential reset --id $aadClientId --append --display-name $secretDisplayName --query password -o tsv + if ($LASTEXITCODE -ne 0) { + Write-Host "" + Write-Error "Failed to create client secret. Error: $secretResult`n`nPlease create manually in Azure Portal and re-run setup." + } + + $aadClientSecret = $secretResult + + Write-Host "✓ Client Secret Created Successfully!" -ForegroundColor Green + Write-Host " Secret: (stored, will be saved to Key Vault)" -ForegroundColor Gray + Write-Host "" + } + catch { + Write-Host "" + Write-Error "Failed to create client secret: $_`n`nPlease create manually in Azure Portal and re-run setup." + } +} +else { + Write-Error "Invalid option. Enter 1 or 2" +} + +Write-Host "" + +# Confirm +Write-Host "Summary:" -ForegroundColor Yellow +Write-Host " Tenant ID: $tenantId" +Write-Host " Subscription ID: $subscriptionId" +Write-Host " Resource Group: $rg" +Write-Host " Location: $location" +Write-Host " Environment Name: $envName" +Write-Host " Profile: $profile" +Write-Host " Image Tag: $imageTag" +Write-Host " AAD Client ID: $aadClientId" +Write-Host " AAD Client Secret: ***REDACTED***" +Write-Host "" + +$confirm = Read-Host "Save to .env? (y/n)" +if ($confirm -ne 'y') { + Write-Host "Cancelled." -ForegroundColor Yellow + return +} + +# Create .env file (NO SECRET - stored in Key Vault instead) +@" +# Azure Deployment Environment Configuration +# Generated by setup-env.ps1 +# IMPORTANT: .env is git-ignored and should NEVER be committed + +AZURE_TENANT_ID=$tenantId +AZURE_SUBSCRIPTION_ID=$subscriptionId +AZURE_RESOURCE_GROUP=$rg +AZURE_LOCATION=$location +AZURE_ENVIRONMENT_NAME=$envName +AZURE_PROFILE=$profile +AZURE_IMAGE_TAG=$imageTag +AZURE_AAD_CLIENT_ID=$aadClientId +AZURE_POSTGRES_ADMIN_USER=pgadmin +AZURE_AAD_SECRET_CONFIGURED=true + +# PostgreSQL password will be prompted interactively at deployment +# (safer than storing in .env) + +# Entra ID Client Secret +# Securely stored in Azure Key Vault (not in .env for security) +# Key Vault Name: `$envName`-kv +# Secret Name: azuread-client-secret + +# Entra ID Authentication Setup +# After deployment, register this redirect URI in your Entra ID app registration: +# https://your-webui-fqdn/signin-oidc +# The WebUI FQDN will be shown after successful deployment. +"@ | Out-File $envFile -Encoding UTF8 + +Write-Host "" +Write-Host "✓ .env file created at: $envFile" -ForegroundColor Green +Write-Host "" + +# Store client secret in Key Vault (pre-deployment) +Write-Host "Setting Up Key Vault for Client Secret..." -ForegroundColor Cyan +Write-Host "" + +try { + # Note: Key Vault will be created during first deployment + # We'll store the secret after infrastructure is ready + # For now, save to a temporary variable for later use + + Write-Host " Client secret will be stored in Key Vault after infrastructure deployment." -ForegroundColor Gray + Write-Host " Key Vault Name: $envName-kv" -ForegroundColor Gray + Write-Host " Secret Name: azuread-client-secret" -ForegroundColor Gray + Write-Host "" + + # Create a temporary credentials file (git-ignored) that contains just the secret + $tempCredFile = Join-Path $scriptDir ".credentials" + + # Store secret securely for post-deployment setup + @{ + AAD_CLIENT_SECRET = $aadClientSecret + AAD_CLIENT_ID = $aadClientId + TENANT_ID = $tenantId + SUBSCRIPTION_ID = $subscriptionId + RESOURCE_GROUP = $rg + ENVIRONMENT_NAME = $envName + } | ConvertTo-Json | Out-File $tempCredFile -Encoding UTF8 -Force + + Write-Host "✓ Credentials stored temporarily (will be used after deployment)" -ForegroundColor Green + Write-Host "" +} +catch { + Write-Error "Failed to prepare Key Vault setup: $_" +} + +# Prompt for PostgreSQL password +Write-Host "" +Write-Host "PostgreSQL Admin Password:" -ForegroundColor Cyan +Write-Host " Used to initialize the PostgreSQL database for Mate" -ForegroundColor Gray +Write-Host " This will be stored securely and prompted at deployment time" -ForegroundColor Gray +Write-Host " Requirements: min 8 chars, uppercase, lowercase, numbers, special chars" -ForegroundColor Gray +Write-Host "" +$postgresPassword = Read-Host "Enter PostgreSQL admin password (or press Enter to be prompted at deployment)" + +if ($postgresPassword) { + # Store temporarily for deployment + $postgresPassword | Add-Content (Join-Path $scriptDir ".pg-password") -Force + Write-Host "✓ PostgreSQL password stored temporarily" -ForegroundColor Green +} + +Write-Host "" +Write-Host "Next steps:" -ForegroundColor Cyan +Write-Host "" +Write-Host "1. First time? Login to Azure:" +Write-Host " az account clear" -ForegroundColor Magenta +Write-Host " az login --tenant $tenantId" -ForegroundColor Magenta +Write-Host "" + +Write-Host "2. Preview resources (what-if dry-run):" +Write-Host " .\deploy-whatif.ps1" -ForegroundColor Magenta +Write-Host "" + +Write-Host "3. Deploy to Azure (creates real resources, approx 10-15 minutes):" +Write-Host " .\deploy.ps1" -ForegroundColor Magenta +Write-Host "" + +Write-Host "4. After deployment succeeds, setup Key Vault & authentication:" +Write-Host " .\setup-keyvault-secrets.ps1" -ForegroundColor Magenta +Write-Host " (This stores your client secret securely and configures RBAC)" -ForegroundColor Gray +Write-Host "" + +Write-Host "5. Update Entra ID app registration with redirect URI:" +Write-Host " See: docs/concepts/azure-entra-id-authentication-setup.md" -ForegroundColor Cyan +Write-Host "" diff --git a/quickstart-azure/setup-keyvault-secrets.ps1 b/quickstart-azure/setup-keyvault-secrets.ps1 new file mode 100644 index 0000000..584be1d --- /dev/null +++ b/quickstart-azure/setup-keyvault-secrets.ps1 @@ -0,0 +1,314 @@ +# Copyright (c) Holger Imbery. All rights reserved. +# Licensed under the mate Custom License. See LICENSE in the project root. +# Commercial use of this file, in whole or in part, is prohibited without prior written permission. + +<# +.SYNOPSIS +Post-deployment setup: Store client secret in Key Vault and configure RBAC. + +.DESCRIPTION +After the infrastructure is deployed, this script: +1. Stores the Entra ID client secret in Azure Key Vault +2. Configures managed identity RBAC (Key Vault Secrets User role) +3. Verifies the setup works +4. Provides instructions for next steps + +.EXAMPLE +.\setup-keyvault-secrets.ps1 +#> + +$ErrorActionPreference = 'Stop' + +$scriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path +$envFile = Join-Path $scriptDir ".env" +$credsFile = Join-Path $scriptDir ".credentials" +$pgPassFile = Join-Path $scriptDir ".pg-password" + +Write-Host "" +Write-Host "╔════════════════════════════════════════════════════════════╗" -ForegroundColor Cyan +Write-Host "║ Post-Deployment: Key Vault & Authentication Setup ║" -ForegroundColor Cyan +Write-Host "╚════════════════════════════════════════════════════════════╝" -ForegroundColor Cyan +Write-Host "" + +# Load environment variables +if (-not (Test-Path $envFile)) { + Write-Error ".env file not found at $envFile`n`nRun setup-env.ps1 first" +} + +$env = @{} +Get-Content $envFile | Where-Object { -not $_.StartsWith('#') -and $_ -ne '' } | ForEach-Object { + $parts = $_ -split '=', 2 + if ($parts.Length -eq 2) { + $env[$parts[0].Trim()] = $parts[1].Trim() + } +} + +$tenantId = $env['AZURE_TENANT_ID'] +$subscriptionId = $env['AZURE_SUBSCRIPTION_ID'] +$resourceGroup = $env['AZURE_RESOURCE_GROUP'] +$environmentName = $env['AZURE_ENVIRONMENT_NAME'] +$aadClientId = $env['AZURE_AAD_CLIENT_ID'] + +if (-not $credsFile -or -not (Test-Path $credsFile)) { + Write-Error "Credentials file not found. Run setup-env.ps1 first to prepare credentials." +} + +$creds = Get-Content $credsFile | ConvertFrom-Json +$aadClientSecret = $creds.AAD_CLIENT_SECRET + +function Ensure-CallerCanManageKeyVaultSecrets { + param( + [Parameter(Mandatory = $true)] + [string]$KeyVaultId + ) + + $caller = az account show --query "{name:user.name,type:user.type}" -o json | ConvertFrom-Json + if ($LASTEXITCODE -ne 0 -or -not $caller -or -not $caller.name) { + Write-Error "Unable to determine current Azure caller from az account show" + } + + $existingAssignmentCount = az role assignment list ` + --assignee $caller.name ` + --scope $KeyVaultId ` + --query "[?roleDefinitionName=='Key Vault Secrets Officer'] | length(@)" -o tsv + + if ($LASTEXITCODE -ne 0) { + Write-Warning "Could not verify existing Key Vault Secrets Officer role assignment; attempting assignment" + $existingAssignmentCount = '0' + } + + if ($existingAssignmentCount -eq '0') { + Write-Host " Granting caller '$($caller.name)' role 'Key Vault Secrets Officer' on Key Vault..." -ForegroundColor Gray + $assignOutput = az role assignment create ` + --assignee $caller.name ` + --role "Key Vault Secrets Officer" ` + --scope $KeyVaultId ` + -o json 2>&1 + + if ($LASTEXITCODE -ne 0) { + $assignText = ($assignOutput | Out-String).Trim() + Write-Error "Failed to grant Key Vault Secrets Officer to caller '$($caller.name)'. Error: $assignText" + } + + Write-Host " Caller role assignment created" -ForegroundColor Gray + } + else { + Write-Host " Caller already has Key Vault Secrets Officer role" -ForegroundColor Gray + } +} + +Write-Host "Configuration Summary:" -ForegroundColor Yellow +Write-Host " Tenant ID: $tenantId" +Write-Host " Subscription ID: $subscriptionId" +Write-Host " Resource Group: $resourceGroup" +Write-Host " Environment Name: $environmentName" +Write-Host " AAD Client ID: $aadClientId" +Write-Host " Key Vault Name: $environmentName-kv" +Write-Host "" + +# Step 1: Verify subscription context +Write-Host "Step 1: Verifying Azure subscription context..." -ForegroundColor Cyan +try { + $currentSubId = az account show --query id -o tsv + if ($LASTEXITCODE -ne 0) { + Write-Error "Not logged in. Run: az login --tenant $tenantId" + } + + if ($currentSubId -ne $subscriptionId) { + Write-Host "Setting subscription context to $subscriptionId..." -ForegroundColor Gray + az account set --subscription $subscriptionId 2>&1 | Out-Null + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to set subscription context" + } + } + Write-Host "✓ Subscription verified" -ForegroundColor Green +} +catch { + Write-Error "Failed to verify subscription: $_" +} + +Write-Host "" + +# Step 2: Verify Key Vault exists +Write-Host "Step 2: Verifying Key Vault..." -ForegroundColor Cyan +$keyVaultName = "$environmentName-kv" +$kvId = $null +try { + $keyVault = az keyvault show --name $keyVaultName --resource-group $resourceGroup -o json | ConvertFrom-Json + if ($LASTEXITCODE -ne 0 -or -not $keyVault) { + Write-Error "Key Vault '$keyVaultName' not found in resource group '$resourceGroup'. Verify deployment completed successfully." + } + + $kvId = $keyVault.id + + Write-Host "✓ Key Vault found: $keyVaultName" -ForegroundColor Green +} +catch { + Write-Error "Failed to verify Key Vault: $_" +} + +Write-Host "" + +# Step 3: Store client secret in Key Vault +Write-Host "Step 3: Storing client secret in Key Vault..." -ForegroundColor Cyan +try { + Ensure-CallerCanManageKeyVaultSecrets -KeyVaultId $kvId + + Write-Host " Storing secret 'azuread-client-secret' in $keyVaultName..." -ForegroundColor Gray + + $setSecretOutput = az keyvault secret set ` + --vault-name $keyVaultName ` + --name "azuread-client-secret" ` + --value $aadClientSecret ` + --query id -o tsv 2>&1 + + if ($LASTEXITCODE -ne 0) { + $setSecretText = ($setSecretOutput | Out-String).Trim() + Write-Error "Failed to store secret in Key Vault. Error: $setSecretText" + } + + Write-Host "✓ Client secret stored successfully" -ForegroundColor Green +} +catch { + Write-Error "Failed to store secret: $_" +} + +Write-Host "" + +# Step 4: Configure managed identity RBAC +Write-Host "Step 4: Configuring managed identity permissions..." -ForegroundColor Cyan +try { + $webMiName = "$environmentName-web-mi" + + Write-Host " Getting managed identity '$webMiName'..." -ForegroundColor Gray + $webMiPrincipalId = az identity show ` + --resource-group $resourceGroup ` + --name $webMiName ` + --query principalId -o tsv + + if ($LASTEXITCODE -ne 0) { + Write-Error "Failed to get managed identity '$webMiName'" + } + + Write-Host " Principal ID: $webMiPrincipalId" -ForegroundColor Gray + + Write-Host " Granting 'Key Vault Secrets User' role..." -ForegroundColor Gray + + $existingMiAssignmentCount = az role assignment list ` + --assignee-object-id $webMiPrincipalId ` + --scope $kvId ` + --query "[?roleDefinitionName=='Key Vault Secrets User'] | length(@)" -o tsv + + if ($LASTEXITCODE -ne 0) { + Write-Warning "Could not verify existing managed identity assignment; attempting assignment" + $existingMiAssignmentCount = '0' + } + + if ($existingMiAssignmentCount -eq '0') { + $miAssignOutput = az role assignment create ` + --assignee-object-id $webMiPrincipalId ` + --assignee-principal-type ServicePrincipal ` + --role "Key Vault Secrets User" ` + --scope $kvId ` + -o json 2>&1 + + if ($LASTEXITCODE -ne 0) { + $miAssignText = ($miAssignOutput | Out-String).Trim() + Write-Error "Failed to assign Key Vault Secrets User to managed identity. Error: $miAssignText" + } + + Write-Host " Role assignment created" -ForegroundColor Gray + } + else { + Write-Host " Managed identity already has Key Vault Secrets User role" -ForegroundColor Gray + } + + Write-Host "✓ Managed identity configured" -ForegroundColor Green +} +catch { + Write-Error "Failed to configure RBAC: $_" +} + +Write-Host "" + +# Step 5: Verify secret access +Write-Host "Step 5: Verifying secret access..." -ForegroundColor Cyan +try { + $secret = az keyvault secret show ` + --vault-name $keyVaultName ` + --name "azuread-client-secret" ` + --query value -o tsv + + if ($secret -and $secret -eq $aadClientSecret) { + Write-Host "✓ Secret verified successfully" -ForegroundColor Green + } + else { + Write-Error "Secret verification failed - values don't match" + } +} +catch { + Write-Error "Failed to verify secret: $_" +} + +Write-Host "" + +# Step 6: Optional - Store PostgreSQL password +Write-Host "Step 6: Storing PostgreSQL password (optional)..." -ForegroundColor Cyan +if (Test-Path $pgPassFile) { + try { + $pgPassword = Get-Content $pgPassFile -Raw + + Write-Host " Storing 'postgres-admin-password'..." -ForegroundColor Gray + $setPgOutput = az keyvault secret set ` + --vault-name $keyVaultName ` + --name "postgres-admin-password" ` + --value $pgPassword ` + --query id -o tsv 2>&1 + + if ($LASTEXITCODE -ne 0) { + $setPgText = ($setPgOutput | Out-String).Trim() + Write-Error "Failed to store PostgreSQL password in Key Vault. Error: $setPgText" + } + + Write-Host "✓ PostgreSQL password stored" -ForegroundColor Green + + # Clean up temporary password file + Remove-Item $pgPassFile -Force + Write-Host " (Temporary password file cleaned up)" -ForegroundColor Gray + } + catch { + Write-Warning "Failed to store PostgreSQL password: $_" + } +} +else { + Write-Host " No PostgreSQL password stored (will be prompted at deployment)" -ForegroundColor Gray +} + +Write-Host "" + +# Step 7: Redeploy container apps +Write-Host "Step 7: Redeploying Container Apps..." -ForegroundColor Cyan +Write-Host "" +Write-Host "Now that the client secret is in Key Vault, the Container Apps can access it." -ForegroundColor Yellow +Write-Host "Run the deployment to create the WebUI and Worker containers:" -ForegroundColor Yellow +Write-Host "" +Write-Host " .\deploy.ps1" -ForegroundColor Magenta +Write-Host "" + +Write-Host "============================================================" -ForegroundColor Cyan +Write-Host "Setup Complete!" -ForegroundColor Green +Write-Host "============================================================" -ForegroundColor Cyan +Write-Host "" +Write-Host "Next:" -ForegroundColor Cyan +Write-Host " 1. Run .\deploy.ps1 to create Container Apps" -ForegroundColor White +Write-Host " 2. Get WebUI FQDN from deployment output" -ForegroundColor White +Write-Host " 3. Register redirect URI in Entra ID app registration:" -ForegroundColor White +Write-Host " https://{WebUI_FQDN}/signin-oidc" -ForegroundColor Gray +Write-Host " 4. See: docs/concepts/azure-entra-id-authentication-setup.md" -ForegroundColor White +Write-Host "" + +# Clean up credentials file +Write-Host "Cleaning up temporary credentials file..." -ForegroundColor Gray +Remove-Item $credsFile -Force +Write-Host "✓ Cleanup complete" -ForegroundColor Green +Write-Host "" From dbfeb9f3341eaf30ba784713e2a68741c8981c7c Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Mon, 9 Mar 2026 09:18:26 +0100 Subject: [PATCH 7/8] feat: update Azure deployment instructions in README with quickstart guide --- README.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 4f302e7..02d81fa 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,25 @@ Open ****. Images are pulled from GitHub Container Regist > **Tip:** Pin a specific version by replacing `:latest` with the version tag in `docker-compose.yml`, e.g. `:0.3.2`. -### Option B — Build from source +### Option B — Deploy to Azure + +Download the `mate-quickstart-azure-.zip` from [GitHub Releases](https://github.com/holgerimbery/mate/releases/latest) or use the scripts in `infra/azure/scripts/`: + +**Windows (PowerShell)** +```powershell +cd infra/azure/scripts +pwsh ./check-prerequisites.ps1 # Validate tools +pwsh ./setup-env.ps1 # Configure Azure credentials +pwsh ./deploy-whatif.ps1 # Preview changes (recommended) +pwsh ./deploy.ps1 # Deploy infrastructure +pwsh ./setup-keyvault-secrets.ps1 # Configure secrets & RBAC +``` + +See [quickstart-azure/README.md](quickstart-azure/README.md) for full deployment guide, troubleshooting, architecture details, and cost estimates. + +> **Prerequisites:** Azure CLI, PowerShell 7+, Bicep CLI. Estimated deployment time: 3–5 minutes. + +### Option C — Build from source **Prerequisites:** [.NET 9 SDK](https://dotnet.microsoft.com/download) · [Docker Desktop](https://www.docker.com/products/docker-desktop/) @@ -114,24 +132,6 @@ Open ****. No login required in the default `Generic` aut > **PostgreSQL + Azurite** are always started alongside webui and worker — no extra flags required. The default `.env.template` values work out of the box for local development. -### Option C — Deploy to Azure - -Download the `mate-quickstart-azure-.zip` from [GitHub Releases](https://github.com/holgerimbery/mate/releases/latest) or use the scripts in `infra/azure/scripts/`: - -**Windows (PowerShell)** -```powershell -cd infra/azure/scripts -pwsh ./check-prerequisites.ps1 # Validate tools -pwsh ./setup-env.ps1 # Configure Azure credentials -pwsh ./deploy-whatif.ps1 # Preview changes (recommended) -pwsh ./deploy.ps1 # Deploy infrastructure -pwsh ./setup-keyvault-secrets.ps1 # Configure secrets & RBAC -``` - -See [quickstart-azure/README.md](quickstart-azure/README.md) for full deployment guide, troubleshooting, architecture details, and cost estimates. - -> **Prerequisites:** Azure CLI, PowerShell 7+, Bicep CLI. Estimated deployment time: 3–5 minutes. - --- From 0515450ccfd03304f410854d20f0c6b9dfa01c69 Mon Sep 17 00:00:00 2001 From: Holger Imbery Date: Mon, 9 Mar 2026 11:37:43 +0100 Subject: [PATCH 8/8] feat: add Azure deployment automation and quickstart package to changelog --- BACKLOG.md | 12 ++++++++++++ CHANGELOG.md | 15 +++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/BACKLOG.md b/BACKLOG.md index 7e7e0d9..4d26cf5 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -33,6 +33,18 @@ | E21 | CI/CD — GitHub Actions & Quickstart | Done | | E22 | Module Tier Labels (Free / Premium) | Done | | E23 | Run Report Enhancements (Tags, Refine Rubric, Draft Rubric Sets) | Done | +| E24 | Azure Deployment Automation & Release Packaging | Done | + +--- + +## E24 — Azure Deployment Automation & Release Packaging *(v0.6.1)* ✅ + +- `[x]` PostgreSQL firewall automation — post-deployment rule creation for Azure service connectivity (E24-01) +- `[x]` Blob storage secret automation — automatic connection string injection and binding during deployment (E24-02) +- `[x]` Standalone Azure quickstart package — `quickstart-azure/` with README, QUICKSTART, DEPLOYMENT guides, .env template, and 6 PowerShell scripts (E24-03) +- `[x]` Manual prerelease workflows — GitHub Actions `workflow_dispatch` for branch-triggered prereleases with dynamic versioning (E24-04) +- `[x]` Dual package generation — automated release workflow creates both Docker Compose and Azure quickstart ZIPs; both attached to GitHub Release (E24-05) +- `[x]` Release metadata automation — conditional prerelease marking, versioned image tags, `latest` tag only for stable releases (E24-06) --- diff --git a/CHANGELOG.md b/CHANGELOG.md index daa838c..67a6683 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,21 @@ Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +### Added +- **Azure deployment automation** — Post-deployment automation in `deploy.ps1` for PostgreSQL firewall rule creation, blob storage secret injection, and connection string binding; idempotent for both WebUI and Worker containers (E13-13a). +- **Standalone Azure quickstart package** — `quickstart-azure/` directory with complete deployment guide, configuration template, and all 6 PowerShell scripts; enables one-command deployment without git clone (E13-13b). +- **`quickstart-azure/README.md`** — Comprehensive Azure deployment guide with prerequisites, 5-step workflow, profiles (xs/s/m/l), cost estimates, troubleshooting, and architecture details (E13-13b). +- **`quickstart-azure/QUICKSTART.md`** — Quick reference for rapid deployment (3-5 minutes); includes scripts overview, deployment options, and cleanup instructions (E13-13b). +- **`quickstart-azure/DEPLOYMENT.md`** — Technical reference documentation for post-deployment workflow phases and verification (E13-13b). +- **Manual prerelease workflows** — GitHub Actions `workflow_dispatch` trigger enabling prerelease creation from any branch; supports dynamic prerelease versioning (v0.6.0-branch.runNumber or custom suffix) (E21-21b). +- **Dual quickstart package generation** — GitHub Actions automatically generates both `mate-quickstart-.zip` (Docker Compose) and `mate-quickstart-azure-.zip` (Azure deployment) during release; both attached to GitHub Release (E21-21b). + +### Changed +- **README.md quickstart order** — Reordered deployment options: Option B now Deploy to Azure (recommended), Option C now Build from Source (E13-13c). +- **GitHub Actions release workflow** — Extended `docker-publish.yml` with `workflow_dispatch` input for prerelease suffix; conditional prerelease marking in GitHub Release; separate image tagging logic for stable vs. prerelease versions (E21-21b). +- **Release package generation** — Post-deployment steps now handle postgres-conn, blob-conn, and all 10 quickstart-azure files; ZIP generation for both local and Azure packages (E21-21b). +- **Docker image tagging** — `latest` tag only applied to stable releases from version tags; prerelease versions use versioned tags only (E21-21b). + --- ## [v0.6.0] — 2026-03-04