# Infra **What it is.** Everything that builds, ships and hosts ARC. One container image (Go API + Vue SPA static assets) runs as an ECS Fargate service behind an internet-facing ALB, talking to an Aurora PostgreSQL Serverless v2 cluster, two SQS work queues (saveoff, passback) and an S3 documents bucket. Terraform lives in `infra/` (one root per environment, shared modules), CI/CD in `.github/workflows`, and the local developer stack is `docker-compose.yaml` (Postgres + LocalStack) plus `air` hot reload. ## Topology (per environment) | Terraform module | Resources | Notes | |---|---|---| | `security_groups` | `alb-sg` (443 from 0.0.0.0/0), `ecs-sg` (8080 from alb-sg), `rds-access-sg` (5432 from ecs-sg and bastion-sg), `bastion-sg` (22 from 159.182.0.0/16 = Pearson VPN) | infra/modules/security_groups/main.tf | | `bastion` | t3.small EC2 from latest `pcm-amzn2023x86_64-prod-*` AMI (owner 056952386373), EIP, IMDSv2 required, encrypted root, DNS `bastion..arc..tech` | SSH tunnel to RDS for developers; infra/modules/bastion/main.tf | | `rds_cluster` | Aurora PostgreSQL 16.11, provisioned engine mode with `db.serverless` instances (default 2 = writer + reader), ACU 0–2 default (prd/perf 6–24), auto-pause 3600 s, `manage_master_user_password` (Secrets Manager), deletion protection, storage encrypted, 7-day backups, Performance Insights, `auto_minor_version_upgrade=false` | Secret rotation pinned to 1000 days; infra/modules/rds_cluster/main.tf | | `ecs_cluster` | ECS cluster `arc-` with Container Insights, FARGATE + FARGATE_SPOT capacity providers, log group `/aws/ecs-cluster/arc-` (731-day retention) | infra/modules/ecs_cluster/main.tf | | `s3_documents` (`s3_bucket`) | `arc--documents`: public access blocked, SSE AES256, versioning | infra/modules/s3_bucket/main.tf | | `ecs_task` | ALB + HTTPS:443 listener (`ELBSecurityPolicy-TLS-1-2-2017-01`, ACM cert), target group :8080 with health check `GET /health`; task def `arc--task` (container `app`, port 8080, container health check `curl -f localhost:8080/health`); service `arc--service` in private subnets; CPU target-tracking autoscaling (default 60%, out cooldown 60 s, in 300 s); Route53 A alias `.arc..tech`; SQS `arc--saveoff` / `arc--passback` (long-poll 20 s, redrive to `-dlq` after 4 receives, DLQ retention 14 days); IAM task role (S3 Get/Put/Delete/List on documents bucket, SQS send/receive/delete on both queues) and execution role (Secrets Manager read of DB secret + New Relic key); `null_resource.force_ecs_deployment` runs `aws ecs update-service --force-new-deployment` + `wait services-stable` on every apply | infra/modules/ecs_task/main.tf | | `cloudwatch_dashboard` | `arc--dashboard`: ECS CPU/memory/task count, RDS CPU/connections/free storage, log widgets for `panic` lines and `lti_launch` volume per 5 min | infra/modules/cloudwatch_dashboard/main.tf | ### Container environment set by Terraform (infra/modules/ecs_task/main.tf:160-248) | env var | value | read by | |---|---|---| | ENVIRONMENT | env name (dev/tst/smk/perf/uat/prd) | `server/controllers/app_controller.go` | | APP_URL | `var.app_url` | `server/config/env_helpers.go`, `server/names_and_roles/auth_service.go`, `server/services/delivery_service.go` | | DB_HOST / DB_HOST_READER | Aurora writer / reader endpoint | `server/database/connections.go` | | DB_PORT / DB_NAME | `5432` / `postgres` | `server/database/connections.go` | | DB_USERNAME / DB_PASSWORD | **secrets**: keys `username` / `password` of the RDS-managed secret (`:username::`) | `server/database/connections.go` | | SQS_SAVEOFF_URL / SQS_PASSBACK_URL | queue URLs created in the same module | `server/sqs/queues.go` | | MODE | `prod` | `server/utils/logger.go` | | NEW_RELIC_APP_NAME | `arc-api-` (hard-coded; the `new_relic_app_name` variable is passed in but unused) | `server/config/new_relic.go` (`newrelic.ConfigFromEnvironment`) | | NEW_RELIC_LICENSE_KEY | **secret** (Secrets Manager `newrelic-*`); agent is disabled when empty | `server/config/new_relic.go` | | EXTENSION_ID | `var.ldb_extension_id` (Chrome lockdown extension id) | `server/config/app_settings.go` | | DOCUMENTS_S3_BUCKET | `arc--documents` | `server/services/s3_service.go` | | SESSION_MAX_AGE_STUDENT | `var.session_max_age_student` (default 14400) | `server/config/session_config.go` | | DB_STATS_INTERVAL_SECONDS / DB_MAX_OPEN_CONNS / DB_CONN_MAX_LIFETIME_MINUTES | module defaults 30 / 25 / 5 (no env overrides them) | `server/database/pool_metrics.go`, `server/database/connections.go` | | DASHBOARD_STATS_ENABLED / DASHBOARD_STATS_INTERVAL_MIN | per-env (default true / 60) | `server/config/cron_config.go` | Not set in ECS (code defaults apply): the other `SESSION_MAX_AGE_*`, `DEBUG_MODE`, `NEW_RELIC_ENABLED`, `CLASS_ROSTER_SYNC_INTERVAL_MINUTES`, `DB_MAX_IDLE_*`, `AWS_ENDPOINT_URL`, `LOCALSTACK_ENDPOINT`. `NEW_RELIC_ENABLED` and `SQS_LOCAL` (both in `.env.example`) are read by nothing in `server/` (grep) — dead settings; New Relic on/off is decided by whether the license key is set. ### Environments (infra/environments/*/vars.tf) | env | AWS account (ECR / state bucket) | app_url | extension id | tasks min–max | task cpu/mem | DB ACU | dashboard cron | |---|---|---|---|---|---|---|---| | dev | 098588166805 (nonprd) | https://arc-dev.pearson.com | fboikdof… (STAGING ext) | 1–3 | 256/512 | 0–2 | on | | tst | nonprd | https://arc-tst.pearson.com | lejnbofk… (PRD ext) | 1–3 | 256/512 | 0–2 | on | | smk | nonprd | https://smk.arc.pearsondev.tech | lejnbofk… | 1–3 | 1024/2048 | 0–2 | off | | perf | nonprd | https://perf.arc.pearsondev.tech | fboikdof… | 3–6 | 1024/2048 | 6–24 | off | | uat | 711962919805 (prd) | https://arc-uat.pearson.com | lejnbofk… | 1–3 | 1024/2048 | 0–2 | on | | prd | 711962919805 | https://arc.pearson.com | lejnbofk… | 3–6 | 1024/2048 | 6–24 | on | Image: nonprd envs pull `098588166805.dkr.ecr.us-east-1.amazonaws.com/arc-nonprd/arc-api:latest`, uat/prd pull `711962919805…/arc-prd/arc-api:latest`. State: S3 backend `prsn-terraform-state-us-east-1-`, key `arc-infra/environments/us-east-1//terraform.tfstate`. Providers pinned: aws 6.56.0 (6.57.0 is broken, see comment), null 3.3.0; Terraform 1.11.4 in CI. ## Build (Dockerfile) Three stages: `golang:1.26-bookworm` builds `arc_server` (linux/amd64, `-s -w`) from `server/`, `main.go`, `internal/jwks`; `node:26.3-alpine` pins npm 12.0.2 and runs `npm run build-only` in `client/`; final `amazonlinux:2023-minimal` image copies `arc_server`, `server/static`, `internal/jwks/{private,public}.key` and `client/dist`, exposes 8080, runs as root. Migrations are embedded in the binary and run at startup (`server/database/connections.go` `MigrateUp`). ## CI/CD (.github/workflows) **dev_pipeline.yml** — on push/PR to `dev`: 1. `build_go` (`go build ./...`, `go test ./...`), `build_client-and-test` (npm ci, `build-only` with sourcemaps, upload `client-dist` artifact, `npm run test:unit` Vitest), `check_migration_order` (see Conventions), `snyk_sca` (test on PR at critical, monitor on dev). 2. Push to `dev` only: `build_docker` on a CodeBuild runner pushes `arc-nonprd/arc-api:latest` + Snyk container monitor. 3. `terraform_smk` → `terraform_dev` → `terraform_perf` (each `init / fmt -check (non-fatal) / validate / plan / apply -auto-approve`, gated by GitHub environments). 4. `newrelic_marker` after dev; `snyk_sast` and `snyk_iac` on dev. **prd_pipeline.yml** — on push/PR to `release/*` or `hotfix/*`: build_go, build_client (no unit tests), `build_docker` (environment `uat`; pushes `arc-prd/arc-api:latest`), `terraform_uat` → `terraform_prd`, New Relic marker. **nonprd_monthly_bastion_patch.yml** — manual dispatch (cron commented out), approval environment `nonprd-bastion-approval`, then `terraform apply -target=module.bastion` for smk, tst, dev to pick up the latest AMI. **dependabot.yml** — weekly npm updates for `/client` and `/lockdown` with 3–7 day cooldowns. ## Local stack - `docker-compose.yaml`: `postgres:16.11-alpine` (volume `pgdata`, port `DOCKER_POSTGRES_EXTENRAL_PORT`) and `localstack/localstack:latest` (services s3,sqs on 4566). `internal/localstack/init.sh` runs on ready: creates SQS `arc-saveoff`, `arc-passback`, S3 bucket `arc-documents`. LocalStack state does not survive a restart: `compose down && compose up -d` recreates the queues. - `.env.example`: copy to `.env`; points DB at localhost, `SQS_LOCAL=true`, LocalStack endpoints, `AWS_PROFILE=localstack`, session max ages, `EXTENSION_ID`, cron settings, `DOCKER_*` compose vars. - `make run` = `run-client` (`cd client && npm run dev`, Vite on https://secure.localhost:40444) + `run-server` (`air -c ./internal/.air.toml`: rebuilds `./internal/tmp/main` on .go/.tpl/.tmpl/.html changes, app on 8080, air proxy 8081). - `mise.toml` pins go 1.26, node 26.3, npm 12, podman, mockery 2.53.6. ## Gotchas - **`allowed_account_ids` is not enforced.** Every root declares `provider "aws" { alias = "arc", allowed_account_ids = … }` but no module uses the `arc` alias, so resources go through the implicit default provider. Nonprd roots list account `947439393957` while their ECR/state/ACM live in `098588166805`; the guard would fail if it were wired. infra/environments/dev/main.tf:1-8. - `null_resource.force_ecs_deployment` triggers on `timestamp()`, so every `terraform apply` redeploys the service and blocks until stable; that is how a new `:latest` image is rolled out (tags are mutable, task def is unchanged). - `aws_ecs_service` ignores `desired_count` changes so applies do not scale in an autoscaled fleet. - `rds_cluster` creates an `rds_sg` security group that nothing attaches (its output is commented out); the cluster uses `security_groups.rds_access` instead. - `rds_cluster` declares both `number_of_instances` (unused) and `num_instances` (used). - `dev` passes `session_max_age_student`; other envs rely on the module default 14400 (same value). - tst `new_relic_app_name` default is `arc-api-smk` (copy-paste) — harmless because the task def hard-codes `arc-api-`. - tst has no pipeline job (only the bastion patch touches it); perf is applied after dev on every dev push. - `perf` and `smk` disable the dashboard-stats cron so it does not pollute load-test numbers; each ECS task runs its own cron, so N tasks ⇒ N runs per interval. - DLQs exist but nothing in the repo reads them; replays use the requeue script in `internal/queue/requeue.go` ([[async-workers]]). ## Dead / legacy - `infra/accounts/nonprd` (bastion + `arc-postgres` SGs, own state key `…/account/terraform.tfstate`) is **legacy**: no environment references it (`grep -rn accounts infra/environments/*/main.tf` → none); the per-env `security_groups`/`bastion` modules replaced it. A stale `.terraform/modules/modules.json` in dev still lists `nonprd_bastion`, `stack`, `platform`, `bastion_host` modules that no longer exist. `infra/accounts/prd/main.tf` is empty. - `internal/deploy/deploy.sh` is **dead**: builds `prism-dev` and polls an AWS App Runner service in account 091146282849 (predecessor project); nothing references it. - `internal/deploy/deploy_{dev,smk,uat,prd}.sh` are **legacy** manual deploys (podman build/push + `terraform apply` from a laptop) superseded by the pipelines. `deploy_prd.sh` applies `infra/environments/uat`, not prd (copy-paste bug). - Makefile `localstack` target starts a standalone localstack via podman socket; superseded by docker-compose + init.sh. ## Security notes - JWKS private key `internal/jwks/private.key` is baked into the image ("temporary until we have a proper way to deploy/store keys", Dockerfile:46-48) — every image holder has the LTI signing key. - Container runs as root (`USER arc` commented out). - ALB TLS policy `ELBSecurityPolicy-TLS-1-2-2017-01` is an older policy (allows CBC ciphers). - RDS master secret rotation is effectively disabled (1000 days). - Container Snyk scan step is commented out; Snyk SAST/IaC run with non-failing exit handling on dev. - docker-compose mounts `/var/run/docker.sock` into localstack (local only). ## Related [[arc-server]], [[arc-client]], [[postgres]], [[saveoff-queue]], [[passback-queue]], [[documents-bucket]], [[new-relic]], [[lockdown-extension]], [[load-testing]], [[propl-data]]