diff --git a/backend/src/main/resources/content/platform/ci-cd/overview.md b/backend/src/main/resources/content/platform/ci-cd/overview.md index 30b9361..96f7c9d 100644 --- a/backend/src/main/resources/content/platform/ci-cd/overview.md +++ b/backend/src/main/resources/content/platform/ci-cd/overview.md @@ -10,31 +10,47 @@ audience: dev > is **stale — do not believe it.** Verified against the workflow YAML and the runners on > **2026-07-14**. -## Do not hand-build images +## ⚠️ ONLY `cd-develop` RUNS. Production is manual. -**Merge → let CI build → deploy the image CI produced.** Do **not** run `./build.sh` yourself. +This is the single most misunderstood fact about our CI. Get it right: -A hand-built image **skips the security gate entirely** (npm audit + Trivy) and **races CI for the -mutable `:dev` tag**, so what ends up in the registry is nondeterministic. On 2026-07-13 a full -day's worth of images were hand-built on the false belief that CI was off; every one of them -reached bcos.dev **unscanned**. +| Path | Who builds | +|------|-----------| +| **`develop` → bcos.dev** | **CI.** `cd-develop.yml` fires on every push. **Never hand-build.** | +| **`main` → production** | **You, by hand.** `./build.sh --registry registry.bcos.cloud --push` | -The narrow exception — **CI is genuinely broken AND the user explicitly asks.** Confirm it is -broken with the actions API first, not from memory: +`cd-main.yml` exists in most repos but is **dormant scaffolding — it has never run.** It targets +`runs-on: production`, and the only runner advertising that label (runner-02) is not an enabled +service. **Zero `cd-main` runs have ever executed in any repo.** Merging to `main` builds nothing. + +**Consequence worth naming:** the npm-audit + Trivy gate exists **only on `develop`.** Production +images are hand-built and therefore unscanned by construction. That is why *"ship only what has been +verified on bcos.dev"* is not a nicety — it is the entire security argument. + +## Do not hand-build for bcos.dev + +**Merge to `develop` → let CI build → deploy the image CI produced.** Do **not** run `./build.sh`. + +A hand-built dev image **skips the security gate** and **races CI for the mutable `:dev` tag**, so +what lands in the registry is nondeterministic. On 2026-07-13 a full day's images were hand-built on +the false belief that CI was off; every one reached bcos.dev **unscanned**. + +The narrow exception — **CI is genuinely broken AND the user explicitly asks.** Confirm it is broken +with the actions API first, not from memory: `GET /repos/hiveiq-src//actions/tasks?limit=3`. ## What the workflows actually do -| Workflow | Trigger | Does | -|----------|---------|------| -| `cd-develop.yml` | push to `develop` | `npm audit --audit-level=critical` → **Trivy** (`--exit-code 1 --severity HIGH,CRITICAL`) → `docker build` → push `:dev` **and `:`** | -| `cd-main.yml` | push to `main` | same, against the production registry | -| `sonar-pr.yml` / `sonar-nightly.yml` | PR / schedule | SonarQube (`192.168.200.230:9000`) | -| `version-check.yml` | PR | guards the version bump | +| Workflow | Trigger | Runs? | Does | +|----------|---------|-------|------| +| `cd-develop.yml` | push to `develop` | ✅ **yes** | `npm audit --audit-level=critical` → **Trivy** (`--exit-code 1 --severity HIGH,CRITICAL`) → `docker build` → push `:dev` **and `:`** | +| `cd-main.yml` | push to `main` | ❌ **never has** | dormant — `runs-on: production`, no enabled runner | +| `sonar-pr.yml` / `sonar-nightly.yml` | PR / schedule | ✅ | SonarQube (`192.168.200.230:9000`) | +| `version-check.yml` | PR | ✅ | guards the version bump | -**Immutable tags (added 2026-07-14):** every build now pushes `:` alongside `:dev`, so a +**Immutable tags (added 2026-07-14):** `cd-develop` now pushes `:` alongside `:dev`, so a deploy can pin a **scanned** artifact instead of chasing a mutable tag. Pin `:` in the -instance `.env` — never deploy from `:dev` or `:latest`. +instance `.env` — never deploy from `:dev`. ### ⚠️ `cd-develop` does NOT deploy for most repos @@ -44,26 +60,27 @@ The old claim that "a green `cd-develop` run redeploys the service to bcos.dev" | Repo | `cd-develop` | Images CI builds | |------|--------------|------------------| | `hiveops-incident` | **builds + DEPLOYS to bcos.dev** | backend + frontend | -| `hiveops-devices` | **builds + DEPLOYS to bcos.dev** | backend + frontend | +| `hiveops-devices` | **builds + DEPLOYS the frontend** (backend image pushed only) | backend + frontend | | `hiveops-recon` | build + push only | backend + frontend | | `hiveops-reports` | build + push only | backend + frontend | | `hiveops-journal` | build + push only | backend | | `hiveops-auth` | build + push only | backend | | `hiveops-dashboard` | build + push only | frontend | -| `hiveops-fleet` | build + push only | **frontend ONLY** | +| `hiveops-fleet` | build + push only | **frontend ONLY** — see below | -For every repo except incident and devices, a green run means **the image exists in the registry — -nothing has been deployed.** Someone still has to pull and restart on `.251`. +For every repo except incident (and the devices *frontend*), a green run means **the image exists in +the registry — nothing has been deployed.** Someone still has to pull and restart on `.251`. -### 🕳️ Known gap — some backends are never built by CI +Where a backend is blue/green, CI deliberately does **not** deploy it: CI cannot know which colour +is live, so the cutover stays a human's job. -- **`hiveops-fleet`: the backend (`hiveiq-fleet`) is built by neither `cd-develop` nor `cd-main`.** - Only the frontend is. Any fleet backend image therefore *has* to be hand-built, which means it - never passes Trivy. -- **`hiveops-devices`: `cd-main` builds only the frontend** — the production devices backend image - is not produced by CI either. +### 🕳️ Known gap — `hiveops-fleet`'s backend is not built by CI -Treat these as **defects, not as licence to hand-build**. Fix the workflow. +**`cd-develop` builds `hiveiq-fleet-frontend` only.** The backend image (`hiveiq-fleet`) is built by +no pipeline at all, so it can only be hand-built — meaning it never passes the gate. Tracked in +**hiveiq-ops/hiveiq-devops#46**, fixed by **hiveops-fleet#96**. + +Treat this as a **defect, not a licence to hand-build.** Fix the workflow. ## The runners diff --git a/backend/src/main/resources/content/processes/release/overview.md b/backend/src/main/resources/content/processes/release/overview.md index caa800c..6322a27 100644 --- a/backend/src/main/resources/content/processes/release/overview.md +++ b/backend/src/main/resources/content/processes/release/overview.md @@ -14,17 +14,22 @@ Milestones section in [[processes.gitea-issues]]. Roll up everything targeted at `GET /repos/issues/search?milestones=2026-07&owner=hiveiq-src`. ## Microservice release + +> **Production is built BY HAND, on purpose.** `cd-main.yml` exists in most repos but **does not +> run** — it targets `runs-on: production`, and no enabled runner serves that label. **Zero +> `cd-main` runs have ever executed.** Do not assume merging to `main` builds anything. +> +> This is the one place `./build.sh` is correct. It is **not** correct for bcos.dev, where +> `cd-develop` builds and scans on every push — see [[platform.ci-cd]]. + 1. Merge to `main` via the [[processes.main]] gate. -2. **CI builds it — you do not.** `cd-main.yml` fires on the merge: npm audit → **Trivy** → - build → push `:latest` **and `:`** to `registry.bcos.cloud`. **Never run `./build.sh` - yourself** — a hand-built image skips the security gate. See **[[platform.ci-cd]]**, which also - lists the repos whose backend CI does *not* build (a defect to fix, not a licence to hand-build). -3. **Confirm the scanned image exists**, then deploy on the production services VM - (`173.231.195.250`) by **pinning `:`** — never `:latest`, which is mutable. - Blue/green rules apply: deploy the **STANDBY** colour, then flip the nginx map - (`scripts/bluegreen-deploy.sh`). **Never `docker compose up -d` the ACTIVE colour** — that - recreates the live container and is an outage, not a cutover (see [[processes.main]] and - [[platform.deployment]]). +2. **Build + push by hand:** `./build.sh --registry registry.bcos.cloud --push` (image `hiveiq-`). + Because CI is not in this path, **nothing has run npm audit or Trivy against this image** — the + security gate only exists on `develop`. Ship from a commit already verified on bcos.dev. +3. Deploy on the production services VM (`173.231.195.250`). Blue/green rules apply: deploy the + **STANDBY** colour, then flip the nginx map (`scripts/bluegreen-deploy.sh`). **Never + `docker compose up -d` the ACTIVE colour** — that recreates the live container and is an outage, + not a cutover (see [[processes.main]] and [[platform.deployment]]). 4. Update **VERSIONS.md** and add a **What's New** entry (below). ## Agent release (`hiveops-agent`)