Profile had no technical.* module. Documents: frontend-only SPA (hiveiq-profile-frontend
:5184, no backend/DB — composes auth/mgmt/messaging), PROFILE_VERSION sha-pinning +
single-instance recreate, the profile.bcos.cloud->www.bcos.cloud root-redirect red herring,
and that Profile is the sole is_default module (browser landing page). Inbox unread badge
cross-links [profile.inbox] rather than duplicating. Captured from the 2026-07-17 prod deploy.
Records what actually governs atms.institution, after feedback #792 (NC000937
kept reverting from Peoples Bank to Five Star).
- Three writers, one lock: updateLastHeartbeat() (guarded, never overwrites),
syncAtmConfig() (agent-reported key; now skips a locked device and WARNs),
moveInstitution() (operator intent — sets atms.institution_locked).
- The fingerprint for debugging "it changed back": `Assigned institution ...` is
syncAtmConfig(); `Auto-assigned institution ...` is the guarded heartbeat path
and is not the culprit. devices#93 blamed the wrong one for weeks.
- The key a device reports is NOT its authenticated identity: the agent reads it
from `incident.institution.key` in its own hiveops.properties, and agent-proxy's
putIfAbsent keeps it over the token-derived key. Fixing agent.auth.token alone
does not stop a mis-imaged box re-asserting the wrong institution.
- Pre-V19 rows ship unlocked; re-run the move to lock one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
hiveops-feedback has been live in production since 2026-07-09 and had no Guide
module; coverage-reconcile.py has been reporting it as MISSING. Coverage is now
green: 127 modules across 19 apps, no missing services/agent modules/SPAs.
Written from source (develop @ ea81eb4), matching the technical.* shape:
single overview.md, tab=Overview, audience=dev, order=10.
Records the things that actually cost time rather than restating the obvious:
- Split DB vars (DB_HOST/DB_PORT/DB_NAME), NOT DB_URL like journal/devices.
- Two auth realms: X-Submit-Key -> ROLE_SUBMITTER for intake; JWT MSP/BCOS for
triage. RELOCATED is terminal and immutable (409 on accept/review).
- Gitea relay: label lookups swallow their exceptions, so a token without
read:organization silently drops org labels (feedback#14).
- Rate limiting trusts X-Forwarded-For blindly and is per-instance Caffeine
(feedback#5). /h2-console/** is permitAll unconditionally (feedback#7).
- The list endpoint has NO institution scoping — every MSP admin sees every
customer's submissions, and QDS log in as MSP_ADMIN (feedback#19, unfixed).
- Accept never worked in prod until 2026-07-16: %2F path -> 405, then an empty
GITEA_TOKEN -> unauthenticated -> Gitea 404 on a private repo -> opaque 500.
- No Kafka, no close-sync with Gitea.
Refs #26
The module described pre-#290 behaviour: rule key resolved to a display NAME,
compared to the ATM's institution name. The code has not worked that way since
2d2dee0 — it compares rule.institution to atm.institutionKey directly,
case-insensitively, and never resolves a name. institutionKeyService is still
injected but the matcher does not call it.
This was not harmless. Investigating incident#241 (a PBNC EMV rule a customer
reported twice), the stale gotcha led to the conclusion that the rule was
silently falling through to global and that #232 was the root cause. Both wrong
— the rule fires correctly in prod (Loki: SUPPRESS 'EMV Fallback Suppression
for Peoples Bank ONLY' on NC000936, institution=PBNC). That wrong conclusion was
posted to a customer-facing ticket and had to be retracted. The Guide exists to
prevent exactly that.
- claude.md — gotcha rewritten to key-to-key; adds a do-not-re-diagnose note
- internal.md — failure-mode row + Matching gotcha section rewritten; keeps the
silent-fallback-to-global warning, which is still real when a
key genuinely mismatches, and marks the name-based version as
historical (#232/#153, fixed 2d2dee0, live 2026-07-16)
- architect.md — evaluation order step 3 corrected
claude.md's "rule institution is stored as the KEY, not a display name" was
already correct and is unchanged.
Refs #29
Cash Stats now reads devices (record of reference) with an incident fallback;
journal's CassetteInventoryService dual-writes incident + journal.cassette-baseline
to devices. Documents the mask-aware RCY parse, the removed duplicate parser, the
new topics, and the nginx /journal/ path quirk.
Refs #156
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The environment cap decided what a Guide instance serves, but defaulted to ALL
four tiers. That makes the safe configuration the one you have to remember: a
missing or misspelled GUIDE_SERVED_AUDIENCES in production would serve the
Architect and Claude tiers to any MSP_ADMIN — and MSP_ADMIN is a customer-facing
role (QDS log in as MSP_ADMIN). One forgotten env line was all it took.
Default is now `customer`. Environments opt UP explicitly:
production -> unset (customer only: modules + Customer tab)
bcos.dev/DLX -> customer,internal,dev,bcos (set in that instance's compose)
bcos.dev's compose was pinned to all four tiers first (hiveiq-openmetal-dev
b2d4e00, already live on .251) so this change cannot narrow it.
Also updates platform.guide-access, which documented the old default and told
readers the opposite of what the code now does.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Task Permissions tab shipped in hiveops-fleet#119 and is live, but had no
guide module at all — the API returned 404 for fleet.task-permissions, and
fleet.tasks mentions permissions once without describing the grant model.
This is the setting that decides which commands a customer may push to their own
ATMs: deny-by-default, per-institution, per-command. Highest-consequence fleet
setting we have, least documented.
Adds the 5 canonical role tabs, grounded in hiveops-fleet source:
- Customer — grant/revoke, why only Reboot + Restart Agent are offered
- Internal — triage for "customer cannot reboot", what we never widen
- Architect — deny-by-default rationale, the two independent gates, replace-not-
merge semantics, why GRANTABLE lives in code
- Testing — role matrix + the 400-on-CONFIG_UPDATE case, against bcos.dev
- Claude — ownership, table, endpoints, gotchas
Verified: all 5 tabs parse under MarkdownGuideService.parse(); reconciler counts
126 modules with no missing role-tabs. Role matrix confirmed live on bcos.dev
(CUSTOMER 403/403/200/403, MSP_ADMIN 200/200/200/200).
Refs #23. Follow-up to hiveops-fleet#119, hiveops-fleet#100.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Behaviour changed in hiveops-auth#56 / hiveops-profile#10 (both live on
bcos.dev), so the Guide has to follow.
profile.notifications
- inboxPollSeconds added to the prefs doc, defaults and validation tables
(0-3600, 0 = off, default 60; no migration, absent key reads as default)
- corrects the "flags are stored only, no consumer" claim: it is still true
of email/push/severity/quiet-hours, but inboxPollSeconds now has a real
consumer (App.svelte arms the poll from it). It is the only setting on
that tab that does anything today
- Customer tab documents the new Messages section; testing checklist gains
rows for the interval, Off, badge appearing, and badge clearing
- internal: 400 cause, Off-by-design and silent-poll-failure rows
profile.inbox
- documents the sidebar badge: stores.ts owns unreadCount/refreshUnread,
poll interval is an auth-owned pref, unread-only semantics, #ef4444 vs
the sub-tab's #2563eb, and why it must not reuse Inbox's page-local tally
Refs hiveops-auth#56, hiveops-profile#10
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document that the flow normalises host_result (SUCCESS->approved, FAIL/TIMEOUT
->declined) and surfaces a decline Reason, matching hiveops-transactions #49.
Note the note-breakdown still needs raw_lines and a structured reason field
remain follow-ups on #48.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Update the msp.institution-keys module (Customer/Internal/Claude/Testing tabs)
and technical.devices to match the new authz:
- Only BCOS_ADMIN can create/delete institution keys; MSP admins edit only and
must submit a request to BCOS to add/remove an institution.
- POST/DELETE /api/institution-keys -> hasRole('BCOS_ADMIN'); PUT stays MSP+BCOS.
- Note the underlying ownership gap (devices mints institutions with no mgmt
mirror; keys have diverged) tracked in msp#29.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'dev' audience tier was gated via isInternal() (MSP_ADMIN || BCOS_ADMIN),
so MSP admins could read developer/architecture content: technical.* and
platform.* Overviews, agent-module internals, dev Internal/Overview tabs.
Remap 'dev' to isBcos() (BCOS_ADMIN only) in visible(). Retag the 66 Testing
docs from 'audience: dev' to 'audience: internal' so MSP keeps them, per
Johannes.
Result: MSP sees Customer + Internal + Testing (66 modules); loses all
Architect/Claude/architecture. BCOS unchanged. Update platform.guide-access
to the new model.
Closes#18
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 162 Architect/Claude docs were tagged 'audience: dev'. The 'dev' tier
resolves to isInternal() = ROLE_MSP_ADMIN || ROLE_BCOS_ADMIN, so every
MSP_ADMIN could read full internal architecture: source paths, DB tables,
service ports, NGINX routes. Verified live on bcos.dev as msp_a@msp1.bcos.dev.
Retag them to 'audience: bcos' (isBcos, BCOS_ADMIN only). Content-only change;
the 'dev' tier mapping is deliberately left alone so the 15 Internal, 35
Overview and 66 Testing docs stay MSP-visible.
Also add platform.guide-access documenting the audience tiers and the
per-role visibility matrix, so the access model is queryable from the Guide
instead of re-derived from source each time.
Verified: MSP module count unchanged (116 across 17 apps); no module loses
all its tabs. Guide is bcos.dev only, not deployed to production.
Closes#16
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"Deploy = smoke test" only told you to curl the endpoint. An SPA returns 200 + index.html for any
path, so a curl proves nothing about a frontend feature — which is how dashboard#15 got called
"verified" on the strength of a container healthcheck and a bundle grep.
Documents hiveiq-src/hiveops-e2e and, more importantly, the trap in front of it: these SPAs cannot
be driven by a plain browser. They need X-HiveIQ-Browser: true (nginx's browser gate 302s anything
else to the marketing site) and Authorization: Bearer <jwt> (the SPAs carry no auth code; the
Electron shell injects it).
The headers must be injected at the network layer, not via Playwright's extraHTTPHeaders — those
land in the CORS preflight, nginx rejects X-HiveIQ-Browser there, and every XHR fails. The app then
renders and says "Failed to load...", which reads as a broken feature and is not.
States plainly that coverage is one spec, so nobody assumes the suite has them covered.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documents the Total shown under each ATM name on Cash Stats, across all five
tabs.
The load-bearing facts, which are easy to get wrong:
- The total is derived client-side in CashStatus.svelte. No endpoint, DTO, or
column backs it, and none should be added.
- The asterisk means the total is a floor, not a balance: mix/retract notes
(no denomination) and cassettes with no reported count are excluded. An empty
retract cassette deliberately does not flag the row.
- An ATM with no reported inventory shows a dash, never $0, which would read as
an empty machine.
- The pills must keep wrapping: real fleets run 5+ cassettes, which under the
old nowrap layout overflowed the table and crushed the ATM column.
Also notes that the bcos.dev simulator emits no mix/retract slots, so the
partial-total path never fires there and must not be mistaken for dead code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous commit (8d7450d) said CI builds production images on merge to main. That is WRONG.
cd-main.yml exists in most repos but is dormant scaffolding: it targets runs-on: production, and
the only runner advertising that label (runner-02) is not an enabled service. ZERO cd-main runs
have ever executed in any repo. Merging to main builds nothing.
Only cd-develop runs. Production is built by hand, on purpose:
develop -> bcos.dev = CI builds + scans. Never hand-build.
main -> production = you build by hand. ./build.sh is CORRECT here.
Consequence now stated explicitly in both modules: the npm-audit + Trivy gate exists ONLY on
develop, so production images are unscanned by construction. That is why 'ship only what has been
verified on bcos.dev' is the entire security argument, not a nicety.
Also corrects the fleet gap to what it actually is — cd-develop never builds hiveiq-fleet — and
drops the devices cd-main 'gap', which was not a gap because that pipeline does not run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
platform.deployment and processes.release both walked the reader through ./build.sh, never
mentioning that cd-develop/cd-main already build and scan. Following either one produces an
UNSCANNED image that races CI for the mutable :dev tag. That is what happened on 2026-07-13:
a full day of hand-built images reached bcos.dev without passing Trivy.
Verified against the workflow YAML and both runners on 2026-07-14:
- cd-develop does NOT deploy for 6 of 8 repos. Only incident and devices deploy to bcos.dev;
everywhere else CI stops at push. The old 'a green run redeploys the service' claim was wrong.
- hiveops-fleet CI builds the FRONTEND ONLY — the fleet backend is built by neither cd-develop
nor cd-main. hiveops-devices cd-main builds only the frontend too. Flagged as defects.
- runner-02 IS active (incident built on it today) — the 'not the active worker' note was wrong.
- CI now pushes an immutable :<sha8> alongside :dev/:latest, so deploys can pin a scanned image.
Also records the runner failure signatures ('lease does not exist', 'network ... not found'),
the daemon-restart + prune runbook, the standing warning never to garbage-collect or delete
/opt/docker-registry, and the fact that merging a batch of PRs at once overwhelms a runner's
4 shared job slots and breaks its Docker daemon.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirrors devops.dev 'Dev Environment (bcos.dev)'. The domain is what people
actually navigate by; OpenMetal is the hosting provider, which is an
implementation detail and is already covered in the body.
Two new BCOS-only modules documenting the two environments and the rule that
separates them:
devops.prod Production Environment (OpenMetal) — hosts, repo layout, health
report + config drift, and the KNOWN DRIFT warning: main's
00-backends.conf names different active blue/green slots than prod
actually serves (devices/transactions/agent-proxy), so deploying it
as-is flips three services to the wrong slot. Do NOT bulk-copy main
over prod.
devops.dev Dev Environment (bcos.dev) — single 16G all-in-one VM at ~93-97%
RAM (recreate JVM services ONE at a time), its own postgres and
registry, the new 06:00 health report (johannes only), and the
health-check gotchas that must not be "fixed" back: SPA frontends
return 200 for /actuator/health so the body must be checked, the
probe must retry, and exited *-green/*-blue are standby slots.
Both audience: bcos.
The rule, now stated in both:
everything bcos.dev -> hiveiq-openmetal-dev
everything prod -> hiveiq-openmetal-prod
shared, env-agnostic -> hiveiq-devops
Also corrects devops.backups and devops.security, which still told readers the
backup/security scripts live in `hiveops-devops/scripts/ops/`. They were moved to
`hiveiq-openmetal-prod/scripts/ops/` today — the Guide is the source of truth, so
a stale path in it is worse than none.
Refs: hiveiq-ops/hiveiq-devops#39, #38
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OWASP now fails builds on three HIGH CVEs in transitive deps. bom 1.0.7 takes the upstream patches
(no suppressions):
postgresql 42.7.11 -> 42.7.13 CVE-2026-54291 (CVSS4 8.2) — channelBinding downgrade / MITM
httpcore5 5.3.6 -> 5.4.3 CVE-2026-54399 (7.5), CVE-2026-54428 (7.5) — DoS
httpcore5 is transitive via httpclient5; Apache ships them as a matched pair, so bom 1.0.7 bumps
httpclient5 5.5.2 -> 5.6.2 which pulls the fixed core.
Verified locally: builds and tests pass on 1.0.7.
Refs bom#6
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace Open/All columns with: open issues with no milestone (unscheduled
backlog) and open issues in 2026-07; linked number is the count.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- 2026-07 page: authoritative counts from org-wide milestone-name search
(38 open across 12 repos; add devices=4, fleet #50, dashboard #51 that the
stale per-repo milestone counters had hidden).
- External http(s) links in rendered guide content now open in a new tab.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- New top-level MILESTONE section (bcos-only), peer to PROCESSES, renders
uppercase via .sec-head; first module is milestone.2026-07 (per-repo
release-scope view with Gitea filter links).
- Nav UX: all sidebar sections start collapsed; no section/app auto-expands
on entry; landing pane shows the HiveIQ logo instead of auto-selecting Agent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document how to view a release milestone's issues in Gitea:
- per-repo web UI filter URL pattern (milestone=<id>)
- cross-repo rollup via issues/search by name
- a copy-paste one-liner that regenerates the current-month
per-repo links for any YYYY-MM (no hardcoded ids)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
incident.journal-events (Claude tab): the state flip is a dual-write — incident
writes its own atm_properties, then mirrors to devices, and it is devices the UI
reads. Adds the best-effort/afterCommit semantics and the nginx-listener URL rule.
technical.devices: document POST /api/internal/atms/{agentAtmId}/service-state,
that devices does not ingest journal events, the now()-stamped supervisor_entered_at,
and that the status badge is derived client-side (agent-offline masks IN_SUPERVISOR).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documents the mgmt portal 5-step New Customer + Institution wizard shipped to
production in hiveops-mgmt#43: the five steps, atomic-submit semantics, the
MSP_ADMIN agent-onboarding handoff message, and the gotchas (uppercased
institution key, unrecallable handoff, skipping step 4 skips the handoff,
MESSAGING_SERVICE_URL required).
Guide had no coverage for hiveops-template, so the copy-don't-rewrite rule
and the TemplatePage-vs-TablePage decision were only in a stale CLAUDE.md.
- decision table for which page to copy; ask the user when not obvious
- assigned dev ports read from the actual vite configs (next free = 5191;
CLAUDE.md still claims 5180, which is adoons)
- canonical .page-right spacing is 0.65rem/0.5rem, not the documented 1rem/0.75rem
- dark-mode filter-header values + collapsed-by-default sidebar
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to hiveops-incident#253 (QDS_ADMIN removed from all 5 backends'
SecurityConfig/InstitutionContext, and from the hiveops-auth Role enum so no
JWT can carry it). The guide KB still quoted QDS_ADMIN in role/allow-list
descriptions, which no longer matched the code.
Removed QDS_ADMIN from 26 references across 23 content files; surviving roles
(USER, CUSTOMER, ADMIN, MSP_ADMIN, BCOS_ADMIN, AGENT, SYSTEM) left unchanged.
No frontmatter touched -> zero module-coverage impact. Doc-accuracy only.
Closes#10
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reword journal Overview to drop the 'legacy SmartJournal dependency' mention
and remove smartjournal from the coverage-reconcile deprecated skip-list.
qds-monitor stays in the skip-list (deprecated, intentionally undocumented).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
QDS_ADMIN was removed from the hiveops-auth Role enum (972a28a); no JWT can carry
ROLE_QDS_ADMIN. Strip the stale SecurityConfig hasAnyRole entry. Zero runtime change.
Refs hiveiq-src/hiveops-incident#253
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
platform.nginx-routing: replace the stale 'flip map + internal callers in sync' guidance with the
new pattern — internal callers point at hiveiq-nginx:<port> and follow the 00-backends.conf map
(10-internal-backends.conf). Document the targeted-flip gotcha (global sed corrupts the derived
active_color table; bluegreen-deploy.sh has this bug) and 'never recreate the active color'.
platform.deployment: add a prod blue/green cutover section + the 2026-07-03 incident lesson.
Refs hiveiq-ops/hiveiq-devops#27, hiveiq-ops/hiveiq-openmetal-prod#20.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the MSP_MANAGED/CUSTOMER_MANAGED incident ownership mode across the
Claude/Internal/Architect/Testing/Customer tabs: the write matrix, the
/institution-keys/{key}/incident-mode endpoint + MSP toggle, controller-layer
assertWritable enforcement, IncidentDTO.institution, V90 migration, and the
known UI follow-up (customer nav still hidden).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a Milestones section to processes.gitea-issues: releases are tracked by per-repo
YYYY-MM milestones (not labels), Next = milestone with no due date, Backlog = no
milestone. Note the retired Release/* labels, cross-repo rollup via issues/search, and
assignment API. Cross-link from processes.release for release scoping.
SGs are the firewall of record — public surface 80/443, SSH/Loki restricted to
internal+admin. All VMs key-only (passwordauthentication no) -> brute-force
mitigated, fail2ban optional. Dead SSH-ingress SG (22->world, unattached) to remove.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The redirect (>> /var/log/x.log) fails before the command runs when the target
is root-owned/absent; job dies with zero output. Caused backups #18/#19 + IDS
#22 outages. Diagnose via redirect-target owner; fix with chown + logrotate create.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
devops.security: #16 remediated (freshclam + clamscan, no resident clamd); 5/6
VMs done incl CDE; proxy EXCLUDED (957MB, install OOM d it -> hard reboot).
platform.gotchas: check VM RAM before installing memory-heavy tools.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First pass had a partial view. Backups are 3 layers (Swift dumps + encrypted
off-cluster borg to DLX + Cinder snapshots), not just pg_dumpall — #15 (DR)
resolved, #13/#14 re-scoped. Security has iptables auto-block (security-monitor)
+ AIDE FIM — #17 re-scoped; ClamAV (#16) remains the one real gap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
johannes
hiveiq