---
module: incident.incident-tracker
title: Incident Tracker
tab: Internal
order: 20
audience: internal
---

> Ops / support / admin view. What to check when the Incident Tracker board is wrong or empty. Grounded in `hiveops-incident` frontend + backend as of 2026-07-01.

## What the board actually is

The Incident Tracker is a **read-only, client-side aggregation**. The Svelte component (`frontend/src/components/IncidentTracker/IncidentTracker.svelte`) makes three GET calls in parallel and builds every row, pipeline stage, severity, and sort order **in the browser**. There is no dedicated "tracker" endpoint or table — nothing is written back.

Three calls per refresh (auto every **60s**, or manual ↻ button):
1. `GET /api/device-summary` — ATM inventory (id + location), authoritative source for the left column.
2. `GET /api/incidents/paginated?status=OPEN,ASSIGNED,IN_PROGRESS&size=1000&groupId=…` — active incidents.
3. `GET /api/incidents/paginated?status=RESOLVED,CLOSED&size=500&sort=updatedAt,desc&groupId=…` — recently closed.

Rows are joined **in JS** on the ATM business key: `incident.atmName` ↔ `device_summary.atm_id`.

## Roles required

- **Viewing the tracker** needs only a **valid JWT** — the two read endpoints (`/api/device-summary`, `/api/incidents/paginated`) carry **no `@PreAuthorize`**. Access is scoped by `InstitutionContext.resolveScope()`:
  - `ROLE_CUSTOMER` / `ROLE_MSP_ADMIN` → **scoped** to their granted institution keys (list comes from the JWT credentials).
  - `BCOS_ADMIN` / `ADMIN` / `QDS_ADMIN` / `USER` → `null` scope = **sees all institutions**.
- **Acting on an incident** (the "View Incidents" drill-down leads to create/assign/transition) requires `hasAnyRole('MSP_ADMIN','BCOS_ADMIN')` — enforced on `POST /api/incidents`, `POST /api/incident-management/{id}/assign`, `/transition`, `/bulk/update`.

## First things to check when it misbehaves

**Board is empty / "All ATMs clear" but you expect incidents**
- The board only shows ATMs with an **open** incident (`OPEN`/`ASSIGNED`/`IN_PROGRESS`) or, with the toggle on, a `RESOLVED`/`CLOSED` one. If every incident is closed and the toggle is off, you correctly see "All clear."
- Confirm the user's **institution scope** — an MSP_ADMIN only sees incidents for their granted institution keys. A wrong/empty scope in the JWT hides everything.
- Check the group filter: `defaultGroupId` store applies `groupId` to the **incident** queries. A stale/wrong default group can filter out all incidents.

**Rows show a blank location or the ATM ID looks raw**
- The left column comes from `device_summary`. If the join key (`incident.atmName`) has no matching `device_summary.atm_id`, the row falls back to a **stub** (`location: ''`). Root cause is almost always a **stale `device_summary` mirror**.
- `device_summary` is Kafka-fed from **hiveops-devices** via topic `hiveops.device.events` (consumer `DeviceEventConsumer`, group `hiveops-incident-device-sync`). If devices didn't publish, or the consumer lagged/failed, incident's mirror goes stale. Check `DeviceEventConsumer` logs (`Upserted device_summary for …` / `Removed device_summary for deleted device …`).

**Severity or pipeline stage looks wrong**
- Both are computed client-side. Pipeline stage = **lowest** stage index among that ATM's active incidents (`Math.min`); severity = the most urgent among active incidents (`CRITICAL > HIGH > MEDIUM > LOW`). This reflects the underlying `incidents.status` / `incidents.severity` values — fix the incident record, not the board.

**Board silently missing some incidents on a huge fleet**
- Active list is capped at `size=1000`, closed at `size=500`, both **unpaginated in the UI**. An institution with more open incidents than the cap will silently truncate. See uncertainties in the architect notes.

**Nothing loads / "Failed to load data"**
- One of the three GETs failed. Check browser network tab for which. Common causes: NGINX `/api/incident/` route down, expired JWT (401), or CORS. Backend port is **8080** behind NGINX path `/api/incident/`.

## Where to look

- Backend service: `hiveops-incident`, DB `hiveiq_incident` (env `DB_URL`, full JDBC URL — not split host/port/name).
- Tables read: `incidents`, `device_summary`.
- Logs: use **Loki** (services VM) for `hiveops-incident`, don't SSH+cat.
