---
module: transactions.reparse
title: Journal Reparse
tab: Internal
order: 20
audience: internal
---

Ops / support / admin view of **Journal Reparse** (Transactions SPA → Settings → *Journal Reparse*, `JournalReparse.svelte`). The screen has two tabs — **ATM Journals** and **TCR Journals** — that show file parse status and trigger a bulk re-parse. The data does **not** come from the transactions service; transactions is a stateless proxy that forwards to **hiveops-journal**, which owns the files, the parsing, and the reparse action.

## Who can do what

| Action | UI | Real endpoint (journal svc) | Role enforced |
|--------|----|-----------------------------|---------------|
| View parse status / file list | Refresh, tab switch, ATM filter | `GET /api/journal/files` | `isAuthenticated()` — any logged-in user |
| Trigger bulk reparse | **Reparse All** button | `POST /api/journal/admin/reparse-all` | **`hasAnyRole('MSP_ADMIN','BCOS_ADMIN')`** |

So a plain user can *see* status but only **MSP_ADMIN / BCOS_ADMIN** can trigger a reparse. If a non-admin clicks Reparse All they get a 403 from the journal service (surfaced as the red error alert). The transactions proxy in front is deliberately loose (`hasAnyRole(USER,CUSTOMER,ADMIN,MSP_ADMIN,QDS_ADMIN,BCOS_ADMIN)` on `/api/**`); the real authorization is enforced downstream by journal's `AdminController`.

## What "Reparse All" actually does

For each matched `journal_files` row (`AdminController.reparseAll`):
1. Deletes all rows in `journal_transactions` for that file (`deleteByJournalFile`).
2. Resets the file: `parse_status = PENDING`, `transaction_count = null`, `parse_error = null`, `last_parsed_size = null` (the null size forces a re-parse even if the file bytes are unchanged).
3. Publishes an async parse request to Kafka; increments the `queued` counter.

Returns `{ message, queued, total }` → UI shows *"Queued X of Y files"*. Files then move **PENDING → PARSING → PARSED** (or **ERROR**) on their own as the per-institution worker picks them up. Nothing is parsed inline in the request.

**Scope of the buttons:**
- **ATM tab** → `journalType=ATM` → matches all files *except* `NH_TCR` (`findByJournalTypeNot("NH_TCR")`).
- **TCR tab** → `journalType=NH_TCR` → matches only TCR files.
- **ATM ID blank** → every ATM of that type (can be thousands of files / several minutes).
- **ATM ID set** → uppercased and matched exactly (`findByAtmIdAndJournalType…`).

## First things to check when it misbehaves

1. **Files stuck in PENDING and never moving to PARSED.** This is the #1 failure mode. Parsing is done by the **worker** containers (`JOURNAL_ROLE=worker`) consuming per-institution Kafka topics. If the worker for that institution isn't running, its files sit PENDING forever. Check the worker container for the institution is up.
2. **PENDING right after a deploy.** The upload service only *produces* Kafka messages; a restart can strand in-flight files. Run the reconcile endpoint (re-queues all PENDING/PARSING):
   ```
   POST /api/journal/admin/reconcile   (MSP_ADMIN/BCOS_ADMIN)
   ```
   It also runs automatically on startup and every 30 min (`recoverStaleParsing` re-queues PARSING > 30 min; a separate sweep re-queues PENDING > 30 min). So most stuck files self-heal within ~30 min — if they don't, the worker is down or the institution has no worker.
3. **"Unassigned" institution.** If a file's ATM has no institution mapping (`jf.getInstitution()` null and `AtmInstitutionSyncService` can't resolve it), it routes to the `journal.parse-requests.UNASSIGNED` topic and is parsed by `hiveiq-journal-worker-unassigned`. If that worker is down, those files hang. Fix the ATM→institution mapping in devices/incident and re-run reparse.
4. **ERROR status with a message.** The **Error** column shows `parse_error`. Usually a rules problem (bad/renamed journal format) — check the rules for that vendor type before re-running. TCR files use a separate ruleset (`tcr-rules.yml`).
5. **Kafka down.** `parseJournalFileAsync` swallows send failures (logs a warn) and the file is already reset to PENDING with its transactions deleted — the 30-min PENDING sweep or a manual `/reconcile` will re-queue once the broker is back.

## Common failure modes → fix

| Symptom | Likely cause | Fix |
|---------|-------------|-----|
| Reparse All → 403 / red error | User is not MSP_ADMIN or BCOS_ADMIN | Use an admin account |
| Files parsed=PENDING, never advance | Institution worker container down / not deployed | Bring up `hiveiq-journal-worker-{KEY}`; then `/reconcile` |
| Files PENDING after deploy | Kafka messages stranded on restart | `POST /api/journal/admin/reconcile` (or wait ≤30 min) |
| Transactions dropped to 0 and stay 0 | Reparse deleted rows but no worker re-parsed | Same as above — reparse **deletes first**, so a missing worker leaves data empty until it runs |
| Status pills/table empty | Wrong tab (ATM vs TCR) or ATM filter typo | Clear filter, confirm tab matches file type |
| Single file needs re-parse, not the whole fleet | — | `POST /api/journal/files/{id}/reparse` (per-file, `isAuthenticated()`); diagnostics-only sync: `POST /api/journal/admin/parse-sync?fileId=X` |

## Good to know

- **Reparse is destructive-then-rebuild:** transactions are deleted *before* re-parsing. During a fleet-wide reparse, affected transactions are temporarily gone from the Transactions browser until workers finish.
- **Auto-refresh** in the UI polls `GET /api/journal/files` every 15s (client-side only; toggle persisted in `localStorage`). It does not drive parsing.
- Downstream of a successful parse, journal emits `journal.file-parsed`, which re-triggers **gap detection** and **cassette inventory** — a large reparse also regenerates those.
