---
module: transactions.processing-rules
title: Journal Processing Rules
tab: Architect
order: 30
audience: dev
---

How the **Journal Processing Rules** feature is built. The screen is served by the Transactions SPA, but the feature is **owned end-to-end by hiveops-journal**. See [platform.service-ownership].

## Services involved

| Concern | Service | Notes |
|---------|---------|-------|
| UI (`ProcessingRules.svelte`) | hiveops-transactions (frontend, port 5178) | Settings screen; also a sibling `TcrProcessingRules.svelte` for TCR |
| Request routing | Transactions BFF (hiveops-transactions backend, port 8098) / nginx | Proxies `/api/journal/**` verbatim to journal; forwards only `Authorization` + `Content-Type` |
| Rules storage + parser | **hiveops-journal** (upload role, port 8087) | `AdminController` + `JournalRulesService` + `JournalParser` |
| Downstream consumers | devices, recon, analytics | Consume journal Kafka events (not the rules themselves) |

The Transactions backend holds **no** data for this feature — it is a stateless proxy (see [technical.transactions]). All state is the YAML file plus parsed rows in the journal DB.

## Where the rules live

- **File, not a table.** Rules are persisted to `journal-rules.yml` at `JOURNAL_RULES_PATH` (`journal.rules.path`). If unset, the classpath copy `src/main/resources/journal-rules.yml` is read-only (writes throw `IllegalStateException`).
- The file is parsed into a `JournalRules` model, cached in-memory (`cachedRules`), and compiled into per-journal-type `CompiledRules` (regex patterns) on first use. `reloadRules()` clears both caches. See [platform.data-architecture].
- YAML shape: `transaction.transaction_types` is a map of `NAME → { jpr_contains, op_code_matches[] }`; `system_events` is a list of `{ pattern, event_type }`. Per-journal-type overrides live under `journal_types.<TYPE>.system_events` etc.

## Read/write path (the config screen)

```
ProcessingRules.svelte
  GET  /api/journal/admin/rules/structured   → JournalRulesService.getStructuredRules()  → StructuredRulesDTO {version, transactionTypes[], systemEvents[]}
  PUT  /api/journal/admin/rules/structured   → JournalRulesService.saveStructuredRules(dto)
        → replaces transaction_types map + system_events list in JournalRules
        → serializes to YAML (NON_NULL, no doc-start marker), writes JOURNAL_RULES_PATH
        → reloadRules() (cache + compiled-pattern flush) → returns new version
```

Both endpoints are `@PreAuthorize("hasAnyRole('MSP_ADMIN','BCOS_ADMIN')")`. Save is a **full replace**, not a merge — the DTO's ordering becomes the on-disk ordering, which is the parser's match priority (first match wins).

Adjacent admin endpoints on the same controller: raw YAML (`GET/PUT /api/journal/admin/rules`), `POST /reload-rules`, `GET /types`, `POST /reparse-all`, plus the TCR variants under `/tcr-rules*`.

## Runtime: how rules are applied to journals

The rules configured here drive the parse pipeline, which follows the executor → Kafka → record-store pattern (see [platform.kafka]):

1. Agent uploads a journal file (agent-facing, no JWT) → row in `journal_files`.
2. `JournalParsingService.parseJournalFileAsync()` resolves the ATM's institution and publishes a parse request keyed by ATM to a per-institution topic `journal.parse-requests.<INSTITUTION>` (prefix `journal.parse-requests.`; legacy single topic `journal.parse-requests` during transition).
3. A **worker-role** container (`@KafkaListener` in `JournalParseConsumer`) consumes it and runs `JournalParser.parseJournalFile()` using `CompiledRules`:
   - **Transaction Types** — inside `TRANSACTION START…END` blocks, classify via `jpr_contains` / `op_code_matches`; write `journal_transactions` rows.
   - **System Events** — outside transaction blocks, each `system_events` pattern match increments `event_count` on the file. (It is counted only — no per-event record/emit; see the Internal tab's known-gap note.)
4. After parsing, the consumer emits Kafka events:
   - `hiveops.journal.transactions.recorded` (`TOPIC_TRANSACTIONS`) — for `WITHDRAWAL`/`DEPOSIT`/`REVERSAL` only.
   - `journal.replenishments` — replenishment transactions.
   - `journal.file-parsed` — one per file; consumed internally for gap detection and by analytics.

## DB tables (journal DB `hiveiq_journal`)

- **Written by parsing:** `journal_files` (status, counts), `journal_transactions`.
- **Read for config:** none — the config screen is file-backed, no table.
- `journal_events` exists in the **incident** DB and is written only by `CassetteInventoryService` (CASSETTE_INVENTORY backfill), not by the system-event rules on this screen.

## Kafka topics (see [platform.kafka])

| Topic | Direction | Purpose |
|-------|-----------|---------|
| `journal.parse-requests.<INSTITUTION>` | journal upload → journal worker | dispatch parse work per institution |
| `hiveops.journal.transactions.recorded` | journal worker → recon/analytics/devices | financial transactions |
| `journal.replenishments` | journal worker → consumers | replenishment events |
| `journal.file-parsed` | journal worker → gap detection / analytics | file parsed marker |

## Cross-references

- [platform.data-architecture] — file-backed config vs DB-backed transaction store
- [platform.kafka] — per-institution parse dispatch, executor→Kafka→record-store
- [platform.service-ownership] — journal owns rules + parsing; transactions is a proxy
- [technical.transactions] — the stateless BFF that fronts this screen
