Files
hiveops-guide/backend/src/main/resources/content/devops/security/overview.md
T
johannes 80e4bbe415 docs(guide): ClamAV remediation status + VM-sizing OOM gotcha
devops.security: #16 remediated (freshclam + clamscan, no resident clamd); 5/6
VMs done incl CDE; proxy EXCLUDED (957MB, install OOM d it -> hard reboot).
platform.gotchas: check VM RAM before installing memory-heavy tools.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 10:58:00 -04:00

3.3 KiB

module, title, tab, order, audience
module title tab order audience
devops.security Host Security & Anti-Virus Overview 20 dev

DRAFT · internal. Rewritten 2026-07-02 after auditing the prod VMs. The first pass reported a bare posture ("no firewall/fail2ban") — that was mis-scoped; there IS custom IDS + FIM. Scripts: hiveops-devops/scripts/ops/security/.

Controls actually in place

Control How Hosts PCI
Intrusion detection + auto-block security-monitor.sh (every 5 min): scans nginx logs for attack patterns (.env, .git, id_rsa, /actuator/env, /actuator/heapdump, webshells…) → iptables -I INPUT -s <ip> -j DROP; Slack alerts; daily/weekly reports .250, .252 Req 1/10/11 (compensating)
File-integrity monitoring aide-check.sh (daily 03:15): AIDE --check → email alerts@bcos.cloud on any change .250, .188 Req 11.5
Firewall iptables (managed via security-monitor blocks) + almost certainly OpenStack security groups at the network layer all Req 1
TLS acme.sh auto-renew .250, .252

So iptables auto-blocking and AIDE FIM are live — the box is not wide open.

Anti-virus (ClamAV) — remediated 2026-07-02 (#16)

Audit 2026-07-02 found AV broken/missing on most hosts (PCI Req 5 gap, incl. the CDE) — someone had disabled clamav-freshclam + clamd on 2026-06-09 (resource reasons). Remediated the same day.

Approach (script: scripts/ops/security/clamav-remediate.sh): freshclam (signature updates, cheap) + a daily 04:30 clamscan of writable/ingress dirs. Do NOT enable the resident clamd daemon — it holds the ~2 GB signature DB in RAM permanently (that's why it was disabled); clamscan loads it only transiently per run.

VM ClamAV Notes
services .250, cdn .252, database .188 (CDE) freshclam + clamscan CDE PCI gap closed
share .254, ollama .7 installed + freshclam + clamscan ample RAM (3.9 / 16 GB)
proxy .253 EXCLUDED 957 MB VM — too small; installing it OOM'd the VM (see gotcha below). Agentless scan if AV ever required.

Daily health check (platform.production-checks §4) flags per-VM state; also reads /var/log/clamav/scan-status.log for detections. Alerting on detection currently uses local mail (unreliable — see #21).

⚠️ Gotcha — check VM RAM before installing heavy tools. bcos-proxy is only 957 MB. Installing ClamAV there (freshclam + a 2 GB clamscan) OOM'd the guest — it hung mid-apt and needed a hard reboot (openstack server reboot --hard) to recover; soft reboot doesn't work on a hung guest. Core services were unaffected. Always free -m a VM before adding memory-heavy agents. Tiny VMs (proxy) → agentless/remote scanning, not on-host clamscan.

Residual (re-scoped from #17)

  • No ufw (iptables is used directly via security-monitor; confirm OpenStack SG rules are the network firewall of record and document them).
  • No fail2ban — but security-monitor covers web attack auto-blocking; SSH brute-force specifically is the residual gap (SSH is key-only, which mitigates). → hiveiq-devops#17 (re-scoped)

See platform.production-checks, devops.backups, platform.topology.