docs(technical.agent): document installCheck install-time network+credential check
New gotcha section: a fresh agent install can run fine yet never appear in the fleet when outbound HTTPS is blocked (SSL-inspection firewall resetting TLS to api.bcos.cloud) or credentials in hiveops.properties are wrong. installCheck (installCheck.cmd curl-based + installCheck.ps1 for curl-less ATMs, auto-selected by install.cmd) catches both at install time and can be re-run. Records the La Capitol 2026-07-22 case and the ps1 real-hardware gotchas (pure ASCII, explicit TLS 1.2, HttpWebRequest not raw sockets, CRLF). Session: 1 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -8,6 +8,48 @@ audience: dev
|
||||
|
||||
# Agent gotchas & diagnostics
|
||||
|
||||
## New install runs fine but never appears in the fleet — run installCheck
|
||||
|
||||
The agent can install cleanly, start, and log happily while the device **never shows up in the
|
||||
fleet**, because registration/heartbeat depends on outbound HTTPS to the backend that the ATM's
|
||||
network may silently block. Two failure classes cause it, and they look identical from the ATM:
|
||||
|
||||
1. **Egress blocked** — most often an **SSL-inspection firewall** on the site's network that
|
||||
intercepts and **resets the TLS connection** to `api.bcos.cloud:443`. The agent's HTTPS calls
|
||||
fail; nothing reaches nginx, so the device is simply absent. (La Capitol, 2026-07-22: 8 ATMs
|
||||
invisible — `XB7357` had 5951 nginx hits, `XB7358` had 0. Fix was on their side: bypass SSL
|
||||
inspection for `api.bcos.cloud:443`; the agents then registered on their own, no reinstall.)
|
||||
2. **Bad credentials** — a wrong/blank `agent.institution.key` or `agent.auth.token` in
|
||||
`hiveops.properties`. The token is validated by the agent-proxy `AgentAuthFilter`; a bad one is
|
||||
a 401 the installer never sees.
|
||||
|
||||
**`installCheck` catches both at install time.** `install.cmd` copies it onto the ATM and runs it
|
||||
automatically after install (and it can be re-run any time); a FAIL prints the exact cause and fix
|
||||
and warns that the device will not appear until the network is fixed. Two implementations ship in
|
||||
the Windows dist (`deployment/windows/` in `hiveops-agent`):
|
||||
|
||||
- **`installCheck.cmd`** — curl-based (needs `curl.exe`: Win10 1803+ / Server 2019+).
|
||||
- **`installCheck.ps1`** — PowerShell / .NET, for **older ATMs without curl** (pre-1803 / Server 2016).
|
||||
|
||||
`install.cmd` picks automatically: `where curl` → `.cmd`, else
|
||||
`powershell -ExecutionPolicy Bypass -File installCheck.ps1`. Run either by hand the same way after a
|
||||
network fix.
|
||||
|
||||
**What it checks:** (1) agent API reachable — an authenticated GET task-poll + PUT heartbeat;
|
||||
(2) CDN reachable; (3) credentials — `institution.key` / `auth.token` / `atm.id` present in
|
||||
`hiveops.properties`, plus an **authenticated token probe** (401 → token rejected, 2xx → accepted).
|
||||
Any HTTP response (even 401) proves the network path is open — a **connection reset / send failure
|
||||
during TLS** is the SSL-inspection signature and is called out as such.
|
||||
|
||||
**ps1 gotchas (hard-won on real hardware, AT000999 + XB7358):**
|
||||
- **Pure ASCII only.** PowerShell 5.1 reads scripts as ANSI, so any non-ASCII char (em-dash, smart
|
||||
quote) corrupts parsing and functions silently fail to load. Keep the file 7-bit ASCII.
|
||||
- **Explicit TLS 1.2** (`ServicePointManager.SecurityProtocol`) — old SChannel otherwise throws
|
||||
"A call to SSPI failed".
|
||||
- **Use `HttpWebRequest`, not raw `TcpClient`/`SslStream`.** ATMs reach the backend through a system
|
||||
proxy (like the agent does); a raw socket bypasses the proxy and false-fails while HTTP succeeds.
|
||||
- **CRLF line endings** (`.gitattributes` `*.ps1 eol=crlf`) so it runs correctly once on Windows.
|
||||
|
||||
## Windows JAR self-update needs an external updater process
|
||||
|
||||
Windows holds a file lock on any JAR loaded by a running JVM, so a self-update **cannot** copy over
|
||||
|
||||
Reference in New Issue
Block a user