#Honcho unreachable causes agent initialization to hang

1 messages · Page 1 of 1 (latest)

pearl swift Jun 2, 2026, 1:08 AM

This evening honcho experienced an outage. On my end the behavior experienced was hermes TUI hanging on initializing agent. Initial analysis is that calls to Honcho are synchronous, and 5x30sec timeout results in a hang of 150sec.

Logs:
Report https://paste.rs/HM1Sw
agent.log https://paste.rs/oH46B
gateway.log https://paste.rs/njnQi

Analysis from my hermes:

Honcho Outage Analysis — 2026-06-01

Incident window: ~20:30–20:41 (init blocked); recovery ~20:49
Prepared by: Hermes Agent (Ted)
Date: 2026-06-01

Executive Summary

Honcho (cloud, via HONCHO_API_KEY) became unreachable around 20:19. Existing sessions started seeing 30s timeouts on sync and dialectic calls. The gateway restarted at 20:29. After that, new agent inits hung for ~11 minutes before completing. Once Honcho recovered (~20:49), init returned to normal (~2s).

This was not a hard crash. Hermes blocked on synchronous Honcho HTTP during agent initialization. Each call waited the full 30s timeout before continuing. Multiple sequential calls stacked into multi-minute delays.

What Happened

Phase	Time	Behavior
Healthy baseline	18:59, 19:24	Honcho init → session retrieved in ~1s
Degradation	19:18–20:28	Partial outage ("no route to host"), then repeated 30s timeouts
Gateway restart	20:29:00	Gateway exits code 1
Blocked init #1	20:30:20–20:34:57	Sequential 30s timeouts on add_peers, context, MEMORY.md upload
Blocked init #2	20:35:06–20:41:14	Overlapping retry; USER.md timeout; finally "activated"
Recovery	20:49:01+	New init completes normally in ~2s

Log Timeline

Sources: ~/.hermes/logs/agent.log, ~/.hermes/logs/errors.log

Healthy baseline (earlier that evening)

18:59:00 — Honcho init → session retrieved in ~1s
19:24:18 — same, ~1s

Degradation

19:18:48 — "No route to host" on dialectic/sync (partial outage, init still succeeded)
20:10:09 through 20:28:09 — repeated "Request timed out after 30.0s" on sync/dialectic during active sessions

Gateway restart

20:29:00 — gateway exits code 1

Blocked init #1

20:30:20 — Agent init starts, Honcho client initializing
20:31:53 — add_peers timeout (30s)
20:33:25 — context fetch timeout (30s)
20:34:57 — MEMORY.md upload timeout (30s)
(no "activated" log yet — still in initialize())

Blocked init #2 (overlapping retry)

20:35:06 — second agent init starts while first may still be stuck
20:36:38 — add_peers timeout
20:38:10 — context timeout
20:39:42 — MEMORY.md timeout
20:41:14 — USER.md timeout
20:41:14 — "Memory provider 'honcho' activated" (initialize_all finally returned)

Recovery

20:49:01 — new init attempt
20:51:11 — session retrieved (43 messages), activated normally
20:54:23 — investigation session started

@proper dock for visiblity

compact nacelle Jun 2, 2026, 2:20 AM

That analysis lines up with the current Honcho startup path.

Your logs show the provider blocking during agent init on several sequential Honcho calls:

add_peers timed out after 30s
session context fetch timed out after 30s
MEMORY.md upload timed out after 30s
USER.md upload timed out after 30s

So this is not TUI-specific. The agent is waiting on Honcho memory initialization before it becomes usable, and during a Honcho outage those 30s waits can stack.

There is already an open PR for this exact class of issue: https://github.com/NousResearch/hermes-agent/pull/24847

That PR makes Honcho startup fail open: Hermes resolves the session key, starts Honcho setup in the background, and lets the agent continue without Honcho context until the backend is ready. The PR checks are green.

Short-term workaround if this happens again before that lands:

hermes config set honcho.timeout 5

Then restart Hermes/gateway. That only reduces the stacked wait; it does not remove the synchronous-startup shape. To avoid the init path entirely, temporarily disable the external Honcho provider:

hermes config set memory.provider ""

Then restart Hermes/gateway.

cc @boreal silo or @wheat sparrow (review?)