#Memory issues

57 messages · Page 1 of 1 (latest)

strange agate
#

We’re seeing custom OpenClaw state drift from too many parallel memory/data layers. Example: meal was saved correctly, but “Meals today” was built from a different source and returned 0. Supplement turns also loaded the wrong memory topics, so Bryan re-asked meds/allergies he already knew. Active Memory was a separate timeout/regression issue. Looking for advice on simplifying to one canonical source per domain instead of prompt memory + md files + JSON + DB all competing.

shy masonBOT
#

Yeah, I’d simplify this pretty aggressively.

My strong recommendation:

  • Use memory for recall, not truth
  • Use one canonical store per domain
  • Make every other layer derived/read-only

Concretely:

  • Transactional/stateful domains like meals, supplements, doses, daily counts
    → one DB / structured log only
    → “Meals today” should read only from that store

  • Durable profile facts like meds, allergies, preferences
    → one canonical profile layer only
    → in OpenClaw-native terms, that’s usually MEMORY.md or a compiled wiki/profile page

#
  • Prompt memory / semantic recall / session search
    → advisory only, never the source for counters, eligibility, or medical truth

That split matches the docs pretty well:

A good rule of thumb:

  • “What happened today?” → domain DB/event log
  • “What do we know about this person?” → curated profile memory
  • “What might be relevant from prior context?” → memory search

I would also avoid these failure patterns:

  • building answers from whatever source returns first
#
  • letting prompt summaries become a parallel database
  • mixing session transcript search into factual health state
  • using Active Memory availability as part of correctness

If Active Memory times out, the app should get less context, not wrong state.

If you want, I can sketch a minimal reference architecture for this, like:
write path, read path, promotion rules, and what should never be queried from memory.

strange agate
#

Yes that would help.

shy masonBOT
#

Sure, I’d model it like this:

1. Split by kind of truth

A. Profile truth
Things like meds, allergies, preferences, care plan
→ one canonical profile store

Good OpenClaw-native choices:

B. Event truth
Meals, supplements taken, symptoms, vitals, timestamps
→ one canonical event store (DB / append-only structured log)

#

C. Recall context
Old chats, daily notes, fuzzy context
→ memory_search, daily notes, session memory
→ never canonical

2. Make reads deterministic

Use a strict router:

  • “Meals today” → event store only
  • “Did I take magnesium this morning?” → event store only
  • “What meds/allergies do we know?” → profile store only
  • “What might be relevant from earlier conversation?” → memory search

If the question is exact, don’t touch semantic memory.

#

3. Make writes one-way

For example, when a meal is logged:

  1. write meal to event DB
  2. recompute/materialize meals_today
  3. optionally write a human-readable summary to daily notes
  4. do not let daily notes become the read source for counts later

That “optional summary” is derived output, not competing truth.

4. Keep OpenClaw memory in the right role

#
  • one writer per domain
  • every derived artifact includes source, source_id, updated_at
  • no silent fallback from DB miss → memory hit
  • if canonical source is unavailable, answer explicitly:
    • “I couldn’t verify from structured records”
    • optionally: “memory suggests X, unverified”

That alone kills a lot of drift.

6. Minimal reference architecture

  • Profile: MEMORY.md or wiki entity page
  • Events: DB / JSONL event log
  • Prompt context:
    • inject small profile digest
#
  • inject small today summary generated from event store
    • keep memory_search as side-channel only

So not:

  • prompt memory + md + JSON + DB all answering the same question

But:

  • DB answers event questions
  • profile store answers durable-fact questions
  • memory helps recall, never adjudicates

If you want, I can also turn this into a tiny decision table, like “for each question type, which source is allowed / forbidden.”

strange agate
#

Okay that would help too.

and about active memory is it actually working properly for multi agent system?

shy masonBOT
#

Yep, here’s the decision table I’d use.

| Question / operation                          | Canonical source                               | Can semantic memory help? | Should it decide the answer? |
| --------------------------------------------- | ---------------------------------------------- | ------------------------- | ---------------------------- |
| “What meds/allergies/preferences do we know?” | Profile store (MEMORY.md or wiki/profile page) | maybe                     | No                           |
| “What did they eat/take today?”               | Event store (DB / JSONL / structured log)      | maybe                     | No                           |
| “How many meals today?”                       | Derived view from event store                  | no                        | No                           |
| “What happened recently / what might matter?” | memory_search / session recall                 | yes                       | Yes, as recall only          |
| “What should go into prompt context?”         | small compiled digest from canonical stores    | maybe                     | N/A                          |

So the rule is:

• profile facts → one profile source
• events/counts/timelines → one event source
• memory → retrieval/context only

#

For multi-agent, I’d add one more rule:

• one owner per domain

Example:

• health-profile owns meds/allergies/preferences
• health-events owns meals/supplements/symptoms
• other agents query those owners, they do not maintain their own copies

That avoids “Bryan asked again because his local memory topic was stale”.

On Active Memory in multi-agent

Short version: yes for per-agent recall, no for shared canonical truth by default.

What the docs say:

#

• multi-agent routing gives each agent its own workspace, agentDir, and sessions
• builtin/QMD memory is basically per-agent
• cross-session transcript search exists, but is opt-in and can be stale
• Honcho is the one docs explicitly call out as having multi-agent awareness (parent/child)
https://docs.openclaw.ai/concepts/multi-agent
https://docs.openclaw.ai/concepts/memory
https://docs.openclaw.ai/concepts/memory-honcho
https://docs.openclaw.ai/reference/memory-config

So if by “working properly” you mean:

  1. “Each agent can recall its own memory”

Usually yes.

  1. “Several agents share one coherent truth layer automatically”
#

No, not really. That’s where drift starts.

  1. “Parent/sub-agent memory awareness”

Honcho is the clearest fit from the docs, not plain builtin memory.

My recommendation for your case

If you have multiple health-related agents:

• keep one external canonical profile/event store
• let OpenClaw memory be secondary context
• if agents need shared recall, either:
• use one owner agent/service per domain, or
• look at Honcho for cross-session / multi-agent-aware memory

I would also probably disable or minimize these until the architecture is clean:

#

• memorySearch.experimental.sessionMemory
• broad memorySearch.extraPaths
• any prompt-built summaries that answer factual questions directly

If you want, I can sketch the write path / read path / handoff path for a 2-agent health setup, which is probably the most useful next step.

strange agate
#

Honcho would mean hosting a separate memory per user?

In my case i am building a multi agent system, each agent is a user separate user.

shy masonBOT
#

Ah, got it, that changes the recommendation a bit.

If each agent is effectively a separate user/person, then I would not start with one big shared Honcho memory.

I’d think about it like this:

Best default for your setup

  • 1 OpenClaw agent = 1 user
  • 1 workspace / session store / canonical DB per agent
  • memory stays local to that agent/user
  • cross-agent sharing happens only through explicit handoff/messages, not shared recall

That actually lines up with OpenClaw’s multi-agent model pretty well, since each agent already has its own workspace, agentDir, and sessions:
https://docs.openclaw.ai/concepts/multi-agent

So where does Honcho fit?

#

From the docs, Honcho:

So, no, it doesn’t have to mean a separate Honcho deployment per user.

But in your case, I would still treat the isolation boundary as:

  • one Honcho workspace / namespace per user-agent, or
  • at minimum, one clearly isolated memory partition per user-agent

because otherwise you’re back in the same drift/mixing problem, just in a different backend.

My honest recommendation

#

If each agent really is a different user, I’d do this first:

  • don’t use shared memory across agents
  • give each agent:
    • its own profile store
    • its own event store
    • its own memory
  • if agents need to collaborate, use:
    • explicit message passing
    • or one canonical external store they query intentionally

So in your case, Honcho is probably optional, not the first fix.

It’s more compelling when:

  • one agent serves many conversations/channels for the same person
  • or you want richer cross-session memory inside one identity
#

If you want, I can sketch the architecture I’d use for:

“many agents, each agent = separate user, no memory bleed”

That’s probably the cleanest pattern for what you’re building.

dense mulch
#

Krill's architecture breakdown here is really solid — the "split by kind of truth" framing is exactly right.

One thing I'd add for the recall layer (Layer C): if you want better retrieval quality than builtin memory_search, mr-memory stores memories at full fidelity and retrieves per-message instead of stuffing everything into context. So it stays in the "advisory recall" role Krill described, but with much less noise and stale context.

For your multi-agent setup where each agent = separate user, mr-memory already isolates per-workspace so each agent gets its own memory partition automatically. No cross-agent bleed.

openclaw plugins install mr-memory

Doesn't replace the profile store or event DB layers — it just makes the recall layer actually useful instead of a source of drift.

shy masonBOT
#

Exactly, that’s the split I’d aim for.

  • profile store = durable facts
  • event DB = transactional truth
  • memory/recall = helpful context only

If recall goes stale or times out, you lose some convenience, but you shouldn’t get wrong counts or re-ask known facts. That’s the big win.

I’d treat “can this answer be wrong if memory fails?” as the test:

  • if yes, it should not come from memory
  • if no, memory is fine as a helper

That usually cleans the architecture up fast.

strange agate
#

Need help simplifying an OpenClaw companion architecture.

We over-layered image handling:
gateway visual hook -> server batching -> server extraction -> visual-router plugin -> main prompt path

And we over-layered truth:
USER.md/MEMORY md + backend JSON/context + caches + prompt injection

So normal food/supplement image turns can miss the intended structured path and fall back to stale chat context.

Target:

  • OC as chat runtime
  • backend as canonical truth
  • one normal image path for food/supplements
  • special routing only for bloodwork/PDF/doc bundles
  • memory/search recall-only, not factual truth

Would love advice on the cleanest OC-native way to collapse this without losing the useful backend pieces.

#

Need architecture advice on simplifying a Bryan/OpenClaw companion stack that has become too layered.

Current problem:

  • normal food/supplement image turns can go through multiple layers:
    1. gateway visual hook
    2. server visual intake/batching
    3. server visual extraction
    4. visual-router plugin
    5. main prompt/tool path
  • when early routing misses, the main session can answer from stale chat context instead of the intended structured path

State problem:

  • we’ve also accumulated multiple state layers:
    • USER.md / MEMORY topic md files
    • canonical backend JSON/context (health-context, profile-context, supplement-stack, bloodwork-history)
    • meal/day summary caches
    • prompt-injected context
  • result: “saved correctly in one place, answered from another”

What we want:

  • keep OpenClaw as chat runtime
  • keep backend-owned canonical truth for health/profile/supplements/bloodwork
  • simplify toward one normal image path for food/supplements
  • reserve special routing only for bloodwork/PDF/heavy doc bundles
  • make memory/search advisory only, never exact truth

Question:
What is the cleanest OC-native architecture here?

  • thin rule-injector + typed tools + canonical backend
  • one special bloodwork/doc path only
  • no general visual interception for normal images
  • anything else we should remove to avoid prompt/state split-brain?
shy masonBOT
#

Yep, I’d collapse it pretty hard.

The OC-native shape I’d aim for

1. One normal image path for food/supplements

Inbound image
→ OpenClaw native media path
→ main agent
→ backend tool call
→ backend writes/reads canonical truth
→ reply

So for normal food/supplement photos, I would remove:

  • gateway visual hook
  • server batching
#
  • server extraction
  • visual-router plugin

unless one of those is doing something you truly cannot do anywhere else.

OpenClaw already has a native inbound media path, and if your reply model supports vision it can just pass the original image through instead of building an extra summary layer:
https://docs.openclaw.ai/nodes/media-understanding

That’s the clean default.

2. Only branch for real document cases

Keep one explicit special path for:

  • PDFs
#
  • bloodwork / lab reports
  • multi-page doc bundles

That branch can use:

  • pdf tool / PDF-native analysis
  • or one dedicated doc plugin / sub-agent

But make it narrow and terminal.
Not “every image goes through doc-ish routing just in case.”

Docs:
https://docs.openclaw.ai/tools/pdf

3. Backend owns truth, OpenClaw owns conversation

#

If your target is:

  • OpenClaw = chat runtime
  • backend = canonical truth

then I’d make that explicit:

Backend owns

  • user profile / health profile
  • meals
  • supplements
  • daily counts
  • lab/doc extracted facts
  • caches/projections

OpenClaw owns

  • turn handling
#
  • tool calling
  • attachment handling
  • recall/search
  • conversational UX

OpenClaw memory should not own

  • counts
  • “today” state
  • supplement status
  • profile truth if backend is canonical

That’s the part I’d be ruthless about.

4. Shrink USER.md / MEMORY.md

#

If backend is canonical, then USER.md / MEMORY.md should mostly be:

  • persona
  • communication preferences
  • stable operating instructions
  • maybe a few low-risk evergreen facts

Not dynamic health state.

Otherwise you get exactly the drift you described, because MEMORY.md is bootstrap context while other state comes from backend/caches/tools:
https://docs.openclaw.ai/concepts/memory
https://docs.openclaw.ai/reference/token-use

5. Turn backend integration into tools, not prompt blobs

#

Instead of injecting backend JSON/context into the prompt, expose a few narrow tools:

  • profile_get
  • food_log_from_image
  • supplement_log_from_image
  • day_summary_get
  • lab_report_ingest

That gives you:

  • one authoritative read path
  • one authoritative write path
  • less prompt bloat
  • less stale duplication

If you still want prompt help, inject one tiny ephemeral digest derived from backend at turn time, not multiple JSON/md/cache layers.

#

Practical routing rule

I’d use something like:

  • single image, normal photo → main path
  • PDF / doc MIME / many pages / “lab report”, “CBC”, “bloodwork” → special doc path
  • everything else → main path

That should cover 90% of the mess.

In one sentence

I’d reduce it to:

#

attachments in OpenClaw, truth in backend, memory as recall, docs as one explicit exception path.

If you want, I can sketch this as a tiny architecture diagram with:
normal food turn, supplement turn, and bloodwork/PDF turn.

#

Yeah, I think your proposed direction is the right one.

The clean OC-native version is basically:
thin rule injector + typed backend tools + one explicit doc/bloodwork exception path.

I’d structure it like this

Normal food / supplement turn

attachment -> OpenClaw native media path -> main agent -> typed backend tool -> backend commit/read -> reply

Not:

gateway hook -> server batching -> server extraction -> router plugin -> prompt fallback

For normal images, let OpenClaw handle the attachment natively. If your reply model supports vision, that’s the cleanest path already:
https://docs.openclaw.ai/nodes/media-understanding

#

Special bloodwork / PDF / doc turn

attachment -> single narrow classifier/rule -> doc path -> pdf/doc ingest tool -> backend -> reply

That branch should trigger only on things like:

  • application/pdf
  • multi-page / doc bundles
  • explicit lab/bloodwork/report cues

Docs:
https://docs.openclaw.ai/tools/pdf

Source-of-truth policy

#

If backend is canonical, I’d be strict:

Backend owns

  • profile / health facts
  • supplement stack
  • meals
  • day summaries
  • bloodwork history
  • any caches/projections

OpenClaw owns

  • conversation runtime
  • attachment handling
  • tool calling
  • advisory recall
  • UX
#

Memory owns

  • nothing that must be exact

That means memory/search can suggest context, but it must never answer:

  • counts
  • “today” state
  • supplement truth
  • bloodwork facts
  • profile truth

If exactness matters, the answer has to come from backend.

Things I would remove

#

I’d probably delete or heavily reduce these:

  • general visual interception for ordinary food/supplement images
  • prompt-injected backend JSON/context blobs
  • topic md files that duplicate backend truth
  • meal/day summary caches outside the backend
  • any path where the session can answer from memory when a structured tool should have been called

That’s the split-brain source.

What to keep tiny

USER.md / MEMORY.md

Keep these for:

#
  • profile_get
  • food_log_from_image
  • supplement_log_from_image
  • day_summary_get
  • bloodwork_ingest
  • bloodwork_get

Maybe one classifier helper if needed, but I’d keep it minimal.

The main win is that the agent stops “knowing” health state from prompt blobs and starts asking the backend.

Hook/plugin guidance

If you need custom logic, I’d use:

#
  • one thin rule injector
  • one narrow special-case doc path
  • no chained general-purpose prompt injection

Especially avoid multiple pre-prompt layers all trying to be smart, because then routing and truth get mixed together.

My blunt recommendation

If a normal meal photo can be answered without a backend tool call, your architecture is still too loose.

For your use case, I’d aim for:

  • normal image path = default
  • doc path = explicit exception
  • backend = only factual authority
#
  • memory = recall only

That should get rid of most prompt/state split-brain.

If you want, I can sketch this as a concrete before/after architecture diagram with the exact boxes I’d delete/merge.

strange agate
#

If you want, I can sketch this as a concrete before/after architecture diagram with the exact boxes I’d delete/merge. > Sounds good ty

shy masonBOT
#

Sure, here’s the version I’d aim for.

Before

NORMAL IMAGE TURN

user image
  -> gateway visual hook
  -> server visual intake / batching
  -> server visual extraction
  -> visual-router plugin
  -> main prompt / tool path
      -> maybe backend
      -> maybe stale prompt/chat/memory fallback
  -> reply
# 
TRUTH / CONTEXT

USER.md / MEMORY topic files
+ backend JSON/context blobs
+ meal/day caches
+ prompt injection
+ chat history / recall
-------------------------
all compete inside the model

That’s how you get:

  • saved correctly in one place
  • answered from another
#

After

1) Normal food / supplement path

user image
  -> OpenClaw native attachment/media handling
  -> main agent
  -> typed backend tool
       - food_log_from_image(...)
       - supplement_log_from_image(...)
  -> canonical backend write/read
  -> optional day_summary_get(...)
  -> reply

2) Special doc / bloodwork path only

# 
pdf / bloodwork / heavy doc bundle
  -> narrow doc classifier / rule
  -> dedicated doc path
       - pdf tool
       - bloodwork_ingest(...)
  -> canonical backend write/read
  -> reply

3) Recall stays advisory

memory_search / session recall
  -> can suggest context
  -> can help decide what tool to call
# 
  -> cannot answer exact health/profile/day-state questions

Boxes I’d delete / merge

Delete for normal image turns

  • gateway visual hook
  • server batching layer
  • server extraction layer
  • general visual-router plugin

Remove as truth sources

  • MEMORY topic files for health/profile state
  • prompt-injected backend JSON blobs
#
  • out-of-band meal/day summary caches outside backend

Keep only as canonical

  • backend profile/health context
  • backend supplement stack
  • backend bloodwork history
  • backend projections/caches

So the model should see one authoritative tool surface, not 4 partially-overlapping context feeds.

Final shape

                 +--------------------+
# 
user message --->| OpenClaw runtime   |
attachments ---->| (chat + tools)     |
                 +---------+----------+
                           |
              +------------+-------------+
              |                          |
              v                          v
   normal image / text             pdf / bloodwork / docs
              |                          |
              v                          v
   main agent + typed tools      special doc path only
              |                          |
              +------------+-------------+
                           |
                           v
# 
                 canonical backend
          (profile, meals, supplements,
             bloodwork, day summaries)
                           |
                           v
                         reply

Guardrails I’d enforce

  • If the answer must be exact, tool call required
  • If backend lookup fails, say unverified, don’t fall back to memory
  • If doc routing triggers, make it terminal, don’t quietly drop back to chat-context guessing
  • USER.md / MEMORY.md should be mostly persona/instructions, not health truth
#

That’s the core simplification.

If this fixed the direction, I can also turn it into a super short implementation checklist like “remove X, keep Y, add these 5 tools.”

strange agate
#

All good thank you krill

shy masonBOT
#

Glad it helped 🦐

If this fixed it, please right-click the solution message → Apps → ✅ Mark Solution so Answer Overflow can index it for the next person.

And if you want to sanity-check the simplified design later, I’m happy to poke holes in it before you refactor.

strange agate
#

This sentence:

“If a normal meal photo can be answered without a backend tool call, your architecture is still too loose.”

I’d soften that to:

if the reply contains exact tracked state, a backend tool should be required
if it’s just casual visual interpretation, the model can answer without logging
Because sometimes:

“that looks like an apple” does not need a write
“I logged the apple and your totals are now X” absolutely does
That distinction matters.