Embedding issues

[placid flare]

Resume note for next thread:

Original issue: you couldn’t see OpenClaw “thinking”. Main takeaway:
/think changes reasoning effort,
/reasoning only shows reasoning if the backend returns it,
/verbose is the better knob for seeing tool/activity traces,
,

Memory/embedding investigation:
~/.openclaw/workspace/memory contains ~7.2k .md files,
OpenClaw indexes Markdown from workspace memory files into ~/.openclaw/memory/main.sqlite,
We verified the DB contained the target file and chunks via sqlite3,
,

Big problem discovered:
old DB expected 4096-dim vectors,
new llama-server embedding endpoint at 127.0.0.1:8081 was at one point returning 2560,
that mismatch explains broken search,
later you said the current model is now returning 4096, which is promising,
,
👍🏻
Click to react
💪🏻
Click to react
✅
Click to react
Add Reaction
Reply
Forward
More
[2:54 PM]Tuesday, April 21, 2026 at 2:54 PM
Failed reindex:
a forced reindex later failed on embedding input size (2071 tokens > 2048),
that temporarily left the DB empty,
you restored the old DB successfully,
,

Current desired setup:
main chat model on 127.0.0.1:8080,
embedding model on 127.0.0.1:8081,
current check from you: embedding endpoint now returns 4096 for the active model,
,

Last recommended verification steps:
openclaw memory status --agent main --deep | rg 'Provider:|Model:|Vector dims:|Indexed:|Dirty:'
openclaw memory search --agent main --query "I also have my card in Trading 212" --max-results 5 --min-score 0 --json
,

Interpretation:,
👍🏻
Click to react
💪🏻
Click to react
✅
Click to react
Add Reaction
Reply
Forward
More
[2:54 PM]Tuesday, April 21, 2026 at 2:54 PM
if search now works, the root cause was the embedding-dimension mismatch
if it still returns [] even with live 4096 and DB 4096, then it likely needs a GitHub bug report,
,

That should be enough to resume cleanly in a new thread.

[oak wraith]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[placid flare]

its 4096 now but even a simple word cant find anything, why=

[sage dirge]

Yep, that’s a solid resume.

The next thing I need is these 3 outputs, in this order:

openclaw --version
openclaw memory status --agent main --deep | rg 'Provider:|Model:|Vector dims:|Indexed:|Dirty:'
openclaw memory search --agent main --query "I also have my card in Trading 212" --max-results 5 --min-score 0 --json

A couple notes:

• Don’t force a reindex yet, since the last one hit the 2048-token embedding limit and briefly emptied the DB.
• If status shows 4096 dims and search starts returning hits, then the root cause was almost certainly the embedding dimension mismatch.
• If status is 4096/healthy and search is still [], I’d treat that as likely bug-report territory.

Relevant docs:

• FAQ: https://docs.openclaw.ai/help/faq
• Memory CLI: https://docs.openclaw.ai/cli/memory
• Memory search: https://docs.openclaw.ai/concepts/memory-search

Paste the outputs and I’ll help interpret them.

[placid flare]

i dont understand how can this be so broken

matias@matias-System-Product-Name:~$ openclaw --version
openclaw memory status --agent main --deep | rg 'Provider:|Model:|Vector dims:|Indexed:|Dirty:'
openclaw memory search --agent main --query "I also have my card in Trading 212" --max-results 5 --min-score 0 --json
OpenClaw 2026.4.15 (041266a)
Provider: openai (requested: openai)
Model: Qwen3-Embedding-4B-Q6_K
Indexed: 7268/7271 files · 20083 chunks
Dirty: no
Vector dims: 4096
{
"results": []
}

[sage dirge]

Most likely, because 4096 only tells you the vector length, not that the vectors live in the same embedding space.

So the two big suspects are:

1. You restored an old DB built with a different embedding model

   • same dims, different model = query vectors and stored vectors are incompatible
   • search looks “empty” even though the DB has chunks

2. FTS is unavailable/off

   • then even simple exact-word matches won’t rescue you

So right now my bet is: same dims, wrong model space, or FTS isn’t actually working.

Please paste these:

openclaw --version
openclaw memory status --agent main --deep
sqlite3 ~/.openclaw/memory/main.sqlite "select model, count(*) from chunks group by model order by count(*) desc;"

If status --deep shows fts unavailable/off, that explains the simple-word failure.

If FTS is ready, but the chunk model in SQLite is not the same model you’re querying with now, then that’s the issue. In that case, the real fix is a clean rebuild with the exact same embedding model, but I would not force it yet until we understand the 2048-token reindex failure.

Docs:

• https://docs.openclaw.ai/reference/memory-config
• https://docs.openclaw.ai/concepts/memory-builtin

[placid flare]

matias@matias-System-Product-Name:~$ openclaw --version
openclaw memory status --agent main --deep
sqlite3 ~/.openclaw/memory/main.sqlite "select model, count() from chunks group by model order by count() desc;"
OpenClaw 2026.4.15 (041266a)

🦞 OpenClaw 2026.4.15 (041266a)
It's not "failing," it's "discovering new ways to configure the same thing wrong."

│
◇
Memory Search (main)
Provider: openai (requested: openai)
Model: Qwen3-Embedding-4B-Q6_K
Sources: memory
Indexed: 7268/7271 files · 20083 chunks
Dirty: no
Store: ~/.openclaw/memory/main.sqlite
Workspace: ~/.openclaw/workspace
Dreaming: 0 3 * * * · limit=10 · minScore=0.8 · minRecallCount=3 · minUniqueQueries=3 · recencyHalfLifeDays=14 · maxAgeDays=30
Embeddings: ready
By source:
memory · 7268/7271 files · 20083 chunks
Vector: ready
Vector dims: 4096
Vector path: ~/.npm-global/lib/node_modules/openclaw/node_modules/sqlite-vec-linux-x64/vec0.so
FTS: ready
Embedding cache: enabled (40012 entries)
Batch: disabled (failures 0/2)
Recall store: 307 entries · 0 promoted · 307 concept-tagged · 0 spaced · scripts=307 latin
Recall path: ~/.openclaw/workspace/memory/.dreams/short-term-recall.json
Recall updated: 2026-04-21T07:36:53.434Z
Dreaming artifacts: diary present · 1 corpus files · ingestion state present
Dream corpus: ~/.openclaw/workspace/memory/.dreams/session-corpus
Dream ingestion: ~/.openclaw/workspace/memory/.dreams/session-ingestion.json
Dream diary: ~/.openclaw/workspace/DREAMS.md

/home/matias/llama.cpp/models/Qwen3-Embedding-8B-Q8_0.gguf|20083
matias@matias-System-Product-Name:~$

[sage dirge]

Yeah, that’s genuinely busted, and it’s probably not you.

Given:

• Indexed: 7268/7271 files · 20083 chunks
• Dirty: no
• Vector dims: 4096
• search still returns []

we’re basically down to two real explanations:

1. The restored DB was embedded with a different model/vector space

   • same 4096 dims, but still incompatible
   • status shows the current configured model, not necessarily the model stored in every chunk

2. FTS / exact-text search is not actually working

   • that would explain why even simple words return nothing

Please run these two:

openclaw memory status --agent main --deep
sqlite3 ~/.openclaw/memory/main.sqlite "select model, count(*) from chunks group by model order by count(*) desc;"

How I’d read it:

• FTS unavailable/off → that explains the exact-word failure
• chunk model != Qwen3-Embedding-4B-Q6_K → restored DB is from a different embedding space
• FTS ready + chunk model matches + still empty → I’d call that a real bug and file an issue with those outputs

If it gets to that last case, file here and include the command output:
https://github.com/openclaw/openclaw/issues

Memory CLI docs:

https://docs.openclaw.ai/cli/memory

Paste those two outputs, I think we’re very close to the actual reason.

Aha. We found the smoking gun.

Your runtime is querying with:

• current model: Qwen3-Embedding-4B-Q6_K

but your DB was built with:

• stored chunk model: /home/matias/llama.cpp/models/Qwen3-Embedding-8B-Q8_0.gguf

for all 20083 chunks.

So yes, this is broken, but now it makes sense:

• same 4096 dims
• different embedding model
• therefore different vector space

• therefore vector search returns garbage / nothing

So the immediate problem is 4B queries against an 8B-built index.

Also, I’d call this an OpenClaw bug or at least a bad edge case, because Dirty: no is misleading here. It should have noticed the model mismatch instead of pretending the index was clean.

What I’d do next

Fast sanity check

Point your embedding endpoint on 127.0.0.1:8081 back to the same 8B model that built the DB, then rerun:

openclaw memory search --agent main --query "I also have my card in Trading 212" --max-results 5 --min-score 0 --json

If that starts working, we’ve confirmed it.

Proper fix

If you want to stay on the 4B embedding model, you need a full rebuild of the index with 4B. But first you need to solve the earlier reindex failure (2071 > 2048). The likely path is to lower agents.defaults.memorySearch.chunking.tokens, then rerun a full reindex.

Docs:

• https://docs.openclaw.ai/reference/memory-config
• https://docs.openclaw.ai/cli/memory

One more important note

FTS: ready means the lexical machinery exists, but it does not rule out restore weirdness.
So if search is still empty even after switching the embed model back to the matching 8B one, then I’d file a bug here:

https://github.com/openclaw/openclaw/issues

If you want, send the bit of config for your memorySearch block / embedding endpoint, and I’ll help you pick the cleanest path back to a working setup.

[placid flare]

matias@matias-System-Product-Name:~$ openclaw --version
openclaw memory status --agent main --deep
sqlite3 ~/.openclaw/memory/main.sqlite "select model, count() from chunks group by model order by count() desc;"
OpenClaw 2026.4.15 (041266a)

🦞 OpenClaw 2026.4.15 (041266a)
It's not "failing," it's "discovering new ways to configure the same thing wrong."

│
◇
Memory Search (main)
Provider: openai (requested: openai)
Model: Qwen3-Embedding-4B-Q6_K
Sources: memory
Indexed: 7268/7271 files · 20083 chunks
Dirty: no
Store: ~/.openclaw/memory/main.sqlite
Workspace: ~/.openclaw/workspace
Dreaming: 0 3 * * * · limit=10 · minScore=0.8 · minRecallCount=3 · minUniqueQueries=3 · recencyHalfLifeDays=14 · maxAgeDays=30
Embeddings: ready
By source:
memory · 7268/7271 files · 20083 chunks
Vector: ready
Vector dims: 4096
Vector path: ~/.npm-global/lib/node_modules/openclaw/node_modules/sqlite-vec-linux-x64/vec0.so
FTS: ready
Embedding cache: enabled (40012 entries)
Batch: disabled (failures 0/2)
Recall store: 307 entries · 0 promoted · 307 concept-tagged · 0 spaced · scripts=307 latin
Recall path: ~/.openclaw/workspace/memory/.dreams/short-term-recall.json
Recall updated: 2026-04-21T07:36:53.434Z
Dreaming artifacts: diary present · 1 corpus files · ingestion state present
Dream corpus: ~/.openclaw/workspace/memory/.dreams/session-corpus
Dream ingestion: ~/.openclaw/workspace/memory/.dreams/session-ingestion.json
Dream diary: ~/.openclaw/workspace/DREAMS.md

/home/matias/llama.cpp/models/Qwen3-Embedding-8B-Q8_0.gguf|20083
matias@matias-System-Product-Name:~$

[sage dirge]

Yep, this confirms it.

Your DB is 8B, your live query model is 4B.
That index is not valid for the current embedder, even though both are 4096-d.

So I’d stop trying to debug search on that DB. The mismatch is the reason.

Your two real options

1. Fastest recovery

Put 127.0.0.1:8081 back on:

/home/matias/llama.cpp/models/Qwen3-Embedding-8B-Q8_0.gguf

Then rerun the same openclaw memory search ... test.

2. Proper fix for your desired setup

Keep the 4B model on 8081, but do a full rebuild of memory with that model.

Because your earlier rebuild failed on the 2048 input limit, I’d first lower:

agents.defaults.memorySearch.chunking.tokens

then run:

openclaw memory index --agent main --force

Docs:

• https://docs.openclaw.ai/reference/memory-config

• https://docs.openclaw.ai/cli/memory

My recommendation

Do #1 first, just as a sanity check.
If search works again with 8B, that fully proves the root cause.

Then decide whether you actually want to stay on 8B, or do the clean 4B rebuild.

Also, yes, this deserves a bug report

Dirty: no should not be shown when the stored chunk model and current embedding model differ.

Suggested issue title:

Memory search reports Dirty: no and returns empty results after restored index uses a different embedding model than current config

Bug tracker:
https://github.com/openclaw/openclaw/issues

If you want, I can help you write the exact memorySearch config for the clean 4B rebuild.

[placid flare]

should we try this with a smaller amount of files before going in all the way

[sage dirge]

Yes, absolutely. That’s the smart move.

I would not go straight back to all 7.2k files yet.
Do a tiny fresh repro first, with no restored DB and only the 4B model you actually want.

I’d test like this

1. Make a small temp corpus, like 3 to 10 markdown files
2. Put one very unique phrase in one file, for example:
   • banana trading212 octopus wallet
3. Use a fresh test workspace / test agent / fresh DB
4. Point memory at only that small set, for example via memorySearch.extraPaths
5. Index with the 4B embedding model only
6. Test:
   • exact phrase search
   • one-word search
   • partial phrase search

Why this is better

• isolates whether 4B + your endpoint + current config actually works
• avoids another expensive full reindex
• gives you a clean bug repro if it still fails
• removes the old 8B DB contamination from the picture

My recommendation

• First: tiny fresh corpus
• Then: maybe 50 to 100 files
• Only then: full corpus

If the tiny corpus already fails, that’s actually great news, because it means you have a clean minimal bug report instead of a giant messy one.

Relevant config docs:
https://docs.openclaw.ai/reference/memory-config

If you want, I can sketch the cleanest minimal test setup next.

[placid flare]

matias@matias-System-Product-Name:~$ openclaw memory status --agent main --deep

🦞 OpenClaw 2026.4.15 (041266a)
It's not "failing," it's "discovering new ways to configure the same thing wrong."

│
◇
Memory Search (main)
Provider: openai (requested: openai)
Model: Qwen3-Embedding-4B-Q6_K
Sources: memory
Indexed: 0/22 files · 0 chunks
Dirty: yes
Store: ~/.openclaw/memory/main.sqlite
Workspace: ~/.openclaw/workspace
Dreaming: 0 3 * * * · limit=10 · minScore=0.8 · minRecallCount=3 · minUniqueQueries=3 · recencyHalfLifeDays=14 · maxAgeDays=30
Embeddings: ready
By source:
memory · 0/22 files · 0 chunks
Vector: ready
Vector path: ~/.npm-global/lib/node_modules/openclaw/node_modules/sqlite-vec-linux-x64/vec0.so
FTS: ready
Embedding cache: enabled (0 entries)
Batch: disabled (failures 0/2)
Recall store: 307 entries · 0 promoted · 307 concept-tagged · 0 spaced · scripts=307 latin
Recall path: ~/.openclaw/workspace/memory/.dreams/short-term-recall.json
Recall updated: 2026-04-21T07:36:53.434Z
Dreaming artifacts: diary present · 1 corpus files · ingestion state present
Dream corpus: ~/.openclaw/workspace/memory/.dreams/session-corpus
Dream ingestion: ~/.openclaw/workspace/memory/.dreams/session-ingestion.json
Dream diary: ~/.openclaw/workspace/DREAMS.md

matias@matias-System-Product-Name:~$

okay how do we index, running embedding 8b q8 now

[sage dirge]

Yep, for this small test corpus, the indexing command is:

openclaw memory index --agent main --force --verbose

Then verify with:

openclaw memory status --agent main --deep

and test search with your known phrase.

One important warning

Your pasted status still says:

Model: Qwen3-Embedding-4B-Q6_K

So before you trust the rebuild, make sure that Model: line reflects the 8B Q8 model you actually want.

If status still says 4B, then OpenClaw is still indexing/querying as 4B, even if you changed the llama-server underneath.

Recommended sequence

openclaw memory status --agent main --deep
openclaw memory index --agent main --force --verbose
openclaw memory status --agent main --deep
openclaw memory search --agent main --query "your unique test phrase here" --max-results 5 --min-score 0 --json

If the index step errors again with the > 2048 limit, stop there and paste that output. Since you’re only on 22 files now, this is a much cleaner repro 👍

Docs:
https://docs.openclaw.ai/cli/memory

[placid flare]

15:12:04 [memory] sync: indexing memory files
Memory index updated (main).

🦞 OpenClaw 2026.4.15 (041266a)
Powered by open source, sustained by spite and good documentation.

│
◇
Memory Search (main)
Provider: openai (requested: openai)
Model: Qwen3-Embedding-4B-Q6_K
Sources: memory
Indexed: 22/22 files · 55 chunks
Dirty: yes
Store: ~/.openclaw/memory/main.sqlite
Workspace: ~/.openclaw/workspace
Dreaming: 0 3 * * * · limit=10 · minScore=0.8 · minRecallCount=3 · minUniqueQueries=3 · recencyHalfLifeDays=14 · maxAgeDays=30
Embeddings: ready
By source:
memory · 22/22 files · 55 chunks
Vector: ready
Vector path: ~/.npm-global/lib/node_modules/openclaw/node_modules/sqlite-vec-linux-x64/vec0.so
FTS: ready
Embedding cache: enabled (55 entries)
Batch: disabled (failures 0/2)
Recall store: 307 entries · 0 promoted · 307 concept-tagged · 0 spaced · scripts=307 latin
Recall path: ~/.openclaw/workspace/memory/.dreams/short-term-recall.json
Recall updated: 2026-04-21T07:36:53.434Z
Dreaming artifacts: diary present · 1 corpus files · ingestion state present
Dream corpus: ~/.openclaw/workspace/memory/.dreams/session-corpus
Dream ingestion: ~/.openclaw/workspace/memory/.dreams/session-ingestion.json
Dream diary: ~/.openclaw/workspace/DREAMS.md

wasnt able to find banana trading212 octopus wallet

[sage dirge]

Yep, I think this is now a real bug, and you’ve got a pretty good minimal repro.

Why I’d call it here:

• tiny corpus, 22 files
• fresh index, 55 chunks
• Embeddings: ready
• Vector: ready
• FTS: ready
• unique phrase still not found
• Dirty: yes even right after Memory index updated

That’s enough to stop blaming your setup.

One last sanity check, optional but useful

If you want the issue to be airtight, run this:

sqlite3 ~/.openclaw/memory/main.sqlite "select path, start_line, end_line, substr(text,1,160) from chunks where text like '%banana trading212 octopus wallet%' limit 5;"

If that returns rows, then the phrase is definitely in the indexed chunks and OpenClaw search is failing to retrieve it.

I’d file this now

Bug tracker:
https://github.com/openclaw/openclaw/issues

Suggested title:

Memory search returns no results on fresh small index even though chunks are indexed and FTS/vector are ready

Suggested details to include:

• OpenClaw version: 2026.4.15 (041266a)
• provider: openai against local llama-server
• reported model: Qwen3-Embedding-4B-Q6_K
• fresh small corpus: 22 files, 55 chunks
• Embeddings: ready, Vector: ready, FTS: ready
• Memory index updated (main)
• Dirty: yes immediately after indexing
• query for unique phrase returns nothing
• optional sqlite proof if the LIKE query above finds the phrase in chunks.text

One extra note, your status output still says 4B, not 8B, so I’d mention that too. It may be part of the same bug / stale model-state problem.

If you paste the sqlite output here, I can help you tighten the issue text before you file it.

[placid flare]

it found it

[sage dirge]

Perfect. That actually makes the case much stronger.

If the sqlite query found the phrase, then:

• the text is in the indexed chunks
• indexing did happen
• OpenClaw search is failing to retrieve it

So yes, that’s a clean bug report now.

I’d file it here:
https://github.com/openclaw/openclaw/issues

Suggested title:

Memory search returns no results even though the phrase exists in indexed chunks

Suggested summary:

• OpenClaw 2026.4.15 (041266a)
• local openai-compatible llama-server embedding setup
• small fresh corpus, 22 files / 55 chunks
• Embeddings: ready, Vector: ready, FTS: ready
• openclaw memory search ... does not find a unique phrase
• direct sqlite query against chunks.text does find the phrase
• Dirty: yes immediately after Memory index updated

That’s enough for maintainers to work with.

If you want, paste the exact sqlite output and I’ll help turn it into a tight issue body you can drop straight into GitHub.

[placid flare]

steps to reproduce

[sage dirge]

Use this:

Environment

• OpenClaw 2026.4.15 (041266a)
• memorySearch provider: openai
• local OpenAI-compatible llama-server embedding endpoint
• fresh small corpus, no restored DB

Steps to reproduce

1. Put a small set of markdown files in the memory workspace, for example 22 files under ~/.openclaw/workspace/memory/.
2. In one file, add a unique phrase such as:

   banana trading212 octopus wallet
   

3. Run a fresh index:

   openclaw memory index --agent main --force --verbose

4. Confirm status reports the index as built and search backends ready:

   openclaw memory status --agent main --deep
   

   Example observed state:
   • Indexed: 22/22 files · 55 chunks
   • Embeddings: ready
   • Vector: ready
   • FTS: ready
5. Search for the unique phrase:

   openclaw memory search --agent main --query "banana trading212 octopus wallet" --max-results 5 --min-score 0 --json
   

6. Observe that search returns no results.