#clarification on agents.*.memorySearch.fallback.

1 messages · Page 1 of 1 (latest)

quick finch
#

Hey Krill — feature request / clarification on agents.*.memorySearch.fallback.

The current schema lets me set provider: "gemini" and fallback: "openai", but model, outputDimensionality, and remote are shared between primary and fallback. That means the fallback provider inherits the primary's model name (e.g. gemini-embedding-001 hits the OpenAI endpoint and 404s) and dimensionality (3072 vs OpenAI's 1536 → dim mismatch on reindex).

Additionally, enabling multimodal hard-rejects any fallback (memory-search-Bcj3hLJp.js:201), which forces a choice between image embeddings and provider redundancy.

Is there a supported pattern for cross-provider embedding fallback (e.g. Gemini primary + OpenAI fallback on custom provider), or is the current fallback only intended for same-provider/same-dim scenarios? If cross-provider is in scope, would per-fallback model / remote overrides be on the roadmap?

Context: running OpenClaw 2026.4.22 (00bd2cf), macOS.

vocal phoenixBOT
tranquil pastureBOT
#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

vocal phoenixBOT
snow bayBOT
#

One quick detail I wanted to verify was whether provider-level base URLs can differ when memorySearch.remote is left unset.

#

I checked docs plus the current fallback wiring in source.

Short answer: I would not treat cross-provider memory fallback as a supported pattern today, especially for Gemini ↔ OpenAI and especially with custom remote settings.

What the code/docs show:

  • Multimodal explicitly disables fallback

  • Fallback reuses shared config

    • resolveMemoryFallbackProviderRequest(...) passes the fallback provider:
      • the same remote
      • the same outputDimensionality
      • fallback: "none"
    • So agents.*.memorySearch.remote.* is effectively global for primary + fallback.
#

  • remote.baseUrl / apiKey / headers are not per-fallback

    • resolveRemoteEmbeddingBearerClient(...) uses memorySearch.remote.baseUrl first.
    • That means if you set a custom remote.baseUrl, both primary and fallback will use it.

  • Model handling is only partially better on current source

    • Current main has resolveEmbeddingProviderFallbackModel(...), so fallback can pick the fallback adapter’s default model.
    • But that does not solve shared remote or shared outputDimensionality.

So your read is basically right:

  • Gemini primary + OpenAI fallback on a different/custom endpoint: not cleanly supported
  • Same fallback with mismatched dims: also not safe
  • Multimodal + fallback: explicitly unsupported today

Practical guidance today:

  1. Treat fallback as “compatible-provider only”
#
  • same endpoint family, or
    • no shared remote override, and
    • compatible dimensions

  1. Do not rely on Gemini → OpenAI fallback with memorySearch.remote.baseUrl set

  2. If you need both:

    • multimodal, and
    • provider redundancy

    there is no supported config for that right now.

On roadmap, I don’t see any existing schema/docs for per-fallback model or remote overrides, and I didn’t find a documented hidden config for it. This looks like a legit feature gap, not user error.

If you want, I’d file it as a GitHub issue with:

  • your exact provider / fallback
  • shared remote.baseUrl problem