having trouble with indexing

[sand current]

i can run memory index from cli works fine
but the gateway gets repeated batch starts but no finished
i'm using local embeddings, but had same behaviour with openai

i get these every time the gateway starts

{"0":"{"subsystem":"memory"}","1":{"provider":"local","items":1,"timeoutMs":600000},"2":"memory embeddings: batch start","_meta":{"runtime":"node","runtimeVersion":"22.22.0","hostname":"unknown","name":"{"subsystem":"memory"}","parentNames":["clawdbot"],"date":"2026-01-29T00:33:39.999Z","logLevelId":2,"logLevelName":"DEBUG","path":{"fullFilePath":"file:///home/inestyne/.nvm/versions/node/v22.22.0/lib/node_modules/clawdbot/dist/logging/subsystem.js:158:16","fileName":"subsystem.js","fileNameWithLine":"subsystem.js:158","fileColumn":"16","fileLine":"158","filePath":"/home/inestyne/.nvm/versions/node/v22.22.0/lib/node_modules/clawdbot/dist/logging/subsystem.js","filePathWithLine":"/home/inestyne/.nvm/versions/node/v22.22.0/lib/node_modules/clawdbot/dist/logging/subsystem.js:158","method":"logToFile"}},"time":"2026-01-29T00:33:40.000Z"}

but they just keep repeating

[calm vault]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[blazing spindle]

Based on the docs and recent releases, this is a known pattern where memory indexing gets stuck waiting for batch jobs to complete. Here are the troubleshooting steps:

Quick fixes to try:

1. Check if the batch is actually stuck or just slow:

   moltbot logs --follow | grep -E "batch|memory.*embeddings"
   

2. Disable batch mode (most reliable fix):

   {
     "agents": {
       "defaults": {
         "memorySearch": {
           "remote": {
   

             "batch": {
               "enabled": false
             }
           }
         }
       }
     }
   }

Then restart the gateway.

3. For local embeddings - check the model is downloaded:

   ls -la ~/.cache/node-llama-cpp/  # or your local.modelCacheDir
   

4. Increase timeout if you want batch mode:

   {
     "agents": {
       "defaults": {
         "memorySearch": {
           "remote": {
             "batch": {
               "timeoutMinutes": 30
             }
           }
         }
       }
     }
   }
   

Common causes:

• OpenAI/Gemini batch API taking longer than expected (can be 10+ minutes)
• Local model not downloading properly
• The batch job completed but Moltbot didn't poll for completion correctly

Try this first:

moltbot memory index  # Run manually to see if it completes

If that works but the gateway still loops, the issue is likely in the background sync. Disabling batch mode is the safest workaround for now.

Source: Memory docs + recent releases mention fixing CLI hangs and adding timeouts.

[sand current]

with batch enabled=false

inestyne@celery1:~/clawd$ clawdbot logs --follow | grep -E "batch|memory.*embeddings"
2026-01-29T00:33:40.000Z debug memory {"subsystem":"memory"} {"provider":"local","items":1,"timeoutMs":600000} memory embeddings: batch start
2026-01-29T01:13:15.316Z error Invalid config at /home/inestyne/.clawdbot/clawdbot.json:\n- agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:15.317Z warn gateway/reload {"subsystem":"gateway/reload"} config reload skipped (invalid config): agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:25.985Z error Invalid config at /home/inestyne/.clawdbot/clawdbot.json:\n- agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:27.282Z error Invalid config at /home/inestyne/.clawdbot/clawdbot.json:\n- agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:27.331Z error - agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:33.527Z error Invalid config at /home/inestyne/.clawdbot/clawdbot.json:\n- agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:33.575Z error - agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:39.769Z error Invalid config at /home/inestyne/.clawdbot/clawdbot.json:\n- agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:39.818Z error - agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:46.033Z error Invalid config at /home/inestyne/.clawdbot/clawdbot.json:\n- agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:13:46.085Z error - agents.defaults.memorySearch.local: Unrecognized key: "batch"
2026-01-29T01:14:11.130Z debug memory {"subsystem":"memory"} {"provider":"local","items":1,"timeoutMs":600000} memory embeddings: batch start
Log tail truncated (increase --max-bytes).

[blazing spindle]

Ah, that's the problem! You set batch under local, but batch config only applies to remote embeddings (OpenAI/Gemini). For local embeddings, you need a different approach.

Correct fix for local embeddings:

{
  "agents": {
    "defaults": {
      "memorySearch": {
        "provider": "local",
        "local": {
          "modelPath": "hf:ggml-org/embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf"
        },
        "fallback": "none"
      }
    }

  }
}

Or if you want remote instead (simpler):

{
  "agents": {
    "defaults": {
      "memorySearch": {
        "provider": "openai",
        "fallback": "openai"
      }
    }

  }
}

Check what's actually happening with local:

# See if the model is downloading/loading
clawdbot logs --follow | grep -E "local|llama|model"

The "batch start" log with local provider is confusing - it's just the generic logging, not actually using batch API. The real issue is likely:

1. Local model not downloaded yet
2. node-llama-cpp build issues
3. The 600000ms timeout (10 min) suggests it's waiting for model load/inference

Quick diagnostic:

# Check memory index status
clawdbot memory status

# Or test manually
clawdbot memory index --verbose

If local is stuck, try switching to OpenAI embeddings temporarily to confirm the rest works:

# Quick test with OpenAI
CLAWDBOT_MODELS_PROVIDERS_OPENAI_API_KEY="your-key" clawdbot memory index

[sand current]

config

we just switch to local trying to fix it, openai was the same problem'

manual indexing works

[blazing spindle]

Your config looks fine! The issue is specific to local embeddings in the gateway. The CLI works, but the gateway environment is different.

Quick diagnosis:

# Check if the model is downloaded and accessible
ls -la ~/.cache/node-llama-cpp/ 2>/dev/null || echo "No cache dir"

# Check gateway logs for llama-cpp errors
clawdbot logs --follow | grep -E "llama|node-llama|error" | head -20

Most likely causes:

1. node-llama-cpp not built for this Node version - Gateway runs as a service, may need rebuild:

   cd ~/clawd && pnpm approve-builds  # Select node-llama-cpp
   pnpm rebuild node-llama-cpp

2. Different Node version - Check gateway vs CLI:

   moltbot --version
   node --version
   

3. Missing build for production - If running from source vs npm:

   cd ~/clawd && pnpm build
   moltbot gateway restart
   

Quick test - switch to OpenAI temporarily:

{
  "agents": {
    "defaults": {
      "memorySearch": {
        "provider": "openai",
        "fallback": "openai"
      }
    }
  }
}

If that works, the issue is definitely node-llama-cpp in the gateway environment.

Also check:

# Is the model file there?
find ~/.cache -name "*.gguf" 2>/dev/null | head -5

# Gateway process environment
clawdbot gateway status --deep | grep -i node

[blazing spindle]

That's the key clue! CLI works, gateway doesn't. The gateway runs as a service with a different user/environment.

Quick diagnostic:

# Run gateway manually to see the real error
cd ~/clawd && pnpm build && moltbot gateway --verbose

Most common fixes:

1. Rebuild node-llama-cpp for the gateway's Node:

   cd ~/clawd
   pnpm rebuild node-llama-cpp
   moltbot gateway restart
   

2. Check model cache permissions:

   ls -la ~/.cache/node-llama-cpp/
   # Gateway user might not have access
   

3. Set explicit cache dir accessible to gateway:

   {
     "agents": {
       "defaults": {
         "memorySearch": {
           "local": {
             "modelPath": "hf:ggml-org/embeddinggemma-300M-GGUF/embeddinggemma-300M-Q8_0.gguf",
             "modelCacheDir": "/home/inestyne/.cache/node-llama-cpp"
           }
   

         }
       }
     }
   }

4. Or just use OpenAI (easiest fix):

   {
     "agents": {
       "defaults": {
         "memorySearch": {
           "provider": "openai"
         }
       }
   

     }
   }

Try option 4 first to confirm everything else works. If you need local, run moltbot gateway --verbose and paste the error output.

[sand current]

thanks buddy 😛