#LocalAI setup with qwen3-vl-8b-instruct only returns JSON

1 messages · Page 1 of 1 (latest)

calm spindle
#

I've setup my model provider like this:
⁨⁨```
"models": {
"providers": {
"localai": {
"baseUrl": "http://localhost:9999/v1",
"apiKey": "xxxxxxx",
"auth": "api-key",
"api": "openai-completions",
"authHeader": false,
"models": [
{
"id": "qwen3-vl-8b-instruct",
"name": "qwen3-vl-8b-instruct",
"api": "openai-completions",
"reasoning": true,
"input": [
"text"
],
"cost": {
"input": 0,
"output": 0,
"cacheRead": 0,
"cacheWrite": 0
},
"contextWindow": 200000,
"maxTokens": 8192
}
]
}
}


It simply responds with: 
⁨⁨```answer
No output — tool completed successfully.
```⁩⁩

On telegram it responds with:
⁨```Unexpected non-whitespace character after JSON at position 157```⁩ etc.
weary sparrowBOT
#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

calm spindle
#

@glacial surge

glacial surgeBOT
# calm spindle <@1457407575476801641>

⚠️ Agent failed before reply: No API key found for provider "anthropic". Auth store: /srv/clawdbot/state/agents/main/agent/auth-profiles.json (agentDir: /srv/clawdbot/state/agents/main/agent). Configure auth for this agent (clawdbot agents add <id>) or copy auth-profiles.json from the main agentDir.
Logs: clawdbot logs --follow

#

⚠️ Agent failed before reply: No API key found for provider "anthropic". Auth store: /srv/clawdbot/state/agents/main/agent/auth-profiles.json (agentDir: /srv/clawdbot/state/agents/main/agent). Configure auth for this agent (clawdbot agents add <id>) or copy auth-profiles.json from the main agentDir.
Logs: clawdbot logs --follow

calm spindle
#

OK I can see responses being made, I switched to Gemma (it was the same with Qwen though)
⁨```
local-ai | Feb 02 20:08:51 DEBUG Reply received from LLM(finetuned) message="Hey there! I’m OpenClaw, a friendly AI assistant. I'm running on the Gemma-3-4b-it-qat model, but the default is Gemma-3-4b-it-qat. What can I help you with today?" caller={caller.file="/build/core/http/endpoints/openai/chat.go" caller.L=812 }
local-ai | Feb 02 20:08:51 DEBUG Sending chunk chunk="{"created":1770062919,"object":"chat.completion.chunk","id":"c06eaf59-07c8-4f48-bfe9-a340ccbfa27f","model":"gemma-3-4b-it-qat","choices":[{"index":0,"finish_reason":null,"delta":{"content":"Hey there! I’m OpenClaw, a friendly AI assistant. I'm running on the Gemma-3-4b-it-qat model, but the default is Gemma-3-4b-it-qat. What can I help you with today?"}}],"usage":{"prompt_tokens":12150,"completion_tokens":79,"total_tokens":12229}}" caller={caller.file="/build/core/http/endpoints/openai/chat.go" caller.L=566 }

#

But in the chat I just get "Answer ✔️ Completed"

glacial surgeBOT
#

⚠️ Agent failed before reply: No API key found for provider "anthropic". Auth store: /srv/clawdbot/state/agents/main/agent/auth-profiles.json (agentDir: /srv/clawdbot/state/agents/main/agent). Configure auth for this agent (clawdbot agents add <id>) or copy auth-profiles.json from the main agentDir.
Logs: clawdbot logs --follow

glacial surgeBOT
#

⚠️ Agent failed before reply: No API key found for provider "anthropic". Auth store: /srv/clawdbot/state/agents/main/agent/auth-profiles.json (agentDir: /srv/clawdbot/state/agents/main/agent). Configure auth for this agent (clawdbot agents add <id>) or copy auth-profiles.json from the main agentDir.
Logs: clawdbot logs --follow

calm spindle
#

Tried with IBM Granite, same issue

next vine
#

Did you ever get past this?

gleaming elm
#

I was able to setup llama-3.2-3b on lmstudio. At first "no output" increase tokens and bam ... a whole convo

#

From my understanding this qwen model is to be used for tools and doesnt have enough reasoning. I tried it at first.

modern bane
#

I'm running into the same thing. I can get an answer call, with the actual answer, so the information is being processed via LocalAI, but I get the same JSON error, and nothing but a tool call in the chat.

glacial surgeBOT
# modern bane I'm running into the same thing. I can get an `answer` call, with the actual ans...

That error combo (“No output — tool completed successfully.” + Unexpected non-whitespace character after JSON at position …) is almost always tool-calling JSON blowing up with local / OpenAI-compatible backends.

What’s happening in practice:

  • OpenClaw sends your model a big tool schema (functions).
  • The model (or LocalAI’s OpenAI-compat layer) returns a tool call whose arguments string is not valid JSON (often it has trailing text, comments, multiple JSON objects concatenated, etc.).
  • OpenClaw then does JSON.parse(arguments) → Unexpected non-whitespace character after JSON…
  • On Discord you may only see the “tool completed” placeholder, because the actual “final assistant text” never cleanly materializes after the failed tool call / parse.

Quick workaround (get replies working): disable tools for LocalAI

This forces the model to answer with plain assistant text (no tool calls → no JSON parsing).

In ~/.openclaw/openclaw.json:

{
# 
  tools: {
    byProvider: {
      "localai/qwen3-vl-8b-instruct": { deny: ["*"] },
      // or: "localai": { deny: ["*"] } to disable tools for all localai models
    },
  },
}

Docs for tool profiles / byProvider restrictions:
https://docs.openclaw.ai/gateway/configuration

Also fix this in your model entry

For most local/OpenAI-compat models, set:

  • reasoning: false (unless you know the backend implements OpenAI-style reasoning cleanly)
#

So your model def should be more like:

{
  id: "qwen3-vl-8b-instruct",
  name: "qwen3-vl-8b-instruct",
  reasoning: false,
  input: ["text"], // (or ["text","image"] if you actually want vision)
  cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
  contextWindow: 200000,
  maxTokens: 8192,
}

Local models guidance (and the “known-good” LM Studio setup):
https://docs.openclaw.ai/gateway/local-models