#Load model from hugging face

1 messages · Page 1 of 1 (latest)

viscid maple
#

What I'm trying to do: use Qwen3.6-27B-Q6_K.gguf from hugging face as main model in local in openclaw
So far I have only used local model with ollama as provider.

lethal flintBOT
#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

abstract estuaryBOT
viscid maple
#

What I'm trying to do: use Qwen3.6-27B-Q6_K.gguf from hugging face as main model in local in openclaw
So far I have only used local model with ollama as provider.

cedar talonBOT
#

Short version: yes, but not by pointing OpenClaw directly at the .gguf file.

For the main model, OpenClaw expects a provider/server in front of it, like:

  • LM Studio (easiest for a Hugging Face GGUF)
  • Ollama
  • another OpenAI-compatible local server (llama.cpp, inferrs, etc.)

One important gotcha: the GGUF file path / hf: stuff in OpenClaw is for local memory embeddings, not the main chat model.

Easiest path for your case: LM Studio

  1. Load Qwen3.6-27B-Q6_K.gguf in LM Studio
  2. Start the local server, usually http://localhost:1234
  3. Set a token, even a placeholder if auth is off:
    export LM_API_TOKEN="placeholder-key"
#
  1. Run:
    openclaw onboard
    then choose LM Studio
  2. Or set it after discovery:
    curl http://localhost:1234/api/v1/models
openclaw models set lmstudio/<exact-model-key>

Use the exact key returned by LM Studio, don’t guess it.

If you want fully manual config

Use a local OpenAI-compatible provider and set:

  • agents.defaults.model.primary
  • models.providers.<your-provider>.baseUrl
# 
curl http://localhost:1234/api/v1/models

I can give you the exact openclaw models set ... line.

#

If you want to stay with Ollama as the provider, then not directly.

OpenClaw does not point the main model at a raw Hugging Face .gguf file. It only sees whatever the provider exposes.

So your choices are:

  1. Keep Ollama

    • import/create that GGUF inside Ollama
    • then OpenClaw uses it as:
      openclaw models list --provider ollama
openclaw models set ollama/<your-model-name>

  2. Use LM Studio
    Better fit if you specifically want to run an exact HF GGUF file.

    • load the GGUF in LM Studio
#

and paste:

openclaw --version

I’ll give you the exact next step.

viscid maple
#

et VLLM?

cedar talonBOT
#

Oui, vLLM works too.

Same rule as with Ollama/LM Studio though: OpenClaw talks to the vLLM server, not directly to the Hugging Face .gguf file.

With vLLM

OpenClaw expects an OpenAI-compatible endpoint, usually:

http://127.0.0.1:8000/v1

Then:

export VLLM_API_KEY="vllm-local"
openclaw models list --provider vllm
# 
openclaw models set vllm/<model-id>

If auto-discovery works, OpenClaw reads model IDs from:

curl http://127.0.0.1:8000/v1/models

Important nuance

For your specific case, if your starting point is a raw HF GGUF file, LM Studio is usually the simpler fit.
vLLM is supported by OpenClaw, but OpenClaw still wants the served model ID, not the GGUF path.

Docs