#How do I add Mistral AI as my custom LLM?

1 messages · Page 1 of 1 (latest)

wild ravenBOT
#
AI-generated Answer for yves_69297

Short answer:
Use the “Custom LLM” integration. Point ElevenLabs to an OpenAI‑compatible chat completions endpoint, supply your API key as a Secret, and set the model ID. Mistral works if accessed via an OpenAI‑compatible provider (e.g., Together AI) or via your own adapter that mirrors the OpenAI chat/completions schema. (elevenlabs.io)

Two ways to add Mistral:

  1. Via Together AI (documented path that includes Mistral models)
  • Create a Together API key.
  • In your Agent, add a Secret for that key (Secrets → Add Secret → Save).
  • Choose “Custom LLM” for the Agent, then set:
    • Server URL: https://api.together.xyz/v1
    • Model ID: a Mistral model that supports tool use, e.g. mistralai/Mistral-7B-Instruct-v0.1 or mistralai/Mixtral-8x7B-Instruct-v0.1
    • Select your Together API key Secret, then Test. (elevenlabs.io)
  1. Direct (any Mistral endpoint that is OpenAI‑compatible)
  • Ensure your endpoint implements OpenAI’s chat completions request/response format (including function calling if you want ElevenLabs system tools to work).
  • In your Agent: Secrets → Add Secret (store your provider API key) → in the Agent’s LLM settings pick “Custom LLM,” enter your server URL and model ID, select the Secret, and save. If you need to pass extra parameters, enable “Custom LLM extra body.” (elevenlabs.io)

Notes

  • Custom LLMs bypass ElevenLabs’ fallback/cascading; failures are retried against your Custom LLM only. Plan reliability accordingly. (elevenlabs.io)
  • ElevenLabs supports “bring any LLM” and Custom LLM configuration by specifying an endpoint plus credentials. (elevenlabs.io)

If you want, tell me whether you’re using Together AI or Mistral’s own endpoint and I can list the exact fields you should enter in the Agent UI. (elevenlabs.io)

Sources:
Integrate your own model | ElevenLabs Documentation
Together AI | ElevenLabs Documentation
Integrate your own model | ElevenLabs Documentation
LLM Cascading | ElevenLabs Documentation
Models | ElevenLabs Documentation
Together AI | ElevenLabs Documentation

cedar flume
#

I am using Mistral's own endpoint.

wild ravenBOT
#
AI-generated Answer for yves_69297

Great—using Mistral’s own endpoint works as a Custom LLM as long as the endpoint is OpenAI “chat completions” compatible.

What to do in ElevenLabs:

  • Ensure your endpoint matches OpenAI’s create chat completion request/response (including SSE streaming and the choices[].delta/content shape). If it doesn’t, place a small proxy in front that adapts to this format (the docs include a FastAPI shim you can reuse). (elevenlabs.io)
  • In your agent, go to Secrets → Add Secret, choose Custom LLM, then configure:
    • Server URL: your OpenAI‑compatible base URL for Mistral
    • Model ID: your target model name
    • Secret: the API key you stored
    • Enable “Custom LLM extra body” if you need to pass additional provider parameters
    • (Optionally) set “Limit token usage” for safety
      Then save and use “Test AI Agent.” (elevenlabs.io)

If your agent uses tools:

  • Your LLM must emit OpenAI‑style function calls; ElevenLabs will send tool definitions in the tools field and process function_call outputs. (elevenlabs.io)

If your model is slow:

  • Stream “buffer words” first (e.g., an initial chunk ending with “… ”) to keep speech natural while the model continues thinking. (elevenlabs.io)

Behavior with fallbacks:

  • When you pick a Custom LLM, ElevenLabs’ normal LLM cascading is bypassed; retries stay on your Custom LLM rather than falling back to hosted models. (elevenlabs.io)

Notes:

  • ElevenLabs supports “any OpenAI‑API compatible provider” for Custom LLMs (which includes providers like Mistral when served via an OpenAI‑compatible endpoint). (elevenlabs.io)

If you want, share the exact Mistral base URL and model ID you’re using, and I’ll sanity‑check them against the required OpenAI‑compatible shape using the steps above. (elevenlabs.io)

Sources:
Integrate your own model | ElevenLabs Documentation
Integrate your own model | ElevenLabs Documentation
Integrate your own model | ElevenLabs Documentation
Integrate your own model | ElevenLabs Documentation
LLM Cascading | ElevenLabs Documentation
HIPAA | ElevenLabs Documentation
Integrate your own model | ElevenLabs Documentation

cedar flume
#

I still get the following error: This conversation failed with the following reason:

custom_llm generation failed