Mistral Models 502 errors | OpenRouter | Page 1

spring oak Apr 5, 2024, 8:13 PM

#

I get the following error with all Mistral models. Other models work just fine. Any idea?

  error: {
    message: '{\n' +
      '  "message":"An unexpected error occurred",\n' +
      '  "request_id":"8b9b6b5a67672e62459d8d40d1a3fd37"\n' +
      '}',
    code: 502
  }
}

finite chasm Apr 5, 2024, 8:17 PM

#

They’re working on the playground- what parameters are you sending?

spring oak Apr 5, 2024, 10:23 PM

#

yh this is purely from API; all other models work fine which is weird.

{
  model: 'mistralai/mistral-small',
  messages: [ { role: 'user', content: [ { type: 'text', text: '12*13' } ] } ]
}

fails

{
  model: 'openai/gpt-3.5-turbo-0125',
  messages: [ { role: 'user', content: [ { type: 'text', text: '12*13' } ]
 } ]
}```
while something like this works

mint gale Apr 6, 2024, 11:51 AM

#

hi, im getting a 502 error via sillytavern as well dunno if this is a good place to post this but i didnt wanna make a whole new thread

#

Endpoint response: { error: { message: '{"object":"error","message":"Service unavailable.","type":"internal_server_error","param":null,"code":"1000"}', code: 502

mint gale Apr 6, 2024, 1:34 PM

#

this is happening with all mistral models for me

finite chasm Apr 6, 2024, 3:15 PM

#

Looking into it. (This error is coming from Mistral platform itself.) in the meantime try using Mixtral 8x7B Instruct, which is hosted outside of mistral platform

mint gale Apr 6, 2024, 7:25 PM

#

is there any ETA on this being fixed? i hope this isnt a permanent problem with mistral lol

#

wonder whats going on at their end

finite chasm Apr 6, 2024, 8:06 PM

#

trying to reproduce it first - if i don't have any luck then we'll email them

#

cc @tall knot

finite chasm Apr 6, 2024, 9:20 PM

#

I wasn't able to reproduce this, but I just deployed a possible mitigation of this. Can you guys try now?

If it didn't work, can you see if ST is sending a response_format parameter in your terminal for some reason? (it's not for me)

mint gale Apr 7, 2024, 2:15 AM

#

nope, not working for me

#

and im not getting a response_format parameter either

finite chasm Apr 7, 2024, 2:17 AM

#

what countries are you two in?

mint gale Apr 7, 2024, 2:17 AM

#

im within southeast asia

#

should i like try a VPN?

finite chasm Apr 7, 2024, 2:18 AM

#

try it to see if that helps

mint gale Apr 7, 2024, 2:24 AM

#

Kk hold on

#

nope, not working

tall knot Apr 7, 2024, 2:37 AM

#

Which VPN region did you try?

mint gale Apr 7, 2024, 2:37 AM

#

the vpn was connected to japan

tall knot Apr 7, 2024, 2:37 AM

#

Try singapore or Australia

#

Or somewhere in EU

mint gale Apr 7, 2024, 2:41 AM

#

er okay hold on im using protonVPN and their free plan is annoying as shit

#

but i can confirm that the US vpn server doesnt work either

#

netherlands isn't working either

tall knot Apr 7, 2024, 2:42 AM

#

hmmm

mint gale Apr 7, 2024, 2:42 AM

#

should i try using another frontend? like venus.ai or something

#

maybe its a sillytavern problem or smth

tall knot Apr 7, 2024, 2:43 AM

#

Working for me, can you try on our playground:

#

https://openrouter.ai/playground?models=mistralai/mistral-small

OpenRouter

A router for LLMs and other AI models

mint gale Apr 7, 2024, 2:43 AM

#

working here

tall knot Apr 7, 2024, 2:43 AM

#

(I'm also in SEA)

#

hmmm

mint gale Apr 7, 2024, 2:49 AM

#

yeah its weird

#

seems to be just the API

tall knot Apr 7, 2024, 2:51 AM

#

Is it possible that

#

when calling from ST, it's calling us directly from your machine

#

and your VPN doesn't shield outbound localhost request?....

#

That's my hunch...

mint gale Apr 7, 2024, 2:56 AM

#

oh what the hell

#

i just tried venus.ai out of curiosity, it works just fine there

#

thats so weird

tall knot Apr 7, 2024, 2:56 AM

#

Yeah but venus is hosted

#

ST is all local right?

mint gale Apr 7, 2024, 2:56 AM

#

yup

#

but it works on venus without a vpn

#

all good on that front

#

apparently theres an update, gonna update ST and see if that fixes anything

#

nope lol, damn

tall knot Apr 7, 2024, 3:18 AM

#

I'd check your VPN setting and see what's up :-?

mint gale Apr 7, 2024, 3:46 AM

#

hm ill try another one, see if that helps at all

mint gale Apr 7, 2024, 11:15 PM

#

hm yeah idk nothing really seems to be working

#

im using 8x7b rn but i really liked using mistral medium

spring oak Apr 8, 2024, 1:26 PM

#

yh still fails for me (from the API, playground is fine)

#

located in the USA

finite chasm Apr 10, 2024, 6:11 PM

#

@spring oak which parameters are you sending via api?

spring oak Apr 11, 2024, 3:01 AM

#

nothing crazy, just what is default in docs (all other models work) — Mistral: Mixtral 8x22B (base) works (as does Mistral 7B Instruct
It's mistral tiny/small/medium/large that fail.
Just tried and I get:

{
  error: {
    message: '{\n' +
      '  "message":"An unexpected error occurred",\n' +
      '  "request_id":"d26d4c2b14bcae3dd952431f33c4a00b"\n' +
      '}',
    code: 502
  }
}
``` with those

#

code is some version of this, but again, works for all models but mistral tiny/small/medium/large @finite chasm

const openai = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: API_KEY || process.env.OPENROUTER_API_KEY,
  defaultHeaders: {
    "HTTP-Referer": "XYZ",
    "X-Title": "XYZ", 
  },
});
let request = {
  model: model_name,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "text",
          text: textInput,
        },
      ],
    },
  ],
};
let completion = await openai.chat.completions.create(request);

tall knot Apr 11, 2024, 3:13 AM

#

spring oak code is some version of this, but again, works for all models but mistral tiny/s...

What happens if you remove response_format?

spring oak Apr 11, 2024, 3:40 AM

#

tall knot What happens if you remove response_format?

ah sorry that shouldn't be there, only passed in for the few models that support it for evals; editing above

#

above @finite chasm seems to imply that it was coming from mistral ("This error is coming from Mistral platform itself."); I don't think it's an issue with how I'm querying the API

#

which would explain why Mistral: Mixtral 8x22B (base) works but not Mistral tiny/small/medium/large

spring oak Apr 18, 2024, 3:13 PM

#

@tall knot flagging that this is still an issue, getting

"\"{\\\"error\\\":{\\\"message\\\":\\\"{\\\\n  \\\\\\\"message\\\\\\\":\\\\\\\"An unexpected error occurred\\\\\\\",\\\\n  \\\\\\\"request_id\\\\\\\":\\\\\\\"6c0a0c05b22e0231ff34b16b608ca757\\\\\\\"\\\\n}\\\",\\\"code\\\":502}}\""

back for Mistral hosted models. All other models working as expected.

tall knot Apr 18, 2024, 3:14 PM

#

spring oak <@353228093420208131> flagging that this is still an issue, getting ``` "\"{\\\"...

yeah that's an error coming from upstream :d...

finite chasm Apr 18, 2024, 3:14 PM

#

We’ll email them

#Mistral Models 502 errors