#Why does gpt work and mixtral and others won't with Openrouter?

27 messages · Page 1 of 1 (latest)

dreamy python
#

#MODEL_NAME=gpt-4-turbo-preview
MODEL_NAME=mixtral-8x22b-instruct-preview

(pilot-env) ➜ pilot git:(main) ✗ python main.py

There was a problem with request to openai API:
API responded with status code: 404. Request token size: 23 tokens. Response text: {"error":{"message":"Model toppy-m-7b is not available","code":404},"user_id":"user_2eyVfxm7xyHbl30CQmbkGstKUvE"}
? Do you want to try make the same request again? If yes, just press ENTER. Otherwise, type "no".

pure wolf
#

Where are you getting these model names from?

#

You should use complete model ids (including the company prefix), eg mistralai/mixtral-8x22b

fickle beacon
#

Model's API name can be copied from it's page (click on that clipboard button)

image.png
dreamy python
#

i tried that but no luck.
MODEL_NAME=fireworks/mixtral-8x22b-instruct-preview

(pilot-env) ➜ pilot git:(main) ✗ python main.py

------------------ STARTING NEW PROJECT ----------------------
If you wish to continue with this project in future run:
python main.py app_id=727a66b4-b865-4df1-ac68-3db7172dd8b9

There was a problem with request to openai API:
API responded with status code: 404. Request token size: 23 tokens. Response text: {"error":{"message":"Model mixtral-8x22b-instruct-preview is not available","code":404},"user_id":"user_2eyVfxm7xyHbl30CQmbkGstKUvE"}

#

this works though: MODEL_NAME=gpt-4-turbo-preview

#

so does this:
MODEL_NAME=openai/gpt-4-turbo

fickle beacon
#

Very strange. Quickly wrote a script to test this in Python with OpenAI library.

client = OpenAI(
  base_url="https://openrouter.ai/api/v1",
  api_key="INSERT_API_KEY_HERE",
)

completion = client.chat.completions.create(
  model="fireworks/mixtral-8x22b-instruct-preview",
  messages=[
    {
      "role": "user",
      "content": "Say this is a test",
    },
  ],
)
print(completion.choices[0].message.content)

Works just fine. What are you using for API requests?

fickle beacon
pure wolf
#

Yeah it sounds like you don’t have the openrouter base url set

dreamy python
#

it seems to be the base my app wants to use.

#

I seem to be getting more of a modal description result when i use it...

:false},"pricing":{"prompt":"0.00000027","completion":"0.00000027"}},"router":null,"updated_at":null,"created_at":"2023-12-10T00:00:00.000Z","description":"A pretrained generative Sparse Mixture of Experts, by Mistral AI, for chat and instruction use. Incorporates 8 experts (feed-forward networks) for a total of 47 billion parameters.\n\nInstruct model fine-tuned by Mistral. #moe","base_model_slug":"mistralai/mixtral-8x7b-instruct","slug":"mistralai/mixtral-8x7b-instruct","variant":null,"author":"mistralai","versions":["mistralai/mixtral-8x7b-instruct"],"deleted":false,"deprecated":false,"short_name":"Mixtral 8x7B Instruct","name":"Mixtral 8x7B Instruct","group":"Mistral","instruct_type":"alpaca","weight_url":"https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1\",\"default_transforms\":[\"middle-out\"],\"default_stops\":[\"###\",\"\u003c/s\u003e\"],\"context_length\":32768,\"modality\":\"text\"},{\"id\":null,\"endpoint\":{\"model\":{\"id\":null,\"endpoint\":null,\"router\":null,\"updated_at\":null,\"created_at\":\"2023-12-21T00:00:00.000Z\",\"description\":\"This is a 16k context fine-tune of Mixtral-8x7b. It excels in coding tasks due to extensive training with coding data and is known for its obedience, although it lacks DPO tuning.\n\nThe model is uncensored and is stripped of alignment and bias. It requires an external alignment layer for ethical use. Users are cautioned to use this highly compliant model responsibly, as detailed in a blog post about uncensored models at erichartford.com/uncensored-models.\n\n#moe #uncensored","base_model_slug":"cognitivecomputations/dolphin-mixtral-8x7b","slug":"cognit

fickle beacon
fickle beacon
dreamy python
#

If you paste that in your script do you get the same result?

#

i had to add: from openai import OpenAI # Assuming this is a correct import, though usually, it's import openai

fickle beacon
dreamy python
fickle beacon
#

Ah I see

image.png
fickle beacon
dreamy python
fickle beacon
#

Unless you just send a request with a HTTP client

dreamy python
#

Snippet provided.

https://openrouter.ai/models/fireworks/mixtral-8x22b-instruct-preview?tab=api

import requests
import json

response = requests.post(
url="https://openrouter.ai/api/v1/chat/completions",
headers={
"Authorization": f"Bearer {OPENROUTER_API_KEY}",
"HTTP-Referer": f"{YOUR_SITE_URL}", # Optional, for including your app on openrouter.ai rankings.
"X-Title": f"{YOUR_APP_NAME}", # Optional. Shows in rankings on openrouter.ai.
},
data=json.dumps({
"model": "fireworks/mixtral-8x22b-instruct-preview", # Optional
"messages": [
{"role": "user", "content": "What is the meaning of life?"}
]
})
)

Mixtral-8x22B Instruct OH by fireworks | OpenRouter

Fireworks Mixtral 8x22b Instruct is the first instruct-tuned version of the latest MoE model from Mistral - Mixtral 8x22B.

This model was finetuned on ~10K entries from OpenHermes dataset.

#moe