Can't get TogetherAI as custom endpoint to work

[quaint lichen]

in .env file I added 1 line

my librechat.yaml.

endpoints:
  custom:
    # Together AI API
    - name: "TogetherAI"  # Unique name for the endpoint
      # For `apiKey` and `baseURL`, you can use environment variables that you define.
      # recommended environment variables:
      apiKey: "${TOGETHER_AI_KEY}"
      baseURL: "https://api.together.xyz"

      # Models configuration
      models: 
        # List of default models to use. At least one value is required.
        default: ["mistralai/Mistral-7B-Instruct-v0.2"]
        # Fetch option: Set to true to fetch models from API.
        fetch: true  # Defaults to false.
      # Title Model: Specify the model to use for titles.
      titleModel: "mistralai/Mistral-7B-Instruct-v0.2"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Summarize setting: Set to true to enable summarization.
      summarize: false

      # Summary Model: Specify the model to use if summarization is enabled.
      summaryModel: "mistralai/Mistral-7B-Instruct-v0.2"  # Defaults to "gpt-3.5-turbo" if omitted.

      # Force Prompt setting: If true, sends a `prompt` parameter instead of `messages`.
      forcePrompt: false

      # The label displayed for the AI model in messages.
      modelDisplayLabel: "TogetherAI"

my docker-compose.override.yml

# # USE LIBRECHAT CONFIG FILE
  api:
    volumes:
      - ./librechat.yaml:/app/librechat.yaml```

then `docker compose up -d`

it runs 1 warning/error
`! api The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested`

but the web starts fine, I entered the together ai key. 
then entered message and send. 

I've tried to change baseUrl to `https://api.together.xyz/v1` too but also same error

Does Together AI is not supported yet? If so sorry I thought as long its OpenAI API compatible it should work

[feral pagoda]

I do use it, nothing added in the .env and this added in the librechat.yaml file

I'll send you the link to the whole file, it's too long to put in a comment

together.ai starts at line 134 and ends at line 193

https://github.com/fuegovic/lc-config-yaml/blob/main/librechat.yaml

Don't need anything in the .env, you provide the API key from the UI as-is

---

if you prefer to add your key in the .env file, replace this line

apiKey: "user_provided"

with this:

apiKey: "${TOGETHER_AI_KEY}"

and enter your api key in the .env file like this: TOGETHER_AI_KEY=your-secret-api-key-here

[quaint lichen]

thank you for your help.
I tried your method but still not work

maybe because there is an error/warning here so that's why?

! api The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

[feral pagoda]

this is unrelated to the addition of together.ai... I saw something about that somewhere, let me see if I can find it

[quaint lichen]

I've resolved that warning/error by adding platform in override file

  api:
    platform: linux/amd64
    volumes:
      - ./librechat.yaml:/app/librechat.yaml```

but still cant chat with together ai

[feral pagoda]

Glad you were able to fix that!

do you see together in the endpoint list?

[quaint lichen]

yes

[feral pagoda]

I'm using this config file without issue, do you have errors in the console or some kind of error in the UI?

[quaint lichen]

here is the log and error

[feral pagoda]

hmmm, that's weird, the only difference from the default config in my install is that I've set the host to 0.0.0.0 in the .env file, not sure this should make a difference...

I assume you're not behind a VPN either?

[quaint lichen]

no, I'm not on VPN.

I'm also running chat-ui from huggingface
https://github.com/huggingface/chat-ui
with the same together ai key without problem.

[quaint lichen]

@feral pagoda
OMG. it worked. it works!
I just restart my pc and it works.
Thanks a lot for your help!! ❤️