pls add support for perplexity.ai custom endpoint and api

[quiet zinc]

https://docs.perplexity.ai/docs/getting-started

i want to use these models in Librechat -- Mistral 7B, Llama 13B, Code Llama 34B, Llama 70B, and Replit 3b.

it can be achieve via perplexity api

it will be very helpful feature for users like me

thx in advance

[brazen lion]

Setting this up as a reverse proxy would work

[quiet zinc]

it did not

fuegovic tried it

[spark otter]

they don't use v1 in their address and that seems to cause problems. Without v1 the model list wont update, even when filling it manually.

https://api.perplexity.ai/chat/completions

[placid marsh]

hey there!

i noticed you mentioned some issues with the api address for perplexity

it seems like not using the v1 in the address is causing problems, right?

that's interesting!

it's important to make sure the api address is correct so that the model list can update properly

have you tried adding the v1 to the address and seeing if that resolves the issue?

sometimes even a small change can make a big difference

let me know how it goes!

[brazen lion]

Strange, I can make a proxy API, like https://github.com/techwithanirudh/AnirudhGPT-API which would make it work

[spark otter]

yes, I tried adding v1, it didn't work 😅

[placid marsh]

oh well

[brazen lion]

If you'd like I can create the API

[spark otter]

I'm sure @quiet zinc would appreciate!

[brazen lion]

Cool! I don't have replit, so I don't know how to invite someone 😅 to it.

I'll do it on my own, and yeah

done

[spark otter]

lol, that was fast!

[brazen lion]

lol

also what's your replit username?

[spark otter]

I think it's also fuegovic

yes, @fuegovic

[brazen lion]

done

check replit

[quiet zinc]

wait what happaned here i am in the dark because i am the code nooby xD

[spark otter]

not sure if I did something wrong of if it's just not working 🤔

[brazen lion]

Hmm

[brazen lion]

The base url is set

[spark otter]

I'm not sure whay the error, I can see it's calling the right url 🤷‍♂️

[brazen lion]

V1 is added, 🤔

[brazen lion]

Yeah

[brazen lion]

Check the headers

Is there any docs for the API?

[spark otter]

https://docs.perplexity.ai/reference/post_chat_completions

[placid marsh]

hey there!

[brazen lion]

Does it support streaming?

Yes it does

Wait got it

It worked!

Check the invited repl

https://replit.com/@techwithanirudh/Perplexity-API-NodeJS-API-Usage-Example#index.js

[brazen lion]

Does it work in librechat?

[brazen lion]

Yeah, does libre send messages to the API?

[spark otter]

wdym?

[brazen lion]

True, got it, what's the OPENAI_REVERSE_PROXY URL set to?

[brazen lion]

Messages is sent to OpenAI endpoints

[spark otter]

||https://perplexity-api.techwithanirudh.repl.co/v1/chat/completions||

[brazen lion]

Okay

[brazen lion]

model name

got the issue

Librechat is thinking this is not a openai model e.g gpt-3.5-turbo and not passing messages

I modified the api to handle that

[brazen lion]

Make the model names gpt- prefixed with the name

it should work then

[spark otter]

gpt-llama-2-70b-chat,gpt-llama-2-13b-chat,gpt-codellama-34b-instruct,gpt-mistral-7b-instruct

like this?

[quiet zinc]

so it means ||https://perplexity-api.techwithanirudh.repl.co/v1/chat/completions|| using this we can now set reverse proxy in Librechat and make it work ?

[brazen lion]

Yup

[brazen lion]

Yes

[quiet zinc]

wow

thank you so much guys i will test it soon

[brazen lion]

@spark otter Got it to work!!!

[spark otter]

nice!

[mint aurora]

if the only difference is to support /api/chat/completions then this is an easy fix

[brazen lion]

Not that, many other things

[spark otter]

here's the documentation: https://docs.perplexity.ai/reference/post_chat_completions

[brazen lion]

req.body.model = req.body.model.replaceAll('gpt-', '') // enabling sending messages
delete req.body.stop; // deleting stop tokens handling
delete req.body.presence_penalty;
delete req.body.frequency_penalty; // greater than 0

[mint aurora]

it looks pretty similar to OpenAI at first glance

[spark otter]

they say "The API is conveniently OpenAI client-compatible for easy integration with existing applications."

[brazen lion]

custom handling for other types of text based models like replit code which requires prompts are also required, messages should be converted to prompt: 'User: message, Assistant: message and system and so on and soforth'

[mint aurora]

librechat supports prompt payloads, that would be an easy fix too

[brazen lion]

Cool!

[mint aurora]

perplexity ai is $20/mo?

[spark otter]

as far as I can tell 😬
I dm'd you a key for testing if you need

[brazen lion]

Yeah

Should I delete perplexity api then? @spark otter

[spark otter]

It's for @quiet zinc, I won't use it 🤷‍♂️

[brazen lion]

Ah, ok! I think danny's working on it, so idk.

I'll keep it

[quiet zinc]

i got this error 😦

Something went wrong. Here's the specific error message we encountered: Failed to send message. HTTP 422 - {"detail":[{"loc":["body"],"msg":"ApiChatCompletionsRequest.init() missing 1 required keyword-only argument: 'messages'","type":"type_error"}]}

[brazen lion]

What model are you using?

[quiet zinc]

gpt-llama-2-70b-chat,gpt-llama-2-13b-chat,gpt-codellama-34b-instruct,gpt-mistral-7b-instruct

these models

[brazen lion]

All of these work

Strange

[quiet zinc]

i guess this link is not workin on my network that may be the cause