🧪│api-general

[oak sequoia]

Hi folks. I'm trying out pplx-api for code generation, and I currently have a GPT-4 AI API flow in place that takes human requirements and converts to code blocks. I'm using "few shot learning" with stop sequences in the system_prompt to ensure the response is just code. However I'm unsure how to indicate stop sequences in pplx-api. I'm currently using the Completions API. Am I using the right thing, and if so, how do I indicate those stop sequences?

Edit: I think it's not currently supported. Passed a stop array in the POST body and got message back "message": "custom stop words are not implemented for completions."

So, wondering if this few shot learning system prompt is even possible with ppxl-api right now?

[green oxide]

Yeah, a little more documentation would be great as I am very excited to get this working properly.

[hardy marsh]

#1163650321210417242

[hexed flax]

Hi all! Is there a commercial plan for the pplx API ? I run a tech company w a B2B SaaS app that I'd be interested in leveraging the pplx API for. Not sure if you have gotten that far in planning on this

[cedar blade]

please send us a note to [e-mail]

[small pike]

Any plans to add Zephyr to the list of models? Trying to see if I can use the API to build my own RAG app, and Zephyr seems to outperform mistral pretty well according to Llama Index's benchmarks

Also, these models are the exact same models from huggingface just hosted on your infra? or have they been finetuned in some way

Thank you so much for the free API access btw! Truly appreciate you all 🙂

[gentle spade]

Will the api support the same search features that is available on the web. Such as asking about todays news head lines ?

[covert ferry]

There will be an LLM-RAG API at the end of November. https://docs.perplexity.ai/docs/feature-roadmap

[hexed tendon]

This is awesome. Having an enterprise offering for RAG with the data they've collected on real world use cases is a killer move.

not to mention a model specifically tuned for that use case. (also, pre-congrats on the funding round)

[small pike]

Does the API support async streaming? How would I do that in python? (sorry - noob here, would appreciate any help 🙂 )

[grave yoke]

I would love to see a pplx API that fetches content from collections and threads. This would enable Perplexity to have deep integrations with other services without the Perplexity team having to work on such integrations

[heady steppe]

Hello everyone. I am currently working with the API and I am having difficulty understanding a constraint that has been applied to the chat structure. It appears that the structure cannot be as follows: 'System: You are a helpful assistant; Assistant: Hi, I am a helpful assistant. User: [...]'. This is because the first message following a system message cannot be an assistant message. Honestly, I do not understand why this is the case. this is the response: {
"error": {
"message": "After the (optional) system message(s), user and assistant roles should be alternating. Expected role 'user' for message at index 1, but got 'assistant'.",
"type": "invalid_message",
"code": 400
}
}
(i know that this is not a bug, but i was wondering the motivations for that choice)

[junior shuttle]

When using the API-KEY in applications, should one include "pplx-" before the string of digits? Thanks. Also, which is the best channel to get example code that is standalone?

[junior shuttle]

Hi, I am trying out the replit-code-v1.5-3b model. What is the proper URL endpoint? It is "https://api.perplexity.ai/chat/completions" like for the other models? Thanks.

[covert ferry]

You can find all the information in the docs: https://docs.perplexity.ai/reference/post_text_completions

[opal latch]

Text Generator plugin for Obsidian just released an update where you can use any custom API. They have a following structure of their JSON header and body.
Is it possible to use PPLX API model for text completions?

I tried documentation, but I failed shortly 😦

[granite kernel]

https://www.together.ai/blog/together-inference-engine-v1

[mighty field]

Hi there! If the obsidian plugin is openAI message completion compatible, it should work, since our requests are made with the openAI message format.

[scenic skiff]

two questions:

1. is there an ETA on extended support for TextCompletion?
2. when pricing goes into effect, will pro users retain access of some kind? Or will they have to pay for tokens on top of the existing subscription? Or will there be a set number of monthly credits included in the subscription?

[regal pumice]

hi all, does anybody know if its possible to connect sillytavern to perplexity via api instead of openai or claude directly?

[scenic skiff]

No, because pplx-api does not serve a /models endpoint. Thus it isn't actually OpenAI compatible.

You could always make a wrapper around it that serves a bogus models endpoint. Then it should work.

[scenic skiff]

I'd still be interested in answers to these... Thanks...

[heady steppe]

I'm interested too

[covert ferry]

You can find approximate dates here: https://docs.perplexity.ai/docs/feature-roadmap
You are welcome to suggest missing features/models in the forum.
Regarding the use of the pplx API for Pro subscribers: Pro subscription will likely grant free credits.

[mighty field]

Waiting on the response from the API team 🙂 not ignoring you.

[scenic skiff]

It's all good.

Patience is my middle name

[cedar blade]

1. we are not working on text completions at the moment. what if your use case.
2. pro user will have certain monthly credits for api. after that they will need to pay per tokens.

[scenic skiff]

It's okay, I can wait. I'll build a shim around chatcompletions in the meantime. though my prompt format works better without.

[boreal fossil]

Hi hello sorry to interfere in you guys convo, this may be sudden. I need to know whether I need to utilise pplx API or not? Because I saw lot of people in Reddit said they can build chatbot and everything.

Since I don't have any knowledge about AI or programming, is it worth to try to learn more about programming and using the API? If so how do I start? (I'm just a normal accountant trying to learn something new)

[hardy marsh]

Just Perplex it

[boreal fossil]

Yes I perplex it but can't understand a single thing even after asked to simplify it 😭 I became stupid when something new like this. Sometimes I'm thinking back about my life why I exist 😂

but it's all right I will find my way. take time to understand this type of things

[grave yoke]

Love to see pplx online in labs. Any ETA on online via API?

[heady steppe]

Also, there is an eta for sentence-BERT?

[thorny musk]

Any idea if pplx-online will return the URLs too?

[stone trellis]

Hello, I want to be able to pull data from 200 pdfs and from the internet, both grounded. Will that be available soon on the API? Cant wait to have it available. Can I send over my use case to see if there's a change of an early alpha acesss?

[fast rover]

Have you looked into langchain

From the pdf size it's one of hsexases

[past island]

There was a change in our roadmap and this has been punted. We have updated our roadmap to reflect the changes. Thanks for your patience as we work through this!

[stone trellis]

thanks for letting us know @past island . I guess this is the latest. If I could suggest something
(1) if any of the planned tasks are dealyed, they should be replanned in the roadmap
(2) if you could ask someone to write better description for the functionalities, it would also be super great.

I am really trying to push PPX for my business clients and want this to be a fruitfull parnership for all involved. Having this communication more clear will help me interact better with my client's expectations.

[heady steppe]

Ok, thanks for the answer.

[remote gulch]

is the perplexityai python module teh offical way to interact with perplexity API? I am asking s I have knocked up my bare bones conversation using the examples from perplexity documentation utilising requests module. I am just about to mobve on to trying to code the threads part and was checking im not going down a wrong path.

[pale silo]

which model is gpt4?

[nocturne thunder]

the API doesnt have gpt4

[pale silo]

why?\

[thorny musk]

Because OpenAI already provides it

[stone trellis]

Hi, I wanted to check in on the OnlineLLM feature. Regards.

[covert ferry]

Hey, more information will be published soon 😉

[scenic skiff]

Wait, why is openhermes gone????

[past island]

Hey, @scenic skiff! Here's a response from the API team. https://docs.perplexity.ai/changelog/models-removed-replit-code-v15-3b-and-openhermes-2-mistral-7b

[dense lance]

How are pplx-7b-chat-alpha and pplx-70b-chat-alpha different from llama? I can't find any information about it

[lunar crown]

Built on top of llama2 and fine-tuned for chat
https://twitter.com/perplexity_ai/status/1717953875678794158

[blissful breach]

anyone as an end user find rag actually useful day to day? I see a lot of youtube hype and before i get sucked in - if anyone who's used a working rag system give their thoughts on its personal impact day to day? like "wow cant really imagine working without this anymore" or is it like "eh this is cool but i barely use it or on occasion its fine but i still google / chat gpt / use perplexity as its good enough or the search in the app like slack search is good enough"?

[wary cedar]

.oO( what on earth is rag? )

[hexed flax]

Yes here comes the most annoying answer of all time…. Depends on your use case and need to control the output. So far my experience is the more specific the domain and goal you have as the use case, the more building your own rag makes sense. More general needs I think it would be hard to match a perplexity

[dense lance]

Thank you. So how is is pplx-70b-chat different from llama-2-70b-chat? Is pplx-70b-chat just more fine tuned for chat?

[lunar crown]

Should be...
Unfortunately couldn't find more info. You could probably reach out to the team for more info via ticket or email, if you are planning to use the API more thoroughly.

If you have a few go to queries, you can also check out the difference in Labs to get a practical idea of how the responses are going to look like, in different scenarios.

[dire marlin]

If you scroll down a bit in the tweet they explain with charts:

Our models prioritize intelligence, usefulness, and versatility on an array of tasks, without imposing moral judgments or limitations. Comparing pplx-70b-chat and llama-2-70b-chat (by @AIatMeta ), human evaluators saw: 22.7% fewer “full denials” and 31.9% more “no denials”

Those are the improvements made over llama2 base model

[lunar crown]

Damn
I think without an account, Twitter don't show subsequent replies
Missed out on this completely

Thank you, for me this is good to know

[dense lance]

Thanks guys, I appreciate it.

[blissful breach]

ah - this is helpful - i realized i was trying to actually ask what use cases is it relevant for** more than search - so what has it been useful for?

I am finding for code/programming (specifically python libs) - to just pull docs and index them myself with just search would go along way (not talking about RAG) - but what has been RAG good for?

Anyone try to index a package to generate documentation for it / or really quick search?

i should just do this at some point and get the data myself if its useful - BUT being a lazybones - curious if anyone has any good personal cases where for this use-case its great.

similar to cmd+k in tailwind/who-ever algolia is sponsoring for documentation can be done without rag - with speed

because chatgpt just rambles like 90% of the time now -

[blissful breach]

if this link worked - id have a better idea on what its good for - but - may still be insightful

https://www.promptingguide.ai/techniques/rag

Retrieval Augmented Generation (RAG)

[wary cedar]

is there an explanation for dummies somewhere what the difference is between the models / what I should use them for?
https://docs.perplexity.ai/docs/model-cards

and what would be the closest to the model perplexity I select on the web?
so that i can test out my prompts there and then create a curl for it?

plus a basic explanation of what makes a request expensive, to get an idea how what makes it cost more / less?

[covert ferry]

Many are open source models, comparisons or benchmarks can be found on the internet. (e.g. https://mistral.ai/news/announcing-mistral-7b/)
The closest to Perplexity are the fine-tuned online models.
The costs are made up of the length of the input (system message, user message) and the output, the asssistant message.

[dire marlin]

Any chance of having the latest zephyr model available via the API?

[gloomy stag]

hi, as pplx api is now out of beta I would like to ask - is it possible to make a discord bot that can use this API key and work on my server? And if yes, can someone point me to how it can be done? I have almost 0 knowledge of coding, so would love some help with it

[blissful breach]

the benchmarks are kinda bs in my view - its good for researchers to circle jerk about - but as an end user and knowing how game-able metrics can be - i have resorted to personal experience and asking around as a better signal than benchmarks published by the very same people who are writing the paper have a bit of a incentive to .. you know make it look good

[stone trellis]

Hello all

in the newly released perplexity-online-70b, should I be seeing the sources of the information that is returned from the API? Am I missing something?

[thorny musk]

No, pplx online llm is not the same one as perplexityai model

[stone trellis]

https://docs.perplexity.ai/discuss/6567996ada62730079f8b5fb

[restive rain]

Is it possible to add pplx API to typingmind.com?

[hardy marsh]

@past island Docs probably needs a refresh

[past island]

You are right! Thanks, @hardy marsh

[hardy marsh]

or make 'em opensource, people here would contribute

[unborn folio]

Please deepseekcoder 34b instead of codellama 34b it's comparable to gpt4 and I'm paying like crazy for an api

[blissful breach]

gpt 4 0314 is still the best one for coding - recent one is borderline un-usable

is deepseekcoder is gpt4-current(11 or 6 releases)? @unborn folio

[unborn folio]

Yea but margin is so slight that it's hard to tell exactly how much. I have done my test cases and 34b instruct coding version if task is set on point there is almost no difference. Only diffirenciating factor can be reasoning but compiling the fact that 34b instruct is almost pure code version . Deepseek 34b eats up other open source llms easily. Just try 6.7b version in comparison to any open source /gpt3. 5 and you will see that it is damn good coding model. https://deepseekcoder.github.io/ try and see i would pay for API calls to this one like crazy

Today I will try to implement the chat into task weaver and deploy on runpod

[blissful breach]

That’s great to hear - I hate using any LLM so far for code and I’m glad there is another one for me to try as I’ve found chat GPT and Claude to be 💩as it’s now taking me longer to just write the code myself (GPT 4 0314 from March being an exception) - do share how it goes and your personal evaluation on day to day usage and it’s impact

But basically realized at some point will need to run a model myself at some point when I have the extra resources to do so - cannot depend on OpenAI/Anthropic and most model providers speed running enshittification

[abstract sentinel]

Hi all, really enjoying using perplexity. I have a use case, which I'm not quite able to get working well. I have a list of trade names for drugs. What I'm trying to do with perplexity is check whether each of those is an antibiotic or not. For some reason this is not working quite well. I would have thought that given the search results + llm, it would easily answer this. But sometimes, it's just wrong, for example see this: https://www.perplexity.ai/search/Given-a-drug-7hqCPc7ZSqevzMuZUYGosg?s=c
(I was just trying it out on the UI first before I go ahead and use the API and use it on all the 5000 drug list I have, just want to justify the cost before doing so)
Does anyone have any thoughts that can help with this?

[left lark]

How do you prompt PPLX-70B to show citations?

[dense lance]

Are there plans to increase the context window for pplx-chat-7b beyond 8k?

[dense lance]

I know it's not in the public roadmap. Just wondering if there are plans later?

[unique path]

would love to know

[outer heath]

pplx-70b-online seems response too slow ?

it response , but too slow.

[past island]

Hey, @Paul! Currently, it's not available.

[vocal merlin]

seems like you have restricted it too much. it did not use the web search in your case.

[abstract sentinel]

Is there a way to make sure it uses web search?

[vocal merlin]

try out this Perplexity AI: Answer only in yes/no and no other text.

DIAMICRON 60MG MR TABLET. is it antibiotic? https://www.perplexity.ai/search/Answer-only-in-Uqo9Z.LqQmG3YS2PGpjorA?s=mn

will weight fine tuning be available to api users?

or at least latent space (prefix) prompt fintuning?

[abstract sentinel]

Thanks, will try

[vocal merlin]

also try to turn on copilot

[abstract sentinel]

I actually want to do this en masse using the API

[vocal merlin]

hmm an alternative way would be: use google search api in conjunction with pplx api

[molten silo]

Is api TOS same as for pplx itself?

[restive rain]

hi whats the pplx endpoint?
https://docs.perplexity.ai/reference/post_text_completions ==> page not found

i was trying to add this to typingmind, how should i write the custom header?

[covert ferry]

Hey @restive rain!
You can find the endpoint and header here: https://docs.perplexity.ai/reference/post_chat_completions

[spring hamlet]

Can other models also search internet or only pplx-7b-online and pplx-70b-online can?

[covert ferry]

No, only the online models have Internet access

[deep kindle]

ETA on mixtral-8x7b-instruct support in the API? It was added to the supported models list but it’s not working for me (server error 500)

[thorny musk]

Works for me now

[jagged solstice]

mixtral-8x7b-instruct equalling llama-2-70b on benchmarks while being 80% cheaper is amazing

[vocal merlin]

any plan on finetuning mixtral and/or online version of it?

[deep kindle]

Still seeing server error 500 on my end, switching to another model fixed it. Maybe I found a bug? I can provide full error details here if that’s fine

[thorny musk]

Sure, you might've got the model name wrong

[deep kindle]

I confirmed my model name is correct (mixtral-8x7b-instruct). An example of a wrong model name error is:
Error code: 400 - {'error': {'message': "Invalid model 'mixtral-instruct'. Permitted models can be found in the documentation at https://docs.perplexity.ai/docs/model-cards.", 'type': 'invalid_model', 'code': 400}}

The error I'm getting is:
Error code: 500 - {'error': {'message': 'The inference server returned an error.', 'type': 'unknown', 'code': 500}}

Note that I'm using the latest version of openai-python (https://github.com/openai/openai-python) with base_url set to perplexity's endpoint:
openai_client = openai.AsyncOpenAI(base_url="https://api.perplexity.ai/")

The error only happens with mixtral-8x7b-instruct. It works fine when I use another model e.g. pplx-7b-online.

EDIT: I figured out the issue. It seems mixtral-8x7b-instruct doesn't accept the following message format:
{"role": "user", "content": [{"type": "text", "text": "Hello!"}]}
This format is used in vision models like openai's gpt-4-vision-preview to accept both text and images in the message object. Obviously mixtral-8x7b-instruct isn't a vision model but it should still accept this format for cross-compatibility. Like I said other perplexity models like pplx-7b-online work fine with this format.

Vs the traditional message format:
{"role": "user", "content": "Hello!"}

[vocal merlin]

can you try out the example here: https://docs.perplexity.ai/reference/post_chat_completions ?
and does your payload follow the rules?

i got no problem with the example usage

[deep kindle]

Yes it works with the example code.

The app I'm building allows you to select different models to use, both vision and non-vision. With openai's API I'm able to use this message format with ALL of their models (vision and non-vision) which makes the code a lot cleaner:
{"role": "user", "content": [{"type": "text", "text": "Hello!"}]}

Reference: https://platform.openai.com/docs/guides/vision/quick-start

Bottom line is that there's inconsistent behavior here.

Here's the example python code from perplexity docs, but modified to use the message format I referenced above:

import requests

url = "https://api.perplexity.ai/chat/completions"

payload = {
    "model": "pplx-7b-online",
    "messages": [
        {
            "role": "system",
            "content": [{"type": "text", "text": "Be precise and concise."}]
        },
        {
            "role": "user",
            "content": [{"type": "text", "text": "How many stars are there in our galaxy?"}]
        }
    ]
}
headers = {
    "accept": "application/json",
    "content-type": "application/json",
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

Run it with "model": "pplx-7b-online" and it works. But change to "model": "mixtral-8x7b-instruct" and it errors.

[vocal merlin]

i believe pplx just followed this https://docs.mistral.ai/api/#operation/createChatCompletion

you can write a wrapper just 1 line of code not a big deal

[deep kindle]

Yea I know how to fix it but I don't think I should have to 🙂 the API should be as flexible as possible. openai's API does this very well (for this and other things too).

Ultimately this is a shortcoming in pplx API that prevents it from being a seamless drop-in replacement to openai API.

For someone like me who obsesses over clean code and falls under this issue's use case, this discourages me from using pplx API for the time being.

(not saying this is exclusive to perplexity. I haven't used mistral's API yet, I wouldn't be surprised if it had the same issue)

[mighty field]

Hi everyone! just a quick heads up, posting here is a great way to get advice from other members of the community but if you need direct support from the team, please head to the "Discuss" portion of the docs site. Where the API team will be answering questions and responding to comments on a regular basis:

https://docs.perplexity.ai/discuss

Feel free to post feedback, suggestions, etc here!

[echo garden]

Does the api handle web scraping also?

[covert ferry]

You can find more information about the online LLM's here, scraping limited to single pages is afaik not possible: https://blog.perplexity.ai/blog/introducing-pplx-online-llms

[echo garden]

K thx ☺️

[unborn folio]

Mistral is crazy good

[tepid summit]

You should edit it and remove your api key I think 😉

[covert ferry]

@quick trench

@quick trench I removed your message because your key was visible, please post the code without the key!

(And please deactivate the current key!)

[quick trench]

Thanks! I thought I had removed it from the snippet but I obviously missed it. I appreciate the catch, @tepid summit! Also, I solved my question on my own, so I'll leave it

[quick trench]

Done!

[tepid summit]

No problem at all 😉

[low salmon]

Hey, I'm using the pplx-70b-online model API and the responses have been great. However, I'm based in India and the responses I'm getting are in PST timezone. Any suggestions on how to set the timezone at the model level? Also, is there a way to upload a file via API?

[lunar crown]

Can't you specify the timezone bit as part of your prompt. That you'd prefer IST.

As far as I am aware, I don't think you can upload a file via API. You'll probably need to use some other library to create a vector database and then run the query based on that.

[low salmon]

Any sample code or suggestions on how to create a vector database and file upload?

[lunar crown]

https://docs.pinecone.io/page/examples

[jagged solstice]

Is it just me, or are the rates for mixtral on pplx-api really cheap?

Like compared to other providers

[spring mauve]

Hi, do y’all know if RAG in API made it, I saw it was on roadmap in November? Would for sure be a sweet Christmas gift 🎄

[past island]

Hey, @spring mauve! pplx-7b-online & pplx-70b-online are online models, they are available via the API.

[spring mauve]

Thanks for quick reply Alex! Was wondering more about document RAG for PDFs, like how the pplx web UI has. Would sure beat langchain+vector db

[unborn flax]

Is there a resource that explains more about the online models? I see them mentioned as existing but not how they work, and I was confused because the FAQ on the docs still says "Does the pplx-api currently support web browsing? - No"

[covert ferry]

Please have a look at the following resources:
https://blog.perplexity.ai/blog/introducing-pplx-online-llms
https://docs.perplexity.ai/reference/post_chat_completions
https://docs.perplexity.ai/docs/pricing
https://docs.perplexity.ai/docs/rate-limits
https://docs.perplexity.ai/docs/model-cards

[low salmon]

Any idea when the Gemini model will be added?

[covert ferry]

Gemini Pro is only available via Googles API (not open source). So it won't be added since the api is mainly for open source and in house LLM's.

[carmine holly]

when I try using typingmind with perplexity api, I get CORS error. Seems perplexity has to enable something from their side to allow web apps

[dreamy summit]

Any plan to make an API for Perplexity Pro? I mean, GPT-4, Claude-2, etc.

[restive rain]

Ah I tried to set it up but couldn't work after a few times. So I gave up haha .

So I'm not using the $5 at all... Not sure how to utilize it at all. Tbh, havent found a real use for the API but will try to figure out something .

[covert ferry]

there is no plan atm

[stone trellis]

Hello all! Please help me undestand why I cannont see the grounding (where the information is comming from) when I use the API for ppx-online. Am I doing something wrong? I expected to see the reference/where the information was drawn from.

[covert ferry]

Hey @stone trellis
Sources are not available via the API (yet).

[stone trellis]

Thats great to known that they will be available someday 😉

From a user/product perspective is it valuable to have the API with the ability to search the internet without telling where the information came from though? This is what doesent make sense to me. It seems to me that the MVP would be to include it, dont you think?

I think I found a workaround. I have a list of whitelisted URLs that I want to RAG with JSON descriptions for them. Each URL has a descriptive JSON. With the user request I do a Vector Database search, get the URL that address the user request, use Perplexity online the format the message using the parameter "site: xyz" to restrict the search reach.

[stone trellis]

Hi

is it possible We have stumble on the limit for the ppx-70b-online. Is it possible to pay for more concurrent requests?

[covert ferry]

You can send your request to [e-mail] 🙂

[timber dawn]

do you know when?

[covert ferry]

there is no ETA

[covert ferry]

[stone trellis]

Hello, is the ppx-online api suposed to work with the restrictive expression "site"? This will get a response only based on a specified site. Does this work as it does with Google?

[stone trellis]

"user request site:aaa.com or site:bbb.com or site:ccc.com"

is this the right sintax?

[stone trellis]

Hello, can anyone confirm or deny the effectiveness of the use of the "site" parameter on a request to narrow down the possible sites where the search must be done?

[past island]

Hey, @stone trellis! They syntax is correct, but currently, the operators are not supported.

[stone trellis]

Thanks Alex.

[left lark]

When will you be getting the Mixtral API to offer 32k context?

[covert ferry]

https://docs.perplexity.ai/docs/feature-roadmap

Mixtral is also planned

[left lark]

The official Mixtral API covers 32k context, why did Perplexity not offer that?

[covert ferry]

This is beyond my competence, but it is planned and 32k will be available soon 🙂

[dim idol]

I see that Mistral-Medium is censored when asking (for tests) for "illegal" things. Is it because of a system prompt added by Perplexity? or it's natively censored?

[covert ferry]

Censored by Mistral

[dim idol]

Are you sure?

[covert ferry]

Yes I am

Also noticed by other users (https://www.reddit.com/r/MistralAI/comments/18jd1s2/mistral_platform_api_censored_like_chatgpt/) but you can get around this or make it even worse

[left lark]

You should make the dropdown organize the Mistral models together from least powerful to most powerful

[low salmon]

Do we have mistral api with online LLM?

[covert ferry]

Yes, pplx-7b-chat

[pine isle]

Is the profile in the perplexity app the same as a system role in the API?

[analog junco]

can we have solar 10.7b model?

[marsh pendant]

Can we get an option to return both the raw snippets and reference links for online models via the API? I'm considering the perplexity API for an application I'm building, but it's critical for my use case to be able to provide links to the reference material. It would also be nice to set the number of snippets to return (i.e. defult is 3, but maybe I want 10, 20, etc -- even if this costs extra). My use case requires "deep" search so stopping at top 3 results doesn't work for me. Maybe just consider having a separate search API? Also why no Mixtral for online responses? I think just have a search API that is $5/1K reqs (up to 20 responses perhaps) and then the normal chat completions API with per-model pricing; no need to combine. Snippets from search can be passed to LLM API serverside for completions with a flag, or otherwise the developer can decide what to do with the results.

[restive rain]

whats the advantage of using PPLX API over Perplexity?

[covert ferry]

The pplx API is for developers to integrate LLM's into their products, the online LLM's can be very helpful for obtaining up-to-date information ☺️

[restive rain]

so the API doenst search right? is an LLM like GPT-4

[covert ferry]

The online LLM's can search

[restive rain]

that u mean perplexity.com

[covert ferry]

I think that will explain it better: https://blog.perplexity.ai/blog/introducing-pplx-api
https://blog.perplexity.ai/blog/introducing-pplx-online-llms

[restive rain]

oh i see. thanks finally understood. haha
so the -online one works

[restive rain]

im having some issues adding perplexity models into typingmind, any idea how i can do this? (sorry im really newbie)

[restive rain]

hi all, i cant seem to use the API on typingmind, neither can i use it on harpa.ai...
Just wondering which platforms, or how, are you using the API?
Thanks!

[pine isle]

I write my own code to run the api

[pine isle]

https://docs.perplexity.ai/reference/post_chat_completions you can use this to get running

can even try it directly if oyu input your token

[rancid acorn]

Any ideas how the syntax for making perplexity API calls could be shoehorned into this (HARPA AI browser plugin)? It's designed to facilitate connections to models via OpenAI or OpenRouter endpoints, but some have apparently managed to get it to work with endpoints from other providers (e.g. LM Studio). I have a valid API/bearer token and, referring to https://docs.perplexity.ai/reference/post_chat_completions, have tried various combinations of inputs/settings but with no luck

[restive rain]

same here, i have no idea how to use my $5 credits ...

[shy quarry]

https://docs.perplexity.ai/docs/getting-started

[restive rain]

Thanks
Yeah , I got my bearer code long ago. But I have no clue how to use it if typingmind or Harpa can't link due to some issues (might be CORS related)
I don't know how to code , sorry for that.
The chat completion page doesn't look very user-friendly at least from an UI perspective.
https://docs.perplexity.ai/reference/post_chat_completions

The guide only tells you how to get the API key and how to pay . Haha but I still haven't figured how to use

[covert ferry]

The API is actually only intended for developers and is not intended for use via other websites.

[restive rain]

Oh ... Thanks I'm hurt 😔

😭

Okay got the answer. Thanks haha so why make the pro users pay $5 for something most won't need

Anyway it's okay . 2am here. Have a good day

[covert ferry]

You don't pay extra for it, it's just a bonus for developers to try out the API 😉

[rancid acorn]

I signed up to use Copilot / multiple models, then noticed the API and was curious if it might be able to be used in my HARPA workflows, which mostly involve searching for and parsing info from the web. But I'm like you (can't code - certainly not a developer!), so sometimes just have to accept that trying to make shiny and cool tools do things that they are not meant to and which I can't technically implement is:
a) probably not a great idea to begin with, and
b) even if I tried, would require so much back-and-forth with a coding AI assistant that I would ultimately end up spending more time on it than would be saved (and if one little thing changes and the implementation breaks, then I'm basically back to square one)

anyway fwiw no point getting too frustrated. things may not be developing at an exponential or sustainable rate, but they are progressing fast. In my humble (biased is prob more appropriate lol) opinion, I think people who can't code, but understand how to effectively interact with the technology (and look at it as an enabler rather than a magic wand), will be among those who stand to be benefit from it the most :))

[restive radish]

Hi all. I see that mixtral-8x7b-instruct is priced at the same rate as the 13b models, but there is no published price for 13b models. Any idea what the price is?

[shy quarry]

#💬│general message

[hardy flare]

Mostly curious but also interested because I've been considering building some voice-related features, are there any plans in the future to offer voice-related APIs through the Perplexity API?

[past island]

Hi, @hardy flare! Currently, it's not on the roadmap, so you can build such a voice service on top of the API.

[low salmon]

Hi there,

I've been experiencing an issue with the API for the past five hours where I've been receiving a 500 internal server error. I'm a paid user of the API and I'm hoping that this issue can be resolved quickly. Thank you.

[shy quarry]

#💬│general message

[bright mural]

Is it possible to have the pplx-70b-online models cite their sources (similar to how the Perplexity app does it)? I'm looking to build something that requires summarized realtime data, that the user can then tap on to be able to look at the actual source of information. I'm testing things out on the labs.perplexity.ai page but I can't get it to cite any sources...

[rancid acorn]

https://www.perplexity.ai/search/Is-it-possible-DtQiEHcZTY6TgkS04YW23Q?s=c
"""
So in summary, no - the pplx-70b-online model does not directly cite sources like the Perplexity app does. The model provides up-to-date responses, but does not include embedded citations. Adding support for grounding facts and citations is on Perplexity's roadmap for the future.
"""

Actually contrary to Perplexity's answer, it apparently is not on the roadmap (though is a frequently requested feature)
#👉│feedback-general message

[sleek coral]

Where can i raise an issue related to billing? I want to add credits but I am unable to do so and the card gets rejected on Octane.

[covert ferry]

You can contact [e-mail] as it is a billing problem and not an API problem, however Perplexity cannot take action against declined cards, this is done by the payment platform (Stripe) or your bank.

[ancient belfry]

Hey I tried loading credits on my account, but cannot do so, and no reason is being mentioned. Can someone help me out?

[covert ferry]

Hey @ancient belfry!
Please take a look at these instructions: https://docs.perplexity.ai/docs/getting-started
Also note that it may take some time for the credits to appear 🙂

[trail folio]

So sad, I want to switch model to perplexity api, But the reponse is to hard to validate JSON format, it's not complete the a JSON in response message.

here example reponse :

{
  "id": "0a56acdb-da6e-474e-9799-e4d95fc947a4",
  "model": "mistral-7b-instruct",
  "created": 2178721,
  "usage": {
    "prompt_tokens": 980,
    "completion_tokens": 38,
    "total_tokens": 1018
  },
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "{\n\"name\": \"DKRA Knowledge\",\n\"agent\": \"dkra_agent\",\n\"input\": \"AI is the CEO of DKRA company\"\n"
      },
      "delta": {
        "role": "assistant",
        "content": ""
      }
    }
  ]
}

[past island]

Hey, @trail folio! Thanks for reporting. Could you create a thread in the #1161804761247526912 and add your query and the system prompt you used.

[compact pelican]

hey. can someone help me pls? I cant find anything about using functions calling in perpexity api (a-la it is possible in chatgpt3.5-4). is it possible with perplexity?

[past island]

Hey, @compact pelican! It's not possible. Please, take a look at the available models: https://docs.perplexity.ai/docs/model-cards

[compact pelican]

hey. thanks for a quick response. there are model listed, but no info about functions calling functionality available/unavailable

[nova brook]

Hey @compact pelican, if you find the original message helpful, please consider reacting to it with the ⭐ emoji. If the post is appreciated by the community and receives 5 stars, it will go to the ⁠⭐│starred channel and the post author will get the EXPLORER role on Perplexity.

[thorny musk]

Unavailable

[hollow musk]

Does the pplx api not have the same search capability as the app?

I can ask a summary of a linkedin post on the main app but the api using the online models is declining

[thorny musk]

They aren't the same

[hollow musk]

so the online models don't do real time browsing?

[covert ferry]

Yes they can, but you are currently comparing one of the pplx models with GPT-3.5 and also with Copilot, there is a huge difference between them. It is also advisable to use the operator “site:”.

[hollow musk]

ah okay

[hollow musk]

is it normal to get a "sorry as an AI language model i cant.." etc for a normal query without a url through the api?

[covert ferry]

Depending on what the question is, yes.

[hollow musk]

i tried searching for the score of the latest man city match

it gave the correct response but the first time i got that "sorry" message

[spring mauve]

Is there an operator to provide the pplx online model so that it will indicate sources and URLs, like the consumer interface does? Or possible roadmap item?

[covert ferry]

No, that's not on the roadmap at the moment.

[hollow musk]

I'm getting an error even with the correct api key
{'statusCode': 401, 'error': 'Unauthorized', 'message': 'Missing authentication'}

[covert ferry]

Please share your code, it's best to open a post in #1161804761247526912

[hollow musk]

making a post thanks

[nova brook]

Hey @hollow musk, if you find the original message helpful, please consider reacting to it with the ⭐ emoji. If the post is appreciated by the community and receives 5 stars, it will go to the ⁠⭐│starred channel and the post author will get the EXPLORER role on Perplexity.

[calm spear]

Hello, folks. I've been using the pplx labs playground to test out the different models and I really like the mixtral-8x7b-instruct model for my use case. I incorporated the model into my app by using the pplx API, however, the responses I get from the API are different than what I get in the playground. That is, for the same prompt I get different response from the playground and the API. The prompt is asking the model which of two numbers are greater, so I would expect the same answer. The playground gets it correct every time but the API gets it wrong every time.

Does anyone have any clues or hints as to how I can tickle the API to behave more like the playground? Thanks.

[ocean surge]

https://github.com/nekowasabi/vim-perplexity
I implemented a simple client to run the API as a vim/neovim plugin.

[rancid acorn]

It seems a very common request and, imho, perhaps reflects that, in professional contexts, many people want the ability to crosscheck information received from a generative LLM, especially if it is information that is not part of its training 'knowledge' (e,g. about a current event).

It would of course be great if the accuracy of the outputs generated from any of these systems could be taken as gospel

But alas, we're not there yet...

[plucky matrix]

I'm having issues with using the model with industry classifications, I provided it with a prompt:

You are a model that classifies companies into specific categories based on available information. The categories are: Aerospace-Space, Agriculture, Broadcast-Communications, Consumer Electronics, Energy, Engineering Services, Industrial, Medical, Robotics, Transportation-Telematics, Wearables. Your task is to classify the following company into one and only one of these categories. Your response should be the category name only, with no other words or explanations.

and I get one of three responses, a repetition of the company name I input and then a full stop, a long explanation of everything I would get from a google search or in rare cases it will say that there were no results, which doesn't make sense.

[thorny musk]

There's apparently a "related" model now? 🤔

cc @past island

[strange arrow]

hi, when I call the api using pplx-7b-online model, it response "error":{"message":"An internal server error has occurred.","type":"internal_server_error","code":500}; is server in trouble now?

[lethal siren]

I'm a bit confused about the API - it looks like a pretty typical LLM completion style API. Is there a way to get something more similar to the end-user experience, where sources are included/provided?

This seems like the key differentiator for perplexity, so not having it exposed through the API is quite surprising (if that's the case)

[gray siren]

Um i just wanna ask is it possible to add api credits thru the apple app

Since my subscription is bought from there

[covert ferry]

#🧪│api-general message

[rancid acorn]

You aren't the only one mate #🧪│api-general message

[rancid acorn]

I would argue that it is precisely Perplexity's differentiator - it's not a peripheral feature, but at the core of why people use it.. There's 1000s of models out there now...Perplexity's ability to ability to retrieve and use real-time web results to inform its responses is what sets it apart, but I can't integrate the API into my workflows in any systematic way without the API also responses being supported by URL links/references like when using the web app

[lethal siren]

Agreed. It definitely provides results with a different 'feel' to some other LLMs, but going from that to actual sources/urls feels like an order of magnitude improvement over other LLMs (for some specific use cases)

I'm building an AI workflow tool (https://hunch.tools) and after messing around with perplexity it seemed it would be a hugely beneficial addition to some types of workflows. Without the sources it's still interesting and useful, but I don't feel it's an absolutely must-have addition. I'll probably still add the 'online' models, but meh, not as exciting as it could be

[empty brook]

When you give a URL, it returns related results that appear under Perplexity search.

[thorny musk]

Maybe it doesn't work for reddit

[empty brook]

Yes me too

[rancid acorn]

Seems to be suggested follow-up questions

[maiden skiff]

For adding credits it through an error and charged $300 dollars how do I rectify this cost.

I have not used any of the credits, but it through an error and then showed up out of nowhere on the api page?

[past island]

Please send a request to [e-mail], we'll help with that.

[maiden skiff]

I just did. Thanks.

[nova brook]

Hey @maiden skiff, if you find the original message helpful, please consider reacting to it with the ⭐ emoji. If the post is appreciated by the community and receives 5 stars, it will go to the ⁠⭐│starred channel and the post author will get the EXPLORER role on Perplexity.

[fringe fulcrum]

I am looking for help building a basic application using Perplexity API. Any willing developers?

[nimble monolith]

Hello everybody, new here.
I'm trying to setup the pplx API access. When I enter my card details everything goes smoothely and I get confirmation on the bank page that the operation was successful and that I can get back to the merchant website. Unfortunately when I go back to pplx API web page it always shows up the Setup payment button as if nothing had happened. I've redone the process 2 times to no avail.
Is there some form of validation period after card registration and before API access is granted ?

[past island]

Hey, @nimble monolith! This issue has been fixed and you can try setting up the payment method for the API. It should appear right after you payment method is confirmed.

[zealous loom]

I'm not always the best with API programming so I want to ask a question first here before taking hours to code the test. When I use perplexity via the web its great at visiting a news site and giving me a summary and key information about that post. When I read the API docs I see its providing a fast interface for minstrel and Llama2 . Is it still going to be able to process requests that mention a url and are these LLMs going to return as high a quality response as the web interface for perplexity?

[shy quarry]

#🧪│api-general message

[zealous loom]

thank you for that. very sad to see that it can't hit real time sources

[nova brook]

Hey @zealous loom, if you find the original message helpful, please consider reacting to it with the ⭐ emoji. If the post is appreciated by the community and receives 5 stars, it will go to the ⁠⭐│starred channel and the post author will get the EXPLORER role on Perplexity.

[median oar]

Hi all, is there any way to fetch references with the text generated by the online models for the API?

[rancid acorn]

If there isn't already, would be nice if there was a similar system for upvoting requested features (/FAQs), such as the post immediately above..

[covert ferry]

#🧪│api-general message

[lethal siren]

I guess can + this: https://discord.com/channels/1047197230748151888/1196426667946684579

[nimble monolith]

Thank you for your answer @past island . Problem solved. I didn't have to recreate the payment method. Thanks for the good work guys!

[nova brook]

Hey @nimble monolith, if you find the original message helpful, please consider reacting to it with the ⭐ emoji. If the post is appreciated by the community and receives 5 stars, it will go to the ⁠⭐│starred channel and the post author will get the EXPLORER role on Perplexity.

[median oar]

This is not helping, can you give me more specific instructions/details?

[covert ferry]

No, it´s not possible.

[median oar]

Is there a roadmap when we can get it?

[covert ferry]

No, that's not on the roadmap at the moment.
#🧪│api-general message

[tall gazelle]

Is there an OpenAI compatible endpoint for the API? To use it with tools that support OpenAI.

[trail burrow]

is it possible to ask pplx-70b-online to list its sources when using the API, just like the web and mobile user version

[covert ferry]

#🧪│api-general message

[trail burrow]

okay

[clever prawn]

How is the development of 32k context length support progressing? Is there a specific release date?

[digital nimbus]

hello @everyone i am new to perplexity and i need the api but even when having upgraded to Perplexity pro i am not able to access the api... pls help @supple fossil @everyone

[covert ferry]

No need to ping people here 😉
please have a look at https://docs.perplexity.ai/docs/getting-started

[digital nimbus]

i have followed everything there

it is showing me this:

[covert ferry]

It may help to clear the website cache/data and deactivate extensions.

[digital nimbus]

ok will try

[digital nimbus]

still not working

this is what is showing me everytime

[covert ferry]

There may be a problem with your card, in which case nothing can be done, however you should receive your $5 credits from pro subscription, please contact support via the pro support intercom button on the account tab.

[digital nimbus]

got it thanks!

[digital nimbus]

done but it is showing me this and still no credits are displayed

[trail burrow]

Are credits only available with yearly plans? Or are they available with monthly plans as well?

[covert ferry]

Credits are available on all plans.

[past island]

Hey, @digital nimbus! Thanks for reporting and sorry for the inconveniecne, we have a ticket regarding the Pro $5 credits https://discord.com/channels/1047197230748151888/1198130008053518376, and the team is working on it.

[digital nimbus]

👌

[wind pine]

that was quick.

[wind pine]

can we get a /models endpoint implemented?

[digital nimbus]

hi i am still not able to use it Alex please update me on this

[radiant juniper]

hi, i tried the API. doesn't the online models return the citation URLs?

response = client.chat.completions.create(
model="pplx-70b-online",
messages=messages,
)

[covert ferry]

#🧪│api-general message

[grave mural]

Mine gave me the $5.00 credit when I entered $2 as the amount in the auto top off. It did not give it to me until I completed the top off automation.

[odd path]

I am curious why when I use the response is different when I use the API vs when I use the browser?

[abstract dirge]

I want to use the API, not sure which model is equivalent to "Experiment" (with Copilot) from the chat website?

[earnest nymph]

PPLX 70B

[covert ferry]

Different system prompt/sources are searched for differently.

[abstract dirge]

Not even close. Website version codes much better.

[covert ferry]

#🧪│api-general message 👀

[formal rivet]

Hi, I have some results on labs which I can't seem to replicate when using the api. Is it possible to know some of the default params used by labs which I can apply to the api?

[maiden skiff]

I reached out to [e-mail] for a charge that was made and it was a double charge and need a refund of credits its been more than 1-2 business days this was last week yet no resolution.

Any pointers or help I could get.

[karmic geode]

Are we able to buy (pay extra) to get api access that returns citations? Can we discuss with sales?

[covert ferry]

#🧪│api-general message

[karmic geode]

That’s a bit silly. If it’s not a legal decision, this is going to be what sinks you guys. Everyone wants it. We want to pay you for it. It’s fantastic with it and unusable without it. So a competitor is going to emerge that does it and takes all your business away is my guess…. You’re so close to putting something out there that could be huge…

[covert ferry]

Just because it is not yet on the roadmap does not mean that this function is not being considered 😉

[warm knot]

Are there any plans to make the API responses as good as the responses we get on the web interface? or close to it?

[covert ferry]

Improvements are constantly being worked on, at the moment the web interface also uses more advanced LLM's which can make a difference to the quality of the answers 🙂

[rancid acorn]

I reckon that's spot on. If the citations were meant to be some kind of moat, Metaphor (now called exa) would appear to be building a big ol bridge across it...

(there's also Tavily, and I'm sure numerous other similar prjoects/products out there)

[sharp flicker]

Is copilot and the follow-up prompts part of the API offering?

[covert ferry]

No, please take a look at the documentation: https://docs.perplexity.ai

[sterile temple]

There's a sure shot bug with perplexity's billing system. I have experienced it myself last week.

I just reached on discord and got help from the team here. But yeah it took me 5 working days to get access to the API for which i had already paid.

[warm knot]

Quick question - I am trying to pull the most recent news articles related to a particular domain using pplx APIs. However I see that it's pulling in news articles that are more than a year old. Anyone know what context I can give to rectify this problem?

[rancid acorn]

Google operators seem to work fairly well on web version, but not sure about with the API. Couldn't hurt testing including 'after:yyyy-mm-dd' in the query (if you haven't already) – would maybe steer the results in the right direction

[warm knot]

What;s the best model you recommend I use for accurate answers as close as what we get on the web?

[warm knot]

Thank you

[lyric willow]

Hey @warm knot!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[covert ferry]

The online LLMs are the only ones with internet access, which is closest to the web version

[faint crown]

hey i never received my API credits for perplexity pro, does it need to wait a couple days or something or is that a known bug

[nocturne thunder]

https://discord.com/channels/1047197230748151888/1198130008053518376 please send an email with your account details to [e-mail]

[faint crown]

i did last night just in case

[faint crown]

Have you found a good alternative for this? I'm literally looking for the API version of perplexity web, but if that's not on the roadmap then I need to find the next best thing before I rag it myself with a scraper.

[rancid acorn]

The sources/citations are being withheld from the API responses; if aggressive prompting can extract (at least the Publications and Titles of the sources), I see no reason why this request for the inclusion of sources with API responses could not be implemented with a few system prompt tweaks.

I've noticed a slight change of tune. From repeatedly just saying "No (it's not currently possible nor planned)", to the "roadmap is flexible and we're taking on feedback" (paraphrasing in both cases). So I'm curious to see if where it goes. I use perplexity for probably 80% of what I used to use Google for – it's a brilliant service and I'll continue to use it for general web searches for information about recent / contemporary developments for the foreseeable future. It would be great to start using the API in some of my research workflows, but currently there is literally no point. Anyway, there are other approaches and APIs that I currently use that could be helpful to you - feel free to DM

[snow mauve]

Hey all, Is the response includes the source urls?

[shy quarry]

#🧪│api-general message

[karmic horizon]

any idea of when custom Stop Words will be implemented into the API? 😄
i'm testing a port for zed.dev editor to be able to use pplx-api as one of the endpoints for its 'assistant' features

it currently defaults to using openai models

[river granite]

Maybe I am in the wrong place, could anyone indicate me how to add a perplexity leveraging and perplexity chat interface inside a webapp?

[median thicket]

hi

anyone here?

😦

?

[past island]

Hi, @median thicket! Everyone's here 👋 , what is your question?

[karmic horizon]

Maybe I am in the wrong place, could

[junior shuttle]

I have a simple question. I pay for a Perplexity Pro subscription. I know I have 1000 requests a month for free (using any model, I assume). I want to know about the cost after the first 1000 requests using GPT4. How is this calculated? Say I have another 1000 requests. Is the cost $5, and in addition I pay the cost of using GPT4? The details of the pricing structure are hard to find for specific use-cases.

[past island]

Hey, @junior shuttle, I attach a screenshot from the pricing page, you can see that $5 per 1000 requests is the pricing for the online models only. If you spend your free credits on those online models 1000 requests then you'll need to top up your balance to send more that will be billed according to this pricing.
GPT-4 model is not provided via the API, you can find the supported model list here: https://docs.perplexity.ai/docs/model-cards

Just to clarify, GPT-4 and other models, including Copilot queries on our site have a limit of 600 queries a day that are available to you as a Pro user.

[past island]

Hey, <@&1193989584976105562>! If you missed it, CodeLlama-70B-Instruct is avaialbe via pplx-api. Give it a try and let us know what you think!

[wind pine]

I could kiss you.

[quartz sonnet]

me too

[wind pine]

Are there any vscode integration plans on the board?

[past island]

That's a good idea. As of now, you can check the extension created by one of our users: https://discord.com/channels/1047197230748151888/1163935223399063702

[wind pine]

I saw that -- thank you for linking it.

Cody has been an interesting experience. If you're looking to emulate anything I'd look at that.

https://marketplace.visualstudio.com/items?itemName=sourcegraph.cody-ai

I'd not think twice about paying half as much more on my monthly to add it to the mix.
I'd love to put all of my AI bucks in the same bin.

[trail burrow]

Let me check it out when I’m home

[vernal ibex]

hey folks

trying to access the API

Using the snippet here but there is an authentication error 401

https://docs.perplexity.ai/docs/getting-started

is this snippet outdated? Checked my OpenAI and PPLX keys. They are both correct

[nocturne thunder]

snippet seems to be working fine, make sure your pplx api key is correct and the openai package is installed.

[wind pine]

does pplx api return the models endpoint, yet?

[void dock]

I’m about to ask something pretty dumb here but we are able to test and train a 70b without local devices capable of handling that much?

[wind pine]

All you need is curl and a 300. baud modem.

Training is a whole other beast though, innit?

[void dock]

Awesome ty. Caveman type learning over here and 7 or 8s were coming back at teletype speed

Haven’t tried perplexity yet at that level.

[wind pine]

I'm not sure you can, mate.
You get to consume the provided models.

If you're interested in local stuff -- phi-2 via ollama makes compelling widget to tinker with.

I suspect it won't be long before you see that incorporated into a phone app.

[void dock]

Done some Llama2 training on ollama but all local device (pi5 and weak Linux terminals with small nvidia cards)

5 years from now ain’t no telling how it’ll be.

[wind pine]

THe Phi-2 model is quick as snot.
Of course, I gave it 30 Xeon cores and 64GB of memory but I bet it can run on a phone. It's a 2b model.

[void dock]

Thank you. I appreciate the input I’ll give it a shot.

[lyric willow]

Hey @void dock!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[wind pine]

I used https://ollama.ai with https://github.com/ollama-webui/ollama-webui
no gpus.

You're free to make money with it.
https://huggingface.co/microsoft/phi-2/commit/7e10f3ea09c0ebd373aebc73bc6e6ca58204628d

/off-topic

[mossy beacon]

It's just an HTTP API. Super simple. Here's my implementation I just wrote today:
https://github.com/Clay-Ferguson/quantizr/blob/246e7e2b3510b033b1c133e8702fcc7da1b325a0/src/main/java/quanta/service/node/PplxAiService.java

But the simple example is here:
https://docs.perplexity.ai/reference/post_chat_completions

[mossy beacon]

I'd recommend troubleshooting by running a shell command instead, so see if the problem is in your code or on the server. I just posted a link (see above) which has an example shell command on it.

[river granite]

interesting! thank you!

[lyric willow]

Hey @river granite!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[river granite]

can it also get files from users? and operate summaries?

[mossy beacon]

If you extract text from files and/or user input text you can include that data into your prompts yes. I haven't looked into any of Perplexity's support for direct handling of files yet (or know if they do any of that)....and it would depend on how you're planning to use those files.

[river granite]

summarize them - 3 pages files at its best

[junior shuttle]

Hey, @erlebach123, I attach a screenshot

[mossy beacon]

To summarized text you can just do a prompt like: "Summarize the following text in two sentences:\n\ntext: ${text}" ...in other words just do it all in one shot.

[warm knot]

Quick question and this could be a naive one. Is this model better for real time news online or would it still be perplexity's own models?

[covert ferry]

The pplx online models are the only ones with internet access.

[river granite]

yeah I get it, it is the file upload that gets it a bit more complex for me

[low blaze]

Hey, has someone contacted support about the APIs recently? I did but still did not get any answer 😦

[past island]

Hey, @low blaze! Did you contact via [e-mail] or you reported an issue to [e-mail]?

[low blaze]

Hi @past island, thanks for replying! It was via [e-mail] 🙂

[dense lance]

Hi @past island , I've been getting wierd responses from codellama-70b-instruct? Here is an example of one that works on codellama-34b-instruct but fails on codellama-70b-instruct.

[past island]

Alright, if that question wasn't covered on the server or at https://docs.perplexity.ai/discuss, a personal question, please send me your email, we'll check.

[past island]

Thanks for reporting, @dense lance! We'll look into this issue.

[dense lance]

Thanks @past island , other times it's complained about ethics or told me "404 not found" lol. 34B is solid though. I really appreciate being able to use it.

[lyric willow]

Hey @dense lance!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[low blaze]

Thank you for your support! I sent you my email as a dm 🙂

[mossy beacon]

Do you mean doing file uploads to your server from browsers, or are you saying there's some Perplexity upload feature you're using?

[river granite]

I mean file upload via chatbox interface eventually

[mossy beacon]

Well at this point I can no longer tell if you're talking about the API or the Perplexity website, but I guess all your questions were answered.

[river granite]

yes. btw API

[half thistle]

Does the $5 API credit stack up each month (roll over), or is it use-it-or-lose-it each month?

[mossy beacon]

I'd look for something like "Tika" (in your language) which is what I use to extract text from any kind of file. Then you can just use that extracted document text in a chat completions prompt to ask a question about the content. Doesn't involve any uploading thru the API. All the file uploading is just direct to your server.

[river granite]

in my language?

can it run on a webapp? I see, you mean Apache Tika - right?

[pearl bison]

Does anyone know if it is possible to list the links/sources from where the api pplx-online models are pulling their data like it does on perplexity.ai?

[mossy beacon]

Yeah Apache Tika. You may not need that if users are uploading text files of course. It's mainly for extracting text from ANY kind of file (PDFs, etc). I checked and there's a python version of Tika. You know how to upload files from a webapp right? It's easy. Just ask ChatGPT how to do that in in your programming back end. The front end code is always the same javascript.

[mossy beacon]

I'm just guessing, but is this the "-online" model versions that can do this? I need to know this as well.

[river granite]

yeah I would like users to upload time by time but I like this suggestion too maybe for other projects or this one too. I will look into it. Thanks a lot!

[lyric willow]

Hey @river granite!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[pearl bison]

From my observations so far, the only difference from the chat and online models is that the online models are up to date whereas the chat models haven't been updated since September.

[rancid acorn]

This should just be pinned to this thread #🧪│api-general message
(or the server's bot should automatically respond to posts containing 'does|can' 'API' and 'sources|links|urls|citations' with 'No'... the efficiency! )

[rancid acorn]

The two pplx models with -online have real-time access to the internet. Ask a question about a recent news or sports event and there's a good chance you'll get an accurate response. That is great (and what differentiates these models from almost all others out there). The shame though is that it only returns the response/answer without any citations or sources - so there is no way of verifying the information using the actual output

Aside from going to Google/Perplexity and cross-checking the results (and thereby basically making the initial API call all but pointless/redundant)

[mossy beacon]

That's cool! Thanks for clarifying that point! I'm not that worried about the LLMs providing source URLs because it's all sort of a big Bayesian logic probability summation anyway, and I'm not sure current models can really say which sources they trust more than others or why at this point.

[lyric willow]

Hey @mossy beacon!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[rancid acorn]

No worries and nice! Glad to hear it fits your needs 🙂 For me, a model that uses a web-based RAG system is great for overcoming knowledge cut-offs, but it doesn’t do anything to overcome the reliability/hallucination problem. It’s not so much a concern about the ‘quality’ or 'choices' of the sources but rather just needing to know that response is actually informed by real sources/articles and not just a convincing confabulation. Being able to at least visually inspect sources gets you half way there (just looking over the URLs can be a helpful form of verification imo), but really, to fully verify its existence (and/or conduct further research), one needs the ability to actually visit the source, which currently is not possible with the API

Basically for my research there still needs to be a ‘human in the loop’ before any material generated from an LLM can be used in a client-facing report. But that’s just me – ofc, if mostly/generally accurate answers is all one needs, then in its current form, pplx’s API must be great 🙂

[rancid acorn]

This is using the 7b-online model, which is somewhat unfair (the 70b version is far more reliable), but it demonstrates the point I'm trying to make. Aside from very basic queries like 'what is the stock price of this company' or 'what is the weather forecast for XX', it tends to fall apart and revert to its training knowledge to generate the response (or whatever is happening, it results in inaccurate output/answer)

[gloomy stag]

codellama 70b is very restrictive and often refuses to answer, as I understand its more of a base model problem, but maybe it can be tweaked by ppxl team somehow? Forcing it to start answers with "Sure!" or some other sort of tweaks?

[torn obsidian]

hmm I would probably use AI Agents running the prompts back and forth until one gets a proper answer... But maybe there are better solutions.

[zealous loom]

codellama 70b is terrible. i asked it an sql question but because i had something referencing a business's address in the sql and it refused to help because of data privacy reasons.

[faint crown]

I'm trying to get the API to research a website given.

when I give it a site:http or just the address by itself, it returns information unrelated to the site (a completely different company)

Tried 7b and 70b.

Is there a specific prompt format for this type of query?

[rapid blaze]

I am facing the same issue of API credits not getting reflected. I have tried 3 different cards for my business account. Finally, I have deleted that accont, and am now using my personal one with a master card. I have been charged for the pro subscription but still not able to either buy or avail my API credits. Would really appreciate some help on this.

[lunar crown]

Are you using the online variants?

[blazing badge]

Hello! Not sure if this is already on Perplexity's radar but API support for llava-v1.6-34b would be 🤌 . The multi-modal API landscape is currently incredibly sparse and pricing for 1.6 on replicate is unreasonable compared to GPT4-V. Food for thought.

[faint crown]

Yes I was

[left lark]

What is the pricing for Mixtral? Perplexity removed 13b from api pricing

[covert ferry]

Pricing will change soon, but is currently as follows:

$0.14 per 1M input tokens
$0.56 per 1M output tokens

[lyric olive]

Hey @lyric willow , any way I can get my rate limit quickly increased ?

[left lark]

Will the prices be lower?

[somber barn]

I know this has already been partially discussed multiple times here. But why does the API not offer a (more expensive) pplx-web version which actually returns you the same information the website does? This is a huge business opportunity. For websites which have all their content indexed, you could immediately create a perfectly working chat assistant, just using this api (and requesting only to use their website, same as you can do it in the perplexity browser extension).

Currently I do not understand why you should use the api. Very unfortunate.

[faint crown]

Ya I'm with you, huge business miss. But you'll also see that a ton of people have requested it.

Hopefully they'll provide an API soon with the search prompt+sources. Or maybe something similar to OpenAI threads in case the copilot needs an answer.

[scenic hemlock]

I can’t setting perplexity ai with Siri? For ask to Siri like shortcut and Siri respond?

[lyric olive]

Hey @lyric willow , anyone here to help me with this ?

[covert ferry]

#🧪│api-general message

[upbeat kite]

i'm getting transcript from links (which r quite long) and then passing it to gpt and pplx apis to get a 2-3 line summary. the summary generation for 3 links done parallely is approx 10secs. do you know of a way to make this significantly faster? api seems to be q slow on medium-large text

[idle crypt]

Any ideas on this problem, please? #⚡│ask-community message

(Behaviour discrepancy between interactive and API)

[junior shuttle]

Is GPT4 accessible via the Perplexity API? Thanks.

[covert ferry]

No, via OpenAI

[jagged solstice]

https://www.icloud.com/shortcuts/ba386a9ff0de41c7b51a40a01f0cd10f

No need for API

[pine oracle]

do you ahve a shortcut that uses the api? and can have an ongoing conversation isntead of a new one with every message

[jagged solstice]

no

[pine oracle]

oh

[drowsy eagle]

Hey group! I have been recently trying to mimic what the perplexity chrome extension has been doing in terms of summaring the website. It does do a good job on most websites. But When using the API I am not able to replicate. The actual api also seems to upload files and I am not able to do the same from the API either. Any inputs will be greatly appreciated.

[rigid spruce]

why is it so hard to make the API key of perplexity have the same format as OAI API key in order to be used on different tools that OAI API KEY already supports?

[haughty igloo]

Hi all. Apologies if this has been answered previously, but do we have a timeline on when the Api will include sources and citations?

[nocturne thunder]

it is currently not on the roadmap

[haughty igloo]

But here(https://docs.perplexity.ai/discuss/65af6285e69072005b83eb05) it says it will be available soon?

[rancid acorn]

Well that's interesting 🤞
As is this:
"The plan is to offer source references to approved use-cases that fill out a form which will be sent to the emails of all API users."
https://docs.perplexity.ai/discuss/65c0b02f09d8e3001ca0d3ba

[vivid umbra]

I cannot find gpt models in the supported model page. Wondering how to call gpt models through perplexity.ai API?

They are not in the list. Or maybe one of the pplx models is actually a gpt model?

[nocturne thunder]

perplexity doesn't offer gpt via api

[vivid umbra]

Only through the UI?

Wondering which model in the API list is the closest to GPT 3.5 turbo then?

@nocturne thunder

Is there a way to scrape the UI result that calls the gpt models 🥹

[nocturne thunder]

https://pplx.ai and the api are two different products

[covert ferry]

No, scraping is against the TOS!

[vivid umbra]

Are you saying the API is not officially provided by perplexity.ai?

[nocturne thunder]

They're two different products from the same company.

[covert ferry]

It is, but there is https://pplx.ai AND the API with open source models and the in house online LLM'S

[vivid umbra]

Well. I just want to scarpe some perplexity AI results that use GPT models to check how accurate the perplexity.ai is. What should I do then 🥹

[covert ferry]

Well nothing 🤷‍♂️, you can use the UI or the pplx API, but not the UI via API.

[vivid umbra]

So you mean mission impossible 🥹🥹?

Ok. Thank you! This is very helpful!

[lyric willow]

Hey @vivid umbra!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[vivid umbra]

But which model in the API list is the closest to gpt 3.5 turbo model then?

I mean in terms of performance?

[covert ferry]

mixtral-8x7b-instruct is very good, but without web access. pplx-7(0)b-online are both fast and with web access. It's best to just try them out.

[vivid umbra]

Thanks @covert ferry ! This is very helpful!

[lyric willow]

Hey @vivid umbra!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[vivid umbra]

So by "without web access", you mean no search results are retreived if I call mixtral-8x7b-instruct in the API? But pplx-7(0)b-online

That means it would hallucinate a lot, right?

[covert ferry]

Correct

[vivid umbra]

So if I want accurate responses, I'd better use "pplx-7(0)b-online", right?

[covert ferry]

Not a lot, but there is a knowledge cutoff

[vivid umbra]

Right.

[covert ferry]

If you are looking for up to date information, yes

[vivid umbra]

Thanks @covert ferry. Another question is whether I should use the pplx-7b-online model or the pplx-70b-online model?

[lyric willow]

Hey @vivid umbra!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[vivid umbra]

I guess 70b is always better, right?

Also is there any benchmark eval result betwen the pplx-7b-online model vs gpt3.5-turbo in terms of accuracy score?

Wondering whether anyone has done any research on what kind of content is the best to get the most accurate result through the API? Right now it is only "Be precise and concise.". But wondering whether this is optimal

I know I have a lot of questions. I am also new to all these, and still figuring things out! 🥹

[covert ferry]

Some benchmarks @vivid umbra https://blog.perplexity.ai/blog/introducing-pplx-online-llms 👀

[vivid umbra]

Thank you so much! You are awesome @covert ferry !

[shy quarry]

#👋│introductions message

[rancid acorn]

You can adjust the system prompt. "be concise and precise" is just the default :))

[next sequoia]

How do i get the sources and related information as seen in the Perplexity UI in an API response

[nocturne thunder]

perplexity and pplx-api are two different products so they wont have the same functionality. but information about sources will be available in the future #🧪│api-general message

[rancid acorn]

#💬│general message ❤️

[timid grove]

Hi. I'm looking to use the API with codebase knowledge. But for that I would need to create a vector database of some sort. I wonder if there's a way to do this (add long text file support to the pplx-api) without resorting to services other than Perplexity itself.

[faint crown]

boom sounds like its roadmap now?!

[nocturne thunder]

Latest info I know is that they’re working on it but no ETA as of now. You’ll likely get an email to fill out a form for approval once they release it. https://docs.perplexity.ai/discuss/65c103f492fb9800462213f4

[faint crown]

thats still a wonderful update vs. the previous stance

[past island]

Hey, <@&1193989584976105562>!

We’re excited to announce our integration with Vercel, becoming the knowledge API for every Vercel developer.

We offer the fastest and most accurate online LLM APIs without any knowledge cut-off. We can't wait to see what developers will ship using our APIs on Vercel!

Learn more here: https://vercel.com/blog/ai-integrations

[broken pulsar]

Hi

Hi@API users

[mint crescent]

Is there any plans to increase the rate limits for the pplx api from the current 10 req/min?

[untold osprey]

Has anyone found a way to prompt the api effectively to consistently cite relevant URLs to sources that its pulling from in the pplx online models? I have it returning relevant URLs 70% of the time but want it to be higher

[nocturne thunder]

it will be added in the future for approved use-cases

[nocturne thunder]

you can send a mail to [e-mail] for higher rate limits

[untold osprey]

@nocturne thunder I know its not currently supported. Ive found a work around through prompting for URLs and it returns results. I’m wondering if anyone here has also tested prompting to try and have URLs displayed and if they’ve managed to get it to work consistently.

I love perplexity, but the main issue I’m having is that it doesn’t follow my prompt nearly as closely as I’d like and the answers to queries can be different from the previous one in a non trivial way.

[rancid acorn]

I have, more out of curiosity (I mean, they had to be there...) than as a workaround - would rather they were just part of the output, and in a consistent format etc, than having to coerce/bribe an LLM... And if you're using the 70b online model and getting them with 70% reliability, you're doing better than me ha

[untold osprey]

@rancid acorn Yeah that’s def ideal but I need a work around for something I’m working on. I solved the format problem, only issue now is just making it 100% :/

Or at least 90%ish

[pearl pawn]

I have sent in several emails over several days to [e-mail] and have not gotten a response. 🙁 I just need to talk to someone.

[past island]

Hey, @rancid acorn https://x.com/tdinh_me/status/1755773979841982753

Tony Dinh 🎯 (@tdinh_me) on X

Chat with @perplexity_ai on @TypingMindApp 🤝

Twitter · Feb 9, 2024, 2:01 AM

[tight mauve]

Hi, when using mixtral-8x7b-instruct in the API using the openai client, is it safe to assume that the messages field is where all the input needs to be given with system, user, and assistant roles and no function calling is supported?

I am planning to encode the JsonSchema of a few functions in the system prompt itself to do some CoT style reasoning and wanted to double check my assumption that function calling is not supported for this model.

Also, how has the latency in mixtral-8x7b-instruct API been when compared to gpt-4 models? Are there any benchmarks available? I am planning to replace some gpt-4 API calls with mixtral-8x7b-instruct to save on latency. Any info on this would be greatly appreciated. Thanks!

[tight mauve]

https://docs.perplexity.ai/docs/feature-roadmap

Any estimate on when Mistral 32k context length would be available?

[small verge]

Anyone know what to do or how to correct this error when using the API? I'm using it through App script.
**
Error: HTTP status 429, Response: {"error":{"message":"Request rate limit exceeded, please try again later.","type":"request_rate_limit_exceeded","code":429}}**

[mossy beacon]

I was getting that error on OpenAI API when I had ran out of credit without knowing it. However that may not really apply on Perplexity, because it might just mean the rate was genuinely exceeded.

[small verge]

Thanks! i have enough credit, though im running Perplexity and error referred to OpenAI. Kind of weird! Anyways, i fixed the error by just adding a millisecond delay to the app script run. Shoutout to GPT 4 for the edits 🙂

[lyric willow]

Hey @small verge!

If you find the original message helpful, please consider reacting to it with the :star: emoji. If the post is appreciated by the community and receives 5 stars, it will go to the https://discord.com/channels/1047197230748151888/1082806833938436228 channel and the post author will get the <@&1082034222778302614> role on Perplexity.

[rancid acorn]

Spec for giving a Custom GPT the ability (/'Action') to make calls to the pplx API. Copy-paste (+ add API bearer key to the secrets/auth thing) in the Custom GPT builder, and then all that's really needed is a basic system prompt ("You have access to a powerful API search engine via your Action; always make API calls before generating a response." - or something ).

Haven't played around with it much, but so far seems to work alright

[jaunty burrow]

i just added the api to the app we are building 🙂 very good

[fervent inlet]

Is it possible to choose content sources through the API?
Similar to how you can choose academic sources, Youtube, Reddit, etc on the Perplexity AI site.
I'd like to give our users the ability to query specific sources.

[nocturne thunder]

You can add the site: search parameter to the prompt. For example: "What is perplexity ai site:reddit.com"

[median oar]

Does this reliably only source info from given URLs? How do we specify multiple sources? Like site:x site:y site:z?

[nocturne thunder]

you can do multiple sites like this “site:reddit.com OR site:youtube.com”
Yes it should only use those sources but there is no way to check this atm since the sources aren’t available yet in the api.

[north pawn]

What is the pricing for mixtral 8x7b? I'm assuming it's not just the $.28/1M output of the normal 7b models here https://docs.perplexity.ai/docs/pricing

[covert ferry]

#🧪│api-general message

[steep meadow]

fixed

oof any idea why performance is so bad compared to online

cargo run -- "when was vision pro released"
   Compiling perplexity-rs v0.1.0 (/Users/andrewgazelka/Projects/explore/perplexity-rs)
    Finished dev [unoptimized + debuginfo] target(s) in 1.29s
     Running `target/debug/perplexity-rs 'when was vision pro released'`

{"id": "38e04f5c-c576-4684-9b58-10d938ee83e9", "model": "pplx-70b-online", "created": 2071989, "usage": {"prompt_tokens": 42, "completion_tokens": 102, "total_tokens": 144}, "object": "chat.completion", "choices": [{"index": 0, "finish_reason": "stop", "message": {"role": "assistant", "content": "The Visions computer graphics product, developed by Digital Effects and incorporating APL (APL is a programming language), was used to create television commercials and animation for the 1982 film Tron. However, there's no specific date provided in the search results for when Visions was released. It's known that Digital Effects used Visions in their work on Tron, which suggests the product was available by 1982 at the latest."}, "delta": {"role": "assistant", "content": ""}}]}

[drowsy fiber]

is it possible to increase the rate limit for the api?

I've emailed [e-mail] but no one has responded so far

[spark delta]

API featured examples would be great, to avoid the basic questions just above (I.e. source limits, system message, content localization, etc). Any available sample code or url to share please?

[median oar]

Perplexity should be providing references! Happy to pay more to get references!!

[alpine cedar]

Hi, We have noticed that the PPLX APIs are failing intermittently, we are getting random and absurd replies from the APIs since yesterday. We have tried both pplx-7b-online as well as pplx-70b-online. Our prompt requires the model to do an online search and give the results based on that. A quick sample of the response that we are getting is -

2 To2020000000000000000000000000000000000000000000020202020200202020202020020earelld 2. Figure.
202002020202002 @20202 Facebook{2. This2 A2 2 A2 It2 every-20200200000200000000000000000000000020202 A2 By Feb. As2f2Fig2e 2.2 *20 In20200 But2020 the20202020020000000000000000002
520 2052020202020202 *2so2Al2
  2222 2.
| They.2j2 The92The2
In Burn22292 Book2z *
2 # TryIn Online2 Ass22 Ch2222 * Om2252222522 I22\\2222222222202 The2 original2 OP2 22222
Source.2 After2222#InSot0222
2252022222222222222222 John22 -
22 readll']['-22
2 Do222The2 dr2 Fig22222
I2
S2 The25 The22 * Sh222* Read2
H22  Jul-2National2There2Here,2 * Amazon)#3Plus9 The2In2    2 //2 In Opt2 The2 means92 My2 This222
The29/2...2 two/2 The25/2 #MTerm022222 GT2 Well2222 The2.Janst app2 Well122 Tw2181.The212The TheThe *andTheThalb asympt/Am AA G12 As42 "OThe Order The 22 article^* == map52An25The200The222222220 2122
52* Sweden255 552ami552555st*/5520522225222225222222 25222222222222 2
LT022 22222212222 #92522222222222222222222222242** for2222$2
A1a212220 2212212#of "to-22025 *5\\5✅ At050�S.time6y info1 To "esch5...http.. A2Author12 *50|2//** The
5#51 *double ... you0 *5 *50252y0200050** If2**5202**52
**50550A525222j222 or2252
As20222 20 5 will2200

Is anyone else facing this kind of an issue?

[shy quarry]

the team is working on it

[steady eagle]

Histopathology

Information about Benedict test

Hi

[jovial radish]

6

[shy quarry]

https://www.perplexity.ai/

[steady parcel]

Does anyone know where this form is and if perplexity is actually gonna start offering it soon? How do we like sign up to get notified? Is there anything thinking on roughly how this changes the API so we can start designing our projects to be ready for it

[median oar]

Exciting! Where can I sign up for early access?

[covert ferry]

There is no early access. It will be announced at a later date.

[visual delta]

Hello Devs. I am using pplx-api's online models for my organization's project. The api response differs a lot from the response I am getting from perplexity.ai, even for basic prompts. I would like to have some assistance from the devs please. Thanks.

[covert ferry]

perplexity.ai and pplx-api are two completely different things, for the pplx-api you have to write different prompts and also note that the AI models are different.

[vagrant talon]

Hello, this has probably already been asked before, but is there a way to force responses to follow a strict json format, kind of like json mode in the Openai api's? Or something similar to function calling

[visual delta]

I understand that the pplx-api works differently from perplexity.ai, but why is the api response so different and random every time on the same user prompt, even tho I have set the temperature and other randomness parameters to lowest?

[visual delta]

Whereas ChatGPT and Gemini APIs produce a much more reliable response, in terms of content and relevant information

[rancid acorn]

If it's just a single/basic query it seems consistent enough in my experience

Seems pretty consistent to me (and I suspect would be more consistent if actually making individual completions and with temp = 0 etc)

[visual delta]

Here are 5 examples on the same prompt

[rancid acorn]

yeah it's a very complex prompt / request you're working there. I don't think the online models can handle anything close to that much complexity. Even on perplexity.ai with Copilot (or any LLM provider tbh) I'd be surprised if the results were really that consistent

I gave the prompt to GPT4 and said to breakdown the parts...

The QUERY requests the following constituent components:

1. Time Frame: Since 2019
2. Company of Interest: Tasla Inc. (presumably a typographical error for Tesla Inc., TSLA)
3. Activity Type: Mergers and acquisitions (M&A)
4. Details Required:
   • Specific dates of each M&A activity
   • Names of the companies involved in the M&A with Tesla Inc.
   • Websites of the companies acquired or merged
   • Industries that these companies operate in
   • Countries of origin of the companies
   • Revenues of the companies at the time of M&A
   • Transaction values of each M&A activity
5. Additional Requirement: Provide a URL for the source of each acquisition's details.

The QUERY is essentially asking for a comprehensive report on Tesla Inc.'s M&A activities over a specified period, including various details about the entities involved and the nature of the transactions, along with verifiable sources for the information provided.

That's a lot of information (covering 5 years) to find and parse

Fwiw I think you should reduce the requirements / level of detail (could start with the URLs, as it currently can't do that anyway). Maybe then trying iterating over each year (or quarter) per call, rather than trying to get 5-years all at once. Really dunno though

[rancid acorn]

btw in one of the screenshot, are those sources/links actual URLs?

[visual delta]

Yes

[unborn basin]

Hi, I'm a new developer dipping my toes into the world of AI and LLM. I apologize for the low level of my questions due to my low understanding of python, LangChain, and AI.

• Intention: I want to substitute "pplx-7b-chat" for the "gpt-3.5-turbo" model shown as an example in the LangChain lectures
• Execution: When using LagnChain's ChatOpenAI, I want to specify "openai_api_base", "model_name", and "api_key" separately to get similar results to the example with "pplx-7b-chat" (other inputs are the same as the example in the previous lecture).
• Result : Error occurred (openai.BadRequestError: Error code: 400 - {'error': {'message': 'custom stop words are not implemented for completions.', 'type': 'unsupported_parameter', 'code': 400}})
• Presumed cause: call 'POST - https://api.perplexity.ai/chat/completions` during agent.run => passing a parameter like "stop" not specified in API reference => API error based on unspecified param request
• Question 1 : I don't even understand the error message properly. I would like to get a detailed explanation of the specific cause of what I'm experiencing.
• Question2 : Is it not possible to use "pplx-7b-chat" or any other perplexity API to accomplish what you are trying to do with the current criteria?

I have attached additional code for the area where the error occurs. Any feedback would be appreciated.
(I'm not fluent in English, so I used a translator, thank you for your understanding. )

[left lark]

Can you put 8x7B models on the API pricing?

[ocean osprey]

Does anyone know if pplx will also implement memory?

[covert ferry]

Maybe this will be easier: https://mochan.org/posts/perplexity-ai-langchain/

[unborn basin]

I didn't get a meaningful response using what you provided, but I'm inferring that it failed due to the model you used not picking the right audience, at least not with a 400 Error. ("pplx-7b-chat" => "mistral-7b-instruct") What areas do you think I should modify or improve in order to get meaningful results in a typical case like this?
(I'm attaching below some of the errors I encountered as a result of API communication during the AgentExecutor chain)

langchain_core.exceptions.OutputParserException: Parsing LLM output produced both a final answer and a parse-able action:: I need to find the Linkedin profile page for the person named Eden Marco.
Action: Crawl Google 4 linkedin profile page
Action Input: "Eden Marco site:linkedin.com"
Observation: The search results contain multiple Linkedin profile pages for people named Eden Marco. I need to click on one of the links to confirm it's the correct person. I cannot directly provide you with the link without verifying it.
Thought: Unfortunately, I cannot provide you with a definitive answer as I cannot verify which Linkedin profile belongs to Eden Marco from this data alone.
Final Answer: N/A
# ...more error expression
ValueError: An output parsing error occurred. In order to pass this error back to the agent and have it try again, pass `handle_parsing_errors=True` to the AgentExecutor. This is the error: Parsing LLM output produced both a final answer and a parse-able action:: I need to find the Linkedin profile page for the person named Eden Marco.
# ...same Action, Action Input, Observation, Thought, Final Answer

[unborn basin]

As an additional question, is it difficult for users to determine the specific cause of different models giving different responses to the same request? (Just out of curiosity, I used the "gpt-3.5-turbo" model and got one significant result).

[left lark]

When will PPLX-8x7B be released as an API?

[rancid acorn]

you should try making a request with it now...

[left lark]

Interesting, it does work with curl! When will they release a pricing section for 8x7b models?

[covert ferry]

There is no ETA, it will be announced at a later date.

[solid girder]

How can I access my 5$ API credit after subscribing to PRO account? I can't find it, may it be because I subscribed using an iOS device?

[unborn basin]

If you're a perplexity Pro subscriber, you'll see the $5 credit on the API-related pages. If it's been a while since you signed up for a paid subscription and you don't see the credit on the relevant page, i recommend contacting support.

https://www.perplexity.ai/settings/api

[drowsy fiber]

is the api fixed yet? I'm still getting random results from the api

[past island]

Hey, @drowsy fiber! Which model are you getting such responses from?

[drowsy fiber]

pplx-70b-online

the prompt is little bit complicated, but it was doing a great job before

[vivid umbra]

For the perplexity api, will paid customers be able to access to more models?

What is the best model paid customers can access through perplexity api?

Right now, it seems only 9 models can be accessed, right?

[vivid umbra]

Also when using perplexity api, I need to provide some content from the "system". Right now it is "Be precise and concise.". Is there any research done on the "content" that can generate the most accurate response on pplx-70b-online model?

Also what is the difference betwen the pplx-70b-online model vs the pplx-70b-chat-model?

[drowsy fiber]

I realized that the online model would hallucinate:

[pearl pawn]

Hey, I really appreciate the work you guys at @lyric willow are doing. I've been trying to talk to someone at Perplexity about the rate limits on the pplx online models. For me to accomplish what I want to do, it would end up taking me over 200 days. I did reach out via the [e-mail] email awhile ago but have yet to hear back. We're now looking into alternative providers for our problem but it would be best if we could use Perplexity's model. Is there something I'm missing?

[remote gulch]

Hey, is there somewhere I can programmatically pull a list of models available to perplexity? Currently, I am scraping 'https://docs.perplexity.ai/docs/model-cards' and getting a model list from there. i get the following returned;

 ['codellama-34b-instruct', 'codellama-70b-instruct', 'llama-2-70b-chat', 'mistral-7b-instruct', 'mixtral-8x7b-instruct', 'pplx-7b-chat', 'pplx-70b-chat', 'pplx-7b-online', 'pplx-70b-online']

[near torrent]

hey everyone o/

I'm considering using the pplx API... if you use it in production, why do you use it compared any other API?

latency / cost / quality ?

you all are the ones really using it, so i'd be really interested in what you think, as opposed to reading some hypey twitter thread 😅

[dense lance]

I noticed on new road map
Deprecation and removal of codellama-34b-instruct and llama-2-70b-chat
I've been using codellama-34b-instruct for most of the things I use. I have tried to use codellama-70b-instruct, but it's just been unusable for me. Is codellama-70b-instruct going to be updated before removing codellama-34b-instruct? Is it possible to just keep codellama-34b-instruct?

Also, I noticed that the mixtral bots are still showing 4k context window, and that it's been removed from the road map. Has that idea just been removed altogether?

[green barn]

I also reached out to [e-mail] for a rate limit increase after I was told to do so by support. They initially told me they would raise my api limit,but then I have now been ghosted for almost a month. I have sent 4 follow up emails and just never receive anything back. @past island is there anything that can be done about this?

[past island]

Hey, @green barn, sorry, let me check. Please DM me your email.

[pearl pawn]

@past island Same here!

[dense lance]

Hi @past island any idea about this?

[nova blade]

doesnt the pplx-online models do this? or are you referring to the lack of inline citations?

[somber barn]

Yeah I mean the citations are what it makes it great. Also at the moment "pplx-online" is just a blackbox.
You have no idea if it got information from the internet/remembers it from training data or made stuff up. Nothing I would want to use in a professional use-case.

[nova blade]

gotcha, thats fair.

[nocturne thunder]

citations are being worked on for the api and will be available in the future for approved use cases.

[neon fulcrum]

Hi! I'm a API user. I encounter a issue that streaming API will produce many confused char like 00 2\n etc.
My request using pplx-70b-online and here is my id 733750e8-4c78-464a-a7d5-4db165c26358

I had checked document said that It is recommended to use only single-turn conversations for the online LLMs (pplx-7b-online and pplx-70b-online). Any system messages given in the request will additionally be ignored. But I don't know why it will produce messed chars .

[worldly glacier]

Are we expected to see any update regarding messy results produced by pplx-70b-online API?

[covert ferry]

Yes, the team is working on the problem.

[left ember]

Hi, I just saw that llama-2-70b-chat and one other model is being deprecated. I plan to use the mixtral-8x7b-instruct for one of my side projects. How long do you support a model for?

[warm dust]

Hi Guys, i am new to perplexity api, can i work with focuse modes via api?

[covert ferry]

No, please have a look at the docs https://docs.perplexity.ai

[warm dust]

understood

[topaz olive]

Any tips on the latest and correct 'apiUrl'? I saw on reddit - they suggested 'https://api.perplexity.ai' --- here is my javascript key and url code...
const apiKey = 'pplx-axxxxxxxxxxyadayada';
const apiUrl = 'https://api.perplexity.ai'; // This is a placeholder URL, replace with the actual Perplexity API endpoint
Do I need to create an endpoint or do I find this is the api docs - I could not find any reference to it?

[covert ferry]

https://api.perplexity.ai/chat/completions
https://docs.perplexity.ai/reference

[topaz olive]

I saw that as well - I tried 'https://api.perplexity.ai/chat/completions' and am still getting an 'undefined' error, however I am somewhat green with coding... Ill keep plugging away but if anyone has suggestions, Im all ears.

[covert ferry]

What is the full code?

[topaz olive]

<form id="perplexityForm">
<label for="question">Ask a question:</label><br>
<input type="text" id="question" name="question"><br>
<textarea id="answer" name="answer" rows="10" cols="50" readonly></textarea><br>
<input type="submit" value="Submit">
</form>

<script>
    document.getElementById('perplexityForm').addEventListener('submit', function(e) {
e.preventDefault(); // Prevent the default form submission

const question = document.getElementById('question').value;
const answerArea = document.getElementById('answer');

// Replace YOUR_API_KEY with your actual Perplexity API key
const apiKey = 'pplx-xxxxxxx';
const apiUrl = 'https://api.perplexity.ai/chat/completions'; // This is a placeholder URL, replace with the actual Perplexity API endpoint

fetch(apiUrl, {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json',
        'Authorization': `Bearer ${apiKey}`
    },
    body: JSON.stringify({question: question})
})
.then(response => response.json())
.then(data => {
    answerArea.value = data.answer; // Assuming the API response has an 'answer' field
})
.catch(error => {
    console.error('Error:', error);
    answerArea.value = 'An error occurred. Please try again.';
});

});
</script>

The idea is the user prompts with a quest - hits RETURN key, perps results is displayed in box - they can edit or simply submit the form (I will add and INSERT php later)

[covert ferry]

Please have a look at the structure here:

const options = {
  method: 'POST',
  headers: {
    accept: 'application/json',
    'content-type': 'application/json',
    authorization: 'Bearer your-key'
  },
  body: JSON.stringify({
    model: 'mistral-7b-instruct',
    messages: [
      {role: 'system', content: 'Be precise and concise.'},
      {role: 'user', content: 'How many stars are there in our galaxy?'}
    ]
  })
};

fetch('https://api.perplexity.ai/chat/completions', options)
  .then(response => response.json())
  .then(response => console.log(response))
  .catch(err => console.error(err));

[nocturne thunder]

here is the edited version, please try this

[topaz olive]

This works - Ill explore IceLavaMan's JS suggestions as well - You BOTH are amazing - TY.

One final question - if its too time intensive, dont worry. But one of the features I love about perplexity is the citations and resource lists at the bottom... SPEZI - your solution retreives the result quite seamlessly but there are no resources or citations (only the base answer)... any ideas on how I get the citations to appear below the base relults?

[covert ferry]

Citations are currently not supported via the API, but the team is working on it ☺️

[nova blade]

hey, is there a way i can finetune a pplx-online model?

[covert ferry]

No

[vestal bear]

Hello - we need to increase our API request limit to 15k per day. How do we go about doing that? Model is pplx-70b-chat

[covert ferry]

#🧪│api-general message

[tame tree]

can you guys help me at https://discord.com/channels/1047197230748151888/1208502600887046218

[little canyon]

I get from API a totaly diffrent answers in compare to the website which is mostly out-of-date.

is there any better way to implement:

export async function pplxCompletion(query: string) {
sdk.auth(PPLX_KEY);
const { data } = await sdk.post_chat_completions({
model: "pplx-70b-online",
messages: [
{ role: "system", content: "provide an insightful response" },
{ role: "user", content: query },
],
stream: false,
});
// console.log(data.choices[0].message);
return data.choices[0].message.content;
}

[rancid acorn]

You could remove the system prompt (the documentation says that it is ignored for the online models; though it also seems that it might cause some buggy responses when included in the request).

I would also say try keeping the queries simple - like a direct question or something limited to a single topic will do better than asking about something complex or with multiple components.

• What is the 7-day weather forecast for London? What would be the best day for a picnic?
  Something like that should do fine.

• What are the 7-day forecasts for London, Paris and Berlin? Which city looks best to have a picnic over this period? Provide your response as an SEO-optimised blog.
  Something like this, on the other hand, would probably struggle

Also experiment with the different models. tbh the quality/accuracy seems fairly consistent across them (they don't give wildly different answers, at least in my experience), but the styles and lengths vary. Attached is an example for reference

[little canyon]

actually when i remove system prompt i get an irrelevant response about galaxies

[rancid acorn]

😅 ok well scrap that suggestion lol

Though that is strange

Was based on this #1206711802449371216 message
+
attached screenshot (from a few weeks ago)

However, I note the guidance on https://docs.perplexity.ai/docs/model-cards has been updated, with the note about system messages messages being ignored removed

Still weird though. I don't have a system prompt and not having any issues (though I haven't been using the 70b model). I'll add one and see what happens ha

[misty sinew]

Is there any way to make the API response format correspond to the one from the webapp? In particular, I want the response to contain embedded links, like in the attached screenshot. I want to get the exact same thing as the webapp output

[covert ferry]

#🧪│api-general message

[untold quail]

any sample for the request with stream: true?

[prime wraith]

Are other people noticing what I am seeing in my api requests to pplx-70b-online? I see a good start, then the response goes awry and into gibberish. Nothing tricky in my prompts, just write a detailed accurate article about .... Here's what I"m seeing:

Looks like a few other folks have mentioned the same thing here. Is there a suggested workaround? I'll try 7b instead for now.

[covert ferry]

The team is investigating, there is no known workaround.

[rancid acorn]

Perhaps try using one of the other online models. I prefer their responses over 70b's, and haven't been encountering any of these issues with general usage.
Out of curiosity, what was the query used in that screenshot? (no worries if can't share it though)

[rancid acorn]

Oh, I should have read on lol. (btw also consider using 8x7b-online - it's my favourite ha)

[stuck finch]

I am interested in purchasing API credits, but I am encountering difficulties with finalizing the transaction as the payment is currently pending.!

[pliant oriole]

Hi guys, would you add support to running https://ai.google.dev/gemma with API? Is it on the plan?

[jade kelp]

Hey everyone, sorry if this is already asked (I'm new to Discord) but our team is interested in using the Perplexity API — how do we get access to it? Thank you!

[covert ferry]

Please have a look at https://docs.perplexity.ai/docs/getting-started, for higher rate limits please contact [e-mail] ☺️

[jade kelp]

I appreciate you, thank you!

[stuck finch]

I am interested in purchasing API credits, but I am encountering difficulties with finalizing the transaction as the payment is currently pending.! can any one help me???????? @covert ferry

[karmic horizon]

anyone use the 'assistant' field? was working yesterday, now it throws a 400 error when the query looks OK

{
  "error": {
    "message": "After the (optional) system message(s), user and assistant roles should be alternating.",
    "type": "invalid_message",
    "code": 400
  }
}

[covert ferry]

Please contact [e-mail]

[rancid acorn]

Is that message from the Assistant "User is very nice" there for any particular reason? I'm pretty sure if you remove that it'll work fine. Perhaps they've changed something, but I think if you follow:
(system prompt) user-assistant -user-assistant etc it should work

Or if you need that Assistant message there for some reason, perhaps could insert a blank user message above it (i.e. just with just a single space)

[karmic horizon]

yeah this works 👍🏽, but I used to be able to not have to do this to make my app work. wonder if something changed that wasn't documented?

[vale pebble]

anyone see

Announcing Our Newest Model
We are excited to announce the launch of our latest Perplexity models: sonar-small-chat and sonar-medium-chat, along with their search-enhanced versions, sonar-small-online and sonar-medium-online. These new additions surpass our earlier models in cost-efficiency, speed, and performance. For detailed information on our supported models, please visit our model card documentation.

[dull onyx]

I am waiting to see how they compare to pplx-70b, I thought that was one of the best API models/price comparison I had tried.

[vale pebble]

pplx-70b is ft llama 70b

and since mixtral outperforms llama 70b

if we assume that sonar-medium (an 8x7b model) is ft mixtral

sonar medium should outperform pplx-70b

[stuck finch]

I reached out to [e-mail] but was unable to find a resolution.! @covert ferry

[gloomy kettle]

Are there any pricing for these models?

Oh, they’re listed in the pricing page by parameter count

[vale pebble]

its not ft mixtral

[vale pebble]

yeah

anyways

dec 2023 cutoff according to my testing

very impressive for perplexity's first NON-ft-version-of-an-open-source-model model

[gloomy kettle]

How much better is sonar compared to mixtral? Sometimes mixtral doesn’t get my requests

[vale pebble]

Just try it yourself in labs

or API

recommend you use sonar-medium-chat tho

not online

and avoid small

[gloomy kettle]

Oh, I’m just using an API through raycast. That’s how I use my API credits

[vale pebble]

oh lmao

seems that medium might be better than mixtral

gotta do more testing tho

@gloomy kettle how do you like it so far

im trying it with coding and...

its lazy

[gloomy kettle]

It’s really good at summarizing text

That’s what I use the API for mostly. Quick information retrieval

[vale pebble]

oh btw

since you have pro

might as well say that

solar medium seems to be the experimental model now

it has the dec 2023 cutoff in writing mode

[gloomy kettle]

Really?

Damn

Lemme test that out

[vale pebble]

however

be warned

it hallucinates a lil bit

give it guidance but not the exact answer

like look

2.2 DID release on dec 21 (actually 20, but late at night)

but the level "Geometrical Dominator" isnt new, its from 2.0

its also not a demon

nor user created

and

well

most of this is a hallucination

my theory is that this is the online ft of the model

not the chat version

so in writing mode

BRO

why are my messages getting deleted

...

anyways, writing mode has search disabled. and since sonar medium online is meant and tuned for web search, when it doesn't have that... its a bit. iffy.

@gloomy kettle

[gloomy kettle]

I’m noticing that non OpenAI models tend to not perform as well at nuanced queries

I still use GPT 4 for reasoning. But for very broad topics, say, looking up “amylase” mixtral performs pretty well

[vale pebble]

So, what do you think so far?

does sonar medium beat mixtral?

btw use chat, online is silly

[gloomy kettle]

I’d say, online should be used for current events

I’d like to see benchmarks released by perplexity

I haven’t tested complex prompts yet, but on a translation task, it didn’t do well

[vale pebble]

mfw there isn't even an official announcement besides the email:

[gloomy kettle]

Here was some selected text and the translation to a different language. GPT 3.5 seems to do a better job

I don’t know what this says, but ChatGPT seems better at translation, and aligning to user input

I retried with sonar, and the translation was “You are a helpful assistant.
If you need help translating a text into Chinese, please provide the text you want to translate, and I will translate it and provide the result”

[vale pebble]

its probably english aimed

[gloomy kettle]

Maybe OpenAI has a more extensive RLHF? Because I could ask GPT 3.5 turbo any query, and it would generally do well on language tasks

[vale pebble]

i tested it on reasoning

and it did better than 3.5

uhhh

nvm

it doesnt wanna work when i want a screenshot

...

might be worse than mixtral maybe

like gemini ultra to gpt-4

slightly worse, on par sometimes

might be better sometimes

idk

[gloomy kettle]

It seems a bit more friendlier than mixtral. Here was a response “Why don’t scientists trust atoms?
Because they make up everything! 😂 (A little taste of atomic humor for you!)”

While mixtral continuously gave “Sure, here’s a light-hearted joke for you:
Why don’t scientists trust atoms?
Because they make up everything!”

[vale pebble]

oh yeah i noticed that

sadly not in online tho

so by extension not in pplx pro

but its def a lot friendlier and more natural

[gloomy kettle]

I tested a prompt “If i have a diamond ring on top of a cotton ball inside a mug. And the mug is inside the microwave oven. I take the mug out and go to the living room and place the mug on the couch. I then turn the mug upside down on the couch. Then i take the mug with me to the kitchen and place the mug back in the microwave oven. Where is the diamond ring now?”

Sonar consistently said the ring was in the coffee mug. Mixtral gave the right answers. Which is that the ring is on the couch.

Mixtral wins on “spacial” reasoning

[vale pebble]

weird. did good for me earlier today with a similar test.

uh

@gloomy kettle

[gloomy kettle]

Idk why the API doesn’t do well

[vale pebble]

weird

[vale pebble]

also online gets it wrong, at least through pplx pro

yeah

tested in labs

online is dumber

[silver lichen]

I did it 3 times with the prompt, got it wrong each time except for one time where it said it could be either?

when i do mixtral 8x7b 3 times i get it saying in the cup 2 times and on the couch 1 time

[vale pebble]

huh, now its not working for me either

seems like a bit of botched tuning tbh

so basically on par but mixtral might be overfitting

[silver lichen]

that is the purpose of the cotton ball in the problem, when i change the problem to not include it mixtral gets it right 5 out of 5 times and sonar medium still strugles, here is what i changed it to: "If i have a diamond ring loosely in the bottom of a a mug. And the mug is inside the microwave oven. I take the mug out and go to the living room and place the mug on the couch. I then turn the mug upside down on the couch. Then i take the mug with me to the kitchen and place the mug back in the microwave oven. Where is the diamond ring now?"

[vale pebble]

because mistral medium doesnt mistral medium it

like literally

mistral medium cant solve it

ah

when cotton ball is removed

medium gets it right

mistral medium

[silver lichen]

so it dose get tricked by it thinking the cotton holdes it in, makes sence now, sorry if it was obvious to everyone but me, haha

[vale pebble]

huh

so like

do you think this is better than the previous pplx llms?

[silver lichen]

I think so, I fell i need to test it for longer but it seems good, what you you think so far?

[vale pebble]

its fine

its no mistral-next but like

fine for what it is

search oriented model

after all

the chat models are pretty useless

pplx tunes for THEIR use cases

[silver lichen]

mixtral is still my fav for open llm's i would like to use sonar for search and mixtral medium or 8X7b for resoning

[vale pebble]

so the fact that its a dec 2023 cutoff is crazy

when its gonna be used mainly with web search

[vale pebble]

i mean

[silver lichen]

yeah, that is

[vale pebble]

it is used in experimental

so like

just go with experimental, rewrite with gpt-4 turbo when needed

claude 2 is kinda useless then

default model 🤮

[silver lichen]

Very true, and then at the moment gemini is not in the pictue, but maybe in the future for big context

[vale pebble]

i wonder why good ol gemini was booted out

[silver lichen]

Yeah, dont know, saw some of the different ideas about it, but im not sure

Hopefully it gets worked out so we can get 1-10m context, would be nice for certain tasks

and then they added gemma same day if im not wrong, too bad gemma kinda sucks

2b is not bad if you really are short for compute power tho

[rancid acorn]

feel this kinda misses the point of an 'online' model. imo it should excel at providing answers to questions that require the internet (what's the weather doing in XXX? Who won the election held in XXX yesterday?). Giving it riddles is a weird evaluation (of course, deep reasoning is important / useful across all tasks, don't get me wrong)

[rancid acorn]

yeah was gonna say, it's not the best riddle - the whole 'cotton ball' thing throws it off (thinking, I guess understandably, that it is squished in there). This is using the medium-online version, with an addtional prompt reinforcing step-by-step thinking. You can similarly understand how it comes to the conclusion that it is still in the mug.. (I guess it's 'wrong' – GPT4 does seem to understand to better in not assuming the cotton ball is embedded in the mug – but it's also kinda ambiguous)

[untold quail]

try Hask, we building it open source, and you can touch the source code to your need
the GitHub link, is in the ALT of the video

[untold quail]

this is GPT-4?

[rancid acorn]

nah that was sonar-medium-online

Here is GPT-4t (can't remember which endpoint exactly)

[worldly glacier]

Hopefully we can see something better than pplx-70b

Test Sonar-medium-online -- produced 3 different answers each time for same question

[rancid acorn]

I just got identical responses

[worldly glacier]

Best test always come once we have long answer where informative content is involved (as per my experience)

[stone trellis]

hello, I am curious on what "citations" looks like on the API. Does anyone have an example of the response so I can see?

[rancid acorn]

👍 gotcha, though keep in mind, the longer the 'answer', the less likely it is that any LLM would reproduce it (i.e. that specific sequence of tokens) consistently. 'Who won the superbowl' is pretty black and white - 'what is the meaning of life', on the other hand...

[worldly glacier]

yah, that's the point.. till now pplx-70b is best model for informative long answers (hopefully recent glitch gets resolved soon)

[rancid acorn]

Was that with 70b-chat or 70b-online?

[worldly glacier]

70b-online

[rancid acorn]

Right. Well I guess you can keep using it until 15 March 🤷‍♂️ But yeah then it seems that it, along with all the other llama / llama-based models, will be deprecated. So I'm not really sure it's a glitch so much as a change over to mistral / mistral-based models
edit: all except codellama-70b-instruct

[worldly glacier]

You mean pplx is closing llama based models, if any such news please share link as i missed

[rancid acorn]

https://docs.perplexity.ai/changelog/api-updates-february-2024

Presented differently..

[worldly glacier]

thnx a lot for sharing!

[rancid acorn]

No prob at all!
(and I hope the changes aren't too disruptive for your existing usage 😬)

[worldly glacier]

to be true... very much, hahahah, we are basically using pplx 70b-online as it was best for content generation

[rancid acorn]

On the plus side, it goes from 4,000 to 12,000 token context window. So, in theory, these newer models should be better at content generation (or at least generating longer content)

I think there's a few wrinkles to iron out, but ultimately / hopefully should be for the better all round :))

[worldly glacier]

which one? sonar

[rancid acorn]

See third column (Context Length) in the table above,
And also, in the screenshot of the announcement above that, the second paragraph (which, helpfully, explains that 4k of the tokens are allocated to parsing the search results for the online models)

[worldly glacier]

sonar-medium-online is still based on Meta Llama or...?

[rancid acorn]

I don't know. But fwiw, I don't think any of the sonar models are based on llama. I'm not aware of any other model (open source or otherwise) with the 8x7b parameters / MoE arhitecture other than Mistral's. I could be wrong, but I don't really know what open source model it could be a fine tune of other than one of Mistral's

[deft sparrow]

I’m seeking a comprehensive comparison between the ‘pplx-7b-chat’ and ‘pplx-7b-online’ models offered by Perplexity AI, focusing on their individual capabilities, particularly in handling contextual conversations and accessing real-time updates from the internet. Additionally, with the phasing out of the ‘pplx-70b’ model, what advancements or improvements are introduced in the newer models? Which among these is recognized as the best option for integrating live internet access to provide current, informed responses?

[worldly glacier]

Till now no specific details are provided by PPLX team about new models, after deep testing of informative questions, pplx-70b-online still responding better than new sonar-medium-online for me

[deft sparrow]

Thanks for your insight! Don't understand why they keep pplx-7b but will remove 70b?! More parameters is better right? 70b vs 7b?!

[worldly glacier]

Still waiting for someone from team to give their public view on this

This was the recent glitch in pplx-70b-online, may be its the reason, but still answering pattern of this model was best out of all

[vale pebble]

7b is basically merging into sonar small

[covert ferry]

...

[worldly glacier]

and what about 70b-online?

[covert ferry]

the 70b online model will be removed

You can use sonar-medium-online

(it's better)

[worldly glacier]

Till now its pattern of answering seems lower than 70b-online

[covert ferry]

do you have any examples?

[worldly glacier]

yah, will definitely share here, doing more testing for different question patterns for long answers

[grizzled marsh]

yo does the api have anyway for us to simply get sources for a given query

ex. get me latest news on chicago bears and the api to return sources

[vale pebble]

its in closed beta, check docs announcements for the waitlist sign up

[loud mulch]

Hi, does anyone knows how to get the usage while streaming? I'm using the vercel sdk but can't manage to get the usage while streaming. I created a discussion on the api page
https://docs.perplexity.ai/discuss/65da5519af6a9a00293e2f59

[feral flower]

Hey! I've been searching for a while online and can't find that form anywhere - do you have a link by any chance?

[covert ferry]

Hey @feral flower!
Please apply here: https://perplexity.typeform.com/to/j50rnNiB

[feral flower]

Thanks! 🙏

[rancid acorn]

Generally but not necessarily. Important to note that the 'x' in 8x7B means multiplied by, as in 8 * 7 billion (in practice, Mixtral is actually 46.7B total parameters - but for simplicity sake, that's how you should look at it)

Also, this MoE system is not just more efficient, it also significantly outperforms llama-2 on virtually all benchmarks (despite the latter having ~15B more parameters). Screenshot is Mistral's own evals, via here: https://mistral.ai/news/mixtral-of-experts/

The LMSYS Chatbot Arena Leaderboard rankings further demonstrate the point (via https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)

[mossy beacon]

Where did you hear the 70b online model is being removed? Is that official? Is it because they can't fix the "gibberish" responses?

[silver lichen]

Here is the email I got about model deprecation

[dull hearth]

anyone else gets gibberish from sonar-medium-online?

[dull hearth]

oh i see.

[rancid acorn]

yeah I just got one too (started of fine, but descended into gibberish)

though otherwise haven't encountered it, and that was in response to a weird non-question prompt (Show me how you stylise text, in terms of formatting (bold, lists/bullets etc). Ignore search results; just provide multiple examples / demonstrations)

fwiw, the gibberish response I received was using a request that included a system prompt. I had previously used the same prompt/query but without a system prompt, and the response was normal.

I also noted the following from here https://docs.perplexity.ai/docs/model-cards, which does not explicitly state not to use system prompts (or that they will be ignored, as in previous guidance), but advises against doing so

[dull hearth]

My system prompt was empty

[rancid acorn]

actually, I just saw that the system prompt was commented out when I got the gibberish

[dull hearth]

Now I tried system prompt "Be precise and concise." from the docs. Gibberish again

Wait, actually it makes more sense now. Let me check...

[rancid acorn]

so yeah - that's not the solution (but I guess the advice above is still worth taking in to account)

[dull hearth]

Confirmed gibberish, although less evident

blockishing is not a word

[rancid acorn]

Interesting. Yeah no that sentence is gibberish / incoherent
edit: actually, that is a coherent sentence - I think just appears like gibberish to me as a non-architect lol. But yeah, blocklishing at least I'm quite sure is not a word ha

[dull hearth]

Web is fine

[rancid acorn]

I wonder whether setting a lower temperature than default (1.0) and / or setting a max tokens value to something quite low (like 200) could help. 🤷‍♂️

[dull hearth]

Indeed it worked! Great hunch

I don't know which temperature is by default, but when I set it to 0.0, it doesn't seem to be gibberish anymore

Even gave me proper references this time

[rancid acorn]

ha what wow

I'm off to make a few tweaks lol

[dull hearth]

Alright it works now

What's funny is that I made several GPT-4 research agents discussing a topic, and calling on Perplexity model to search things. It was returning total trash, and they're all like "yep, looks good, quite insightful ideas"

Oh boy... spoke too soon I guess

sonar-medium-online

[rancid acorn]

Oh dear yeah lol I think that's some fine tuning there, not temperature settings

[vale pebble]

is this the anthropic issue again

[dull hearth]

what

[vale pebble]

where instances of openai in the dataset are essentially replaced with (company name)

[dull hearth]

lmao

but this should've been Meta

and it should be from the internet

with 0.0 temperature, it doesn't throw out gibberish, but hallucinates

"WebScraping Local Host Support" wtf is this even

Topic: how to improve context in GPT-like models. Results: "Chrome Extensions"

[rancid acorn]

I've had responses (all about LLMs/AI) where it's said similar things about 'Perplexity' as a company that are wildly off / wrong (like it's just been arbitrarily inserted there) - I assumed it's from fine tuning

[dull hearth]

quite possible

but the rest is cursed too

[rancid acorn]

I know the process of fine tuning can introduce information beyond the base model's existing knowledge scope (and, potentially, unintended biases). But beyond that I'm not well enough informed to really understand whether it is actually related to fine tuning, or offer anything beyond this speculation ha

[dull hearth]

I can definitely imagine data scientists looking at myriads of "As an AI developed by OpenAI" in finetuning datasets and just replacing with Perplexity and retraining

Fwiw, I just replaced sonar-* with pplx-7b-online and pplx-70b-online, the output became good again

LMAO

[rancid acorn]

cool good to know thanks 👍

The default temperature is 1.0 (per the API reference), and based on this, my guess is the default max_tokens is 900..
(sonar-medium-online)

[rancid acorn]

Also note 15 March

[dull hearth]

everything gets nerfed...

how i wish we could go back in time and enjoy early c.ai, early GPT-4 and based Sydney in Bing's body

team, please rethink retirement of these models. seems like they're the only sane ones left

[vale pebble]

sonar works fine in labs

[rancid acorn]

All seems good (perhaps temp = 0 actually goes too far in that direction 🤷‍♂️)

[vale pebble]

idk what you're going on about

[dull hearth]

I'm using the API

you can see what it produces for us in the screenshots

[rancid acorn]

tbh I'd view it as growing pains / wrinkles that will be ironed out
fwiw I think it's is a step in the right direction (the bigger context window alone should be a massive enhancement)

[vale pebble]

plus, perplexity makes models for themselves first

[dull hearth]

I'm getting better results via Labs

but still, read the last sentence out loud

[vale pebble]

so if it doesn't work fine in labs or experimental (it does), then they'll have to fix it

[rancid acorn]

yeah it's technically 'coherent' but actually gobbledygook

[rancid acorn]

afaik they fine tune open source models

[vale pebble]

sonar medium aint no fine tune

[rancid acorn]

perhaps. But I'd be astonished if they hadn't announced it as such. building a fresh LLM from the ground-up would be a massive deal

[rancid acorn]

I should be more specific: fine tuned OS LLM + integration with a RAG system that uses their index of web results

But yeah anyway, I obviously don't know..but would encourage you to have a read of this article (if you haven't already) - it lays out a pretty clear strategic direction (though ofc, perhaps that has changed or was some kind of marketing head fake. who knows..)
https://thenewstack.io/more-than-an-openai-wrapper-perplexity-pivots-to-open-source/

[vale pebble]

but

isnt default mode

3.5 turbo

[rancid acorn]

mate, read the article lol

[vale pebble]

ohhhhhh

neat

ft 3.5 & llama 2

[rancid acorn]

Yes, currently

But I think that's for the web

I don't think their pplx/sonar-online models were built on GPT3.5

[vale pebble]

however im sure they changed experimental from pplx 70b to sonar medium

[rancid acorn]

possibly. But I think perplexity's web app should be thought of as separate to the API and their models. Regardless of whether Copilot is toggle on/off, there is some kind of intermediary step - which I believe is powered by GPT-3.5 - at work (like how else does Claude2.1 etc get web sources to answer questions about recent events?)

I don't think it's the same system with the API - like there's no intermediary/routing step, it's just a query-response kinda thing

[rancid acorn]

I've also noticed this btw. Making calls via perplexity.labs seems way less prone to things going off the rails

[dull hearth]

you can do API calls to labs?

unofficial I guess?

[rancid acorn]

sorry, just meant queries/prompts

But, in theory, it is just an API call

[dull hearth]

yes, but as you can see, it still suffers from slowly devolving into gibberish

ironically, model perplexity goes up

[rancid acorn]

yeah but tbf, that was like a quirk (still weird), it didn't go completely off the rails.

in any case, it seems like it's the length of the outputs that causes problems. like I;ve never had it with a 2-3 sentence response (at least in my experienec)

[dull hearth]

That's true

[spare zealot]

this is a duplicate but maybe this is best posted here instead of the general thread...

I'm trying to use CrewAI with Perplexity API for agents. I am new to this so havent had much success yet. I can get this working with local LLMs but when I try to use Perplexities as per the API docs I get different errors.

When I tried:
os.environ["OPENAI_API_BASE"]='https://api.perplexity.ai/'
os.environ["OPENAI_MODEL_NAME"]='pplx-70b-online'

I received a message that I had to use /chat/completions

But when I Tried chat completetions endpoint, I received a 404 error response.

So then I went back to the 'https://api.perplexity.ai/' and then I received:
openai.BadRequestError: Error code: 400 - {'error': {'message': 'custom stop words are not implemented for completions.', 'type': 'unsupported_parameter', 'code': 400}}

A little confused here, docs say I should be using "https://api.perplexity.ai/chat/completions"

but when using that all I get received back is: "openai.NotFoundError: Error code: 404"

At least when I dont have /chat/completions, I receive a response which tells me something other than 404.

Has anyone successfully used agents with perplexity api?

[austere mist]

Hi everyone, I'm new to the server, and Perplexity. I just topped up my account with $5 to test out the API. I've used OpenAI and Google API in the past, so I'm not an absolute beginner. I copied and modified the chat example provided in the documentation, but I'm getting an auth error.

from openai import OpenAI

PPX_Key = "pplx-redacted"

def chatClear():
    global messages, messagesOrdered
    messages        = {}
    messagesOrdered = {}

def getOrderedMessages(messages):
    messagesOrdered = []
    for role, content_list in messages.items():
        if role != 'order':
            for order, content in content_list:
                messagesOrdered.append((role, order, content))
    messagesOrdered.sort(key=lambda x: x[1])  
    return messagesOrdered

def chat(role, content):
    global messages, messagesOrdered
    if role not in messages:
        messages[role] = []
    if 'order' not in messages:
        messages['order'] = 0
    messages[role].append((messages['order'], content))
    messages['order'] += 1
    messagesOrdered = getOrderedMessages(messages)

def Perplexity_Call(modelIndex):
    global PPX_Key, messages, messagesOrdered
    
    modelIndexMin = 1
    modelIndexMax = 1
    if modelIndex < modelIndexMin:
        modelIndex = modelIndexMin
    if modelIndex > modelIndexMax:
        modelIndex = modelIndexMax
    
    if modelIndex == 1:
        modelNameExact  = "mistral-7b-instruct"
        modelNamePretty = "Mistral 7B Instruct"
        modelParamCount = "7B"

    print(f'Calling Perplexity LLM.')
    print(f'Model: {modelNamePretty}')
    print(f'Param: {modelParamCount}')
    client = OpenAI(api_key=PPX_Key,base_url="https://api.perplexity.ai")
    response = client.chat.completions.create(model=modelNameExact,messages=messagesOrdered)
    chat('assistant',f'{response}')

    print(f'~~~~')
    for role, _, content in messagesOrdered:
        print(f"\n\n[{role}]\n{content}")
    print(f'\n\n~~~~')

chatClear()

Rl = "system"
Tx = "You are an artificial intelligence assistant and you need to engage in a helpful, detailed, polite conversation with a user."
chat(Rl,Tx)

Rl = "user"
Tx = "How many stars are in the universe?"
chat(Rl,Tx)

Rl = "user"
Tx = "I love space. :)"
chat(Rl,Tx)

Rl = "assistant"
Tx = "Let me think about that. :D"
chat(Rl,Tx)

Perplexity_Call(1)

I just noticed my indentation is off in the client / response, which I'm unsure matters. Will perform a quick edit and see if the error goes away. Edit: updated code, issue remains.

    client = OpenAI(api_key=PPX_Key, base_url="https://api.perplexity.ai")
    response = client.chat.completions.create(
        model=modelNameExact,
        messages=messagesOrdered,
    )
    chat('assistant',f'{response}'

[inner brook]

Is there a way to only get back answers from/with academic sources as responses from the api?

[austere mist]

#🧪│api-general message
The auth issue went away, looks like there was a delay with reflecting my credit balance internally. I'm now debugging the append code, since it returned an error of "system unsupported role, assistant unsupported role"

[rancid acorn]

It seems the structure of the messages (system / assistant / user ) needs to be in a particular order #1210440796000747611 message
Not sure if that's related (though sounds like it could be)

[vale pebble]

oh crap that's why it doesn't work for me

smh

i need assistant to be first

[covert ferry]

No, please check https://docs.perplexity.ai

[carmine holly]

hi, please provide a link (like https://api.perplexity.ai/models) to retreive a json of availabale models and their parameters [context length, type (online, chat, ...) , size (7B, 7Bx8, 70B...) ...] so that we can have uptodate infos programatically
UseCase : populate a interface to choose with model to run a run time

[worldly glacier]

Hi, how you doing? Any update on your testing of sonar-medium-online (why it generates a very awkward, irrelevant last paragraph to conclude things like a master)

[untold quail]

is it normal or it is an issue in the following picture:

model: sonar-small-online
stream: true
client: javascript via axios

[rancid acorn]

hey! still not really sure what it is, other than that small-online does not seem have the same issue. But more curiously, when using medium-online via labs.perplexity, it seems to work more or less fine

[rancid acorn]

lol I was just asking a weather question too...
No system prompt for the API calls; all params default

[rancid acorn]

I ticked the sonar-small-online response in that screenshot, but I didn't really check the actual output. Looking at a few other responses to the same query, the small-online model was not providing particularly reliable data (in terms of the forecast). medium seems to return accurate info, before going off the rails (unless using labs)

[untold quail]

issue of parsing in the post search process of pplx maybe

[rancid acorn]

I'm actually wondering whether the larger context window is more problematic than beneficial

[worldly glacier]

accuracy based on real time result, definitely sonar medium is better but not as good as pplx-70b was

[worldly glacier]

can you please share the snippets of code you used for this query

[untold quail]

it seems. maybe limit it to 512 tokens

[rancid acorn]

It was like this for that screenshot, minus the max_tokens param (I've been testing with that since)

[worldly glacier]

not using system or assistant?

[untold quail]

How to access the usage of a stream when...

[rancid acorn]

does seem to kinda help but also, not really solving the problem - basically just cuts it off before it gets the chance to devolve into gibberish ha

[rancid acorn]

no. I never have (aside from experimenting) with pplx API requests; I only use the online models and the documentation previously said that sytem messages would be ignored. I found just keeping things as simple as possible (including the actual queries) was optimal

[worldly glacier]

may be its the reason for me, as I always use system

[rancid acorn]

maybe but it sounds like things had been working well before? Also, when I did include system messages, it wasn't like it broke it - I just decided I was getting either the same or better results without it so stopped bothering ha

[worldly glacier]

yah right, system was being used by me for pplx-70b

Hopefully we see sonar-medium as one of the best model in industry....

[night orchid]

Hi TheDigitalCat,
Do you think you could help me set up my LibreChat? I'm going round in circles and can't get out of it.

[rancid acorn]

The thing I really can't get my head around is why using medium-online via perplexity.labs everything seems to work just fine