Using RAG agents with a local hosted LLM

[gilded berry]

Using RAG agents with a local hosted LLM

Whenever I try to use the example for RAG (https://microsoft.github.io/autogen/blog/2023/10/18/RetrieveChat/) but while hosting an LLM locally, I get this error: openai.error.AuthenticationError: No API key provided. You can set your API key in code using 'openai.api_key = <API-KEY>', or you can set the environment variable OPENAI_API_KEY=<API-KEY>). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path = <PATH>'. You can generate API keys in the OpenAI web interface. See https://platform.openai.com/account/api-keys for details.

[gilded berry]

Using RAG agents with a local hosted LLM

[mystic tartan]

According to this page: https://microsoft.github.io/autogen/blog/2023/07/14/Local-LLMs/
And also here: https://github.com/microsoft/autogen/issues/46
You need to provide something to the api_key field. Either "NULL" or "sk-111111111111111111111111111111111111111111111111" or whatever suits you
You also need to be one of the pre release version (don't know which one, take the latest to be sure)

[gilded berry]

my API key is already "sk-111111111111111111111111111111111111111111111111" in config_list, I've been able to run autogen without RAG

[gilded berry]

updated to the latest release,
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
this is the current error

[rigid bridge]

hello, were you able to get it to work? i'm facing the same issues!

[gilded berry]

I haven't, but I still haven't tried with the latest release, only the pre release

[rigid bridge]

i'm sorry but i'm a bit of a newbie! when you say the pre-release, you mean the pre-release of what exactly ? thank you

and which version of the openai package are you using btw ?

[gilded berry]

think of pre release as sort of beta versions of a new autogen release