Recently updated the docker image of LibreChat and found that the interface throws an error message if the message tokens exceed 4096. This was 8192 earlier. Was wondering if this is configurable as I use the 128K model and this is a bit restrictive for me.
#Maximum number of tokens for gpt4-turbo on Azure
fringe orbit
nova glen
Is the model called “gpt-4-turbo” when you select it?
fringe orbit
I'm using Azure OpenAI deployment. The model deployed on Azure is gpt-4-turbo, Here's a screenshot from Azure:
Azure says this should support 128K -
and it does when I use it via the API
nova glen
@fringe orbit I see but when you select it in LibreChat, is it just “gpt-4” in the drop-down or something else?
fringe orbit
It says GPT-4. There is no gpt-4 turbo option there.
My config file is like so: AZURE_API_KEY=MYKEY
#AZURE_OPENAI_MODELS=gpt4-turbo
I had commented the AZURE_OPENAI_MODELS line. Should that be uncommented?
nova glen
My config file is like so: AZURE_API_KEY=MYKEY
#AZURE_OPENAI_MODELS=gpt4-turbo
I...
You should name your deployment “gpt-4-turbo” and then name the model AZURE_OPENAI_MODELS=gpt-4-turbo
Also maybe it worked for you before but you need to follow the guide here on using deployment names as the model names: https://docs.librechat.ai/install/configuration/ai_setup.html#azure-openai
fringe orbit
ok. thanks. Wondering if the new Azure config setup that you're working on will fix this? Or will we have to have deployment names = model names even then?
nova glen
ok. thanks. Wondering if the new Azure config setup that you're working on will ...
it will fix it!
as long as on the app-side, they correspond to the expected model
but the azure configuration can be whatever
fringe orbit
Hi. Sorry to be back again. But I tried this and it still doesn't work. Here's my .env file: AZURE_API_KEY=MYKEY
AZURE_OPENAI_MODELS=gpt-4-turbo
AZURE_USE_MODEL_AS_DEPLOYMENT_NAME=TRUE
AZURE_OPENAI_API_INSTANCE_NAME=openai-gptturbo-vision
AZURE_OPENAI_API_DEPLOYMENT_NAME=gpt-4-turbo
AZURE_OPENAI_API_VERSION=2023-07-01-preview
Here's my Azure setup:
Librechat only shows gpt-4:
I did a docker-compose up --build after changing the .env
nova glen
Librechat only shows gpt-4:
It works for me
if you are using Redis, you need to flush the cache whenever you make changes (or disable it).
fringe orbit
Ok, did a docker-compose build with the -no-cache option and now I get gpt-4-turbo in the options
but the token limit still comes up
nova glen
Ok, did a docker-compose build with the -no-cache option and now I get gpt-4-tur...
ok we're almost at the bottom of this. I see the issue, I'm going to push a patch to it now
but setting to gpt-4-turbo was necessary
fringe orbit
Super. Thanks for the response. Once the patch is done, I need to do a git-pull and then build with no-cache again, right?
nova glen
Super. Thanks for the response. Once the patch is done, I need to do a git-pull ...
yes! Sorry about that, going to put in a unit test as this was something I missed
fringe orbit
No apologies necessary! Thanks for the great work on this app