Hi.. I have been using “gpt-4” for evaluation and noticed the “evaluator” performance drop suddenly. The documentation for the api says the api still points to “gpt-4-0613”. I’m not worried about text variations rather reasoning and drastic scoring changes.
Thanks.
#was gpt-4 API updated recently?
acoustic sail
glass heraldBOT
Hi @acoustic sail. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Based on the information provided in the extracts, OpenAI does regularly release new versions of gpt-4 and gpt-3.5-turbo as mentioned in the deprecations documentation (https://platform.openai.com/docs/deprecations). Each model version is dated with an -MMDD suffix, for example, gpt-4-0613. The undated model name, such as gpt-4, typically points to the latest version. Users of undated model names are usually notified by email about 2 weeks before any change takes place.
However, the extracts do not provide specific information about a recent update to the gpt-4 API. It's also worth noting that changes are always being made in the backend of the model and usually, that information does not reach the end user, as mentioned in a forum post (https://community.openai.com/t/380885).
If you've noticed a sudden drop in the performance of the evaluator, it could be due to these backend changes, or it could be due to other factors. It's recommended to run a fixed set of test cases regularly to monitor the performance, as suggested in the same forum post (https://community.openai.com/t/380885).