#Did anyone notice behavioral changes in AI after not using it for a while?

1 messages · Page 1 of 1 (latest)

hoary geode
#

Hi All,

I’m a student working on a project that uses an Azure OpenAI model to perform certain tasks assigned. My main prompt was stored in vectorDB, and everything was working perfectly when I last tested it about two weeks ago.

I got busy with coursework and didn’t touch the project for a while. When I came back to continue documenting my work, the model suddenly wasn’t behaving the way it used to. It was responding with wrong responses. I checked my code, embeddings, and prompt. There were no errors. I have reinserted my prompt in vectorDB again, but nothing helped. Then the next day, without changing anything, it started working normally again.

Now I’m trying to understand what happened. Does model performance degrade if you don’t use it for a while, or is there another explanation for this kind of temporary behavior change?

Has anyone experienced something similar or knows what might cause this?

grave cove
#

Hey, so three things:

  1. We're a Google discord server, so I would not expect Azure experts around to have answers for you 🙂
  2. You didn't specify what model you were using and in which version. It's possible that there was an update made to the model, that was later rolled back.
  3. Models are just huge collections of data. They are usually unaware of timeflow - if you ask a model what date is it, without providing it with any relevant context, it'll always give you the same date. So it's not like the model was unhappy you didn't talk to it for a while 🙂

-# Note: I'm Google employee, this are only my guesses, I don't know what really happened or how Azure manages their models.

supple latch
#

i forgot how to write

grave cove
supple latch
grave cove
supple latch
#

lmk if you find the form

grave cove
#

I'll look tomorrow if I don't forget as I'm ooo for today and on mobile

supple latch
#

@undone shard

tulip brook
#

Manass: It could be a number of things. I'm assuming that you're using some API to access your model? Sometimes models won't change but they will be affected by safety or gaurdrail changes. Or you said you're using a vector database. It could be that you're working from a cold start. Or it could be that you're using some sort of routing or load balancing so you're not hitting the same model that you think you're hitting.