#Published Model Worse than Preview

1 messages · Page 1 of 1 (latest)

light mulch
#

Bug Report: I was using the new model publishing feature and noticed a huge drop in the context awareness of the model in the build phase compared to after publishing

Steps to reproduce:

  1. Begin to build a model
  2. give it code context
  3. go over a change or feature in the preview you intend to make
  4. it will output a lot of great advice on how to solve the problem specifically
  5. Publish the model

Expected result:

That if i prompt with my requirements again, i can recover roughly what was dicscussed in the model build. Doesn't have to be exact, but i'd expect it is still aware of the file structure and objects we just discussed in the build.

Actual result:

Broad answers that don't take into account the uniqeness of the context just fed to include all memory of the files fed in specifically for the problem and a confusion of the objects. It seems to be vaguely aware, but not anywhere near what was happening in the original build screen beore publishing.

Additional information

Browser: Firefox
OS: Ubuntu 22.04

torpid pebble
# light mulch Bug Report: I was using the new model publishing feature and noticed a huge drop...

Just to clarify: when you talk about publishing a model, are you referring to the custom GPT creation process?

If so, are you using primarily the "Create" chat to build the GPT? If so, and you may already know this, "Create" just fills out the "Configure" tab to the right of that toggle--it doesn't inform any kind of "behind the scenes" instruction or training for the GPT.

In other words: custom GPT behavior is defined completely by the contents of the "Configure" tab, which is to say, it is Instructions + Knowledge files + Actions.

Regarding "to include all memory of the files fed in" -- knowledge files aren't stored in permanent context for a GPT, they're just accessible to the GPT for it to search through and find relevant information in, similar to how it can search on the Internet for something in particular. The hard context limits of the model still apply, but it can do a good job of searching through a file to return relevant information or to follow specific instructions in the file.

light mulch
#

"are you referring to the custom GPT creation process?" yep!

I did not know that about configure vs create. This is the view i want now that i see it so thank you.

Yeah, I'm not sure what is going on with this but at this point they don't seem to work all that great for coding for me. I can see all the files in it's knowledge but it keeps inventing things that are either already implented or the concerns of an exiting object.

torpid pebble