mortal cedar
Interesting error. As the subject states, adjusting top p beyond 0.60 on mixtral instruct causes this to send blanks. The reason I say regularly in the subject is because sometimes you get a message, sometimes you don't. Dropping to .50 never gives blanks at all. There's no rhyme or rythym. The only pattern I've identified is adjusting top p above 0.60
I'll get a gif so you can compare
Ignore the slight NSFW:
https://gyazo.com/556fd59711c68be207ed35a7fe831b0c
https://gyazo.com/ab98e1e817473c242998b78dab0c1373
The only difference was my top p. 0.50 had 5/5 replies with text. 0.75 had 4/7 replies with text. The others were blank
I have absolutely no clue what causes this... only common factor is top p above a certain limit
mortal cedar
This has started occurring with top k aswell