#Guys, do you think its possible that Neuro is a MoE AI?

1 messages · Page 1 of 1 (latest)

thorn turret
#

A MoE AI (Mixture of Experts) consists on a AI that is trained to select one LLM among others (or even another MoE AI) to predict the next token.

Seems possible for me that Vedal added more LLM's on Neuro-Sama from time to time and united everything in one fundamental MoE AI, so Neuro-Sama have actual continuity in her old internal model traits even though she is smarter because of the new LLM's.

inland geyser
#

Ehhh tbh I don't know about MoE much myself, but I think it's not exactly like merging 2 LLMs together