Hi @vale wolf. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
#chat_template
bright zephyrBOT
Your understanding is partially correct, but let me clarify some points based on the information provided in the knowledge sources.
For GPT4All models and official models listed in models3.json, the chat template is indeed stored in the model configuration and used by default. However, for sideloaded or custom models, the process is a bit different.
According to the GPT4All Documentation, chat templates are essential for converting conversations into a format that the language model can understand. Each model is designed to work best with a specific format.
For sideloaded or custom models, you may need to manually specify the chat template. The GPT4All Documentation states that you typically don't need to write your own chat template, except for models that are not in the official model list and don't come with a built-in chat template.
When it comes to GGUF models specifically, the documentation doesn't provide explicit information about reading the template from the .gguf file on first load. Instead, it suggests that for sideloaded models, you might need to configure the chat template manually.
There's an ongoing discussion about potentially changing how templates are handled. A GitHub issue proposes removing defaults for model templates and requiring users to fill out these templates manually for sideloaded or "discovered" models.
To find a chat template for a custom model, you can follow the instructions in the GPT4All Documentation, which provides methods using CLI or Python to extract the chat template from the original model's HuggingFace repository.