prompt

[cunning ravine]

Hi @west mountain. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

Sure, I can provide examples for both Python and Node.js as per the GPT4All API documentation.

For Python, you can use the following code to run inference on a downloaded model:

from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
output = model.generate("The capital of France is ", max_tokens=3)
print(output)

In this example, "orca-mini-3b-gguf2-q4_0.gguf" is the name of the downloaded model. The generate method is used to run inference on the model. The output will be printed on the console. You can replace the prompt "The capital of France is " with your own prompt. Source (https://docs.gpt4all.io/old/index.html#gpt4all)

For Node.js, you can use the following code:

import gpt from "../src/gpt4all.js";

const model = await gpt.loadModel("mistral-7b-openorca.gguf2.Q4_0.gguf", {
    device: "gpu",
});

process.stdout.write("### Generator:");
const gen = gpt.createCompletionGenerator(model, "Redstone in Minecraft is Turing Complete. Let that sink in. (let it in!)");
for await (const chunk of gen) {
    process.stdout.write(chunk);
}

process.stdout.write("\n");
model.dispose();

In this example, "mistral-7b-openorca.gguf2.Q4_0.gguf" is the name of the downloaded model. The createCompletionGenerator method is used to run inference on the model. The output will be printed on the console. You can replace the prompt "Redstone in Minecraft is Turing Complete. Let that sink in. (let it in!)" with your own prompt. Source (https://docs.gpt4all.io/old/gpt4all_nodejs.html#async-generators)

Please replace the model names and prompts in these examples with your own.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

@west mountain kapa.ai is still learning and improving, please let us know how it did by reacting below