#Unable to connect to Ollama using CopilotKit 1.50.1

1 messages · Page 1 of 1 (latest)

reef wyvern
#

I'm running Ollama server with llama3.2 (Listening on 127.0.0.1:11434), but when I send a message from the CopilotKit chat the request doesn't hit the server. Here is my attempt:

import {
    CopilotRuntime,
    copilotRuntimeNextJSAppRouterEndpoint,
    ExperimentalOllamaAdapter,
} from '@copilotkit/runtime';
import {NextRequest} from 'next/server';

const serviceAdapter = new ExperimentalOllamaAdapter({
    model: "llama3.2",
});

const runtime = new CopilotRuntime()

export const POST = async (req: NextRequest) => {
    console.log("📨 POST /api/copilotkit called");
    const {handleRequest} = copilotRuntimeNextJSAppRouterEndpoint({
        runtime,
        serviceAdapter,
        endpoint: '/api/copilotkit',
    });

    return handleRequest(req);
}

Can you please help me to configure it?

chilly frostBOT
#

This solution has been AI generated

It looks like the Ollama adapter and Next.js App Router endpoint need a small configuration adjustment. Use the built-in Next.js App Router handler and ensure the model tag matches what Ollama has (default is 'llama3:latest'). Optionally set OLLAMA_BASE_URL, though 127.0.0.1:11434 is the default.

Recommended Next.js App Router setup

Create app/api/copilotkit/route.ts:

import { CopilotRuntime } from "@copilotkit/runtime";
import { ExperimentalOllamaAdapter } from "@copilotkit/runtime";
import { copilotRuntimeNextJSAppRouterEndpoint } from "@copilotkit/runtime";

const runtime = new CopilotRuntime();
const serviceAdapter = new ExperimentalOllamaAdapter({
  model: "llama3:latest", // optional; defaults to 'llama3:latest'
});

export const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
  runtime,
  serviceAdapter,
});

export async function POST(request: Request) {
  return handleRequest(request);
}
  • Ensure your frontend points to /api/copilotkit.
  • You don't need to pass an endpoint param to copilotRuntimeNextJSAppRouterEndpoint.
  • If desired, explicitly set the base URL (defaults to 127.0.0.1:11434):
export OLLAMA_BASE_URL=http://127.0.0.1:11434

References

Was this helpful?

If this solution worked for you, please click on the appropriate option below to help us improve:

chilly frostBOT
#

Thank you for your feedback! Our team will provide additional assistance. 🔄

reef wyvern
chilly frostBOT
#

Are you passing ollama/llama3:latest in your service adapter? Try using the Ollama adapter with just the model tag (no ollama/ prefix), so just llama3:latest.

reef wyvern
#

No, I'm passing llama3:latest but the error says Unknown provider "ollama" in "ollama/llama3:latest"

chilly frostBOT
#

Let me check with the team on this. In the meantime, could you search your repo for "ollama/" to see if it's being added somewhere? If you find it, try removing it. As a temporary workaround, you could also strip the prefix server-side before calling handleRequest.

reef wyvern
# chilly frost Let me check with the team on this. In the meantime, could you search your repo ...

I wouldn't know how to implement this workaround as "ollama" seems to be harcoded as the provider in ExperimentalOllamaAdapter.

With the following approach I can see the request hitting the ollama server:
env var: OPENAI_BASE_URL=http://localhost:11434/v1

import {
    CopilotRuntime,
    copilotRuntimeNextJSAppRouterEndpoint,
    OpenAIAdapter,
} from '@copilotkit/runtime';
import {NextRequest} from 'next/server';


const serviceAdapter = new OpenAIAdapter({
    model: "llama3.2",
});

const runtime = new CopilotRuntime();

export const POST = async (req: NextRequest) => {
    console.log(":incoming_envelope: POST /api/copilotkit called");

    const {handleRequest} = copilotRuntimeNextJSAppRouterEndpoint({
        runtime,
        serviceAdapter,
        endpoint: '/api/copilotkit',
    });

    return handleRequest(req);
};

but the issue is that this generates a request to /v1/responses instead of v1/api/chat and the comms fail.

chilly frostBOT
#

I've passed this to the team! Could you also open a GitHub issue for this? The team will review and follow up with you next week once they're back from the break. Thanks!

GitHub
CopilotKit/CopilotKit

React UI + elegant infrastructure for AI Copilots, AI chatbots, and in-app AI agents. The Agentic Frontend 🪁 - CopilotKit/CopilotKit

reef wyvern