#streaming response of openai to client from nodejs backend

1 messages · Page 1 of 1 (latest)

normal wolf
#

Im building a chatbot that uses open ai chat completion api, im confused how to send the streaming responses to client side ? Im using nodejs and next js

cold jasperBOT
#

Hi @normal wolf. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

To send streaming responses from the OpenAI API to the client side, you can use the stream: true option when making a request to the OpenAI API. This will return a stream of responses that you can then send to the client side.

Here's an example of how you might do this in a Next.js API route:

import { OpenAI, OpenAIStream } from 'openai';
import { NextResponse } from 'next/server';

// Initialize OpenAI with your API key
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export async function POST(req, res) {
  // Extract the `messages` from the body of the request
  const { messages } = await req.json();

  // Request the OpenAI API for the response based on the prompt
  try {
    const response = await openai.chat.completions.create({
      model: "gpt-4",
      stream: true,
      messages: messages,
    });

    const stream = OpenAIStream(response);

    // Return a streaming response
    return new NextResponse(stream);
  } catch (error) {
    console.error("Error: ", error);
    return new NextResponse(JSON.stringify({ error: error.message }), {
      status: 500,
      headers: { 'Content-Type': 'application/json' },
    });
  }
}

In this example, the POST function is an API route in Next.js that receives a request with a messages array in the body. It then sends a request to the OpenAI API with stream: true to get a stream of responses. The OpenAIStream function is used to convert the response into a stream that can be sent to the client side. The NextResponse class from Next.js is used to send the stream as a response from the API route.

Please note that you need to replace "gpt-4" with the actual model you want to use, and process.env.OPENAI_API_KEY with your actual OpenAI API key.