#How to get finish_reason when streaming the response?

1 messages · Page 1 of 1 (latest)

trim snow
#

I am using Vercel AI SDK with Open AI API to get the response as stream.
I am unable to figure out how can I get the finish_reason property from the response since it will tell me when the stream has ended or not.

I just want something to check if the stream has ended or not, before retuning the data to the frontend.

Here's my code.

import { Configuration, OpenAIApi } from "openai-edge";
import { OpenAIStream, StreamingTextResponse } from "ai";

export const runtime = "edge";

const config = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(config);

export async function POST(req: Request) {
  try {
    const { messages } = await req.json();
    const response = await openai.createChatCompletion({
      model: "gpt-3.5-turbo",
      messages,
      stream: true,
    });
    const stream = OpenAIStream(response);

    return new StreamingTextResponse(stream);
  } catch (error) {
    console.log(error);
    return "Error while generating response from Open AI.";
  }
}

Please help me figure this out.

lone flameBOT
#

Hi @trim snow. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

In the streaming response from the OpenAI API, the finish_reason is a property of the choices object. This property indicates why the model decided to stop generating content. If the response was cut-off due to exceeded context length during generation, the finish_reason will have the value length. If everything goes right, the value will be stop source (https://community.openai.com/t/387245).

In your case, you are using the OpenAIStream function to handle the streaming response. To access the finish_reason, you would need to parse each chunk of the response and extract the finish_reason from the choices object.

However, based on the information provided in the extracts, there isn't a direct example of how to do this with the OpenAIStream function in the Vercel AI SDK. You might need to refer to the SDK's documentation or source code to understand how to handle the streamed response and extract the finish_reason.

In the OpenAI Node API Library, the finish_reason can be accessed from the response.choices**0] as shown in this [example** (https://cookbook.openai.com/examples/how_to_build_an_agent_with_the_node_sdk#running-the-final-app). But please note that this example is not directly applicable to your case as it's using a different SDK.

const { finish_reason, message } = response.choices[0];

If you're still having trouble, I recommend reaching out to the maintainers of the Vercel AI SDK or OpenAI for more specific guidance.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: