Handle "finished_reason": "length"

[river sluice]

Hello, using API, the chatCompletion can have finished early because it reach max length. How to make request to continue back where it left?

    const response = await openai.createChatCompletion({
      model: 'gpt-3.5-turbo',
      messages: [
        {role: 'system', content: systemPrompt},
        {role: 'user', content: userPrompt}
      ],
      n: 1,
    });

    const finishReason = response.data.choices[0].finish_reason;
    if (finishReason === 'length') {
      logger.info('ChatGPT response was too long. Retrying...');
    } else if (finishReason === 'function_call') {
      logger.error('Completion had a function call issue. Starting fresh...');
      continue
    }

[frank harness]

you need to do a new request, passing the data you already got as a assistant message

[river sluice]

Like this?

  const response = await openai.createChatCompletion({
    model: 'gpt-3.5-turbo',
    messages: [
      {role: 'system', content: systemPrompt},
      {role: 'user', content: userPrompt}
    ],
    n: 1,
  });

  const finishReason = response.data.choices[0].finish_reason;
  if (finishReason === 'length') {
    logger.info('ChatGPT response was too long. Retrying...');
    const continuedResponse = await openai.createChatCompletion({
      model: 'gpt-3.5-turbo',
      messages: [
        {role: 'system', content: systemPrompt},
        {role: 'user', content: userPrompt},
        {role: 'assistance', content: response.data.choices[0].messages.content}
      ],
      n: 1,
    });
  } else if (finishReason === 'function_call') {
    logger.error('Completion had a function call issue. Starting fresh...');
    continue
  }
}

So, Do I need to concate the old response and the new one later on?

[meager wasp]

Yes, you will need to concatenate the old response and the new one later on if you want to preserve the entire conversation history.