#OpenAI API Response Data types

7 messages · Page 1 of 1 (latest)

vestal atlas
#

Is there some documentation about the OpenAI API responses data types ?

For example in this response

{
  "id": "cmpl-uqkvlQyYK7bGYrRHQ0eXlWi7",
  "object": "text_completion",
  "created": 1589478378,
  "model": "text-davinci-003",
  "choices": [
    {
      "text": "\n\nThis is indeed a test",
      "index": 0,
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 7,
    "total_tokens": 12
  }
}

What is the data type of the logprobs key when is not null ?
I need that info to handle a null value or a missing key in the response.

magic sentinel
#

The C# library I'm using defines the Logprobs class as:

public class Logprobs
    {
        [JsonProperty("tokens")]
        public List<string> Tokens { get; set; }

        [JsonProperty("token_logprobs")]
        public List<double?> TokenLogprobs { get; set; }

        [JsonProperty("top_logprobs")]
        public IList<IDictionary<string, double>> TopLogprobs { get; set; }

        [JsonProperty("text_offset")]
        public List<int> TextOffsets { get; set; }
    }
vestal atlas
#

I'm using the community libs like that, but how they know that data type ?

#

I'm using the Kotlin one

@Serializable
public data class Logprobs(
    /**
     * The tokens chosen by the completion api
     */
    @SerialName("tokens") public val tokens: List<String>,

    /**
     * The log probability of each token in [tokens]
     */
    @SerialName("token_logprobs") public val tokenLogprobs: List<Double>,

    /**
     * A map for each index in the completion result.
     * The map contains the top [CompletionRequest.logprobs] tokens and their probabilities
     */
    @SerialName("top_logprobs") public val topLogprobs: List<Map<String, Double>>,

    /**
     * The character offset from the start of the returned text for each of the chosen tokens.
     */
    @SerialName("text_offset") public val textOffset: List<Int>,
)

#

But I really wanna read an official Documentation about it

magic sentinel
#

you have me curious, and I can't find anything in the docs that mentions it, but davinci models should return a logprob, which you can reverse engineer https://community.openai.com/t/is-logprobs-being-deprecated-or-will-it-eventually-be-available-for-newer-models/104769

OpenAI API Community Forum

Hi all, Some applications of LLMs involve using them to score, not just generate, completions. For example, we may wish to generate several completions under one prompt, and then rank them according to how likely they are under some other prompt. This is possible using the logprobs option in the completions endpoint, but is not yet supported i...

vestal atlas
#

I'm building a client for the API, but I'm blind with the data types :|. So far I am relying on official clients and the community