#Spike in gpt-4-0613 malformed function calls today

3 messages · Page 1 of 1 (latest)

grim junco
#

We have been developing applications that intend to use GPT 4 functions in production. Today we saw a spike in malformed function call results from prompts that worked reliably a few days ago. Example:

(base) zaptrem@Ryans-MacBook-Pro ~ % curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer THE-TOKEN" \
  -d '{
    "model": "gpt-4-0613",
    "messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello! I am Ryan. Say hello THEN wave to me."}],
    "functions": [{"name": "wave_hello", "description": "waves hello to the user",
    "parameters": {
       "type": "object",
      "properties": {
        "name": {
          "type": "string",
          "description": "The name of the person to wave hello to"
        }
      },
      "required": ["name"]
    }}],
    "stream": false
  }'
{
  "id": "chatcmpl-ID-CENSORED-INCASE-OF-DANGER",
  "object": "chat.completion",
  "created": 1688755766,
  "model": "gpt-4-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello Ryan! Nice to meet you.\n\nNow, let's wave hello.\n\n[ASSISTANT][TRIPLE BACKTICKS CENSORED DUE TO MARKDOWN]n{\n  \"function\": \"functions.wave_hello\",\n  \"parameters\": {\n    \"name\": \"Ryan\"\n  } \n}\n[TRIPLE BACKTICKS CENSORED DUE TO MARKDOWN]"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 75,
    "completion_tokens": 51,
    "total_tokens": 126
  }
}

How do we resolve this?

sinful current
#

I've noticed this too. I'm going to look into strategies to force json-like data into valid json. Let us know if you come up with any solutions 👀 -- it happened around the same time all the other stuff went awol

sinful current
#

actually, i kind of solved this today, not sure if you're using node or deno, however, i've had good luck using JSON5 to parse invalid output with missing commas or other syntax errors