Action getting ClientResponseError though OpenAPI spec is "correct"

[sullen sphinx]

I have a custom GPT that I've given an action that should make a GET request to https www plantuml com.
The OpenAPI spec indicates that the response has Content-Type image/png.
(We're using deflated and encoded versions of the ASCII, that's not the problem, turns out GPT-4o can do that with python before running the action 🥳)
I've tried the URL with very few headers, and the only thing I can generate a 400 response by is to use a wrong Host header value.
The GPT can generate the URL fine by itself by examining the OpenAPI spec, but whenever it executes the action it gets a ClientResponseError.
See spec & request below. Assistance appreciated. 🙂
(Removed :// and . from domains to not break rules)

OpenAPI spec:

openapi: 3.1.0
info:
  title: ...
  description: ...
  version: 1.0.0
servers:
  - url: https www plantuml com
paths:
  /plantuml/png/{encodedDiagram}:
    get:
      operationId: getPlantUmlPng
      summary: "..."
      description: "..."
      parameters:
        - name: encodedDiagram
          in: path
          required: true
          schema:
            type: string
          description: "Encoded PlantUML diagram"
      responses:
        '200':
          description: "A PNG image of the diagram"
          content:
            image/png:
              schema:
                type: string
                format: binary
        '301':
          description: "Redirect, follow Location header"
          headers:
            Location:
             # omitted
        '400': # omitted
        '500': # omitted

GPT request:

{
  "domain": "plantuml com",
  "method": "get",
  "path": "/plantuml/png/{encodedDiagram}",
  "operation": "getPlantUmlPng",
  "operation_hash": "b925a73e3c1bb0724bf434aa22d46909b3fdba14",
  "is_consequential": false,
  "params": {
    "encodedDiagram": "SoWkIImgAStDuNBAJrBGjLDmpCbCJbMmKiX8pSd9vt98pKi1IW80"
  }
}

[strong epoch]

Taking a while to get my head around what you're doing. You're sending a GET request with a file name and folder path and the desired API endpoint response is that it should send a png image file back to your custom GPT?
Once received the GPT should translate that into ASCII to display in your conversation window?

You can ask your GPT to check if it has any files saved to it's Python environment. My initial assumption is that whatever tool performs the GPT actions is only capable of receiving a limited amount of data so may not be able to receive whatever your endpoint is sending. I am aware of this as lengthy log file retrievals can fail. You could test with an absurdly small png first, like a 1 pixel png perhaps, just to see if it can received that. My experience has only been with receiving text, so I'm not sure it will work. I've also tried sending generated images, but this fails. I wouldn't be surprised if the Action tool only deals with text.

[sullen sphinx]

I really hoped you were right about it only handling text, so I tried using the svg endpoint instead and setting response content type to text/plain. Still no go.
My last guess is that it can't handle the 301 from plantuml com to www plantuml com, but who knows.

If it were an issue with binary vs. text, this operation should've worked, right?

  /plantuml/svg/{encodedDiagram}:
    get:
      operationId: getPlantUmlSvg
      summary: "Get SVG diagram from PlantUML"
      description: "Returns a SVG image of the UML diagram, following any redirects"
      parameters:
        - name: encodedDiagram
          in: path
          required: true
          schema:
            type: string
          description: "Encoded PlantUML diagram"
      responses:
        '200':
          description: "A SVG image of the diagram"
          content:
            text/plain:
              schema:
                type: string
        '301':
          description: "Redirect, follow Location header"
          headers:
            Location:
              description: "URL to follow for the redirect"
              schema:
                type: string

Going to the url [domain]/plantuml/svg/SoWkIImgAStDuNBAJrBGjLDmpCbCJbMmKiX8pSd9vt98pKi1IW80
surely gives a nice SVG of 1.6 kB, but GPT still says no.

[debug] Calling HTTP endpoint
{
  "domain": "plantuml com",
  "method": "get",
  "path": "/plantuml/svg/{encodedDiagram}",
  "operation": "getPlantUmlSvg",
  "operation_hash": "1d7f284aa03ae9db8da9a6b5f77dcbf985256a89",
  "is_consequential": false,
  "params": {
    "encodedDiagram": "SoWkIImgAStDuNBAJrBGjLDmpCbCJbMmKiX8pSd9vt98pKi1IW80"
  }
}
[debug] Response received
{
  "response_data": "An error occurred while executing the plugin."
}

It seems there was an issue while attempting to retrieve the UML diagram via the external plugin.

postman is completely happy with the operation

[strong epoch]

I haven't direct experience of using SVG files, but I would think these are not text files? SVG = Scalable Vector Graphics.
I doubt the GPT UI is capable of displaying SVGs.
Try receiving a .txt file instead or a simple string.
The response content in the schema I believe is just an example of what the tool might expect to receive. I think its purpose is to give context to the GPT so it can better select which action to call from the users' query.

If you want to display an actual image to your user rather than formatted text/ASCII art, I think you'd have to redirect them from the GPT to a URL.