Correctness guarentee for AI SDK structured data.

[wintry lintel]

I am testing some things with the AI SDK using GPT-4o. But a part of the docs kind of confuse me a bit. This is it (https://sdk.vercel.ai/docs/ai-sdk-core/generating-structured-data#generating-structured-data):
"The generateObject function uses Zod schemas to specify the shape of the data that you want, and the AI model will generate data that conforms to that structure. The schema is also used to validate the generated data, ensuring type safety and correctness."

Is there any real chance that the generateObject function just fails, because the returned structure from the model response does not match? I am mostly thinking of the responding model just hallucinating another structure than the one given, and therefore the Zod validation fails? Or am I guaranteed that the structure always matches what I expect?

[timber moss]

Post created!

🔎 This post has been indexed in our web forum and will be seen by search engines so other users can find it outside Discord

🕵️ Your user profile is private by default and won't be visible to users outside Discord, if you want to be visible in the web forum you can add the "Public Profile" role in id:customize

✅ You can mark a message as the answer for your post with Right click -> Apps -> Mark Solution
(if you don't see the option, try refreshing Discord with Ctrl + R)

[vernal shell]

I am using generateObject function a lot and yes it will throw an error if the model generates smth which doesn't satisfy the zod schema

[wintry lintel]

How often do you find that it happens? I am kinda scared of using this feature, if it fails all the time...

[vernal shell]

it happens like once every 30 calls for me

but that totally depends on model and prompt

[wintry lintel]

I am planning on using gpt-4o

[vernal shell]

I am using gemini and this count was 1 out of 3 calls but I improved my prompts a lot and it is now once every 50 calls

gpt4o was already pretty good at it and it was already at like once every 20 calls

[wintry lintel]

Alright. I will try and test some things out.
Kinda unrelated, but do you know if it is possible to use this feature with the OpenAI Batch API in some way?

[vernal shell]

no idk how to do it

[wintry lintel]

Alr, thank you!