#Evaluating AI Agents - DeepLearning.AI
analog mesa
I would move this out of DataAnalyst: https://github.com/jpadams/data-analyst/blob/b2f5c9127d19a8382034e880b8bb4dda36fa58ca/src/data_analyst/main.py#L18-L25
fossil locust
I would move this out of `DataAnalyst`: https://github.com/jpadams/data-analyst...
Cool. Into a separate Python file/module? or a Dagger Python module?
analog mesa
Can be the same file. Just above.
Do you have the API docs at hand for client.beta.chat.completions?
Oh I guess you don't need that if you're trying to use our llm integratino.
fossil locust
Oh I guess you don't need that if you're trying to use our llm integratino.
exactly.
I think they're using something like
https://platform.openai.com/docs/guides/structured-outputs?api-mode=chat
analog mesa
Let me know if you need help with something specific.
fossil locust
Let me know if you need help with something specific.
@void meteor Guessing we don't have an implementation for structured outputs today? Or maybe we do under the hood. Have to look.
I figured the closest mapping was (structured JSON output)response_format=VisualizationConfig, -> env().with_<VisualizationConfigObjectType>_output() but not sure how to make that all work with our system.
fossil locust
Let me know if you need help with something specific.
Yeah, it seems the Pydantic format doesn't work with Dagger even after adding pydantic library and running dagger develop which pass. Running dagger causes a runtime error:
TypeError: Module.field() got an unexpected keyword argument 'description'
https://v3.dagger.cloud/dagger/traces/a2a3c1be024c595e19d0a6aaffd11df3
So I thought to turn it into Dagger style "struct" or something...
analog mesa
Do you really need pydantic? Doesn't work with normal @object_type?