LLM breaking changes

[copper walrus]

Hey, sorry about that, these LLM APIs are still experimental and we had to be pretty aggressive with shipping them to have them out in time for a meetup/workshop. 😅 Once they graduate from experimental we'll be much more cautious about maintaining compatibility and communicating those changes. It's effectively a big feedback loop involving everyone that uses it; at the current stage it's better to break things and hear about cases where the new approach is worse, to make sure we land on the right long-term API.

In the past we've gone to great lengths to maintain compatibility with modules written against various engine versions - there's a mechanism for presenting different versions of APIs based on the module's reported engine version. We're just not there yet with LLM. Hopefully will be soon.

[fickle carbon]

sorry about that @hot zodiac

like @copper walrus said llm API is a special case at the moment because it's experimental but already getting significant usage. This is not representative of our general attitude towards API stability I promise!

note @hidden dew is adding a formal "experimental" marker that will formalize this better. should be ready soon

[hot zodiac]

Hey, no worries, I totally understand that the LLM API is still experimental and evolving quickly. I get that breaking things early helps refine the API, and I appreciate the iteration process!

That said, even a simple one-liner with something like [Breaking change] in changelog (https://github.com/dagger/dagger/blob/main/CHANGELOG.md#v0180---2025-03-31) about major breaking changes (like the new env API) would really help folks stay in sync without having to discover them the hard way (the way I discovered this is actually from I vaguely remembered I saw something about new env api in this discord, then I found solomon's env api pr and yves's migration commit by digging discord chat). Not asking for full stability at this stage—just a small heads-up so migration is smoother. Looking forward to seeing where the API lands!

[hot zodiac]

I like that, but could real breaking changes be explicitly marked as "breaking change" rather than just "experimental" ?

[oblique pier]

this is a fair request IMO. I'm totally in for flagging that as a breaking change in the experimental API. Maybe we could have an experimental section and highlight the breaking change there. Not sure how Docker is used to handle this. Probably @hidden dew knows 🙏

edit: maybe having an experimental section in the release is enough to start. Regardless, it'd be nice to see experimental API changes in the release notes

[hidden dew]

yes! we're going to have experimental apis - I like the idea of a dedicated changelog section too

[copper walrus]

This process would be much smoother if we just automated the changelogs based on merged PR titles and used labels like breaking to promote them into a higher category

[fickle carbon]

Normally we use advance deprecation + backwards compat mode for breaking change.

Being more explicit in the changelog is a good idea also

[copper walrus]

GitHub has a solid implementation of this that even lets you control the template and assign meaning to certain labels, we consistently forget to use changie especially when we're moving fast. Seems like the last mile would be syncing the generated release descriptions to CHANGELOG.md for folks that prefer looking at it there

[fickle carbon]

agreed

[spring barn]

I was bit by this too haha. After finding in the code that the API changed, I came here and found this thread. Not a big deal but thanks for marking it experimental. I'll expect it now:-)

[copper walrus]

I can try out the GH release notes thing, since I've used it in the past

[spring barn]

I got to learn about environments so that is nice