#not currently from the Dagger SDKs.

1 messages · Page 1 of 1 (latest)

jagged wigeon
#

Any suggested workarounds? I could collect the buffer and implment smooth streaming myself but the initial delay of the first buffer shot often makes for poor ux

lapis jewel
#

stdout? or somwhere else?

#

are you using modules ?

jagged wigeon
#

i want to expose the stream to an API that is using SSE. I could use modules.

lapis jewel
#

are the LLM responses dag.LLM calls?

jagged wigeon
#

yes

lapis jewel
#

hmm not sure if there's even a workaround for this. cc @cerulean socket mighthave the best context here 🙏

#

@jagged wigeon I know how this could be currently achieved but it's a bit tricky

#

all the OTEL telemtry we emit is real time

#

so the only way I can think of is forwarding the OTEL data somwhere and using that to capture the real time output of the LLM calls

#

it's quite a hacky workaround

cerulean socket
#

yeah the extremely tricky part here is how it actually gets sent over the wire. right now all the SDKs just speak plain GraphQL requests. this would have to be a GraphQL subscription (totally uncharted territory), or perhaps a special endpoint that gets returned to you (some precedent there with how .Terminal() / -i works iirc)