#Using vercel labs ai streaming text response with Remix Actions

1 messages · Page 1 of 1 (latest)

versed topaz
#

Has anyone experienced this issue with using remix actions using the StreamingTextResponse wit vercel's new ai package.

This explanation was provided by a vercel developer in the issue:

Basically Remix uses node-fetch and we're using the spec compliant fetch implementation. If you use the edge runtime on Vercel it should work:

StreamingTextResponse not compatible with Remix action function #199
https://github.com/vercel-labs/ai/issues/199

GitHub
StreamingTextResponse not compatible with Remix action function · I...

When using the package in an action, like so, export const action = async ({ request }: ActionArgs) => { const { messages, transcriptId } = z .object({ messages: z.array( z .object({ content: z....

hasty rain
#

Remix expect actions to return json responses, not stream responses

#

also Remix doesn't use node-fetch, it uses a custom fetch implementation more similar to the standard

steep trench
versed topaz
#

hmmm. ok so either I would need to use loader somehow or just wait for the response in the action

#

i will take a look @steep trench. I am using HuggingFace atm, but there might be something that I can take away from this

hasty rain
#

an action is the same, it's only limited to JSON when you use a form, Form, useSubmit or useFetcher

#

if you do the fetch yourself you could return anything

molten jay
#

I wrote the issue, and provided a workaround inside that requires patching some stuff from the vercel/ai package you can use in remix

#

it is still worthwhile using IMO so that you can take advantage of the useChat hook. It has really nice DX and UX with streaming chat messages that would be annoying to implement by hand

steep trench
#

Ohhh

#

Will have to play with it then! Thanks for sharing the patch with us!

molten jay
#

it's quite terrible, but it works :p just don't look at it too closely haha

#

I've got it deployed on vercel serverless functions just fine, but I have not tested it with edge functions

versed topaz
#

Thanks @molten jay. yeah i am hoping that we can have nice things too without workarounds 😃

uncut thorn
#

streaming from openai with remix (where an endpoint will be intercepting the stream before passing it on to the client as another SSE) is really annoying. The default EventStream stuff only supports GET but you have to POST to the OpenAI chat endpoints, so you have to implement it yourself. I ended up hacking up a microsoft EventStream library and got everything working nicely, but it was a real pain. (If you're not intercepting, it's easy because you can just make the fetch to OpenAI in your action and return the response.)