jaunty vortex
I am trying to self-host copilotkit runtime with openai as llm provider with express js. the difference in response time when using cpk cloud and self-hosting is very high. here i have attached ss for the same.
i have followed this docs:
https://docs.copilotkit.ai/direct-to-llm/guides/self-hosting?endpoint-type=Node.js+Express
@frank ridge @vast grail
paper wedgeBOT
The main reason you might be seeing slower response times when self-hosting the CopilotKit runtime with OpenAI (compared to CopilotKit Cloud) is likely related to serverless function timeouts being too short for CopilotKit's streaming responses.
For instance, platforms like Vercel and AWS Lambda often have default timeouts around 10-15 seconds, which might not be enough and could cause delays or interruptions. You could try bumping up your serverless function timeout - something like setting maxDuration to 60 seconds in your vercel.json for Vercel, or increasing the Lambda and API Gateway timeouts if you're using AWS.
If that doesn't help, it might be worth checking your CopilotKit runtime logs for any performance metrics and sharing those if you're still having issues.
jaunty vortex
hey @vast grail . i have set the timeout in my lambda to be 60 sec. now i am getting this weird output
paper wedgeBOT
Could you please share a log that shows the response times? This will help us compare performance.
jaunty vortex
this is my code
paper wedgeBOT
Thanks for sharing the code! Our team will review it when they have a chance. Would you mind sharing the response time logs as well if possible?
jaunty vortex
this is my cloudwatch log
i am using cpk runtime mostly for autosuggestion using copilotkitTextarea in my project.
jaunty vortex
I am trying to self-host copilotkit runtime with openai as llm provider with exp...
ss here shows the time it takes for cpk cloud vs self hosted for suggestions to appear in the textarea element.
paper wedgeBOT
I've shared this with our team and they'll review it and reply shortly.
jaunty vortex
any update on this?