Can I emulate hitting serverless endpoints locally?

[wind raft]

So far I've been testing my runpod serverless locally by executing the python handler

python -u handler.py

but is there any way to emulate hitting the serverless endpoint locally?

[grim thunder]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[wind raft]

ok so I found out that we can serve it using --rp_serve_api but is there any way to make it mimic serverless workers fully? Key differences that I noticed are:

• webhooks got their response only after I hit the status endpoint
• local webhook response differ from the webhook response of the actual runpod serverless workers, local response consist of only the output field instead of the full json response schema

is there any way I can do to make them as close as how the actual workers behave? or at least on those 2 points

[pale axle]

No, unless you modify the Local part of the SDK yourself by forking it. Ive been considering to improve on the local testing, but I gave up on it, and just created a way for me to test on GPU Pod which is the closest env Ill get to, and you can probably add some manual emulations yourself like sending out your own webhooks all that.

https://github.com/justinwlin/Runpod-OpenLLM-Pod-and-Serverless/blob/main/handler.py

My repo I use to test from GPU pod > to deploying on serverless

[wind raft]

ah I see thanks! actually I just need to ensure that the webhook response in production and staging are both same, because when we run it locally using that rp serve thing, the webhook response are totally different

[hoary crystal]

@wind raft We have a CLI tool that can help iterate over your development process quicker.
It has the ability to sync your changes over a Pod - the only thing you'll need to do, is update the Endpoint you're interacting with.

https://docs.runpod.io/cli/install-runpodctl

[wind raft]

thanks @hoary crystal! but what I need the most is to match this webhook response from local runpod development server and the actual runpod serverless, for example a runpod serverless endpoint webhook would look like this:

but a webhook from the local development server provided would look like this:

that's my main problem rn because I need to use the input field also while parsing the webhook response

[hoary crystal]

Yep, the dev environment creates a Fast API server for you on port 8080 where you can pass in your webhook.
Makes for pretty quick development and you can test your code.

[coral ermine]

@wind raft I can have the webhook match closer, did not realize we were returning input as part of it.

[wind raft]

@coral ermine ah I see, thanks! It'd be great to have both the local dev and prod to return a similar schema, it'd helps our local staging a lot

[midnight bobcat]

The are apparently 2 versions of the webhook, the v2 webhook doesn't return the input, so if you use that, then dev and prod should match.

[wind raft]

@midnight bobcat ah, didn't know about that. how can I use the v1 instead?

[midnight bobcat]

Don't think local testing has the v1 format, sounds like it has the v2 format webhook.