Serverless vLLM changing engine arguments

[crisp loom]

Hi, I got vLLM Serverless worker up and running, but want to change one engine argument (which is not overridable through environment variables), specifically --limit-mm-per-prompt , how could I do that with your custom image runpod/worker-v1-vllm:v2.3.0stable-cuda12.1.0 that endpoints use? Thanks

[wise umbra]

use t he configure button to see if its there

or use docs here https://github.com/runpod-workers/worker-vllm

[crisp loom]

thanks I'll look into it and report back

[crisp loom]

@wise umbra https://github.com/runpod-workers/worker-vllm/issues/155 looks like support for it was implemented, and then someone accidentally removed it lol

Would it be possible to add it back again?

Created PR to add it back https://github.com/runpod-workers/worker-vllm/pull/177

[wise umbra]

Hi, thanks for the pr, i currently dont have any permissions to merge pr's but sure i'll try to notify staffs

@hard parrot

[crisp loom]

@wise umbra Thanks!