#Runpod flash add more flexibility to build from source

2 messages · Page 1 of 1 (latest)

indigo anvil
#

Tool: runpod-flash (CLI flash deploy)

Summary:
When any @Endpoint's dependencies=[...] includes a package whose transitive dependency has no pre-built wheel, the entire multi-endpoint build fails — including endpoints that have no relation to the problematic dependency.

Root cause:
flash deploy runs a single merged pip install --only-binary=:all: across the combined dependency lists of all @Endpoint functions in the app. If even one transitive dependency resolves to a sdist-only package (no wheel available for the target platform), pip rejects it and the whole install aborts.

Concrete example:

sam2_track endpoint declares git+https://github.com/facebookresearch/sam2.git in its dependencies.
SAM 2 requires iopath>=0.1.10, but iopath only ships wheels up to 0.1.9; 0.1.10 is sdist-only.
Result: the merged build fails with ERROR: No matching distribution found for iopath>=0.1.10, and the three other endpoints (hello_gpu, ... which i created) — which don't depend on SAM 2 at all — also fail to deploy.

Expected behavior:
Either (a) build and deploy each endpoint's dependencies independently so a failure in one doesn't block others, or (b) fall back from --only-binary to allow sdist builds for packages with no wheel, or (c) surface a clear pre-flight warning identifying which endpoint's dependency is unresolvable before attempting the build.
B should be ideal

current Workaround:
Move the problematic dependency out of dependencies=[...] and install it lazily at runtime inside the handler function (subprocess.check_call([sys.executable, "-m", "pip", "install", ...])).

Impact:
Any endpoint depending on a research library (SAM 2, many HuggingFace packages, etc.) that hasn't published wheels for all target platforms will make it harder to use runpod flash.

GitHub

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use th...

pearl starBOT