Newbie here — How to run ComfyUI + KoboldCPP + AI Assitant in one pod on RTX 4090 ?

[primal karma]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[oblique star]

There's a whole process involved in this - you would need to begin to build your own docker image since we don't have a template that has all of these things involved

If you're starting from scratch my best advice is to get the basics on building a docker image first here:

https://docs.docker.com/get-started/introduction/build-and-push-first-image/

Then once you have the syntax of that down implement the easiest thing (ComfyUI probably) in that image, test it, and then start adding the other things

There's nothing stopping you from getting all of this running in a pod but you're looking at a fairly large scale project implementing this many things in an image at once, it can be done but best to break it into manageable chunks

[grand patio]

i think there is an easier solution but I would let @lime plover confirm that first

you can use Henky's template with KoboldCPP and since his template is able of storing the important stuff on the persistent drive (/workspace) you would just install wht you need. Will survive restart. So you would just install Comfy ad AI assistant

[lime plover]

Actually no my KoboldCpp template doesn't have persistance

The original post was deleted so I don't know who I need to help

[grand patio]

oh, i see. then forget it henky 🙂

[lime plover]

But if you want KoboldCpp in an image that isn't our own one its quite simple.

curl -fLo koboldcpp https://koboldai.org/cpplinuxcu12
chmod +x koboldcpp

Its my codecanvas image that has the persistance

[lime plover]

And of course feel free to add KoboldCpp to yours 😛