how to stop scripts from replacing my configuration | Runpod | Page 1

scenic urchin Jul 18, 2024, 10:49 AM

#

I have a version of ComfyUI together with a version of python and it's dependendencies in a direcotry named /workspace on a storage volume. Everytime this pre_start script runs it wipes out my lib and starts over. I tried modifying the script but it gets regenerated every time I restart the pod. How to stop this?

fresh glenBOT Jul 18, 2024, 10:49 AM

#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

noble summit Jul 18, 2024, 11:13 AM

#

@scenic urchin just to verify your issues is that you install custom nodes and on stop it gets wiped

scenic urchin Jul 18, 2024, 11:16 AM

#

No, my issue is that the comfy UI docker container runs a script called pre_start and replaces my python lib because it's outside of the mounted storage volume. I placed the python venv inside the workspace but now need that script to change path to /workspace/venv/python

noble summit Jul 18, 2024, 11:17 AM

#

#1257333598642639029 my image solves this issue

scenic urchin Jul 18, 2024, 11:18 AM

#

So I'd have to do a fresh install with Better comfy?

#

Could you explain what your image does and the name? I've been trying to install comfyui for a few days now using the available image SD comfyui @noble summit . Is it loaded with all the custom nodes I need (ip adapter, was suite, idm vton)?

noble summit Jul 18, 2024, 11:24 AM

#

it does save venv to workspace

#

and also it has lot of tools preinstalled that might be usefull for comfy ui

scenic urchin Jul 18, 2024, 11:40 AM

#

got it. I am deploying now

noble summit Jul 18, 2024, 11:43 AM

#

Let me know how it goes anlways happy to get some feedback

scenic urchin Jul 18, 2024, 11:53 AM

#

@noble summit so first thing I noticed was that comfyUi was not deployed.
Second thing is that the python main.py --listen doesn not work

#

Traceback (most recent call last):
File "/workspace/ComfyUI/main.py", line 73, in <module>
import comfy.utils
File "/workspace/ComfyUI/comfy/utils.py", line 1, in <module>
import torch
File "/workspace/venv/lib/python3.10/site-packages/torch/init.py", line 238, in <module>
_load_global_deps()
File "/workspace/venv/lib/python3.10/site-packages/torch/init.py", line 197, in _load_global_deps
raise err
File "/workspace/venv/lib/python3.10/site-packages/torch/init.py", line 178, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/usr/lib/python3.10/ctypes/init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: /workspace/venv/lib/python3.10/site-packages/torch/lib/libtorch_global_deps.so: cannot open shared object file: No such file or directory

#

I am going to try wiping out ComfyUI and starting over since the only thing this installation has is ComfyUi manager

noble summit Jul 18, 2024, 12:20 PM

#

ye you would need reinstall custom nodes so it preinstall required packages

karmic finch Jul 18, 2024, 12:54 PM

#

If you use comfyui manager, install the pip dependencies like you first install comfyui manager then fix all package from the cm-cli of comfyui manager

scenic urchin Jul 18, 2024, 12:56 PM

#

@karmic finch I am running1 x RTX A4000
6 vCPU 23 GB RAM Is it possible that I get an error torch.cuda.OutOfMemoryError if I am doing something like this?

karmic finch Jul 18, 2024, 12:56 PM

#

I don't know never used that node

#

Use the 24gb gpu vram at least

scenic urchin Jul 18, 2024, 1:00 PM

#

is it possbile to switch gpu?

#

without having to setup everything again?

karmic finch Jul 18, 2024, 1:00 PM

#

Sure it's easy enough if you have data saved in a network storage (just use the same in another pod)

#

Or if you have it in a pod with no ns, you need to reinstall all of the files/program again

scenic urchin Jul 18, 2024, 1:01 PM

#

I have secure cloud network storage

karmic finch Jul 18, 2024, 1:01 PM

#

Use the same template on another pod I guess

#

Yeah with the same network storage

noble summit Jul 18, 2024, 1:01 PM

#

if you used my template all thing should be saved on NS

scenic urchin Jul 18, 2024, 1:02 PM

#

I am going to shut down the current machine and use another gpu.

#

let's give it a try

karmic finch Jul 18, 2024, 1:02 PM

#

Okay

scenic urchin Jul 18, 2024, 1:02 PM

#

terminating pod now

karmic finch Jul 18, 2024, 1:03 PM

#

Chill, it's fine don't need to update everything here 👌

scenic urchin Jul 18, 2024, 1:05 PM

#

sorry, it's just my 7th time having to setup everything all over again and wnted to just make sure I did things right. Terminating and stopping are two different things but there was no other option. Just nervous I will have to redo everything again.

#

btw @noble summit There are 20 GB it needs to download everytime you terminate the pod and run it again. Any guess as to what it is downloading?

noble summit Jul 18, 2024, 1:12 PM

#

Its base image + build in models

karmic finch Jul 18, 2024, 1:19 PM

#

scenic urchin sorry, it's just my 7th time having to setup everything all over again and wnted...

All good, if youre not sure better to stop it first if you have the option to(you don't when you use ns)

scenic urchin Jul 18, 2024, 1:25 PM

#

Ok, that seems to have done it. But I did notice that it was more CPU intensive than GPU. Is that because somewhere I am not configuring comfy to use GPU for processing?

noble summit Jul 18, 2024, 1:25 PM

#

nope usually yo do not need do anything to run on gpu

karmic finch Jul 18, 2024, 1:33 PM

#

maybe use gpu sampler for more speeds

#

but it changes abit on the image generation process... ( im not really sure how to explain but you can experiment )

scenic urchin Jul 19, 2024, 11:13 AM

#

@noble summit you may need to adjust your script to not create a venv in the root. and instead use the python instance in the /workspace

noble summit Jul 19, 2024, 11:15 AM

#

@scenic urchin whats the context

karmic finch Jul 19, 2024, 11:15 AM

#

venv in / ?

noble summit Jul 19, 2024, 11:15 AM

#

though on what template we talk about

scenic urchin Jul 19, 2024, 11:16 AM

#

bettry comfy ui

noble summit Jul 19, 2024, 11:16 AM

#

This is correct behaviour

scenic urchin Jul 19, 2024, 11:16 AM

#

when I do any kind of pip install -r requirements.txt it begins installing everything in the /venv/lib/python3.1.0/site-packages

noble summit Jul 19, 2024, 11:17 AM

#

as on secure cloud /workspace is being overwritten

#

did you activated venv first?

scenic urchin Jul 19, 2024, 11:18 AM

#

even if I activate the script first it still uses the /venv

noble summit Jul 19, 2024, 11:18 AM

#

what kinda script you try to run

scenic urchin Jul 19, 2024, 11:19 AM

#

Requirement already satisfied: huggingface-hub in /venv/lib/python3.10/site-packages (from -r requirements.txt (line 1)) (0.23.4)

karmic finch Jul 19, 2024, 11:20 AM

#

hmm yeah aren't you should be using the /venv, thats the point of getting faster speeds on runtime

noble summit Jul 19, 2024, 11:20 AM

#

/venv is only cause it need to sync it to /workspace on secure cloud

hollow mulch Jul 19, 2024, 11:20 AM

#

Problem with /venv is that custom nodes get lost if pod is restarted

karmic finch Jul 19, 2024, 11:21 AM

#

yea, so when does your template syncs back to the more persistent storage ?

noble summit Jul 19, 2024, 11:21 AM

#

it's not as comfy ui uses venv from /workspace

karmic finch Jul 19, 2024, 11:22 AM

#

ohh

#

wait so whats the dependencies on / for?
oh for code server

noble summit Jul 19, 2024, 11:22 AM

#

it's so I can install required packages for comfy ui to start then sync it to /workspace on boot

#

code server is not using venv

karmic finch Jul 19, 2024, 11:23 AM

#

Hm.. so no more download?

scenic urchin Jul 19, 2024, 11:24 AM

#

so pretty much we have 2 instances of python and it confuses the custom_nodes cause half of the dependencies have been installed in one and half on the other

#

is there a way to specify which python to use

noble summit Jul 19, 2024, 11:25 AM

#

nah all custom nodes are installed on /workspace if you use node manager

scenic urchin Jul 19, 2024, 11:27 AM

#

you can't use the comfy ui manager

#

This action is not allowed with this security level configuration.

noble summit Jul 19, 2024, 11:27 AM

#

what it worked for many users do you install some wierd extension

scenic urchin Jul 19, 2024, 11:28 AM

#

#

this is marked as experimental

noble summit Jul 19, 2024, 11:28 AM

#

does not tell me anything

scenic urchin Jul 19, 2024, 11:28 AM

#

No I didn't install weird extensions

#

These are all taken from the latest release

noble summit Jul 19, 2024, 11:29 AM

#

I mean I installed many extensions and all worked so im confused right now

scenic urchin Jul 19, 2024, 11:30 AM

#

you should be able to just ssh into the terminal and use pip to install python packages

noble summit Jul 19, 2024, 11:30 AM

#

though you need to install to venv as comfy ui runs from venv

scenic urchin Jul 19, 2024, 11:31 AM

#

ok, so we are saying /venv is the lib to use

noble summit Jul 19, 2024, 11:31 AM

#

./workspace/venv/bin/python -m pip install -r requirements.txt

#

could be comfy ui have hardcoded /venv

scenic urchin Jul 19, 2024, 11:33 AM

#

I think that may have worked

hollow mulch Jul 19, 2024, 11:39 AM

#

noble summit could be comfy ui have hardcoded /venv

venv hard-code themselves, if you move it somewhere else, you have to update it in the files in bin folder

noble summit Jul 19, 2024, 11:39 AM

#

I mean ComfUI has hardcoded /venv/

hollow mulch Jul 19, 2024, 11:46 AM

#

It does not

#

Why would it do that anyway? Some users are on Windows, Mac etc not all Linux

#

And even for Linux, most users would not run it as root user

scenic urchin Aug 7, 2024, 3:55 PM

#

@noble summit does this image Better ComfyUI CUDA12 Full now copy the venv and ComfyUi you have in your workspace and deploy it?

noble summit Aug 7, 2024, 4:09 PM

#

yes venv is synced to /workspace

scenic urchin Aug 7, 2024, 4:10 PM

#

very nice. That should save me time by loading everything I had in workspace

#

I do notice though that even with 1 x RTX A5000
8 vCPU 29 GB RAM It is terribly slow using CUDA

#how to stop scripts from replacing my configuration