A Way to connect to Remote GPU via SSH Tunnelling? | Invoke | Page 1

sly fulcrum Nov 20, 2022, 5:17 PM

#

Hey there,
I don't have an Nvidia driver and I'd like to use this. Can I rent a GPU and use Putty to Tunnel to a certain port like 9090 and use InvokeAI there?

tiny agate Nov 20, 2022, 5:19 PM

#

i'm doing this with runpod.io, happy to share my invokeai template + docker image but you'll need to know your way around the linux terminal to make it work

sly fulcrum Nov 20, 2022, 5:19 PM

#

My!!!! I wasn't hopeful at all!

#

How!

#

[Throw fishnets over the guy, interrogate him!]

#

@tiny agate

#

I know putty, ssh tunnelling

#

I can access to linux terminal to install and run things

#

I use vast.ai

tiny agate Nov 20, 2022, 5:22 PM

#

docker image is at ghcr.io/damian0815/invoke-ai-runpod:latest

#

it puts an invokeai install in /stable-diffusion

#

to run it you need to start the box, login, conda activate invokeai, download a .ckpt and put it somewhere, edit /stable-diffusion/config/models.yaml to point to it and run python scripts/invoke.py --web --port 9090 --host 0.0.0.0 --cors <external url without trailing />

#

the image is 6gb because it contains pre-populated models cache (just without the SD ckpt itself)

#

idk how vast.ai works but you'll need to open port 9090 or use another one

#

note this is not an "official" image, it's just one i use for development

sly fulcrum Nov 20, 2022, 5:26 PM

#

Erm

#

Am I doing to docker pull this on my own PC, or on runpod 😄

#

@tiny agate

tiny agate Nov 20, 2022, 5:30 PM

#

runpod

#

or vast i guess

#

i can't support this much beyond getting it running i'm afraid

sly fulcrum Nov 20, 2022, 5:31 PM

#

Thank you very much. I'll try figuring things out

#

now that I know it's possible @tiny agate

mental holly Nov 20, 2022, 5:32 PM

#

Yes I do this routinely. Launch the web server on the remote machine and then do an ssh forward from port 9090 on your local machine to the remote machine’s localhost:9090

sly fulcrum Nov 20, 2022, 5:32 PM

#

mental holly Yes I do this routinely. Launch the web server on the remote machine and then do...

You are the creator of the library, right!

#

Hello, how're you doing?

#

Thank you for this code

#

Ok so, let me document

#

I'll go ahead and create a tunnel on putty

#

Add public and private credentials to the server and putty.

#

add login details

#

#

Then go ahead and install Invoke AI onto the machine I rented

#

When things are ready, I'll open localhost:9090 on my local PC and see Invoke AI running there, right?

#

@mental holly

tiny agate Nov 20, 2022, 5:40 PM

#

that looks good, just make sure you pass --host 0.0.0.0 and also append the url you're going to use on your local machine to the --cors argument

#

when you launch invoke.py

sly fulcrum Nov 20, 2022, 5:41 PM

#

@tiny agate Thank you very much.

tiny agate Nov 20, 2022, 5:42 PM

#

np and good luck

mental holly Nov 20, 2022, 5:44 PM

#

Actually this isn’t necessary. Just launch invoke with —web and connect from the desktop with ssh -L9090:localhost:9090 gpu.host.com replacing the last bit with the DNS of the gpu host.

#

The putty config looks correct. cors and host args shouldn’t be necessary but won’t hurt.

sly fulcrum Nov 20, 2022, 7:23 PM

#

There was an Automatic1111 template in the Runpod...
I think I'm going to have to go with that, else I'll be angry at random people for nothing this week. I'm just frustrated with the installation process.

#

I coulnd't git pull or git clone your repo on Runpod due to some GIT_DISCOVERY_ACROSS_FILESYSTEM not set error

#

...also... Jupyter installation guide doesn't work either...

#

i can't install it with the terminal

#

There's no docker file that has the model in it...

#

I hope automatic1111 works

#

before I go to sleep

mental holly Nov 20, 2022, 7:44 PM

#

So sorry you’re having difficulty. The Jupyter notebook does work but the docs are AWOL as you’ve found out.

sly fulcrum Nov 20, 2022, 8:05 PM

#

Oh no please don't feel sorry for me. That's enough sadness for the world of diffusions for today. @mental holly you can instead put nice little video on youtube youtube channel, which I'm subscribed btw, about how to do this thing with remote GPUs.
Cause I didn't like automatic1111, I can't do automation with this stuff. I have more than 20_000 prompts to render...

#

and it seems I can only do it with invokeai

#

(prompts are for educational stuff)

tiny agate Nov 20, 2022, 8:08 PM

#

with my docker image you can run scripts/load_models to download one of the sd1.5 checkpoints, it takes about 2 minutes

#

or put one on mega.nz and download it with mega-cmd

#

the reason we don’t and likely won’t ship ckpt files is because you are supposed to agree to a license to use them, and unlike some other stable diffusion distributions we want to actually respect licensing agreements

sly fulcrum Nov 20, 2022, 8:10 PM

#

I totaly understand and respect that

#

Now let me start over with Runpod

#

I'll select Basic Terminal Access

tiny agate Nov 20, 2022, 8:12 PM

#

no don’t do that

sly fulcrum Nov 20, 2022, 8:12 PM

#

😄

tiny agate Nov 20, 2022, 8:12 PM

#

make a custom template hold on i’ll send you a screenshot

sly fulcrum Nov 20, 2022, 8:12 PM

#

this

#

tiny agate Nov 20, 2022, 8:15 PM

#

#

make sure you go to runpod.io settings first

#

set a ssh-ed25519 key

#

needs to be ed25519

#

https://www.runpod.io/blog/how-to-set-up-terminal-access-on-runpod

sly fulcrum Nov 20, 2022, 8:22 PM

#

do you give it a docker command?

#

like docker run -e HUGGING_FACE=apikey?

#

or environment variables?

tiny agate Nov 20, 2022, 8:33 PM

#

no

sly fulcrum Nov 20, 2022, 8:36 PM

#

2022-11-20T20:34:21Z error pulling image: Error response from daemon: Head "https://ghcr.io/v2/damian0815/invoke-ai-runpod/manifests/latest": denied: denied
2022-11-20T20:34:21Z error creating container: Error response from daemon: No such image: ghcr.io/damian0815/invoke-ai-runpod:latest
2022-11-20T20:34:37Z remove network
2022-11-20T20:35:26Z create pod network
2022-11-20T20:35:26Z create container https://github.com/damian0815/runpod-decent-pytorch/pkgs/container/invoke-ai-runpod
2022-11-20T20:35:26Z error pulling image: invalid reference format
2022-11-20T20:35:26Z error creating container: Error response from daemon: invalid reference format
2022-11-20T20:35:50Z create container https://github.com/damian0815/runpod-decent-pytorch/pkgs/container/invoke-ai-runpod
2022-11-20T20:35:50Z error pulling image: invalid reference format```

#

I tried both this and the github
https://ghcr.io/v2/damian0815/invoke-ai-runpod/manifests/latest

tiny agate Nov 20, 2022, 8:37 PM

#

where is that log coming from?

sly fulcrum Nov 20, 2022, 8:38 PM

#

tiny agate Nov 20, 2022, 8:39 PM

#

weird

sly fulcrum Nov 20, 2022, 8:39 PM

#

My template is like yours

#

Ok it's pulling now

tiny agate Nov 20, 2022, 8:42 PM

#

you certainly don’t need 30gb on the container disk

#

i run with 5

sly fulcrum Nov 20, 2022, 8:43 PM

#

It was saying 18 GB for installation on the docs

#

hmmm I'll reduce one this thing runs

tiny agate Nov 20, 2022, 8:43 PM

#

the docker image is free gb

sly fulcrum Nov 20, 2022, 8:43 PM

#

it's on spot instances anyways

tiny agate Nov 20, 2022, 8:44 PM

#

5gb container, for volume you need basically the size of ckpt you want + the size of images you want to make

sly fulcrum Nov 20, 2022, 8:45 PM

#

that's almost half way done

#

ok now i'm connected

#

How did you manage to put your model into stable-diffusion/models folder after downloading it?

#

through ssh?

mental holly Nov 20, 2022, 8:59 PM

#

I download directly onto the remote machine, but if you need to transfer it look for the SFTP or SCP functions in putty.

sly fulcrum Nov 20, 2022, 8:59 PM

#

#

hahhaha

#

there's no curl for it

#

😄

tiny agate Nov 20, 2022, 9:00 PM

#

put it in /workspace so it persists

#

use the load-models script

#

scripts/load_models.py

sly fulcrum Nov 20, 2022, 9:03 PM

#

A problem occurred during download.
The error was: "Run this script from within the top level of the InvokeAI source code directory, "InvokeAI""

#

I feel like a monkey

mental holly Nov 20, 2022, 9:04 PM

#

Move up one directory level and run python scripts/load_models.py

#

We all feel like monkeys sometimes. Right now I’m craving a banana.

sly fulcrum Nov 20, 2022, 9:06 PM

#

I'm so close

#

I'm so close 🍌

#

it's happening

#

now the final command

#

python scripts/invoke.py --web --port 9090 --host 0.0.0.0

#

I don't know how to manage the cors

#

#

#

inpainting wasn't signed

#

ok

tiny agate Nov 20, 2022, 9:14 PM

#

sly fulcrum I don't know how to manage the cors

on the runpod.io “connect” interface there should be a link to connect to port 9090. copy that, delete the trailing /, append --cors <url>

#

i should write this up as a tutorial i guess

sly fulcrum Nov 20, 2022, 9:15 PM

#

I think I'm going to create a youtube video for this one

sly fulcrum Nov 20, 2022, 9:16 PM

#

tiny agate on the runpod.io “connect” interface there should be a link to connect to port 9...

Also can I buy you a beer?

tiny agate Nov 20, 2022, 9:16 PM

#

hahaha

sly fulcrum Nov 20, 2022, 10:00 PM

#

tiny agate on the runpod.io “connect” interface there should be a link to connect to port 9...

yeah... well...

#

This didn't work either

#

I tried with https

#

http

#

http:/ with one /

tiny agate Nov 20, 2022, 10:02 PM

#

did you do git pull?

#

delete the trailing slash, not the one at http://

sly fulcrum Nov 20, 2022, 10:03 PM

#

no I didn't git pull

tiny agate Nov 20, 2022, 10:03 PM

#

you need to do that to fix the INITFILE error

#

that docker image is synced to my github development branch

sly fulcrum Nov 20, 2022, 10:03 PM

#

I thought everything I needed was in the dockerfile

tiny agate Nov 20, 2022, 10:04 PM

#

it is but the code is a moving target

#

i’m not rebuilding a 6gb image every time i commit

sly fulcrum Nov 20, 2022, 10:05 PM

#

believe me I don't even know how to use github

#

I guess I'll

git pull ghcr.io/damian0815/invoke-ai-runpod:latest

workspace/stable-diffusion

#

I have absolutely no clue what to do next.

#

what am I pulling?

#

(invokeai) root@7697e9b9e2cd:/stable-diffusion# git pull
remote: Enumerating objects: 32, done.
remote: Counting objects: 100% (25/25), done.
remote: Total 32 (delta 25), reused 25 (delta 25), pack-reused 7
Unpacking objects: 100% (32/32), 11.00 KiB | 866.00 KiB/s, done.
From https://github.com/damian0815/InvokeAI
 * [new branch]      play-kaleidoscope -> origin/play-kaleidoscope
Already up to date.

#

but unfortunately

(invokeai) root@7697e9b9e2cd:/stable-diffusion# python scripts/invoke.py --web --port 9090 --host 0.0.0.0 --cors https://xxxxxxxxxxxxxxxxx-9090.proxy.runpod.net
* Initializing, be patient...
An exception has occurred: name 'INITFILE' is not defined

tiny agate Nov 20, 2022, 10:30 PM

#

ahh then maybe i didn’t update it

#

sec

#

ok now just type git pull

#

and try again

#

@sly fulcrum

sly fulcrum Nov 20, 2022, 10:32 PM

#

I closed the instance for today... Damian

#

I thank you for your patience

#

You truly are a hero

#

I'll try it first thing tomorrow. Now I gotta sleep

sly fulcrum Nov 21, 2022, 11:56 AM

#

wow ok

#

It works after git pull

#

>> Initialization file /root/.invokeai not found. Creating a new one...
>> InvokeAI runtime directory is "."
Downloading: "https://github.com/DagnyT/hardnet/raw/master/pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth" to /root/.cache/torch/hub/checkpoints/checkpoint_liberty_with_aug.pth
100%|██████████████████████████████████████| 5.10M/5.10M [00:00<00:00, 66.4MB/s]
>> GFPGAN Initialized
>> CodeFormer Initialized
>> ESRGAN Initialized
>> Using device_type cuda
>> Current VRAM usage:  0.00G
>> Scanning Model: stable-diffusion-1.5
>> Model Scanned. OK!!
>> Loading stable-diffusion-1.5 from models/ldm/stable-diffusion-v1/v1-5-pruned-emaonly.ckpt
>> Calculating sha256 hash of weights file
>> sha256 = cc6cb27103417325ff94f52b7a5d2dde45a7515b25c255d8e396c90014281516 (2.59s)
   | LatentDiffusion: Running in eps-prediction mode
   | DiffusionWrapper has 859.52 M params.
   | Making attention of type 'vanilla' with 512 in_channels
   | Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
   | Making attention of type 'vanilla' with 512 in_channels
   | Using faster float16 precision
   | Loading VAE weights from: models/ldm/stable-diffusion-v1/vae-ft-mse-840000-ema-pruned.ckpt
>> Model loaded in 17.30s
>> Max VRAM used to load the model: 2.17G
>> Current VRAM usage:2.17G
>> Setting Sampler to k_lms

* --web was specified, starting web server...
>> Initialization file /root/.invokeai found. Loading...
DEBUG: static_folder should be at ../frontend/dist
>> Started Invoke AI Web Server!
Point your browser at http://localhost:9090 or use the host's DNS name or IP address.
>> System config requested

#

#

Ta da!

#

So happy!

#

#

This was me yesterday

tiny agate Nov 21, 2022, 12:47 PM

#

sly fulcrum So happy!

yay!

#

a pro tip for using runpod is to symlink the models folder to /workspace so that you don't have to re-download the .ckpt files every time you stop the instance or are kicked off it (if you're using spot pricing)

#

coming in v2.1.4 is in fact a way to tell load_models to do that for you

#A Way to connect to Remote GPU via SSH Tunnelling?