#Can't setup a1111 on serverless.. Service not ready error

49 messages · Page 1 of 1 (latest)

sullen leaf
#

Hi guys, I am wondering if anyone managed to setup a1111 on serverless RunPod without network volumes. I am using the blog post https://blog.runpod.io/custom-models-with-serverless-and-automatic-stable-diffusion/ with the https://github.com/runpod-workers/worker-a1111 but I can't seem to get it to work. I managed to build the image, createthe template and endpoint but I get a Service not ready yet. Retrying... error from a Request Exception. I also mounted the image on a container locally and tried to send a request only to get an error (56) Recv failure: Connection reset by peer which seems lke the server closes the connection. Anyone had success with this?

RunPod Blog
Create a custom AUTOMATIC1111 serverless deployment with your custo...

Have you ever wanted to create your own serverless AUTOMATIC1111 endpoint with a custom model that can scale up and down? Now you can do so without much hassle by following this guide!

Pre-requisites

A computer (local or cloud) with the following:

  • docker installed
  • git installed
  • a relatively fast upload
GitHub
GitHub - runpod-workers/worker-a1111: Automatic1111 serverless worker.

Automatic1111 serverless worker. . Contribute to runpod-workers/worker-a1111 development by creating an account on GitHub.

#

PS. my end goal is to setup a1111 for api use with my custom model saved in the docker image.

trail hinge
upbeat trout
#

for the image*

sullen leaf
sullen leaf
upbeat trout
#

Maybe the images broken too try that Ashley's template first

upbeat trout
sullen leaf
trail hinge
#

Connection reset by peer shouldn't ever happen even if your endpoint isn't working properly. The requests should still go into the queue.

sullen leaf
upbeat trout
sullen leaf
upbeat trout
#

Have you tried the Ashley's template on endpoint?

trail hinge
sullen leaf
upbeat trout
#

Hmm yeah try with that network volume or modify the code

trail hinge
#

You can fork it and make it work without network volumes but probably easier just to get the RunPod one working then.

upbeat trout
#

I never tried local testing BTW so i can't really hekp

upbeat trout
trail hinge
#

There are some examples installing models, ControlNet etc

sullen leaf
#

yes will look today into creating a fresh implementation without Network Volumes

upbeat trout
#

The base premade template *

sullen leaf
#

I've already done a Network Volume implementation previous and it works, I get problems with the non-network volume ones.. :/

#

anw will jump right into it

#

thanks for your comments

upbeat trout
#

Alright

#

Read the Readme and the scripts on github

#

You'll soon figure something out for it

sullen leaf
#

will do thnx

sullen leaf
#

2024-05-18T12:36:03.725845977Z Service not ready yet. Retrying...
2024-05-18T12:36:03.929343220Z Service not ready yet. Retrying...
2024-05-18T12:36:04.111961339Z Traceback (most recent call last):
2024-05-18T12:36:04.112013597Z File "/stable-diffusion-webui/webui.py", line 31, in <module>
2024-05-18T12:36:04.112020304Z import gradio
2024-05-18T12:36:04.112025577Z File "/usr/local/lib/python3.10/site-packages/gradio/init.py", line 3, in <module>
2024-05-18T12:36:04.112030334Z import gradio.components as components
2024-05-18T12:36:04.112034719Z File "/usr/local/lib/python3.10/site-packages/gradio/components.py", line 55, in <module>
2024-05-18T12:36:04.112039137Z from gradio import processing_utils, utils
2024-05-18T12:36:04.112043804Z File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 517, in <module>
2024-05-18T12:36:04.112104181Z class AsyncRequest:
2024-05-18T12:36:04.112135458Z File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 536, in AsyncRequest
2024-05-18T12:36:04.112142558Z client = httpx.AsyncClient()
2024-05-18T12:36:04.112148714Z File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1397, in init
2024-05-18T12:36:04.112545061Z self._transport = self._init_transport(
2024-05-18T12:36:04.112626920Z File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1445, in _init_transport
2024-05-18T12:36:04.112766887Z return AsyncHTTPTransport(
2024-05-18T12:36:04.112789494Z File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 275, in init
2024-05-18T12:36:04.112794809Z self._pool = httpcore.AsyncConnectionPool(
2024-05-18T12:36:04.112799462Z TypeError: AsyncConnectionPool.init() got an unexpected keyword argument 'socket_options'
2024-05-18T12:36:04.132782329Z Service not ready yet. Retrying...
2024-05-18T12:36:04.336979246Z Service not ready yet. Retrying...

#

this is what I am getting no matter what I do... 😦

#

can't seem to get it to work

#

tried all other repos as well with network volumes

#

I'm about to give up on RunPod

#

going to replicate

trail hinge
#

Which one is this? The RunPod one or the ashleykleynhans one? Looks like its the RunPod one using an outdated version of gradio or something.

#

I don't think the RunPod one is maintained, so you probably need to update a few things if you want to use it.

upbeat trout
trail hinge
upbeat trout
trail hinge
#

Probably the RunPod one

#

dependabot broke it

#

Its fixed at a specific commit SHA, but @stoic eagle merged a dependabot PR to bump gradio to an incompatible version.

#

By the way @sullen leaf , diffusers is actually far more performant than A1111 in serverless if you're only going to be using a single model and baking it into your image.