Microsoft Florence-2 model in serverless container doesn't work

[rugged trout]

I'm trying to use Florence 2 models in ComfyUI workflow with serverless container and it returns with error:
raise RuntimeError(f'{node_type}: {exception_message}')\nRuntimeError: DownloadAndLoadFlorence2Model: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate

Accelerate library already installed in venv in network storage where comfyui runs, also I installed it in docker container. Maybe anyone know how to solve this problem? Thanks in advance

[modern crater]

Is accelerate listed in the requirements.txt file? If not you will need to add it there and rebuild the docker image.

[rugged trout]

I added accelerate to requirements.txt, rebuild image and create endpoint and got the same result:

{
"delayTime": 5722,
"error": "Traceback (most recent call last):\n File "/rp_handler.py", line 317, in handler\n raise RuntimeError(f'{node_type}: {exception_message}')\nRuntimeError: DownloadAndLoadFlorence2Model: Using low_cpu_mem_usage=True or a device_map requires Accelerate: pip install accelerate\n",
"executionTime": 34016,
"id": "df70ffee-8bf7-4304-9cdd-f6ef9349139c-u1",
"status": "FAILED",
"workerId": "7l3v5ol024c0kb"
}

[modern crater]

Does your handler have

import accelerate

in the Python source code?

[rugged trout]

accelerate lib imported in transformers.py in ComfyUi core

[modern crater]

I suggest adding

import accelerate

at the top of your rp_handler.py file. As, from your error, it is failing on line 317 of that file because the accelerate module is not loaded.

[rugged trout]

It didn't help, i guess because error raised in transformers.py in core of ComfyUI, but I'm sure that accelerator is installed in all possible environments. And another interesting thing is if i deploy a Pod (not serverless) using the same network volume as in serverless - Florence 2 model would works without any problems

I added some code lines for debug inside a container

[modern crater]

That's odd. Maybe it is a specific version of accelerator that it is looking for? If it was recently updated maybe roll back a version. If not, I'm not sure what else it could be.

[chrome epoch]

nope its cause venv path mismatch

[rugged trout]

How to solve this problem?

[round edge]

responded to zendesk ticket