I have a python-ml-3.11 function and it's failing to deploy, build, etc. it's just broked. Container logs are fine, function logs are fine, it SAYS It built, but Appwrite is determined to tell me it's still building.
#Function Failing to Deploy or Build
obsidian yew
Docker Error:
/usr/src/code/src/Executor/Executor.php:97
#0 /usr/src/code/src/Appwrite/Platform/Workers/Builds.php(407): Executor\Executor->createRuntime('66a88764b1aab86...', '664d429a000824f...', '/storage/functi...', 'openruntimes/py...', 'v3', true, 'src/main.py', '/storage/builds...', Array, 'tar -zxf /tmp/c...')
#1 [internal function]: Appwrite\Platform\Workers\Builds->Appwrite\Platform\Workers\{closure}()
#2 {main}
patent bison
I have a `python-ml-3.11` function and it's failing to deploy, build, etc. it's ...
what are the dependencies and how big are they?
obsidian yew
what are the dependencies and how big are they?
appwrite
pydantic
onnxruntime
PyWavelets
scipy
invisible-watermark
invisible-watermark requires opencv, which shouldn't be an issue
I added commands beforehand, it runs them lol though it errors
patent bison
I added commands beforehand, it runs them lol though it errors
maybe that's why it's not building
obsidian yew
maybe that's why it's not building
no I only added them to try it
I'll rebuild rn just to show
This is the same function I mentioned before
obsidian yew
maybe that's why it's not building
it builds, sort of,
it's been here for about a minute
yeah just hung
Anything I can do?
This is pretty important for me because it's a main piece of the functionality of the site, if I have to recode it I might need to host a separate python API
is it possible the openruntimes needs to be updated more than the self-hosted API states or something?
probably not
patent bison
it builds, sort of,
yup. some package is too big
look at the logs in the build worker
obsidian yew
yup. some package is too big
but like, I can't control what pip downloads
I am
2024-07-30T23:46:28.032999992Z [Job] Received Job (66a97b5407e732.94441764).
2024-07-30T23:46:28.034528794Z Creating build for deployment: 66a97b5403feabaa0d50
why would it be too big?
pytorch alone downloads like 2 gigs or something, if you use embeddings those models can be like 500MB+, usually more towards 1-2 gigs
patent bison
i think appwrite can only handle function code up to 2GB
obsidian yew
i think appwrite can only handle function code up to 2GB
2024-07-30T23:54:50.164580467Z [Job] (66a97b5407e732.94441764) failed to run.
2024-07-30T23:54:50.164608970Z [Job] (66a97b5407e732.94441764) Invalid document structure: Attribute "size" has invalid type. Value must be a valid range between -2,147,483,647 and 2,147,483,647
2024-07-30T23:54:50.164711452Z [Error] Type: Utopia\Database\Exception\Structure
2024-07-30T23:54:50.164807563Z [Error] Message: Invalid document structure: Attribute "size" has invalid type. Value must be a valid range between -2,147,483,647 and 2,147,483,647
2024-07-30T23:54:50.164863367Z [Error] File:
2024-07-30T23:54:50.164929601Z [Error] Line: 0
but
obsidian yew
i think appwrite can only handle function code up to 2GB
why is there a size limit?
like
is it something internal, can I raise it, can I change the docker image? I don't really know what the best solution is haha
patent bison
is it something internal, can I raise it, can I change the docker image? I don't...
you can't tweak this at the moment
obsidian yew
but it also removes my ability to use most AI packages, which sort of defeats the purpose of it doesn't it?
these aren't even AI packages, sadly
patent bison
is it something internal, can I raise it, can I change the docker image? I don't...
Looks like we're going to add an env var: https://github.com/appwrite/appwrite/pull/8396
obsidian yew
oh okay, so, I just need to wait some amount of time okay, hm
I'll see if I can remove any of those deps
patent bison
I'll see if I can remove any of those deps
it might help to see which one of those is pulling in other things that's making it huge
obsidian yew
it might help to see which one of those is pulling in other things that's making...
I mean lol, I'm using invisible-watermark which is basically using dwtDct to encode things into an image :p
it's not really low level stuff, so it needs image manipulation and stuff
patent bison
something is bringing in nvidia_cuda stuff which is almost 1.6GB
obsidian yew
poetry add invisible-watermark
Using version ^0.2.0 for invisible-watermark
Updating dependencies
Resolving dependencies... (1.7s)
Package operations: 27 installs, 0 updates, 0 removals
- Installing nvidia-nvjitlink-cu12 (12.5.82)
- Installing filelock (3.15.4)
- Installing markupsafe (2.1.5)
- Installing mpmath (1.3.0)
- Installing nvidia-cublas-cu12 (12.1.3.1): Pending...
- Installing nvidia-cusparse-cu12 (12.1.0.106): Installing...
- Installing nvidia-cublas-cu12 (12.1.3.1): Installing...
- Installing nvidia-cusparse-cu12 (12.1.0.106): Installing...
- Installing nvidia-cublas-cu12 (12.1.3.1)
- Installing nvidia-cusparse-cu12 (12.1.0.106)
- Installing fsspec (2024.6.1): Installing...
- Installing jinja2 (3.1.4): Installing...
- Installing networkx (3.3): Pending...
- Installing numpy (2.0.1): Downloading... 0%
- Installing fsspec (2024.6.1)
- Installing jinja2 (3.1.4)
- Installing networkx (3.3): Installing...
- Installing numpy (2.0.1): Downloading... 18%
- Installing nvidia-cuda-cupti-cu12 (12.1.105): Installing...
- Installing numpy (2.0.1): Downloading... 30%
- Installing networkx (3.3)
- Installing numpy (2.0.1): Downloading... 90%
- Installing nvidia-cuda-cupti-cu12 (12.1.105)
- Installing numpy (2.0.1): Installing...
- Installing nvidia-cuda-cupti-cu12 (12.1.105)
- Installing nvidia-cuda-nvrtc-cu12 (12.1.105)
- Installing nvidia-cuda-runtime-cu12 (12.1.105)
- Installing numpy (2.0.1)
- Installing nvidia-cuda-cupti-cu12 (12.1.105)
- Installing nvidia-cuda-nvrtc-cu12 (12.1.105)
- Installing nvidia-cuda-runtime-cu12 (12.1.105)
- Installing nvidia-cudnn-cu12 (9.1.0.70): Downloading... 34%
- Installing nvidia-cufft-cu12 (11.0.2.54)
- Installing nvidia-cudnn-cu12 (9.1.0.70): Installing...
- Installing nvidia-cufft-cu12 (11.0.2.54)
- Installing nvidia-cudnn-cu12 (9.1.0.70)
- Installing nvidia-cufft-cu12 (11.0.2.54)
- Installing nvidia-curand-cu12 (10.3.2.106)
- Installing nvidia-cusolver-cu12 (11.4.5.107)
- Installing nvidia-nccl-cu12 (2.20.5)
- Installing nvidia-nvtx-cu12 (12.1.105)
- Installing sympy (1.13.1)
yeah
invisible-watermark
patent bison
maybe you can exclude the cuda stuff.
obsidian yew
it also uses torch haha
patent bison
that's useless, in Appwrite, right?
obsidian yew
yeah, but Python is dumb :\
I'll try seeing
I think we stopped using invisible watermark, so I'll see if I can revamp it if we didn't
heady cargo
I am seeing the same error for Swift 5.8 functions I have tried to deploy/build. My last attempt was to create a new Appwrite project, init a new function, and deploy. But that had the same result (failed). I first noticed after migrating a project from Appwrite 1.4.9 to 1.5.7, my functions stopped deploying/building.
#0 /usr/src/code/src/Appwrite/Platform/Workers/Builds.php(407): Executor\Executor->createRuntime('66b02b15028b1b4...', '66b021f6002c443...', '/storage/functi...', 'openruntimes/sw...', 'v3', true, 'Sources/index.s...', '/storage/builds...', Array, 'tar -zxf /tmp/c...')
#1 [internal function]: Appwrite\Platform\Workers\Builds->Appwrite\Platform\Workers\{closure}()
#2 {main}```what's the size of the deployed function?
heady cargo
what's the size of the deployed function?
3.1 KB
obsidian yew
hm, does the function runtime still exist?
heady cargo
Would that be in the logs somewhere if it doesn't exist?
obsidian yew
Would that be in the logs somewhere if it doesn't exist?
I’m assuming you’ve redeployed it since you upgraded right?