#Nixpack build error with Django app

203 messages · Page 1 of 1 (latest)

lament trellis
#

Hi there, I am a former Heroku user trying to migrate to a Nixpack buildpack given the foreseen discontinuation of the Heroku's, but I am receiving the following error with my Django app:

lament trellis
#

My Dockerfile works fine locally. How should I install the packages that will help psycopg2 installation go through without errors?

tame steeple
#

#12 6.407 pg_config is required to build psycopg2 from source. Please add the directory
#12 6.407 containing pg_config to the $PATH or specify the full executable path with the
#12 6.407 option:
#12 6.407
#12 6.407 python setup.py build_ext --pg-config /path/to/pg_config build ...
#12 6.407
#12 6.407 or with the pg_config option in 'setup.cfg'.
#12 6.407
#12 6.407 If you prefer to avoid building psycopg2 from source, please install the PyPI

#

(Copying error from text file into chat here for search)

#

I don’t have great insight to help further here, but interesting there’d be a difference in build locally vs on railway platform. What’s your dockerfile look like?

lament trellis
# tame steeple #12 6.407 pg_config is required to build psycopg2 from source. Please add...

My Dockerfile is tailored for the local environment. I think my project setup has to be overhauled in order to make it work with Nixpacks remotely. So I am starting from scratch. In order to have Python requirements install on Heroku, I used to have a requirements.txt in the project root with the dependencies needed for the remote Heroku environment only. Is this how it is supposed to be done with Nixpacks as well? Also, is there a Nixpack-compatible Python project example available anywhere?

tame steeple
#

There may be more examples. Note I’m just a user, and haven’t worked much with python before. Hope above links help

lament trellis
#

Thanks. The root requirements.txt thing is bugging me though. I thought the Nixpack buildpack was looking into the Dockerfile only, but it is actually looking into the root requirements.txt first and tries to install those. I don't see this covered in the guide at all.

tiny inlet
#

If you have a Dockerfile it should use that

#

If it's in not the root, you'll wanna set the root directory to where the Dockerfile is is

#

Or set Variable DOCKERFILE_PATH to the dockerfiles path

lament trellis
#

My Dockerfile is in the root indeed, but the requirements.txt also is, and Nixpack is somehow preferring the latter.

tiny inlet
#

Fucky. It should pickup Dockerfile first

#

Is it named correctly?

#

Dockerfile

lament trellis
#

Dockerfile

tiny inlet
#

Capitals matter

lament trellis
#

indeed

tiny inlet
#

Mmmm

#

ProjectID?

lament trellis
#

9a8cdddf-87aa-4caf-848a-69fb07710946

#

I am wondering, does Nixpack support multistaging? Or should I go for using separate Dockerfiles for the local vs remote envs?

tiny inlet
#

What would you need to change in your Dockerfile for local vs remote

lament trellis
#

Locally I am using dependencies (mainly for testing and debugging) that are of no use remotely.

tiny inlet
#

Gotchya

#

We've just added multiple provider support to Nixpacks. So, in the future yes but for now no

lament trellis
#

I see

tiny inlet
lament trellis
#

Ok, thank you.

tiny inlet
#

uhh it appears to be using your dockerfile

lament trellis
#

Mmh, when I do sudo nixpack build . it seems to get the requirements.txt instead. Anyway, I am taking it out for good.

#

A skeleton Django project built with a view to using the Nixpack buildpack would be great to inspect, with an example pyproject.toml and an example artifact the sudo nixpack build . command produces at the end of the build.

misty spindle
#

You need to use psycopg2-binary package instead of psycopg2. That should solve this issue

lament trellis
#

Thanks, but I can't. I have an ARM64 CPU. I'll have to work on my Dockerfile.

lament trellis
# tiny inlet https://railway.app/project/9a8cdddf-87aa-4caf-848a-69fb07710946/service/a16ffe7...

Ok, back at it again. Nixpack building is soo slow. Any clue as to why it's behaving like this?

[+] Building 621.4s (4/14)                                                      
 => [internal] load build definition from Dockerfile                       0.0s
[+] Building 621.5s (5/14)                                                      
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 1.08kB                                     0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for ghcr.io/railwayapp/nixpacks:debian-16654  0.4s
 => [internal] load build context                                          0.4s
 => => transferring context: 25.57MB                                       0.4s
 => CANCELED [stage-0  1/10] FROM ghcr.io/railwayapp/nixpacks:debian-16  621.0s
 => => resolve ghcr.io/railwayapp/nixpacks:debian-1665446966@sha256:48088  0.0s
 => => sha256:4ce525b9479139c4ce71ba043d4d15360ae04216 29.77MB / 29.77MB  21.2s
 => => sha256:48088795aef10698578d862349bf230f0ace63c978f 1.05kB / 1.05kB  0.0s
 => => sha256:ba70d21f951d0969e1375f9320cd8578b2eda8e9a04904f 953B / 953B  0.0s
 => => sha256:a2593eb06f40909eccfe277c5d466d69938da501a67 3.84kB / 3.84kB  0.0s
 => => sha256:a9fe95647e78b5516c7e2327355b6996e2ea295 22.02MB / 30.05MB  621.0s
 => => sha256:adbc07ad76c8682da11b96475f4ed9dcbab30d78 7.34MB / 43.03MB  621.0s
#

FYI, I am peaking at 20 mb/s downloading games off of Steam.

tiny inlet
lament trellis
#

?

tiny inlet
#

Oh are you running this locally?

#

Sorry I thought it was on the builder

#

Unsure then could be GCR

lament trellis
#

Uhm ok, it should take like a few seconds to download 40 megs

#

but I was over 10 mins

tiny inlet
#

Unsure sadly we can't debug local stuff

lament trellis
#

I am wondering, what it supposed to happen at the end of the build? What should be produced?

tiny inlet
#

An image

lament trellis
#

Is that it? A Docker image?

tiny inlet
#

Yes

lament trellis
#

And that image is pushed remotely at the end of the build?

#

Sorry, I don't get how that's supposed to work.

tiny inlet
#

It's on your local

#

So you can test it however you'd like

lament trellis
#

I am already producing an image with my own docker compose

tiny inlet
#

That's not usable on Railway

lament trellis
#

I don't need to do nixpack build .

#

Ok

#

Do you believe it would be possible to only have those files in the repo without doing nixpack build .?

tiny inlet
lament trellis
#

Because from here it looks like an unnecessary chore for those having a docker compose

#

producing a set of images

tiny inlet
lament trellis
#

It looks like an unnecessary chore to have another image

#

just for Railway

#

or am I missing something?

tiny inlet
#

What's your issue?

lament trellis
#

it means there's something more in that image produced by nixpack build . that doing docker compose build doesn't do.

tiny inlet
#

Here's a django example

lament trellis
# tiny inlet What's your issue?

I have a couple:

  1. nixpack is not working at the moment as you can see
  2. I feel like I will be polluting my local system with docker images that'll be used just by Railway
tiny inlet
#

Deploy a Django application correct?

lament trellis
#

Yes

tiny inlet
#

Great

lament trellis
#

thanks for the link

#

That looks like a vanilla Django project with a Heroku Procfile

#

I am mentioning Heroku because it's where I was before

tiny inlet
#

You can delete the Heroku procfile if you'd like

lament trellis
#

Ok

tiny inlet
lament trellis
#

I don't see a Dockerfile in there

tiny inlet
#

That's because there isn't one

lament trellis
#

Ok

#

so what's the screenshot above referring to?

tiny inlet
#

The repo builds just fine and is Django

lament trellis
#

Isn't it supposed to be feeding from a local Dockerfile?

tiny inlet
lament trellis
#

Uhm, ok

tiny inlet
#

You can either write a Dockerfile, or have Nixpacks run detection on your code

lament trellis
#

So you don't need a local Dockerfile

tiny inlet
#

You do not

lament trellis
#

Ok

tiny inlet
#

UNLESS, you want to write one

lament trellis
#

Ok

tiny inlet
#

In which case we also support that

lament trellis
#

I'd like to keep a Dockerfile.local but maybe I don't need a Dockerfile for Nixpack

#

(btw, thanks for taking the time to give support)

tiny inlet
tiny inlet
#

Otherwise, Railway will try and use it

lament trellis
#

Ok

#

And Railway will recognize a requirements.txt file placed in the root only, correct?

tiny inlet
#

So if you have a monorepo and it's backend/requirements.txt, set root to backend/

lament trellis
#

When you say "set root in settings", which settings are you talking about?

tiny inlet
#

Service -> Settings -> Root Directory

lament trellis
#

Uhm ok maybe I don't need one

#

Anyways, what you meant before is that nixpack build . is just for local development?

tiny inlet
#

Well

#

Sorta in a way I suppose

#

You can run it locally to test Nixpacks

It's what we run when you build stuff in production using nixpacks

lament trellis
#

Yeah

#

you advise to do nixpack build , locally to test out the project before deploying. But it's not mandatory to run that command in order to deploy.

tiny inlet
#

Just trigger the deploy and see the logs

#

Should be fast enough to do that

lament trellis
#

Thanks, I understand the philosophy behind nixpacks better now.

#

I managed to deploy the app, but for some reason statics are not being served

#

it's 404 all over the place

lament trellis
# tiny inlet Should be fast enough to do that

How come in the example Django repo you have psycopg2 (and not psycopg2-binary) as a requirement, but when I push it it fails with

#12 13.65       Error: pg_config executable not found.
#12 13.65
#12 13.65       pg_config is required to build psycopg2 from source.  Please add the directory
#12 13.65       containing pg_config to the $PATH or specify the full executable path with the
#12 13.65       option:
#12 13.65
#12 13.65           python setup.py build_ext --pg-config /path/to/pg_config build ...
#12 13.65
#12 13.65       or with the pg_config option in 'setup.cfg'.
#12 13.65
#12 13.65       If you prefer to avoid building psycopg2 from source, please install the PyPI
#12 13.65       'psycopg2-binary' package instead.
buoyant wagon
lament trellis
#

I do

#

just checked

buoyant wagon
#

Please share your requirements.txt file.

lament trellis
#

Sure

#

My root requirements.txt is just saying:

-r requirements/remote.txt

I am sharing the actual remote.txt contents in a minute

buoyant wagon
#

Thus it fails

#

Try setting NIXPACKS_PKGS=postgresql under variables.

lament trellis
#

Done. It's redeploying... let's see what happens.

buoyant wagon
#

cool

lament trellis
#

It's taking more than 10 mins now

#

but it is indeed doing stuff

#

Ok it appears to have gone past the postgres deps build

buoyant wagon
#

Don't worry, future builds won't take that long. Railway will cache all your deps.

lament trellis
#

Ok great

#

thanks

#

I think it's almost done

#

let's see if it goes through

tiny inlet
#

What dep do you have that needs PG?

#

So we can add it to Nixpacks

buoyant wagon
tiny inlet
#

We scan for that tho :/

buoyant wagon
tiny inlet
#

O

lament trellis
#

Ok deploy successful

#

Thank you

#

Only problem now is

#

statics are not served, but nothing changed on my side

#

I see 404s for every static

buoyant wagon
#

Is your repository open source?

lament trellis
#

not really

#

I could invite you though

buoyant wagon
#

sure ty,

lament trellis
buoyant wagon
#

cool, accepted.

#

Please share your deploy logs.

lament trellis
# 
/opt/venv/lib/python3.8/site-packages/whitenoise/base.py:115: UserWarning: No directory at: /app/static/
warnings.warn(f"No directory at: {root}")
/opt/venv/lib/python3.8/site-packages/whitenoise/base.py:115: UserWarning: No directory at: /app/static/
warnings.warn(f"No directory at: {root}")
/opt/venv/lib/python3.8/site-packages/whitenoise/base.py:115: UserWarning: No directory at: /app/static/
warnings.warn(f"No directory at: {root}")
#

mmh maybe I should be doing python manage.py collectstatic somewhere

#

not sure where though.

buoyant wagon
#

set this as your start command python manage.py collectstatic && gunicorn niccolomineo.asgi:application --log-level=warning -b 0.0.0.0:$PORT --max-requests 14000 -w 3 -k niccolomineo.workers.UvicornDjangoWorker

lament trellis
#

Thanks. I would like to do without the Procfile if possible

buoyant wagon
#

Service -> Settings -> Start Command

lament trellis
#

Do you think the "release" command could be specified somewhere but in the Procfile?

lament trellis
buoyant wagon
lament trellis
#

Thanks. What should be as the build command?

buoyant wagon
#

nothing in your case because you are using python

#

just set your release command as build command

lament trellis
#

should I just put python manage.py migrate --no-input

buoyant wagon
#

(without &&)

#

yup

lament trellis
#

ok great

#

Thank you guys, it's working great now.

lament trellis
buoyant wagon
lament trellis
#

I am redeploying now, will tell you shortly if it works out of the box.

# 
#12 20.00 ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
#12 20.00     backports.zoneinfo from https://files.pythonhosted.org/packages/1a/ab/3e941e3fcf1b7d3ab3d0233194d99d6a0ed6b24f8f956fc81e47edc8c079/backports.zoneinfo-0.2.1-cp38-cp38-manylinux1_x86_64.whl#sha256=7b0a64cda4145548fed9efc10322770f929b944ce5cee6c0dfe0c87bf4c0c8c9 (from django==4.1.2->-r requirements/remote.txt (line 205))
buoyant wagon
#

I am sorry that I cannot provide definitive answers. I don't have a background in python. 😦 I am more of a c/typescript/golang/rust guy.

lament trellis
#

Don't worry, you've been extremely helpful

#

Cooper and you, that is

#

💯

#

So you were saying I might override NIXPACKS_INSTALL_CMD

buoyant wagon
#

yes

lament trellis
#

I tried both pip3 and pip, but it says command not found

buoyant wagon
#

huh, build logs please

#

mind sharing complete build logs from the start?

lament trellis
#

Sorry, like this it should be better and in full

lament trellis
buoyant wagon
#

Sure, I'll take a look at this in couple of hours.

lament trellis
#

Thank you!