toxic zenith
Hello Railway
I am trying to host my backend (and later frontend) on Railway, but seem to be having some trouble. The repo has a root directory specified to ensure the backend is what's being deployed.
However it seems that despite there being a Procfile, requirements.txt and that the root has a runtime.txt with the specific Python version needed, that it does not see it properly.
Anyone know what could be causing the issue ? Or is able to help
toxic zenith
4cd0f196-cd60-4ec1-bf0d-51e5ec7fdf63
sage basalt
Hello Railway
I am trying to host my backend (and later frontend) on Railway, b...
Your issue is with the Python version, right?
toxic zenith
I got a feeling that it's moreso the repository. I'm told it cannot read gunicorn, but that IS in the requirements.txt so I'm not sure what is going wrong
sage basalt
I got a feeling that it's moreso the repository. I'm told it cannot read gunicor...
Can you show the build and deploy logs?
toxic zenith
Can you show the build and deploy logs?
Sure, how do I copy the logs from there though, don't seem to see a copy button
Guess I can just mark it:
near echo
[Region: us-west1]
==============
Using Nixpacks
==============
context: 0081f12fc1ce8a5c234af1269728346f
╔═════════════════════════ Nixpacks v1.31.0 ═════════════════════════╗
║ setup │ nodejs_18, npm-8_x ║
║────────────────────────────────────────────────────────────────────║
║ install │ npm ci ║
║────────────────────────────────────────────────────────────────────║
║ start │ gunicorn --access-logfile - --error-logfile - run:app ║
╚════════════════════════════════════════════════════════════════════╝
[internal] load build definition from Dockerfile
[internal] load build definition from Dockerfile
[internal] load build definition from Dockerfile ✔ 0ms
[internal] load build definition from Dockerfile
[internal] load build definition from Dockerfile ✔ 11ms
[internal] load metadata for ghcr.io/railwayapp/nixpacks:ubuntu-1736208272
[internal] load metadata for ghcr.io/railwayapp/nixpacks:ubuntu-1736208272 ✔ 101ms
[internal] load .dockerignore
[internal] load .dockerignore ✔ 0ms
[internal] load .dockerignore
[internal] load .dockerignore ✔ 9ms
[stage-0 8/8] COPY . /app
[stage-0 7/8] RUN printf '\nPATH=/app/node_modules/.bin:$PATH' >> /root/.profile
[internal] load build context ✔ 0ms
[internal] load build context
[internal] load build context ✔ 475ms
[stage-0 3/8] COPY .nixpacks/nixpkgs-5624e1334b26ddc18da37e132b6fa8e93b481468.nix .nixpacks/nixpkgs-5624e1334b26ddc18da37e132b6fa8e93b481468.nix ✔ 0ms – CACHED
[stage-0 4/8] RUN nix-env -if .nixpacks/nixpkgs-5624e1334b26ddc18da37e132b6fa8e93b481468.nix && nix-collect-garbage -d ✔ 0ms – CACHED
[stage-0 5/8] COPY . /app/.
[stage-0 5/8] COPY . /app/. ✔ 332ms
[stage-0 6/8] RUN --mount=type=cache,id=s/0d3c87d4-0b8e-4709-b1a1-f5b6127f16be-/root/npm,target=/root/.npm npm ci
npm warn config production Use `--omit=dev` instead.
up to date, audited 1 package in 392ms
found 0 vulnerabilities
[stage-0 6/8] RUN --mount=type=cache,id=s/0d3c87d4-0b8e-4709-b1a1-f5b6127f16be-/root/npm,target=/root/.npm npm ci ✔ 864ms
[stage-0 7/8] RUN printf '\nPATH=/app/node_modules/.bin:$PATH' >> /root/.profile
[stage-0 7/8] RUN printf '\nPATH=/app/node_modules/.bin:$PATH' >> /root/.profile ✔ 136ms
[stage-0 8/8] COPY . /app
[stage-0 8/8] COPY . /app ✔ 224ms
[auth] sharing credentials for us-west1.registry.rlwy.net
[auth] sharing credentials for us-west1.registry.rlwy.net ✔ 0ms
=== Successfully Built! ===
Run:
docker run -it us-west1.registry.rlwy.net/0d3c87d4-0b8e-4709-b1a1-f5b6127f16be:87752f75-561e-48f8-a2d2-f782d9381696
Build time: 7.72 seconds
build seems to "work", but idk if it's cause it just doesn't seem to read the files needed
Not gonna act like an expert, I'm a student so this is new to me in terms of hosting via Railway
sage basalt
Not gonna act like an expert, I'm a student so this is new to me in terms of hos...
Don't worry, I am also a student and not an expert.
sage basalt
build seems to "work", but idk if it's cause it just doesn't seem to read the fi...
The build logs seems to only install node v18, how is your project structured?
toxic zenith
my project is a repo with a frontend and backend folder. The project itself is owned by a company I intern for, but my exam suddenly required having the project hosted, so I am trying to get it hosted with Railway as it seemed to be able to host the areas needed
sage basalt
my project is a repo with a frontend and backend folder. The project itself is o...
In that case https://docs.railway.com/guides/monorepo
toxic zenith
I already told it what root directory to go for. I however did not mention a start command. CGPT told me it would find the files needed itself and run through Procfile but maybe I set it up wrong, since the deploy logs complains about gunicorn, signaling maybe a lack of requirements.txt being run through
sage basalt
I already told it what root directory to go for. I however did not mention a sta...
You can try putting python as a provider under providers and see if that helps.
Since the build logs don't show anything Python related
maybe this will help, but no guarantees
toxic zenith
Wonder if I can specify a version on that, since I need a specific version, though the root directory's runtime.txt has that specified
sage basalt
You can set the Python version via the NIXPACKS_PYTHON_VERSION env https://nixpacks.com/docs/providers/python
junior kite
-# version in runtime.txt also works
toxic zenith
added python, seems to do something in the build now so that's hopeful
seems to run through requirements now
✕ [stage-0 10/13] RUN --mount=type=cache,id=s/0d3c87d4-0b8e-4709-b1a1-f5b6127f16be-/root/cache/pip,target=/root/.cache/pip python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt
process "/bin/bash -ol pipefail -c python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt" did not complete successfully: exit code: 2
Dockerfile:30
-------------------
28 | ENV NIXPACKS_PATH=/opt/venv/bin:$NIXPACKS_PATH
29 | COPY . /app/.
30 | >>> RUN --mount=type=cache,id=s/0d3c87d4-0b8e-4709-b1a1-f5b6127f16be-/root/cache/pip,target=/root/.cache/pip python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt
31 |
32 | # build phase
-------------------
ERROR: failed to solve: process "/bin/bash -ol pipefail -c python -m venv --copies /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt" did not complete successfully: exit code: 2
Error: Docker build failed
says no module named distutils, can't say I remember using that
junior kite
-# set a service variable NIXPACKS_VERSION=1.29.1
toxic zenith
-# set a service variable `NIXPACKS_VERSION=1.29.1`
when you say service variable, do you mean I should set this into my .env file ? Sorry if that's a bad question
toxic zenith
Is there a way to check exactly what version of python the railway service installs? I assume it installed the version I needed, but I'm getting an error that there was no suitable python version for the pandas-stubs.
The company's system runs on Python 3.8.4. and runtime.txt in the root of the repository (not the backend directory), mentions that version exactly
sage basalt
Is there a way to check exactly what version of python the railway service insta...
It's stated in the build logs
toxic zenith
Did try looking through them, but there is a lot of rows of text so kinda hard to find specifically
sage basalt
It's in the table which is like one of the first things it prints out
toxic zenith
Based on this, it seems it runs python 3.11.9 or am I reading it wrong?
/nix/store/k7j3cmdpjc23cxd7axkc8j0yvq5cfhw5-postgresql-15.6-man
/nix/store/xzmgdglbldim3k39pjn16vrgxb3i72as-publicsuffix-list-0-unstable-2024-01-07
/nix/store/k3701zl6gmx3la7y4dnflcvf3xfy88kh-python3-3.11.9
/nix/store/95qcp9a6c8kk1aahclj2g7kjzrvzix49-qrencode-4.1.1
Seems the runtime.txt in the repo root might not be read. For safety I think I'll try adding the variable you mentioned earlier. Couldn't hurt I imagine
✕ [stage-0 6/13] RUN nix-env -if .nixpacks/nixpkgs-e05605ec414618eab4a7a6aea8b38f6fbbcc8f08.nix && nix-collect-garbage -d
process "/bin/bash -ol pipefail -c nix-env -if .nixpacks/nixpkgs-e05605ec414618eab4a7a6aea8b38f6fbbcc8f08.nix && nix-collect-garbage -d" did not complete successfully: exit code: 1
Dockerfile:10
-------------------
8 | RUN nix-env -if .nixpacks/nixpkgs-5148520bfab61f99fd25fb9ff7bfbb50dad3c9db.nix && nix-collect-garbage -d
9 | COPY .nixpacks/nixpkgs-e05605ec414618eab4a7a6aea8b38f6fbbcc8f08.nix .nixpacks/nixpkgs-e05605ec414618eab4a7a6aea8b38f6fbbcc8f08.nix
10 | >>> RUN nix-env -if .nixpacks/nixpkgs-e05605ec414618eab4a7a6aea8b38f6fbbcc8f08.nix && nix-collect-garbage -d
11 |
12 |
-------------------
ERROR: failed to solve: process "/bin/bash -ol pipefail -c nix-env -if .nixpacks/nixpkgs-e05605ec414618eab4a7a6aea8b38f6fbbcc8f08.nix && nix-collect-garbage -d" did not complete successfully: exit code: 1
Error: Docker build failed
Not exactly sure what went wrong here, I assume it didn't accept the version of python ?
sage basalt
```
✕ [stage-0 6/13] RUN nix-env -if .nixpacks/nixpkgs-e05605ec414618eab4a7a6ae...
Can you show the first few lines from your build logs?
e.g.
[Region: us-west1]
==============
Using Nixpacks
==============
context: 502e3d7d84f9e67f22cc31deb908355f
╔══════ Nixpacks v1.29.1 ══════╗
║ setup │ go ║
║──────────────────────────────║
║ install │ go mod download ║
║──────────────────────────────║
║ build │ go build -o out ║
║──────────────────────────────║
║ start │ ./out ║
╚══════════════════════════════╝
Sorry it took a bit, was not at the computer
did re-check variables, and I did write it correctly with NIXPACKS_PYTHON_VERSION equaling 3.8.4
sage basalt
the Python version seems to be right according to the python:setup (python38), I have no idea why postgresql is there tho
toxic zenith
I have some other variables as I inserted my full backend .env to ensure the backend is as close to my local one as possible when trying to deploy
sage basalt
toxic zenith
Like I said, it's a company server that I need to host non-locally for my exam...it's not very well made by them 🙃
But yeah, overall, the python version made it crash before getting to the requirements so I'm confused on what to do from here I will admit
sage basalt
Interesting, so it did work with 3.11?
toxic zenith
well it got to the requirements, where it failed on pandas-stubs due to a lack of a proper python version for what it wanted
sage basalt
Can you paste the full build logs for the latest build (the one where you specified Python 3.8)?
sage basalt
well, I am kinda lost on why it doesn't work so I don't think that will help
if you have the time, please wait for someone that is smarter than me to help you
toxic zenith
No worries, thanks for your help, I appreciate it 
junior kite
are you able to share the repo?
toxic zenith
It's a private company repo, so I'm not sure I'm really "allowed" to
But I can share the requirements and what not if that's what you're looking at
junior kite
have you ran this project locally?
toxic zenith
Yes, over the last 10 weeks, I've worked on my exam project on it. I just need to host it as the final requirement
junior kite
do you need node when running it locally?
toxic zenith
I ran it through 2 git node terminals in Cursor / VS Code, one for frontend running npm run start and one running flask through a virtual environment with flask run
toxic zenith
I do not believe I do no. I don't remember using any node commands to run it, but just venv and flask
junior kite
can you start by removing the node provider then
junior kite
yes you would want to click the deploy button after making that change
toxic zenith
Alright, just wanted to be sure in case of other requested changes
Trying now
I added a new git commit to reflect a requirement removal from the .txt, but how do I attempt another deploy with this git commit in mind? I don't see a deploy with latest commit or similar anywhere, only the last deploy and it having a "Redeploy" button
sage basalt
I added a new git commit to reflect a requirement removal from the .txt, but how...
CMD + K > Deploy latest commit
toxic zenith
Seems the build is successfully running now, but the deploy fails pretty much immediately
toxic zenith
I should have, though I feel like I had it in the .env I posted in there, but give me a moment
junior kite
you dont put json files into .env
toxic zenith
Nono I know that, but the firebase details I meant. It's a recent addition, not made by me, as I've not worked with firebase before, so I could've missed a file I needed to put in
junior kite
you would want to base64 encode the json file, and then put the base64 string into a service variables, then have some code that runs on start to read that environment variable, decrypt it, and write it to the appropriate file, so that the firebase library can read from the file
toxic zenith
I do have a pwa_key.json file which has some firebase data in it, which is ignored by git on commit.
I honestly have no clue how to do what you suggested, but I assume the file is already being read somehow by the system since firebase works with that file locally
junior kite
the file is not being uploaded to railway, thats a good thing, but you'll need to do what i suggested
toxic zenith
Well I will have to figure out how to do that then, cause at the moment I don't know how to, as it is not something I've developed, someone else did
Thank you for your help 🙂 Appreciate it
junior kite
few Google searches away and you'll be good
toxic zenith
Slight update on this issue:
It seems to run now after changing some firebase thing, however I am unable to connect the Redis service I've also created to this. Again I am a student and I've not worked with Redis before so could easily just be me being a goof, but if someone could help me take a quick look at my setup, I'd greatly appreciate it 🙂
I've currently set up the redis service, and have changed the redis_url variable to reflect this new url, but it does not seem to like it. the code that runs the worker itself is not static so it should in theory work, but maybe I forgot to do something.
Any help is appreciated 
toxic zenith
Not gonna ping you due to rules, but I saw another thread regarding python-magic that you helped with some time ago Brody, which seems to be my issue as well, despite it being installed via my requirements.txt, so if you're able, I'd greatly appreciate it
junior kite
happy to help but I'd need you to send me the errors you are getting
toxic zenith
happy to help but I'd need you to send me the errors you are getting
Of course, it mainly seems that libmagic is not being read properly, the error being the following:
[2025-01-19 17:06:33 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2025-01-19 17:06:33 +0000] [1] [INFO] Listening at: http://0.0.0.0:39453 (1)
[2025-01-19 17:06:33 +0000] [1] [INFO] Using worker: sync
[2025-01-19 17:06:33 +0000] [5] [INFO] Booting worker with pid: 5
Temporary folder 'tmp' has been created.
Firebase app initialized
[2025-01-19 17:06:37 +0000] [5] [ERROR] Exception in worker process
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
Traceback (most recent call last):
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 843, in exec_module
File "/opt/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
worker.init_process()
File "/app/run.py", line 5, in <module>
app = create_app(config("CONFIG_NAME"))
File "/opt/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
File "/app/api/app.py", line 123, in create_app
from api.blueprints.fake import fake
self.load_wsgi()
File "/app/api/blueprints/fake.py", line 34, in <module>
File "/opt/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
from api.services.openai_api import generete_tutor_bio, TutorBioResponse
self.wsgi = self.app.wsgi()
File "/app/api/services/openai_api.py", line 20, in <module>
from api.langchain.rag import (
File "/opt/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
File "/app/api/langchain/rag.py", line 3, in <module>
self.callable = self.load()
import magic
File "/opt/venv/lib/python3.8/site-packages/magic/__init__.py", line 209, in <module>
libmagic = loader.load_lib()
File "/opt/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
File "/opt/venv/lib/python3.8/site-packages/magic/loader.py", line 49, in load_lib
return self.load_wsgiapp()
raise ImportError('failed to find libmagic. Check your installation')
ImportError: failed to find libmagic. Check your installation
File "/opt/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
return util.import_app(self.app_uri)
File "/opt/venv/lib/python3.8/site-packages/gunicorn/util.py", line 359, in import_app
mod = importlib.import_module(module)
File "/root/.nix-profile/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
[2025-01-19 17:06:37 +0000] [5] [INFO] Worker exiting (pid: 5)
[2025-01-19 17:06:39 +0000] [1] [INFO] Shutting down: Master
[2025-01-19 17:06:39 +0000] [1] [INFO] Reason: Worker failed to boot.```you said that you saw me help with this In a thread before, did that thread get solved?
toxic zenith
Let me see if I can find it again
toxic zenith
you said that you saw me help with this In a thread before, did that thread get ...
Was a different version of python but it seems a dockerfile was used, which I have also tried, but to no success
junior kite
can you define no success?
toxic zenith
Deployment giving that error
This is the dockerfile so far:
FROM python:3.8.4-slim
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
RUN apt-get update && apt-get install -y \
libmagic-dev \
file \
build-essential \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY . /app
RUN pip install --no-cache-dir -r requirements.txt
RUN python -c "import magic; print(magic.Magic)"
EXPOSE 5000
CMD ["gunicorn", "run:app", "--bind", "0.0.0.0:5000"]
basically just been trying with help from CGPT
As it has been a few years since I last used docker
libmagic 1.0 and python-magic 0.4.27 are specified in my requirements.txt already, but I'm not sure why exactly, but my guess is that one of them doesn't seem to get properly recognized
the build logs does mention their installations though
junior kite
so with that Dockerfile you get the error you sent above?
toxic zenith
with the dockerfile added to the repository, yes
junior kite
how do you know the Dockerfile is being used to do the build?
toxic zenith
I assume it is from seeing libmagic being installed 2 seperate places, one of which using the .gz file as shown in the pic
Honestly though, maybe it's not, but I assume it is
junior kite
check the very top of the logs
toxic zenith
What am I looking for exactly?
toxic zenith
I do have this that mentions the dockerfile
toxic zenith
Oh, how do I ensure it does then ?
junior kite
Do you have your Dockerfile named Dockerfile? capital D and all, and do you have it placed into the same folder as you've set for your root directory
toxic zenith
it's in the backend folder but with a small d
I can change it to capital and try again
junior kite
capital D is needed
toxic zenith
Alright, will try that, one moment
toxic zenith
capital D is needed
It seems to detect my dockerfile now, however, I don't see the nixpacks stuff now, with the setup, install and start commands. Is that something I should be worried about ?
junior kite
no, Dockerfile and nixpacks are different, you only use one or the other
toxic zenith
Alright.
It seems now this happens in the build
Collecting psycopg2==2.9.9
Downloading psycopg2-2.9.9.tar.gz (384 kB)
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-omwe95rq/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-omwe95rq/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-q5cw91q1
cwd: /tmp/pip-install-omwe95rq/psycopg2/
Complete output (23 lines):
running egg_info
creating /tmp/pip-pip-egg-info-q5cw91q1/psycopg2.egg-info
writing /tmp/pip-pip-egg-info-q5cw91q1/psycopg2.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-q5cw91q1/psycopg2.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-pip-egg-info-q5cw91q1/psycopg2.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-q5cw91q1/psycopg2.egg-info/SOURCES.txt'
Error: pg_config executable not found.
pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
If you prefer to avoid building psycopg2 from source, please install the PyPI
'psycopg2-binary' package instead.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>;).
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 20.1.1; however, version 24.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
✕ [5/6] RUN pip install --no-cache-dir -r requirements.txt
process "/bin/sh -c pip install --no-cache-dir -r requirements.txt" did not complete successfully: exit code: 1
Dockerfile:22
-------------------
20 |
21 | # Install Python dependencies
22 | >>> RUN pip install --no-cache-dir -r requirements.txt
23 |
24 | # Verify libmagic is installed correctly
-------------------
ERROR: failed to solve: process "/bin/sh -c pip install --no-cache-dir -r requirements.txt" did not complete successfully: exit code: 1
I imagine it's still related to libmagic despite it failing on psycopg2's installation, but I am not really sure, as neither were installed by me
By installed I mean added to the requirements.txt file
junior kite
can you replace it with the binary version as it suggests?
toxic zenith
right, will try
tried commenting out the psycopg2 installation and using the psycopg2-binary installation that was also in the requirements.txt, which leads to the build succeeding again, but the deploy giving that same error about failing to locate libmagic and being asked to check for installation
Doesn't seem to be found while building either from the dockerfile checking the installation
[6/6] RUN python -c "import magic; print(magic.Magic)"
<class 'magic.Magic'>
[6/6] RUN python -c "import magic; print(magic.Magic)" ✔ 2s
not sure why you are mentioning libmagic, your latest error is something about jwt, when looking at logs please carefully read them
toxic zenith
Ah so it is, I was looking at an earlier deploy somehow, sorry about that
toxic zenith
not sure why you are mentioning libmagic, your latest error is something about j...
It seems to be running now, but is there a console or similar I can use to test it ?
junior kite
no console, isnt this a web app?
toxic zenith
It is, but the frontend is not yet deployed
that was what I wanted to do after I got the backend up and running
toxic zenith
<https://pleasant-wholeness-production.up.railway.app/>
Well I appreciate all the help greatly, it's taken a load off the mental for the exam coming soon I have some CORS to work through before the url is allowed for my application but that should hopefully not be too hard to manage on my own
junior kite
sounds good, am i good to mark as solved?
toxic zenith
Yes you are 🙂
junior kite
!s