docker compose migrations with django

[surreal steeple]

I have a django app inside a docker-compose service and when I looked online on how people run their migrations with docker I saw people doing docker-compose run web python manage.py makemigrations. I was wondering why you migrate inside the docker container instead of outside with python before building the container. Do migrations save back to local if you build inside the container?

[tidal elk]

People should never be running makemigrations inside the container

The migrations files are considered part of the codebase, they should be created at development time and committed to the repo etc. On the production server (in the container at startup, perhaps) is when you run migrate, which will take the migration files that are in the codebase and apply them to the database as necessary.

[surreal steeple]

so docker-compose run web python manage.py migrate after making migrations is acceptable?

[tidal elk]

Yep. We actually have that as part of the entry point command for the container, before it starts up gunicorn.

[surreal steeple]

would it also be advisable to have a command or service that runs migrate on container build? i saw some people using it on stackoverflow but others say to never do that as migrate is dangerous

[tidal elk]

How can you run migrate on container build? It has to talk to the database, which at the time of building the container is not accessible.

[surreal steeple]

not necessarily during build, but something like the first answer on this post https://stackoverflow.com/questions/33992867/how-do-you-perform-django-database-migrations-when-using-docker-compose

using an entrypoint file

[tidal elk]

Yeah, so that's what I said I do above

With docker, you have to be explicit about "build" vs "startup/deploy". They're very different

I've yet to come across any issues with running migrate on container start.

[orchid peak]

nor I

we do that all the time; works fine

since our db persists across container starts, there are rarely any migrations to apply; and when there are, I've tested the bejeezus out of them before deploying

[surreal steeple]

alr thanks

[surreal steeple]

@tidal elk hey recently when trying to makemigrations using python manage.py makemigrations im getting RuntimeWarning: Got an error checking a consistent migration history performed for database connection 'default': [Errno 8] nodename nor servname provided, or not known

the makemigration seems to work successfully as new migration files are created but i am just concerned about the error

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'db',
        'HOST': 'database',
        'PORT': '5432',
        'USER': os.environ.get('POSTGRES_USER'),
        'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
    }
}

the host is called database because i have a database docker container thats connected to a volume folder /database

oh shoot wait nvm im supposed to make host localhost

[tidal elk]

localhost is if the database is running on the same machine/container as Django, otherwise you'll need the hostname of the database container (if you're using Docker) or the database host

[surreal steeple]

where can i find the hsotname of the database container?

[tidal elk]

Are you running docker for local development?

[surreal steeple]

yeah

[tidal elk]

What's your docker-compose?

[surreal steeple]

  database:
    image: postgres:16
    volumes:
      - ./database:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    env_file:
      - path: .env
        required: true

thats just my database though

[tidal elk]

Yeah, though from the Django container's perspective, database should be the hostname

[surreal steeple]

hm

what is causing the error then?

  server:
    image: server
    build:
      context: ./server
      dockerfile: dev.Dockerfile
    volumes:
      - ./server:/server
    ports:
      - "8000:8000"
    env_file:
      - path: .env
        required: true
    depends_on:
      - database

[tidal elk]

Does Django work as normal, is it just makemigrations that throws that error?

[surreal steeple]

let me see

yep runs normally

running the server container i just get the statreloader notif

[tidal elk]

And your app shows data in it etc?

[surreal steeple]

i havent added any rows but migrations work

database shows all the tables ive migrated

[tidal elk]

Hmm

[surreal steeple]

did you figure anything out?

[tidal elk]

Afraid not. I don't use docker for local development, and I don't know why manage.py makemigrations would be different to manage.py runserver

[harsh kayak]

I do, exclusively. @surreal steeple can you please reiterate the current problem you're having?

[tidal elk]

TLDR: this error appears when running makemigrations, but not with runserver:

RuntimeWarning: Got an error checking a consistent migration history performed for database connection 'default': [Errno 8] nodename nor servname provided, or not known

[harsh kayak]

and yet the migrations are successful ?

[tidal elk]

the makemigration seems to work successfully as new migration files are created but i am just concerned about the error
Sounds like it!

[harsh kayak]

Fascinating!

[harsh kayak]

at best I was able to replicate this:

$ docker compose exec backend python manage.py makemigrations
/usr/local/lib/python3.12/site-packages/django/core/management/commands/makemigrations.py:160: RuntimeWarning: Got an error checking a consistent migration history performed for database connection 'default': could not translate host name "database" to address: Temporary failure in name resolution

  warnings.warn(
Migrations for 'core':
  calibration/core/migrations/0001_initial.py
    - Create model KFactorMapping

the only thing I can think about to get the [Errno 8] nodename nor servname provided, or not known is them not running the makemigrations via docker compose, but via docker and without adding the container in the network.

[surreal steeple]

i never tried runserver in my console actualy

because my server runs in another docker container

[surreal steeple]

ill send my docker ocmpose and dockerfiels

[harsh kayak]

How exactly are you running markmigrations?

[surreal steeple]

inside venv: python manage.py makemigrations

docker compose:

services:
  client:
    image: client
    build:
      context: ./client
      dockerfile: dev.Dockerfile
    volumes:
      - ./client:/client
    ports:
      - "3000:3000"

  server:
    image: server
    build:
      context: ./server
      dockerfile: dev.Dockerfile
    volumes:
      - ./server:/server
    ports:
      - "8000:8000"
    env_file:
      - path: .env
        required: true
    depends_on:
      - database
  
  database:
    image: postgres:16
    volumes:
      - ./database:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    env_file:
      - path: .env
        required: true

[harsh kayak]

And where's your venv?

[surreal steeple]

server dockerfile

FROM python:3.12 AS poetry
RUN pip install poetry
WORKDIR /server
COPY pyproject.toml poetry.lock ./
RUN pip install poetry-plugin-export
RUN poetry export -f requirements.txt --output requirements.txt

FROM python:3.12
WORKDIR /server
COPY --from=poetry /server/requirements.txt ./
RUN pip install -r requirements.txt
COPY . .

CMD echo '\e]8;;0.0.0.0:8000\e\\0.0.0.0:8000\e]8;;\e\\\n' \
    && python manage.py runserver 0.0.0.0:8000

[surreal steeple]

its the pycharm venv for my project

i think its just the default virtual environment that gets initialized

wait nvm

im using poetry

so its a poetry environment

[harsh kayak]

So you're running makemigrations outside of your container and it's giving you this warning?

[surreal steeple]

yes

[surreal steeple]

ben mentioned this before

[tidal elk]

Ah then you misunderstood and I wasn't clear

If you're developing inside a container, then by all means run it inside the container

No wonder it doesn't work outside the container 🙂

[harsh kayak]

XD

[surreal steeple]

ohhh

makes sense lol

[harsh kayak]

We can both sleep peacefully now.

[surreal steeple]

would the files also be created on my project directory when i make the migration files inside the container?

[harsh kayak]

If you have a volume mapped yes.

[surreal steeple]

alright

no more runtimeerror everything seems to be working 💯