#Docker + Payload (v3) in a self-hosted setup

13 messages · Page 1 of 1 (latest)

midnight igloo
#

Hey all — wondering if anyone has a solid deployment workflow using Docker + Payload (v3) in a self-hosted setup like DigitalOcean?

Looking for best practices or working examples that cover:
• how to safely run migrations in production (prodMigrations or alternatives)
• how to handle schema updates without data loss
• seeding data into existing DBs
• rollback strategies for both code and database
• how to manage versioned deployments cleanly
• any Docker-specific gotchas with Payload in production

I’ve tried using prodMigrations, but on production startup I keep getting the message 'There are database changes that need to be applied, but it looks like you're running in a development environment.' — and migrations just don’t run. I’m clearly in prod mode (NODE_ENV=production), so not sure what it’s looking for.

Happy to share what I’ve got working so far — I’ve got a functional docker-compose, deploy flow, and CI/CD— but I’m sure it could use extra eyes, and I’m likely missing things others here have already solved.

Thanks in advance!

crude steppeBOT
opal sundial
#

We're running a number of setups in DO with Postgres, all of them as Apps deploying using buildpack (not Docker).

One tip small is this line in the postgresAdapter config:

      ssl: process.env.DATABASE_URI?.includes('localhost')
        ? undefined : { rejectUnauthorized: false },
    },
#

Apart from that little trick everything is 100% straight forward.

#

Migrations with postgres requires a bit of discipline. There are other posts in this forum on that topic. We use both migrate on server start and migrate on CI/CD build. The latter is safer but way more work to set up.

#

Schema updates without data loss is just plain Postgres, some migrations might need manual tweaks after they have been generated.

#

Rollbacks and versioning is not on our plate yet 😉

cosmic epoch
#

Are you by chance using the same database that you were developing with?

When you use the “push” mode to make database changes in development mode, it saves some data in the database to designate that it has been used during development. In theory, you should never use the same (relational at least) database in development and production. In the event that you do, you need to delete the bit in the database that flags it as a dev database. I forget what it’s called, but it’s pretty obvious if you snoop around in the db. You should never run migrations in a database that has been pushed to via dev mode. That is the basic idea.

narrow citrus
midnight igloo
# cosmic epoch Are you by chance using the same database that you were developing with? When y...

This was one thing I was thinking might have caused issues as I am migrating from another CMS to Payload and bringing a lot of data along for the initial launch. One thing I noticed when I did try a migrate:create was that it was in the first row of the db and the dev row was in the 2nd. I’ve removed all migrations doing a fresh one and now see it in reverse order.

Is there a known workflow for these cases? I’d imagine it’s very common case.

#

The end solution might just be to add a middle step to the data migration and instead of inserting from one db to another as I am now when creating the data, save it to json and seed it on whatever environment it needs to exist in.

#

Still unsure though which is the best method for schema changes when using docker through a true ci/cd process.

cosmic epoch
#

The important part is to keep developing and producing databases separate.

I personally recommend using both migrations and local api/seed scripts.

Migrations are auto generated. However, you can manually modify them, or create an empty one to add your own SQL to make changes. If you change something, or move data from one column to another, I’d put it in a migration.

For adding data itself, I would recommend keeping it out of the migration cycle and simply make a seed script, similar to the website template example.

For seeding, you can make a button that triggers it, as the website template does. What I like to do is put seeds in the config’s onInit option and before just throwing the data in the database, I check if a seed env variable is set, as well as if the data already exists. If not, I create it via local API. For example, if you have a product seed, you just do a payload.count on the products collection and if none exist, add them.