#Database Versioning or Branching

6 messages · Page 1 of 1 (latest)

solid sage
#

I have a feature branch that is going to need considerable work before it's code and data will be merged into main, and I still have to do occasional maintenance on main. For code only this is no problem with git branch management, but the feature branch database has significant structure and content differences with main, I can fairly easily use a different database instance for each one, and merge structure changes manually somehow when needed, but what about content data?

Right now, for the quickest and dirtiest flow, I have one db, and just let Payload and Drizzle drop and add tables as I switch between branches, without much stress as I have almost no content. If needed I can use a simple seeding script to always recreate test content for the feature branch.

It's almost entirely new data and structures on the feature branch, so I could, if I have to, code a migration function to pull new data from the dev db to the prod db in a deployment workflow, but I have a feeling Payload has something to offer to help in this regard. It seems like a fairly reasonable use case to me. Could someone more experienced please point me in a direction I can follow to help me automate this workflow as much as possible.

fierce pelicanBOT
patent eagle
#

From my experience, no, Payload does not offer anything specific to solve this problem and I think thats fair because it's not really related to Payload but to working with databases in general. Best solution we found was to use some database that can be branched, just like git:

  1. SQLite: "Branching" with SQLite is very easy because you just copy the database file and use the copy. Thats it. However if you are deploying serverless you cannot really do that but you have to rely on some remote hosting of the SQLite (or libsql more likely) like Turso. Turso supports branching on a platform level but you would have to do it manually or integrate it yourself with your CI. Did that once with a GitHub Action, wasn't exactly straight forward but it worked 😄

  2. Neon Postgres (integrated via Vercel): This is very convenient, especially since payload has an official template for this exact setup. Neon handles all the branching and passing the branched db credentials to the deployment. But you would want a local copy of the db for local dev. Payload recommends running a Postgres docker container on your machine. We added a shell script that connects to the Neon production branch and pulls a full db dump to the local container. That way I can always reset my local db to the "source of truth" remote db. Makes migration testing very convenient and safe.

solid sage
# patent eagle From my experience, no, Payload does not offer anything specific to solve this p...

Ya, thanks. It's not something I would expect of Payload, but I didn't want to accidentally bypass it if their was something. I just noticed yesterday that Neon has branching, which is most convenient because I'm already using Neon. I started with a Supabase db but switched to Neon just be changing the adapter. Could I have missed anything by not starting with the Neon template? Not serious, everything is just peachy as if, I'm just curious.

patent eagle
#

no I dont think you missed anything on the template side, this just gives you the right adapter by default (vercelPostgresAdapter). I am using Neon integrated with Vercel so not sure how it behaves outside of that. But in this combination, the branching is automated with Vercel deployments. I enabled "branching on preview deployments" so every time I create a preview deployment, Neon creates a new db branch (or reuses an existing one) for that git branch. The integration is nice because it lets neon pass the branched db credentials to the deployment before the build.

solid sage