#Issue with dabatase after redeploying when services were down

24 messages · Page 1 of 1 (latest)

pseudo tree
#

HI! after redeploying the mysql database

when I am trying to connect as always from local computer or the application.. I see this issue!

That's something I never needed! I just redeployed....
1.- database mysql
2.- Backend service
3.- frontend service

flint tendonBOT
pseudo tree
#

I need help!!!!!!

#

After Railway's recent redeploy requirement, my MySQL service was upgraded from 8.0 to 8.4 (my source image was set to mysql:8 which resolved to 8.4).

The problem: MySQL 8.4 disables the mysql_native_password plugin by default, but my existing users were created with that authentication method. Now nothing can connect — not my backend, not remote clients, not even Railway's own Database tab (shows 'You have no tables').

Error: ERROR 1524 (HY000): Plugin 'mysql_native_password' is not loaded

I can't fix this myself because:

  • I can't connect to run ALTER USER (same auth error)
  • railway connect fails with the same error
  • MYSQLD_ARGS variable doesn't apply to DB config (Railway warns about this)
  • Downgrading to mysql:8.0 is risky since the data dictionary was already upgraded to 8.4 format

I need Railway to either:

  1. Add --mysql-native-password=ON to my MySQL server startup, OR
  2. Provide a way to run SQL commands on the server without going through the auth plugin

Project: Qrupones App | Environment: production | Service: MySQL

This is blocking all production traffic. Any help would be greatly appreciated!

crisp shoal
#

I suspect you didn't had a pinned image version of MySql so it upgraded upon redeploy

#

Can you search the logs of your last working deployment (the one from 10 months ago)

#

And find the MySql version it was running then?

pseudo tree
#

sure thing! give me 1 min plz

#

maybe mysql 8.3.0?

#

and this is from today's redeployment

#

I thought about pinning to mysql:8.3.0, but the data dictionary was already upgraded to 8.4 format during today's redeploy:

" Data dictionary upgrading from version '80200' to '80300'
Server upgrade from '80300' to '80408' completed
"

So downgrading would likely break since MySQL doesn't support data dictionary downgrades.

Is there any way we can add --mysql-native-password=ON to the server startup command on my service? That would fix everything without any risk to the data, and maybe I can update the way of how my user can connect

crisp shoal
#

Yeah, downgrading after the data dir was already upgraded isn't really supported afaik.
You can edit the start command in the service settings, set it to mysqld --mysql-native-password=ON

#

There is already a start command there, just add --mysql-native-password=ON to it

pseudo tree
#

Let me try to do it and deploy it again

#

ok OMW
fingers crossed!

#

Seems to be working! I am able to connect from my local computer now...

trying to redeploy de backend again

#

awesome!

#

anything else, I should change before leaving this?

crisp shoal
#

Just check if everything is working as it should

#

And make sure the Mysql version is now pinned

pseudo tree
#

Do you think I should take care of this?

2026-02-11T17:44:54.064138Z 8 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'

crisp shoal
#

Yes! Even if you use the pinned version, you don't know when you'll need to update, and that can cause problems with this start command argument we added.

pseudo tree
#

All right! I’ll update it
Thanks for the help @crisp shoal

crisp shoal
#

Np, sorry for the trouble!