All of the volumes in https://railway.app/project/b96c760d-d6df-4c8e-9b31-fc7b3383aadb have been migrating for half an hour now.
Project ID: b96c760d-d6df-4c8e-9b31-fc7b3383aadb
#volume migration stuck
sterile belfry
lean zincBOT
sterile belfry
oh, it's smarter than I thought
wicked canyon
!t
oak garnetBOT
New reply sent from Help Station thread:
This thread has been escalated to the Railway team.
You're seeing this because this thread has been automatically linked to the Help Station thread.
sterile belfry
And now it says this, which does not seem promising
wind torrent
should be fixed?
sterile belfry
seems to be back up, thanks!
eternal zincBOT
sterile belfry
re: is it safe to try and move it again?
wicked canyon
lets try now, monitoring
sterile belfry
redeploying now
seems to be working 🤞
starting to get a little nervous, one of them is still thinking
I'll check back after dinner, if it's not done by then then something's gone terribly wrong
wicked canyon
Gotcha
sterile belfry
not after dinner yet, but it does seem stuck
and the volume that isn't working yet is only about twice the size of the other ones, which only took seconds
sterile belfry
aaaand it failed
wind torrent
i clicked the magic button again
oak garnetBOT
New reply sent from Help Station thread:
Trying again now.
You're seeing this because this thread has been automatically linked to the Help Station thread.
sterile belfry
Well, it says it failed, but the app is working
wicked canyon
cc @sterile belfry, we have gotten reports of more instances here, can you kick another migration and will raise it to the platform on-call.
sterile belfry
cc <@737369271751016518>, we have gotten reports of more instances here, can you...
I can't, it says volume migration failed and won't let me touch the region
wicked canyon
Sorry to use you as a test dummy but thats the perk of the conductor program.
anyway, I am driving the deployment approval fixes, lemme get another thread on this
cinder monolith
@sterile belfry what region are you trying to migrate to?
sterile belfry
us-east
sterile belfry
Any updates?
wind torrent
west to east right?
sterile belfry
Yes
wind torrent
platform on call is not online right now 😦
sterile belfry
Ah
sterile belfry
bump?
wind torrent
lemme check on what happened in our internal thread
sterile belfry
oh it's trying to migrate again
ah you did that
aaand it failed again
wait, what? failed to migrate to us west?
I wanted to move it to us east
I'm very confused now
and the site is up and running again despite all of this
sterile belfry
alright, can try again tomorrow I guess
lavish jungle
alright, can try again tomorrow I guess
Can we try now?
@wind torrent It's going us-west to us-east right?
wind torrent
yes
lavish jungle
kk
wind torrent
yes
lavish jungle
I looked into this. It's very odd as it keeps disconnecting. I can ping it myself but it seems to break
I'm gonna bring it up with infra tomorrow
wind torrent
what even is context
sterile belfry
Any updates?
wind torrent
no updates that i know of
lavish jungle
Unfortunately not ATM