redis connection fails | Railway | Page 1

ocean crown Jul 2, 2023, 3:06 AM

#

Hi! Having some issues with redis. I'm getting

at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16) {
errno: -111,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '34.82.163.149',
port: 7679
}

I already tried restarting Redis or restarting/redeploying the project. Any suggestions? everything was working fine previoously

sterile cobaltBOT Jul 2, 2023, 3:06 AM

#

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

brave ventureBOT Jul 2, 2023, 3:06 AM

#

Project ID: 900023fc-eb3b-4c5e-b8bf-83d6fcb5a72f

ocean crown Jul 2, 2023, 3:06 AM

#

900023fc-eb3b-4c5e-b8bf-83d6fcb5a72f

#

I'm trying to figure out what may be causing this but having some trouble here

#

On closer inspection, something is weird about the port that redis is running on vs the port that Railway thinks Redis is running on.

In the Redis logs I'm seeing:

redis 03:03:19.13 INFO  ==> ** Starting Redis setup **
redis 03:03:19.14 INFO  ==> Initializing Redis
redis 03:03:19.16 INFO  ==> Setting Redis config file
redis 03:03:19.18 INFO  ==> ** Redis setup finished! **
redis 03:03:19.19 INFO  ==> ** Starting Redis **
1:C 02 Jul 2023 03:03:19.238 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 02 Jul 2023 03:03:19.238 # Redis version=6.2.12, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 02 Jul 2023 03:03:19.238 # Configuration loaded
1:M 02 Jul 2023 03:03:19.238 * monotonic clock: POSIX clock_gettime
1:M 02 Jul 2023 03:03:19.239 * Running mode=standalone, port=6379.
1:M 02 Jul 2023 03:03:19.239 # Server initialized

Notice how it says port 6379.

However, in the actual REDIS_URL env var, the port is 7679

tropic delta Jul 2, 2023, 3:11 AM

#

that's completely normal

#

but can you connect to the database with the redis cli?

ocean crown Jul 2, 2023, 3:15 AM

#

i am just in the process of wiping redis and adding it back

#

i was able to do that!

#

it seems like an internal problem to the comms between railway services

tropic delta Jul 2, 2023, 3:16 AM

#

so yes you can connect to it from the redis cli?

ocean crown Jul 2, 2023, 3:17 AM

#

yeah!

tropic delta Jul 2, 2023, 3:17 AM

#

and is your service using variable references for the database?

ocean crown Jul 2, 2023, 3:18 AM

#

yep. Just ${{Redis.REDIS_URL}}

#

nothing changed

#

we haven't deployed in 3 days

tropic delta Jul 2, 2023, 3:19 AM

#

yep sounds like a network issues

#

I will flag for team monday

ocean crown Jul 2, 2023, 3:19 AM

#

i am trying to restart now but we can't wait until monday if it doesn't work

#

i hsouldn't need to wipe my db for this to work

#

it's an incident really

tropic delta Jul 2, 2023, 3:20 AM

#

I'm sorry but the team doesn't usually work on the weekends

ocean crown Jul 2, 2023, 3:20 AM

#

wait what? so what if this was an emergency?

#

we just don't get support?

#

that is unacceptable

#

i'm glad i found out - we will make sure to migrate earlier then

#

i had to reset my redis database and that affected our customers. that's not something that we take lightly

#

btw, wiping the database and starting anew worked

#

but that's really a nuclear option

tropic delta Jul 2, 2023, 3:23 AM

#

for 24/7 support on outages you may want to look into the business class
https://docs.railway.app/reference/support#business-class

ocean crown Jul 2, 2023, 3:24 AM

#

ok we'll reach out

#

i expected to get support on the team plan for that but it seems not

tropic delta Jul 2, 2023, 3:25 AM

#

unfortunately there's no SLOs for the teams plan

#

on a work day, you absolutely would get priority support with the teams plan

ocean crown Jul 2, 2023, 3:25 AM

#

i see

#

thanks for clarifying

#

thanks for the help @tropic delta! I see you here every day btw

#

so we appreciate the fight in the trenches!

#redis connection fails