#Kubernetes Sidecar CrashLoopBackoff
drowsy condor
Hey @red bramble thanks for highlighting the problem. We're checking out at the moment, and will get back to you as soon as possible 👍
drowsy condor
Hey @red bramble , do you mind sharing us the details of your configuration?
red bramble
red bramble
here are the permify environment variables
PERMIFY_RATE_LIMIT: 200
PERMIFY_GRPC_PORT: 3478
PERMIFY_HTTP_ENABLED: true
PERMIFY_HTTP_PORT: 3476
PERMIFY_LOG_LEVEL: debug
PERMIFY_DATABASE_ENGINE: postgres
PERMIFY_DATABASE_AUTO_MIGRATE: true
PERMIFY_DATABASE_MAX_OPEN_CONNECTIONS: 20
PERMIFY_DATABASE_MAX_IDLE_CONNECTIONS: 1
PERMIFY_DATABASE_MAX_CONNECTION_LIFETIME: 1800s
PERMIFY_DATABASE_MAX_CONNECTION_IDLE_TIME: 60s
PERMIFY_DATABASE_GARBAGE_ENABLED: true
PERMIFY_DATABASE_GARBAGE_COLLECTION_INTERVAL: 3m
PERMIFY_DATABASE_GARBAGE_COLLECTION_TIMEOUT: 3m
PERMIFY_DATABASE_GARBAGE_COLLECTION_WINDOW: 720h
PERMIFY_DATABASE_GARBAGE_COLLECTION_NUMBER_OF_THREADS: 1
Thanks for the update @red bramble we are on it! Please allow us to test out, and I'll get back to you once we figure out. Sorry for the inconvience.
wanton siren
@red bramble Hi!
Can you see what print in the log output when you crash in Kubernetes?
red bramble
context deadline exceeded (Client.Timeout exceeded while awaiting headers)
{"level":"info","time":"2023-07-13T07:23:02Z","caller":"/home/runner/work/permify/permify/internal/servers/tenancyServer.go:78","message":"ERROR_CODE_EXECUTION"}
It seems to have happened after Garbage Collector run this time but I don't know if it's linked
wanton siren
is the network communication of the application with permify not healthy because this part confused me a little bit
- name: PERMIFY_GRPC_ENDPOINT
value: localhost:3478
No it's good, in fact at start it runs completely without any trouble but as I said, after a while it does not get any response from permify
wanton siren
Garbage collector cleans the old version tuple data at a certain window interval, so I don't think it will be related to it
red bramble
- name: PERMIFY_GRPC_ENDPOINT
value: localhost:3478
This just define an environment variable for the app to be working with
so it use localhost:3478 to connect to the gRPC endpoint of permify
wanton siren
if you are suspicious of the garbage collector you can turn it off and try again, if you have the same problem again I will try the application with similar configurations and try to find the problem
red bramble
but you don't have my app
wanton siren
but you don't have my app
I will send external request with postman
or with go client
red bramble
Okay