Mounting PVs created with Trident randomly hangs on K8s node

[mellow blaze]

Hey!
Some nodes on our K8s clusters which are using Trident as the storage class sometimes just hang on mounting a specific PV.
Other nodes on the same cluster are able to mount the PV perfectly.
When we restart the node it for some reason fixed the issue..
When I was looking in the network of the node it seems the same as other working nodes. It is just stuck on the "mount" syscall forever. This problem doesn't let pods to terminate at all and just breaks our production cluster.
I don't know how to continue debugging this issue and I'm seeking for help.
Do you have any idea what I should look into?

General info:
We use Trident in version 22.07
K8s version 1.20

Thank you in advance!!

[pallid minnow]

First, you want to enable debugging in Trident.
https://kb.netapp.com/Advice_and_Troubleshooting/Cloud_Services/Astra_Trident/How_to_enable_Trident_debug_and_debugTraceFlag_for_more_verbose_logging
Once you have debugging enabled and you have run into the problem again, create a support bundle
https://kb.netapp.com/Advice_and_Troubleshooting/Cloud_Services/Astra_Trident/How_best_to_collect_Trident_logs_in_a_Kubernetes_or_Openshift_environment
Then open a case with NetApp support and they will look through the issue with you.

Having said all that, if it is hanging at the mount command itself, there is not much help NetApp can give as the mount command is an OS call from Trident. If you aren't completely sure that running the mount command is hanging then go through the steps outlined above and support will assist in finding the cause.