#The pod using iSCSI PVCs fails to start.

1 messages · Page 1 of 1 (latest)

cerulean sail
#

I created an iSCSI (ONTAP-SAN) PVC following the steps in the linked Trident documentation.
Attempting to attach and mount this PVC to a pod may result in a mount error.
This mount error occurs approximately 50% of the time and prevents the pod from starting.

https://docs.netapp.com/us-en/trident-2506/trident-use/vol-provision.html

Deleting and recreating the pod may allow it to mount successfully.
Please advise on possible causes.

Below is the version information and error message (from pod describe) for the environment where the error occurr

[tri@cont ~]$ tail /etc/redhat-release
Red Hat Enterprise Linux release 9.4 (Plow)

[tri@cont ~]$ kubectl version
Client Version: v1.32.8+rke2r1
Kustomize Version: v5.5.0
Server Version: v1.32.8+rke2r1
Trident 25.06.2 (ontap-san)

#

Events:
Type Reason Age From Message

Normal Scheduled 40s default-scheduler Successfully assigned default/pv-pod-iscsi1 to container
Normal SuccessfulAttachVolume 39s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-0b03f5f5-7039-4415-b05f-4098cbd101d2"
Warning FailedMount 2s (x7 over 34s) kubelet MountVolume.MountDevice failed for volume "pvc-0b03f5f5-7039-4415-b05f-4098cbd101d2" : rpc error: code = Internal desc = rpc error: code = Internal desc = failed to stage volume: multipathd is not running
[tri@cont ~]$

pastel nebula
#

multipathd is not running

your iSCSI setup on the nodes seems to be incomplete. Try sceheduling the pod on different nodes to see which nodes are affected (I assume not all are affected since you said it sometimes works)

opaque abyss
#

iscsid/multipathd must be active on all nodes

#

did you add nodes to your RKE2 cluster after the initial setup, which could explain why you would have different behaviors depending on the nodes

cerulean sail
#

Thank you for your reply. I will check it in a test environment based on your advice.