Recover from deleted tridentbackend CR

[earnest ore]

Hi. Due to a human error tridentbackend and tridentbackendconfig CRs was deleted in a test cluster. How do we recover from such a scenario . All PVCs and data on the storage backend (AWS FSx for Netapp) are untouched. the problem is that the tbc is in failed state, because it's referring to a backend with the wrong ID

NAME BACKEND NAME BACKEND UUID PHASE STATUS
fsx01-k8s11-nas fsx01-k8s11-nas 50feed04-60fb-4da1-a2e9-171b6f5e16bd Bound Success
fsx01-k8s11-nas-eco Failed
fsx01-k8s11-san fsx01-k8s11-san 66463408-1d3c-4b58-b00c-e85032cfdf93 Bound Success
fsx01-k8s11-san-eco Failed

NAME BACKEND BACKEND UUID
tbe-gmp6j fsx01-k8s11-nas-eco 7b975d90-95da-4a2c-bc80-ad63d2c80328
tbe-gtcbg fsx01-k8s11-san 66463408-1d3c-4b58-b00c-e85032cfdf93
tbe-n2wlc fsx01-k8s11-nas 50feed04-60fb-4da1-a2e9-171b6f5e16bd
tbe-qmnn7 fsx01-k8s11-san-eco 44b5dfc6-5da2-41fb-8042-842086e28db5

Describing the fsx01-k8s11-nas-eco backednconfig
Name: fsx01-k8s11-nas-eco
Namespace: kube-netapp
Labels: <none>
Annotations: argocd.argoproj.io/tracking-id: kube-kate-apps-k8s11_trident-csi:trident.netapp.io/TridentBackendConfig:kube-netapp/fsx01-k8s11-nas-eco
API Version: trident.netapp.io/v1
Kind: TridentBackendConfig
Metadata:
Creation Timestamp: 2023-12-05T19:19:16Z
Finalizers:
trident.netapp.io
Generation: 1
Resource Version: 673148312
UID: 825845ea-48f4-4e72-a1dd-22355965f614
Spec:
Auto Export CID Rs:
10.4.0.0/16
Auto Export Policy: true
Credentials:
Name: fsx-netapp
Debug: true
Defaults:
Tiering Policy: auto
Management LIF: svm-0759f8947c1af0601.fs-0eff69ff300b9fecc.fsx.eu-north-1.amazonaws.com
Storage Driver Name: ontap-nas-economy
Storage Prefix: k8s11
Svm: k8s11
Version: 1
Status:
Backend Info:
Backend Name:
Backend UUID:
Deletion Policy: delete
Last Operation Status: Failed
Message: Failed to create backend: backend 'fsx01-k8s11-nas-eco' update initiated using an invalid configRef, it is associated with configRef '170dae9a-b860-4e0f-8a13-087cdcdeed06' and not '825845ea-48f4-4e72-a1dd-22355965f614'
Phase:
Events:
Type Reason Age From Message

---

Warning Failed 12m trident-crd-controller Failed to create backend: backend 'fsx01-k8s11-nas-eco' update initiated using an invalid configRef, it is associated with configRef '170dae9a-b860-4e0f-8a13-087cdcdeed06' and not '825845ea-48f4-4e72-a1dd-22355965f614'

describing the tbe-gmp6j fsx01-k8s11-nas-eco backend
Name: tbe-gmp6j
Namespace: kube-netapp
Labels: <none>
Annotations: <none>
API Version: trident.netapp.io/v1
Backend Name: fsx01-k8s11-nas-eco
Backend UUID: 7b975d90-95da-4a2c-bc80-ad63d2c80328
Config:
ontap_config:
... (truncated to save som characters)
Disable Delete: false
Empty Flexvol Deferred Delete Period:
Flexgroup Aggregate List:
Igroup Name:
Labels: <nil>
Limit Aggregate Usage:
Limit Volume Size:
Luns Per Flexvol:
Management LIF: svm-0759f8947c1af0601.fs-0eff69ff300b9fecc.fsx.eu-north-1.amazonaws.com
Nas Type: nfs
Nfs Mount Options:
Password: secret:fsx-netapp
Qtree Prune Flexvols Period:
Qtree Quota Resize Period:
Qtrees Per Flexvol:
Region:
Replication Policy:
Replication Schedule:
San Type: iscsi
Serial Numbers:
Smb Share:
Storage: <nil>
Storage Driver Name: ontap-nas-economy
Storage Prefix: k8s11
Supported Topologies: <nil>
Svm: k8s11
Trusted CA Certificate:
Usage Heartbeat:
Use CHAP: false
Use REST: false
Username: secret:fsx-netapp
Version: 1
Zone:
Config Ref: 170dae9a-b860-4e0f-8a13-087cdcdeed06
Kind: TridentBackend
Metadata:
Creation Timestamp: 2022-09-07T12:36:43Z
Finalizers:
trident.netapp.io
Generate Name: tbe-
Generation: 116
Resource Version: 673148319
UID: e14bb205-80e8-4e55-85f8-8e3dd1e577e0
Online: true
State: online
Version: 1
Events: <none>