SMB/CIFS SVM networking errors | NetApp | Page 1

wooden trench Apr 5, 2023, 9:48 PM

#

Getting a weird error I've never seen before in 9.12.1P1.

Can anyone point me in the right direction?

#

What I've found so far is that the new SVM is trying to get created and talk to DNS over the specific storage network/subnet I created and assigned that isn't routable

last ferry Apr 5, 2023, 10:03 PM

#

do you get the same error in the CLI?

wooden trench Apr 5, 2023, 10:03 PM

#

wooden trench Apr 5, 2023, 10:05 PM

#

last ferry do you get the same error in the CLI?

Have not tried yet. Been doing my best to attempt to do everything in SysMgr in order to send up some feedback

#

It worked earlier when I was using the same .10 subnet, but that doesn't let me mount datastores properly.

.10.0 is 1Gb Hosts subnet
.20.0 is 10Gb storage subnet

#

It worked for NFS, but AD is throwing a fit when trying to replicate for SMB/CIFS

#

wanna jump in a room and I can show ya?

rough lotus Apr 5, 2023, 10:11 PM

#

Does your new SVM have a mgmt-LIF which can reach the DNS-servers?

last ferry Apr 5, 2023, 10:11 PM

#

let me check on my kid and her math homework and i'll join. <5 min

rough lotus Apr 5, 2023, 10:13 PM

#

and do you have a route inside the new SVM to reach the DNS-servers?

last ferry Apr 5, 2023, 10:13 PM

#

wooden trench wanna jump in a room and I can show ya?

ready

wooden trench Apr 5, 2023, 11:25 PM

#

rough lotus Does your new SVM have a mgmt-LIF which can reach the DNS-servers?

I had a mess and needed to clean things up a bit. Tear down SVMs and overhaul ipspaces, Broadcast Domains, and Subnets. Much cleaner now. Gonna try to recreate the svm's again. (Yes, I'm taking notes for PM)

#

last ferry Apr 6, 2023, 1:00 AM

#

i'm back on if you need something @wooden trench

wooden trench Apr 6, 2023, 1:05 AM

#

I’m wondering if I’m just going at this the wrong way. 1Gb for hosts, 10Gb for storage. Completely isolated from one another; and everything that needs/uses 10Gb for storage is multihomed.

last ferry Apr 6, 2023, 1:09 AM

#

I have my lab 2552 setup like so -

📎 message.txt

#

so host accessable NAS SMB/NFS. - runs on my normal network.
VMware NFS and iSCSI are isilonlated in their own vlan.

#

the Synology also connects to the VLAN 10 incase I need to dump vms off the netapp for whatever reason

wooden trench Apr 6, 2023, 1:14 AM

#

Right but I have a 10Gb Synology too. My PC also has it. All multihomed. Why can’t I tell VSC/OTV to mount a datastore over a particular subnet? It’s trying to do everything over the cluster mgmt port

#

I have a wide open export policy where the whole subnet can hit the volume r/w

#

Works great mounting Synology volumes to hosts and even my docker hostVM

#

I know the networking is in good shape, is my point

last ferry Apr 6, 2023, 1:17 AM

#

can ontap ping esx and vise versa?

#

exports all good?

#

i have all my exports as just open cause the only thing that can see them is VMware

wooden trench Apr 6, 2023, 1:18 AM

#

I haven’t gotten that far to start doing vmkping’s from esxcli

river remnant Apr 6, 2023, 2:00 AM

#

Looks like the setup can’t reach any of the domain controllers. Is the route correct? Is the setup choosing an incorrect port?
Remember with routes, using the GUI is not so good. Every route gets a metric off 20. With multiple default routes, they should have an order. The lower the metric the more important the route
There is a way to tell VSC/OTV to use a certain ip of I recall. It’s in the documentation. I think it requires editing a file on OTV.

network interface show
route show

wooden trench Apr 6, 2023, 2:07 AM

#

Gimme a few and I’ll get some outputs back

wooden trench Apr 6, 2023, 3:01 AM

#

  (network interface show)
            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
C250
            C250-01_mgmt up/up    172.16.10.233/24   C250-01       e0M     true
            C250-02_mgmt up/up    172.16.10.232/24   C250-02       e0M     true
            cluster_mgmt up/up    172.16.10.230/24   C250-01       e0M     true
Cluster
            C250-01_clus1
                         up/up    169.254.195.174/16 C250-01       e0c     true
            C250-01_clus2
                         up/up    169.254.41.188/16  C250-01       e0d     true
            C250-02_clus1
                         up/up    169.254.120.165/16 C250-02       e0c     true
            C250-02_clus2
                         up/up    169.254.126.211/16 C250-02       e0d     true
svmNFS
            lif_svmNFS_118
                         up/up    172.16.20.27/24    C250-01       e1b     true
            lif_svmNFS_156
                         up/up    172.16.20.25/24    C250-01       e1c     true
            lif_svmNFS_169
                         up/up    172.16.20.26/24    C250-01       e1c     false
10 entries were displayed.

C250::>

#

Vserver             Destination     Gateway         Metric
------------------- --------------- --------------- ------
C250
                    0.0.0.0/0       172.16.10.1     20
svmNFS
                    0.0.0.0/0       172.16.20.1     20
2 entries were displayed.

C250::>

#

It does look like something to do with routing after all

#

SMB/CIFS SVM networking errors

wooden trench Apr 6, 2023, 4:24 AM

#

Further, I ended up cleaning things up a little bit more with some LACP ifgrps

wooden trench Apr 6, 2023, 4:47 AM

#

#

Hey @stuck moth since you're around, any of this ring a bell with you? Trying to separate host traffic (172.16.10.0/24) from storage 10gig network (172.16.20.0/24). 10gig is on an isolated switch. For some reason I can't get VSC to provision a datastore, and can't get an SMB/CIFS SVM to communicate over .10 to the domain controllers/DNS for auth.

stuck moth Apr 6, 2023, 4:51 AM

#

Hmm does .20 have a default gateway defined?

#

Best to get rid of that inside ontap if so

wooden trench Apr 6, 2023, 4:52 AM

#

#

Negative.

#

  (network interface show)
            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
C250
            C250-01_mgmt up/up    172.16.10.233/24   C250-01       e0M     true
            C250-02_mgmt up/up    172.16.10.232/24   C250-02       e0M     true
            cluster_mgmt up/up    172.16.10.230/24   C250-01       e0M     true
Cluster
            C250-01_clus1
                         up/up    169.254.195.174/16 C250-01       e0c     true
            C250-01_clus2
                         up/up    169.254.41.188/16  C250-01       e0d     true
            C250-02_clus1
                         up/up    169.254.120.165/16 C250-02       e0c     true
            C250-02_clus2
                         up/up    169.254.126.211/16 C250-02       e0d     true
svmNFS
            lif_svmNFS_242
                         up/up    172.16.20.25/24    C250-01       a0b     true
            lif_svmNFS_378
                         up/up    172.16.20.26/24    C250-02       a0b     true
            lif_svmNFS_774
                         up/up    172.16.10.25/24    C250-01       e0M     true
10 entries were displayed.

C250::> route show
Vserver             Destination     Gateway         Metric
------------------- --------------- --------------- ------
C250
                    0.0.0.0/0       172.16.10.1     20
svmNFS
                    0.0.0.0/0       172.16.10.1     20
2 entries were displayed.

C250::>

#

Current status

#

a0b = (4) 10Gb ports on the mezz card e1a/b/c/d

#

I need to tell the SVMs that "Hey! All your .20 neighbors are over here!"

stuck moth Apr 6, 2023, 4:59 AM

#

Hmm ok, that should work for AD auth at least

#

It’s lunch time here and I have family over, can we reconnect in an hour?

wooden trench Apr 6, 2023, 5:01 AM

#

Yea sure I'll probably still be up

#

This is the final nut to crack, getting this thing operational.

whole bluff Apr 6, 2023, 7:30 AM

#

you could try to create the SVM with no protocols then add routes to it and create dns for it

#

after that you can check if dns works for that svm and then enable protocols

#

other way to do it is create a svm with nfs and without CIFS at first ; in this case you will have the lifs and you do the same thing create routes and dns for it , then try to enable CIFS.

#

by the way you need to set NTP also from start

wooden trench Apr 6, 2023, 7:33 AM

#

That was the idea. I was creating separate svm's, one for SMB/CIFS and one for NFS. All of that traffic will be pretty exclusive to the 10gig subnet as well. Which is where the tricky part seems to be coming in.

whole bluff Apr 6, 2023, 7:34 AM

#

yes but you can disable NFS after that since those are just headers over the SVM itself

#

so you have one for NFS that works and you can try create one multiprotocol (NFS / CIFS ) and then disable the NFS on it after the setup

#

i did have yesterday at a client the same issue on A400's 9.11.1p6 but there was the problem with filtering of ports and also routes on the A400's

#

netapp is trying to contact dns on the data lif and if that is different than the original mgmt subnet ( as in your case ) it needs routes

wooden trench Apr 6, 2023, 7:37 AM

#

but there's no gateway to route to?

#

it's just a flat L2 10gig subnet

whole bluff Apr 6, 2023, 7:40 AM

#

i never configured it without a gateway so i dont know if it works in practice

#

for ex your C250/FAS500F nfsSVM has this GW 172.16.10.1 and it is the same as mgmt one

#

can you ping from a 172.16.10.0 to 172.16.20.0 ranges ?

#

from an ip in those subnet to the other subnet ?

wooden trench Apr 6, 2023, 7:43 AM

#

no, the switches are completely isolated

whole bluff Apr 6, 2023, 7:43 AM

#

oki then it needs to be routed somehow

wooden trench Apr 6, 2023, 7:43 AM

#

all hosts/clients are multi-homed that use 10gig

whole bluff Apr 6, 2023, 7:43 AM

#

how can it get to the dns ?

wooden trench Apr 6, 2023, 7:43 AM

#

thats kinda the point, it doesn't need DNS

whole bluff Apr 6, 2023, 7:44 AM

#

https://community.netapp.com/t5/ONTAP-Discussions/Do-I-need-DNS-access-on-SVM-LIFS-to-serve-CIFS/m-p/149476

Do I need DNS access on SVM LIFS to serve CIFS?

Hi everyone! Short version: Do I need direct access to DNS from SVM data LIFS to join the SVM to an AD Domain and serve CIFS? Shouldn't DNS access from the node management interfaces be enough? Long version: We've recently acquired a FAS2750 with OnTap 9.4P1 to replace our aging FAS8020 (still r...

#

i think you need DNS 🙂

#

so you either need to route that or construct a svm mgmt lif

#

that can get to dns

wooden trench Apr 6, 2023, 7:47 AM

#

I was doing that, at the bottom of svm creation there's an option to add a mgmt int for the svm, and I was putting it on the .10 subnet.

#

which routes to 172.16.10.1

whole bluff Apr 6, 2023, 7:47 AM

#

yep

#

but the svm itself needs that route added for 0.0.0.0/0 gw 172.16.10.1

#

it does not need to be 0.0.0.0 obviously

burnt echo Apr 6, 2023, 3:54 PM

#

https://docs.netapp.com/us-en/ontap-tools-vmware-vsphere/configure/add_different_subnets.html

wooden trench Apr 6, 2023, 5:39 PM

#

I appreciate everyone's assistance. Just an update, I believe I was able to get everything wired up properly. Took some manual lif creation/intervention, but I'm able to hit c$ from my desktopPC over 10Gb.

  (network interface show)
            Logical    Status     Network            Current       Current Is
Vserver     Interface  Admin/Oper Address/Mask       Node          Port    Home
----------- ---------- ---------- ------------------ ------------- ------- ----
C250
            C250-01_mgmt up/up    172.16.10.233/24   C250-01       e0M     true
            C250-02_mgmt up/up    172.16.10.232/24   C250-02       e0M     true
            cluster_mgmt up/up    172.16.10.230/24   C250-01       e0M     true
Cluster
            C250-01_clus1
                         up/up    169.254.195.174/16 C250-01       e0c     true
            C250-01_clus2
                         up/up    169.254.41.188/16  C250-01       e0d     true
            C250-02_clus1
                         up/up    169.254.120.165/16 C250-02       e0c     true
            C250-02_clus2
                         up/up    169.254.126.211/16 C250-02       e0d     true
svmNFS
            lif_svmNFS_mgmt
                         up/up    172.16.10.25/24    C250-01       a0a     true
            lif_svmNFS_node02_data10G
                         up/up    172.16.20.26/24    C250-02       a0b     true
            lif_svmNFS_node1_data10G
                         up/up    172.16.20.25/24    C250-01       a0b     true
            lif_svmSMB_mgmt
                         up/up    172.16.10.26/24    C250-02       a0a     true
svmSMB
            lif_svmSMB_330
                         up/up    172.16.20.27/24    C250-01       a0b     true
            lif_svmSMB_833
                         up/up    172.16.20.28/24    C250-02       a0b     true
            lif_svmSMB_node01_data10G
                         up/up    172.16.20.29/24    C250-01       a0b     true
            lif_svmSMB_node01_data1G
                         up/up    172.16.10.27/24    C250-01       a0a     true
            lif_svmSMB_node02_data10G
                         up/up    172.16.20.30/24    C250-02       a0b     true
            lif_svmSMB_node02_data1G
                         up/up    172.16.10.28/24    C250-02       a0a     true
17 entries were displayed.

C250::>

#

Unfortunately, I borked my vCenter instance so it looks like I get to rebuild that now.

wooden trench Apr 6, 2023, 10:24 PM

#

Massive hattip to @unkempt hamlet and @marsh zealot for rescuing my vCenter with me, and big ups as well to @last ferry, @stuck moth, and @mortal oak for getting my head right around the storage system.

Documenting for future searches: The big issue was getting SMB/CIFS to talk to data over a different subnet than the gateway/AD resided.

For layout purposes, I ended up with two subnets...

#

#

...separated across two Broadcast Domains...

#

#

svmSMB was created with the 1G subnets selected in order to get the auth done with AD

#

#

This created a DATA lif per node which was renamed for organizational purposes, and attached to the a0a (e0a | e0b) ifgrp

#

#

Then I used SysMgr to create 2 additional DATA lif's on the 10G subnet

#

#

...using a0b (e1a | e1b | e1c | e1d) ifgrp

#

After this, I was able to access \172.16.20.28\c$ from a windows PC. So a volume was created, and I was successfully able to move data.

#

Footnotes: System Mgr makes a mess of things trying to be slick, and it required a whole heap of going to the cmdline and undoing a lot of the mess manually. I've taken some extreme notes for @exotic wraith and @deft dome that I'm sure their teams will appreciate. 🙂

#SMB/CIFS SVM networking errors