New LIFs unavailable | NetApp | Page 1

iron forge Jul 4, 2023, 11:59 AM

#

Hello, while I'm working my way to make all my LIFs operational . I'm stuck. I tried online tutorial, guides before coming here for help. Where should I start ? (see attachment pictures) Thank you 🙂

jagged frigate Jul 4, 2023, 12:19 PM

#

Hi, start here - check why your ports (e0c-e0f) are down - check cabling - cabling to NW switch, are ports enabled on the switch... after that check your ifgrp - ifgrp show -instance

#

and which ports belong to ifgrp a0a (i guess e0c, e0d) ?

iron forge Jul 4, 2023, 12:38 PM

#

Hello Retlaw, thank you for taking time to help me out. Muchly appreciated. Here's a picture of my setup. As you can see : e0c,e0d,e0e is not used physically on both nodes but is used in a IFGRP e0c, e0d. e0f is used as interconnect / clustering (if I'm using the right terminology, correct me if I'm wrong)

jagged frigate Jul 4, 2023, 1:20 PM

#

is this a FAS2520? i think your cabling is not correct too...

shadow siren Jul 4, 2023, 1:42 PM

#

looks like a FAS2552 - right? see: https://library.netapp.com/ecm/ecm_download_file/ECMP1609081

iron forge Jul 4, 2023, 1:55 PM

#

It's 2 x FAS2552

#

Just letting you know the cabling was suggested by my "SENIOR MENTOR" which thinks he's a smart ass EGO X 1000 so I would be glad to show him that he's wrong and human. 😉

#

Do I need to put interconnect cable as well like I did with e0f ? e0c & e0d on both nodes ?

shadow siren Jul 4, 2023, 2:56 PM

#

see 2C in the link above ...... For option C only: Connect port e0e | 0e to port e0e | 0e and e0f | 0f to e0f | 0f.
and if you need an interface group - do that with e0c & e0d on every node.
i have an old FAS2554 and can make a photo if i find it 😉

iron forge Jul 4, 2023, 3:12 PM

#

@shadow siren I understand.

Tell me then what was the purpose for my senior mentor me to have a QSFP+ cable between e0f node 1 & e0f node 2 ?

From the picture 2C will I still need the network cable on e0a and e0b between both nodes (red CAT6 cable) ? Those are used for my cluster 1 & 2 so I guess I need to keep em.

The issue I'm concerned on the 2C picture is the (2) DATA NETWORK SWITCH, I only have a Standard switch RJ-45.

Thanks !

shadow siren Jul 4, 2023, 3:37 PM

#

connect e0e to e0e and e0f to e0f with a short 10gbit twinax cable <- that is the cluster network!
if you only have a standard rj45 switch with 1gbit then make an interface group on e0a & e0b on every node.
if you have a 10gbit switch then you take e0c & e0d for the interface group.

here is my config on a FAS2554 (only have 10gbit network):
Node: node 1
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status

e0M Default - down 1500 auto/- -
e0a Default - down 1500 auto/- -
e0b Default - down 1500 auto/- -
e0c Default - up 9000 auto/10000 healthy
e0d Default - up 9000 auto/10000 healthy
e0e Cluster Cluster up 9000 auto/10000 healthy
e0f Cluster Cluster up 9000 auto/10000 healthy

Node: node 2
Speed(Mbps) Health
Port IPspace Broadcast Domain Link MTU Admin/Oper Status

e0M Default - down 1500 auto/- -
e0a Default - down 1500 auto/- -
e0b Default - down 1500 auto/- -
e0c Default - up 9000 auto/10000 healthy
e0d Default - up 9000 auto/10000 healthy
e0e Cluster Cluster up 9000 auto/10000 healthy
e0f Cluster Cluster up 9000 auto/10000 healthy

civic moth Jul 4, 2023, 3:55 PM

#

the ports are labelled on the back of the system. You just need to put the correct ports in your interface group and it will work

iron forge Jul 5, 2023, 3:08 PM

#

#

Hello, I connected the 4 twinax cables accordingly as discussed. Now then, I know I will need to unplug e0a on both nodes and e0b also. I changed the ifgrp.

#

Now I want to change the cluster e0a,e0b to e0e and e0f but having trouble at the moment.

civic moth Jul 5, 2023, 3:34 PM

#

lol these are 2Gbit FC cables not 10G ethernet... where did you find those? The last systems that used those were the FAS3020's 😄

#

those cables are probably old enough to buy alcohol 😄

iron forge Jul 5, 2023, 3:56 PM

#

I found these at works. It's possible I might not have 10G Ethernet then.

#

I'll look at work again. If I don't have any spare 10g will I be able to make it work still with the 2 GB FC cables ?

reef marlin Jul 5, 2023, 4:02 PM

#

Wait.... why did you connect ports e0c & e0d between the nodes? 😄

#

simply do this

#

focus on the green cabling

#

you need to get this up and running otherwise your cluster will have issues

#

I think the minimum for the cluster network (green cabling) is to have at least one port (e0f).

iron forge Jul 5, 2023, 4:04 PM

#

I followed Roller's advice.

reef marlin Jul 5, 2023, 4:05 PM

#

But you need 10Gb. 1Gb via ports e0a & e0b is not supported/possible? for the cluster network.

iron forge Jul 5, 2023, 4:05 PM

#

reef marlin But you need 10Gb. 1Gb via ports e0a & e0b is not supported/possible? for the cl...

At the moment it's 1GB and it's working.

reef marlin Jul 5, 2023, 4:05 PM

#

iron forge I followed Roller's advice.

He did not say to connect ports e0c to e0c and e0d to e0d.

#

These ports are for frontend.

#

They need to connect to some switches.

#

But you first need to care about backend.

iron forge Jul 5, 2023, 4:06 PM

#

That's the thing I have one switch standard 1 GB

reef marlin Jul 5, 2023, 4:06 PM

#

Focus on port e0f, you need yourself either a supported twinax/DAC cable with 10Gb.

#

Or two 10Gb SFPs plus the LC-LC cable.

iron forge Jul 5, 2023, 4:08 PM

#

Ok got it I will verify at work. Therefore my e0a and e0b is the Cluster at the moment. Is it difficult to switch to e0e and e0f ?

#

Or I leave it like this ?

reef marlin Jul 5, 2023, 4:08 PM

#

What is this cluster for?

#

Is it running production workload?

iron forge Jul 5, 2023, 4:08 PM

#

Practice

reef marlin Jul 5, 2023, 4:08 PM

#

😄

#

Leave it on e0a & e0b, if it works.

iron forge Jul 5, 2023, 4:09 PM

#

We have a production at work. This netapp was given to me for practice.

reef marlin Jul 5, 2023, 4:09 PM

#

But if you don't have any switch with SFP+ you won't be able to use any of the ports for frontend (since the other RJ-45 ports are used for the cluster backend).

iron forge Jul 5, 2023, 4:09 PM

#

My mentor is rusted and kind of an ass so I'm trying by myself here.

reef marlin Jul 5, 2023, 4:10 PM

#

Or you put all the LIFs on e0M....

iron forge Jul 5, 2023, 4:10 PM

#

Crap more setbacks

#

Ok well I'm eating lunch right now and headed to work to see what I have.

reef marlin Jul 5, 2023, 4:11 PM

#

iron forge We have a production at work. This netapp was given to me for practice.

well you can't do any real practice if you don't have the necessary equipment (cables, sfps, switch)

iron forge Jul 5, 2023, 4:11 PM

#

for sure ! I appreciate your help sir. I will let you know what I have and if so I'll purchase the equipment. Works pays for it anyway

reef marlin Jul 5, 2023, 4:11 PM

#

It's like "here's your new car" and you notice "oh no steering wheel"

iron forge Jul 5, 2023, 4:12 PM

#

Yeah it sucks but I'M still learning a lot with this netapp system it's awesome and complex at the beginning but getting there with the commands

iron forge Jul 5, 2023, 4:36 PM

#

Bingo. Grabbing 2

rn_image_picker_lib_temp_6880f5ba-8f7a-4e6b-b702-cca98d795633.jpg

#

With 2 of these to replace the 2 that you laughed earlier.

rn_image_picker_lib_temp_5d705daa-71af-488a-b1db-17512b795010.jpg

#

I should be set.

reef marlin Jul 5, 2023, 4:58 PM

#

iron forge With 2 of these to replace the 2 that you laughed earlier.

These are correct, X6566B. Use them for e0e <--> e0e & e0f <--> e0f

#

Then move your Cluster LIFs to these ports.

#

You can then connect e0a and/or e0b to your switch and use them for frontend traffic

iron forge Jul 5, 2023, 6:07 PM

#

And the 10GB Cisco QSFP (pink package) won't be of use then ?

reef marlin Jul 5, 2023, 7:55 PM

#

You already got two cables for the cluster backend. Since you don't have any switch with SFP-ports the other Cisco cable does not really help.

#

Also this is no QSFP but a cable with a hardwired SFP+.

#

QSFP is the wider form-factor which is often been used for 40Gb Ethernet. (also for 6Gb SAS in the shelves)

#

In general Cisco DACs (direct-attached-copper) cables like this one usually work. Meaning ONTAP will allow it.

civic moth Jul 5, 2023, 8:25 PM

#

iron forge I'll look at work again. If I don't have any spare 10g will I be able to make i...

Fibre Channel cables do not work for Ethernet, especially not these old ones.

iron forge Jul 6, 2023, 2:11 AM

#

reef marlin Also this is no QSFP but a cable with a hardwired SFP+.

I was not sure understanding the terminology Frontend Traffic & Cluster Backend. I get it now. Did some research.

As for the frontend traffic I don't have any choice at the moment but to use the 1G. My home network
is not in one single entity. Guest room and living room using POE InLine. Also I have a 16 ports standard 1 GB switch.

The NetApp X6530-R6 0.5m Fiber Channel SFP to SFP Direct Attach Cable is specifically designed for Fiber Channel connectivity, which is typically used for storage area networks (SAN) rather than cluster management in NetApp environments.

[ still questioning myself why my EGO colleague told me to hook up e0f to e0f - pointless. What was on his mind that day. He's on vacation at the moment. ]

------------------------.

I will use the X6566B for e0e <--> e0e & e0f <--> e0f (Cluster Backend)

I just need to figure out how I will pull this off. Probably going to down the ports and modify it after. I'll try it tomorrow.

shadow siren Jul 6, 2023, 6:44 AM

#

i think it's easier to start again from scratch. connect the backend with the 10gbe twinax cables and swipe the configuration in the boot menu 😉

iron forge Jul 6, 2023, 10:27 AM

#

My heart rate went straight to 130 ! @reef marlin what's your take on it ?

civic moth Jul 6, 2023, 2:37 PM

#

I would just reconfigure the ports and move the LIFs. But I have done this 100s of times, so it might be that it's just very easy to me

reef marlin Jul 6, 2023, 2:39 PM

#

Yeah, it can still be saved 😄

#

Not sure what's already configured though, any running SVMs?

iron forge Jul 6, 2023, 2:45 PM

#

reef marlin Not sure what's already configured though, any running SVMs?

I have 2 SVM but they are Empty.

iron forge Jul 7, 2023, 11:00 AM

#

reef marlin Not sure what's already configured though, any running SVMs?

If you look at the picture at the starting of this post I have Couple LIFs, service policy custom that I created.

reef marlin Jul 7, 2023, 11:01 AM

#

Yeah, just cable it and move the Cluster LIFs

iron forge Jul 7, 2023, 12:55 PM

#

Looking into it 60 minutes in still haven't figured it out how to move the Cluster LIFs. Easy creating LIF into Vserver but I never changed ports before.

#

The E0E had broadcast domain : DEFAULT-2 and E0F had broadcast domain DEFAULT-1 - I removed them.

#

If this is a long painful process let me know I'll just start over from scratch.

#

#

Probably a way to strategically turning off Cluster lifs and then moving it in a certain order.

reef marlin Jul 7, 2023, 1:36 PM

#

make sure no LIFs are currently on e0e & e0f
check if e0e & e0f are currently in a broadcast- domain: broadcast-domain show
if they are remove them from the broadcast-domain: broadcast-domain remove-ports (you could also try broadcast-domain merge....)
add ports e0e & e0f to the Cluster broadcast-domain: broadcast-domain add-ports -ipspace Cluster -broadcast-domain Cluster

#

Since e0e/e0f and your current e0a/e0b are not in the same layer2, that might be sort of disruptive.

#

Then modify the home-port of the Cluster-LIFs to the e0e & e0f ports. You might also need to "net int migrate" them afterwards.

#

Once no more Cluster-LIFs are on ports e0a & e0b, remove them from the Cluster broadcast-domain.

iron forge Jul 7, 2023, 1:38 PM

#

reef marlin Jul 7, 2023, 1:39 PM

#

a0a is your ifgrp consisting of e0a & e0b right?

#

if yes, you need to destroy that

iron forge Jul 7, 2023, 1:39 PM

#

reef marlin Jul 7, 2023, 1:39 PM

#

ah ok

#

then just go ahead and add ports e0e & e0f to the Cluster broadcast-domain

iron forge Jul 7, 2023, 1:40 PM

#

I had this in mind but here's the thing when I try do that.

#

I can't see Cluster.

reef marlin Jul 7, 2023, 1:42 PM

#

I don't understand... do this:
broadcast-domain add-ports -ipspace Cluster -broadcast-domain Cluster -ports genesis-01:e0e
broadcast-domain add-ports -ipspace Cluster -broadcast-domain Cluster -ports genesis-01:e0f
broadcast-domain add-ports -ipspace Cluster -broadcast-domain Cluster -ports genesis-02:e0e
broadcast-domain add-ports -ipspace Cluster -broadcast-domain Cluster -ports genesis-02:e0f

#

you need to specify the ipspace

iron forge Jul 7, 2023, 1:42 PM

#

Worked. Kneeling before OG1

iron forge Jul 7, 2023, 2:09 PM

#

So far so good. Now need to modify the home port I guess.

iron forge Jul 7, 2023, 2:34 PM

#

Guess it worked :

#

And now I'm cabling e0a and e0b straight to the switch (1gb)

reef marlin Jul 7, 2023, 2:50 PM

#

looks good

#

If you're using Powerline I guess using ifgrps is not necessary. You won't get near 1Gb anyway.

iron forge Jul 7, 2023, 3:37 PM

#

Now I changed the e0a and e0b putting back the Broadcast Domains to Default.

#

Would you recommend changing the ifgrp a0a with the ports e0a and e0b or removing it ? My main purpose is to have some Status Green for my LIFS.

iron forge Jul 10, 2023, 12:29 PM

#

Here's the status right now. e0a, e0b from node1 is connected to the switch and e0a, e0b from node2 is connected to the switch. Now then my ifgrp (a0a) contains the 2 ports : e0c, e0d - what would be next step ?

#

rn_image_picker_lib_temp_8a10d71f-49c3-4afa-a3af-8972b1e7d717.jpg

#

I want to keep my IFGRP just for testing purposes.

reef marlin Jul 10, 2023, 2:57 PM

#

As always with port-channels (ifgrps) you need to configure both sides: ONTAP is already configured, you need to configure your switch. Can't really help you there.

#

If you got a consumer switch it could very well be that it does not even support port-channels.

#

Use ifgrp show -instance to find out what kind of ifgrp you configured (singlemode, multimode or even LACP and which distribution function). Then google to find out how to configure your switch.

#

You need to put ports e0c & e0d from node1 into one port-channel, and ports e0c & e0d from node2 into another port-channel.

#

I would simply remove the ifgrps, makes no sense in your scenario as a I see it, but that's up to you.

iron forge Jul 11, 2023, 9:50 AM

#

Hello @OG1 thank for replying. My consumer switch is a GSS116E – 16-Port Gigabit Ethernet Plus Click Switch which does not support indeed port-channels I was not surprised there. It does not support no modes mentionned above.

#

#

So I see 2 scenarios right now I will mention them below.

#

Scenario 1 : I ask my boss if he wants to buy a switch so I can have the proper setup. Any recommendation ?

iron forge Jul 11, 2023, 10:33 AM

#

Scenario 2 : The boss denies the request and then I try to make it work without IFGRP and then e0a and e0b will be my LIFS ? Now I am more confused. How am I going to pull this off.

#

Something like that ?

shadow siren Jul 11, 2023, 10:56 AM

#

scenario 2:

move all your SVM LIFS to the home-port e0M.
delete the interface group on both nodes.
move back the SVM LIFS to port e0a or e0b.

(problem: you can't move a LIF to a non connected port).

reef marlin Jul 11, 2023, 11:21 AM

#

iron forge Hello @OG1 thank for replying. My consumer switch is a GSS116E – 16-Port Gigabi...

It supports link-aggregation: https://www.netgear.com/images/datasheet/switches/clickswitches/GSS108E_GSS108EPP_GSS116E_DS.pdf

#

"Link Aggregation/Port Trunking (Static Manual LAGs only): 2 LAGs with max 8 members in each LAG"

#

use multimode and not multimode_lacp in ONTAP since it only supports static LAG and no LACP

#

distribution-function shouldn't matter, just choose port or ip

#

page 20: https://www.downloads.netgear.com/files/GDC/GSS108E/GSS108E_GSS116E_GSS108EPP_UM_EN.pdf#G4.1362108

#

You need to up your Google-game 😉

iron forge Jul 11, 2023, 11:25 AM

#

reef marlin You need to up your Google-game 😉

I need to stop relying on ChatGPT who gives me false information

#

I didn't know what I was up against when I started this project. What path did you take to know all of this from day 1 you started.

iron forge Jul 11, 2023, 3:57 PM

#

Crying of joy !!!

#

So the update is this :

I followed OG1 instruction regarding the switch which honestly I didn't there was an interface existing for the GS116E. I was positive it was only a unmanaged switch.

I should dig more next time but you showed me the concept of LAG behind which I'm happy to have learn.

I connected the following :

LAG 1 / node 1 - e0a port 2 / e0b port 3
LAG 2 / node 2 - e0a port 14 / e0b port 15

I also changed my a0a from e0c & e0d to e0a/e0b instead.

In the end all my LIFS are up and alive.

iron forge Jul 11, 2023, 7:41 PM

#

Can we say case is closed ? I think so !

reef marlin Jul 11, 2023, 7:55 PM

#

Yeah, looks much better now

sterile jackal Jul 12, 2023, 5:00 AM

#

thanks for the summary!

iron forge Jul 12, 2023, 1:55 PM

#

Thanks for your help everybody. The fun part can finally begin. 🙂

#New LIFs unavailable