FAS8300 Two-Stack Cabling | NetApp | Page 1

marble stump Mar 26, 2025, 1:45 AM

#

Hello, team! 😊

I am currently operating an FAS8300 with two DS224C shelves for SSDs, two DS460C shelves, and one DS212C shelf for SATA drives.

As shown in the attached image, I have configured the cabling in a Two-Stack setup.

However, the sysconfig output shows "Mixed-path HA," and the event log show displays the alert ALERT ses.multipath.ReqError: SAS disk shelf detected without a multipath configuration.

I have been looking through technical documentation but have found it difficult to locate information specifically about Two-Stack cabling. I would like to know which part of my cabling setup might be causing this issue.

Thank you! 🙏

#

http://docs.netapp.com/us-en/ontap-systems/fas8300/install-detailed-guide.html

This docs shows only single stack cabling

Detailed guide - FAS8300 and FAS8700

This guide gives detailed step-by-step instructions for installing a typical NetApp system. Use this guide if you want more detailed installation instructions.

prime coyote Mar 26, 2025, 7:30 AM

#

Check on which disks you only see one connection to find out which shelf/disks this alert is about:
disk show -fields initiator

marble stump Mar 26, 2025, 8:40 AM

#

disk    initiator
------- -----------
1.0.0   0a,0c,0c,0b
1.0.1   0a,0c,0c,0b
1.0.2   0a,0c,0c,0b
1.0.3   0a,0c,0c,0b
1.0.4   0a,0c,0c,0b
1.0.5   0a,0c,0c,0b
1.0.6   0a,0c,0c,0b
1.0.7   0a,0c,0c,0b
1.0.8   0a,0c,0c,0b
1.0.9   0a,0c,0c,0b
1.0.10  0a,0c,0c,0b
1.0.11  0a,0c,0c,0b
1.0.12  0a,0c,0c,0b
1.0.13  0c,0b,0a,0c
1.0.14  0c,0b,0a,0c
1.0.15  0c,0b,0a,0c
1.0.16  0c,0b,0a,0c
1.0.17  0c,0b,0a,0c
1.0.18  0c,0b,0a,0c
1.0.19  0c,0b,0a,0c
1.0.20  0c,0b,0a,0c
1.0.21  0c,0b,0a,0c
1.0.22  0c,0b,0a,0c
1.0.23  0a,0c,0c,0b
1.1.0   0a,0c,0c,0b
1.1.1   0a,0c,0c,0b
1.1.2   0a,0c,0c,0b
1.1.3   0a,0c,0c,0b
1.1.4   0a,0c,0c,0b
1.1.5   0a,0c,0c,0b
1.1.8   0a,0c,0c,0b
1.1.9   0a,0c,0c,0b
1.1.10  0a,0c,0c,0b
1.1.11  0a,0c,0c,0b
1.1.18  0a,0c,0c,0b
1.1.19  0a,0c,0c,0b

marble stump Mar 26, 2025, 9:00 AM

#

2.10.0 0a,0d,0b
2.10.1 0a,0d,0b
2.10.2 0a,0d,0b
2.10.3 0b,0a,0d
2.10.4 0a,0d,0b
2.10.5 0a,0d,0b
2.10.6 0a,0d,0b
2.10.7 0a,0d,0b
2.10.8 0a,0d,0b
2.10.9 0a,0d,0b
2.10.10 0a,0d,0b
2.10.11 0a,0d,0b
2.10.12 0a,0d,0b
2.10.13 0a,0d,0b
2.10.14 0a,0d,0b
2.10.15 0a,0d,0b
2.10.16 0a,0d,0b
2.10.17 0a,0d,0b
2.10.18 0a,0d,0b
2.10.19 0a,0d,0b
2.10.20 0a,0d,0b
2.10.21 0a,0d,0b
2.10.22 0a,0d,0b
2.10.23 0a,0d,0b
2.10.24 0a,0d,0b
2.10.25 0a,0d,0b
2.10.26 0a,0d,0b
2.10.27 0a,0d,0b
2.10.28 0a,0d,0b
2.10.29 0a,0d,0b
2.10.30 0b,0a,0d
2.10.31 0b,0a,0d
2.10.32 0a,0d,0b
2.10.33 0b,0a,0d
2.10.34 0b,0a,0d
2.10.35 0a,0d,0b
2.10.36 0a,0d,0b
2.10.37 0a,0d,0b
2.10.38 0a,0d,0b
2.10.39 0a,0d,0b
2.10.40 0a,0d,0b
2.10.41 0a,0d,0b
2.10.42 0a,0d,0b
2.10.43 0a,0d,0b
2.10.44 0a,0d,0b
2.10.45 0a,0d,0b
2.10.46 0a,0d,0b
2.10.47 0a,0d,0b
2.10.48 0a,0d,0b
2.10.49 0a,0d,0b
2.10.50 0a,0d,0b
2.10.51 0a,0d,0b
2.10.52 0a,0d,0b
2.10.53 0a,0d,0b
2.10.54 0a,0d,0b
2.10.55 0a,0d,0b
2.10.56 0a,0d,0b
2.10.57 0a,0d,0b
2.10.58 0a,0d,0b
2.10.59 0a,0d,0b

marble stump Mar 26, 2025, 9:24 AM

#

2.11.0 0b,0a,0d
2.11.1 0b,0a,0d
2.11.2 0b,0a,0d
2.11.3 0b,0a,0d
2.11.4 0b,0a,0d
2.11.5 0b,0a,0d
2.11.6 0b,0a,0d
2.11.7 0b,0a,0d
2.11.8 0b,0a,0d
2.11.9 0b,0a,0d
2.11.10 0b,0a,0d
2.11.11 0b,0a,0d
2.11.12 0b,0a,0d
2.11.13 0b,0a,0d
2.11.14 0b,0a,0d
2.11.15 0b,0a,0d
2.11.16 0b,0a,0d
2.11.17 0b,0a,0d
2.11.18 0b,0a,0d
2.11.19 0b,0a,0d
2.11.20 0b,0a,0d
2.11.21 0b,0a,0d
2.11.22 0b,0a,0d
2.11.23 0b,0a,0d
2.11.24 0b,0a,0d
2.11.25 0b,0a,0d
2.11.26 0b,0a,0d
2.11.27 0b,0a,0d
2.11.28 0b,0a,0d
2.11.29 0b,0a,0d
2.11.30 0b,0a,0d
2.11.31 0b,0a,0d
2.11.32 0b,0a,0d
2.11.33 0b,0a,0d
2.11.34 0b,0a,0d
2.11.35 0b,0a,0d
2.11.36 0b,0a,0d
2.11.37 0b,0a,0d
2.11.38 0b,0a,0d
2.11.39 0b,0a,0d
2.11.40 0b,0a,0d
2.11.41 0b,0a,0d
2.11.42 0b,0a,0d
2.11.43 0b,0a,0d
2.11.44 0b,0a,0d
2.11.45 0b,0a,0d
2.11.46 0b,0a,0d
2.11.47 0b,0a,0d
2.11.48 0b,0a,0d
2.11.49 0b,0a,0d
2.11.50 0a,0d,0b
2.11.51 0b,0a,0d
2.11.52 0b,0a,0d
2.11.53 0a,0d,0b
2.11.54 0b,0a,0d
2.11.55 0b,0a,0d
2.11.56 0a,0d,0b
2.11.57 0b,0a,0d
2.11.58 0a,0d,0b
2.11.59 0a,0d,0b
2.12.0 0d,0b,0d
2.12.1 0d,0b,0d
2.12.2 0d,0b,0d
2.12.3 0d,0b,0d
2.12.8 0d,0b,0d
2.12.9 0d,0b,0d
2.12.10 0d,0b,0d
2.12.11 0d,0d,0b

marble stump Mar 26, 2025, 9:30 AM

#

prime coyote Check on which disks you only see one connection to find out which shelf/disks t...

If this so, 2.xx.x doesn't connect with 0c??

jovial wraith Mar 26, 2025, 10:18 AM

#

Something is definitely not correct

What’s the output of
System node run -node * sasadmin expander_map

prime coyote Mar 26, 2025, 10:21 AM

#

Yes, there seems to be some issue with the stack where shelves 10, 11 and 12 are connected.

marble stump Mar 26, 2025, 2:00 PM

#

jovial wraith Something is definitely not correct What’s the output of System node run -node ...

I will trying to collect tmr and will upload

#

so confused with two stack cabling

jovial wraith Mar 26, 2025, 2:18 PM

#

For what it’s worth, your diagram doesn’t even match the output. The diagram has one stack on 0a/0d and the other stack on 0b/0c.

It appears the stack might be interconnected.

The output I asked for will absolutely yield useful information. I use that output all the time to verify my larger cabling jobs

marble stump Mar 27, 2025, 1:22 AM

#

#


Expanders on channel 0a:
Level    1: WWN 500a0980085fbe8d, ID  0, Serial Number ' SHFNC2204000373', Product 'DS22412IOM12A   ', Rev '0281', Slot A
Level    2: WWN 500a09800f315ced, ID  1, Serial Number '    952412000820', Product 'DS22412IOM12    ', Rev '0141', Slot A

Expanders on channel 0b:
Level    1: WWN 500a09800860961d, ID 10, Serial Number ' SHJSG2207000191', Product 'DS46012IOM12A   ', Rev '0281', Slot A
Level  1.1: WWN 5d039ea909edb03f, ID 111, Serial Number '        0027B6C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  1.2: WWN 5d039ea909e0b03f, ID 106, Serial Number '        002782C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  1.3: WWN 5d039ea909e3503f, ID 179, Serial Number '        00278D40', Product 'Traf DCM        ', Rev '0647', Slot A
Level  1.4: WWN 5d039ea909e2003f, ID 130, Serial Number '        00278800', Product 'Traf DCM        ', Rev '0647', Slot A
Level  1.5: WWN 5d039ea90841003f, ID 122, Serial Number '        00210400', Product 'Traf DCM        ', Rev '0647', Slot A
Level    2: WWN 500a0980086097ed, ID 11, Serial Number ' SHJSG2207000193', Product 'DS46012IOM12A   ', Rev '0281', Slot A
Level    3: WWN 500a09800f317c35, ID 12, Serial Number '    722434000552', Product 'DS21212IOM12    ', Rev '0141', Slot A
Level  2.1: WWN 5d039ea918a5503f, ID 128, Serial Number '        00629540', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.2: WWN 5d039ea918a5b03f, ID 182, Serial Number '        006296C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.3: WWN 5d039ea918a5303f, ID 160, Serial Number '        006294C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.4: WWN 5d039ea918a4f03f, ID 119, Serial Number '        006293C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.5: WWN 5d039ea918a5103f, ID 192, Serial Number '        00629440', Product 'Traf DCM        ', Rev '0647', Slot A

#

Level    1: WWN 500a09800f315ccd, ID  1, Serial Number '    952412000820', Product 'DS22412IOM12    ', Rev '0141', Slot B
Level    2: WWN 500a0980085fbe99, ID  0, Serial Number ' SHFNC2204000373', Product 'DS22412IOM12A   ', Rev '0281', Slot B

Expanders on channel 0d:
Level    1: WWN 500a09800f317c5d, ID 12, Serial Number '    722434000552', Product 'DS21212IOM12    ', Rev '0141', Slot B

#


Expanders on channel 0a:
Level    1: WWN 500a09800860986d, ID 10, Serial Number ' SHJSG2207000191', Product 'DS46012IOM12A   ', Rev '0281', Slot B
Level  1.1: WWN 5d039ea909edc03f, ID 223, Serial Number '        0027B700', Product 'Traf DCM        ', Rev '0647', Slot B
Level  1.2: WWN 5d039ea909e0c03f, ID 218, Serial Number '        00278300', Product 'Traf DCM        ', Rev '0647', Slot B
Level  1.3: WWN 5d039ea909e3603f, ID 163, Serial Number '        00278D80', Product 'Traf DCM        ', Rev '0647', Slot B
Level  1.4: WWN 5d039ea909e2103f, ID 114, Serial Number '        00278840', Product 'Traf DCM        ', Rev '0647', Slot B
Level  1.5: WWN 5d039ea90841103f, ID 106, Serial Number '        00210440', Product 'Traf DCM        ', Rev '0647', Slot B
Level    2: WWN 500a098008609835, ID 11, Serial Number ' SHJSG2207000193', Product 'DS46012IOM12A   ', Rev '0281', Slot B
Level  2.1: WWN 5d039ea918a5603f, ID 230, Serial Number '        00629580', Product 'Traf DCM        ', Rev '0647', Slot B
Level  2.2: WWN 5d039ea918a5c03f, ID 134, Serial Number '        00629700', Product 'Traf DCM        ', Rev '0647', Slot B
Level  2.3: WWN 5d039ea918a5403f, ID 112, Serial Number '        00629500', Product 'Traf DCM        ', Rev '0647', Slot B
Level  2.4: WWN 5d039ea918a5003f, ID 176, Serial Number '        00629400', Product 'Traf DCM        ', Rev '0647', Slot B
Level  2.5: WWN 5d039ea918a5203f, ID 144, Serial Number '        00629480', Product 'Traf DCM        ', Rev '0647', Slot B

Expanders on channel 0b:
Level    1: WWN 500a09800f315ced, ID  1, Serial Number '    952412000820', Product 'DS22412IOM12    ', Rev '0141', Slot A
Level    2: WWN 500a0980085fbe8d, ID  0, Serial Number ' SHFNC2204000373', Product 'DS22412IOM12A   ', Rev '0281', Slot A

#

Level    1: WWN 500a0980085fbe99, ID  0, Serial Number ' SHFNC2204000373', Product 'DS22412IOM12A   ', Rev '0281', Slot B
Level    2: WWN 500a09800f315ccd, ID  1, Serial Number '    952412000820', Product 'DS22412IOM12    ', Rev '0141', Slot B

Expanders on channel 0d:
Level    1: WWN 500a09800f317c35, ID 12, Serial Number '    722434000552', Product 'DS21212IOM12    ', Rev '0141', Slot A
Level    2: WWN 500a0980086097ed, ID 11, Serial Number ' SHJSG2207000193', Product 'DS46012IOM12A   ', Rev '0281', Slot A
Level  2.1: WWN 5d039ea918a5503f, ID 128, Serial Number '        00629540', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.2: WWN 5d039ea918a5b03f, ID 182, Serial Number '        006296C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.3: WWN 5d039ea918a5303f, ID 160, Serial Number '        006294C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.4: WWN 5d039ea918a4f03f, ID 119, Serial Number '        006293C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  2.5: WWN 5d039ea918a5103f, ID 192, Serial Number '        00629440', Product 'Traf DCM        ', Rev '0647', Slot A
Level    3: WWN 500a09800860961d, ID 10, Serial Number ' SHJSG2207000191', Product 'DS46012IOM12A   ', Rev '0281', Slot A
Level  3.1: WWN 5d039ea909edb03f, ID 111, Serial Number '        0027B6C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  3.2: WWN 5d039ea909e0b03f, ID 106, Serial Number '        002782C0', Product 'Traf DCM        ', Rev '0647', Slot A
Level  3.3: WWN 5d039ea909e3503f, ID 179, Serial Number '        00278D40', Product 'Traf DCM        ', Rev '0647', Slot A
Level  3.4: WWN 5d039ea909e2003f, ID 130, Serial Number '        00278800', Product 'Traf DCM        ', Rev '0647', Slot A
Level  3.5: WWN 5d039ea90841003f, ID 122, Serial Number '        00210400', Product 'Traf DCM        ', Rev '0647', Slot A

#

Still I don't know how to diagnose my connection 🤪🤪

jovial wraith Mar 27, 2025, 11:05 AM

#

I’m looking on my phone. Definitely wrong. I’ll need to get my PC out for a better look

#

Can you take downtime or not?

#

If you can take downtime...recable...correctly

#

Here is what I see from your output:

#

LXFF-01 0a -> Shelf 0, IOMA -> Shelf 1, IOMA 0c -> Shelf 1, IOMB -> Shelf 0, IOMB 0b -> Shelf 10, IOMA -> Shelf 11, IOMA -> Shelf 12 IOMA 0d -> Shelf 12, IOMB **-> Nothing ** LXFF-02 0b -> Shelf 1, IOMA -> Shelf 0, IOMA 0c -> Shelf 0, IOMB -> Shelf 1, IOMB 0a -> Shelf 10, IOMB -> Shelf 11, IOMB (missing Shelf 12) 0d -> Shelf 12, IOMA -> Shelf 11, IOMA -> Shelf 10 IOMA

jovial wraith Mar 27, 2025, 11:39 AM

#

What it should look like

`LXFF-01
0a -> Shelf 0 IOMA-port 1 -> Shelf 0 IOMA-port 3 -> Shelf 1 IOMA-port 1
0d -> Shelf 1 IOMB-port 3 -> Shelf 1 IOMB-port 1 -> Shelf 0 IOMB-port 3
0c -> Shelf 10 IOMA-port 1 -> Shelf 10 IOMA-port 3 -> Shelf 11 IOMA-port 1 -> Shelf 11 IOMA-port 3 -> Shelf 12 IOMA-port 1
0b -> Shelf 12 IOMB-port 3 -> Shelf 12 IOMB-port 1 -> Shelf 11 IOMB-port 3 -> Shelf 11 IOMB-port 1 -> Shelf 12 IOMB-port 3

LXFF-02
0a -> Shelf 0, IOMB-port 1 -> Shelf 0 IOMB-port 3 -> Shelf 1 IOMB (port 1)
0d -> Shelf 1, IOMA-port 3 -> Shelf 1 IOMA-port 1 -> Shelf 0 IOMA (port 3)
0c -> Shelf 10 IOMB-port 1 -> Shelf 10 IOMB-port 3 -> Shelf 11 IOMB-port 1 -> Shelf 11 IOMB-port 3 -> Shelf 12 IOMB-port 1
0b -> Shelf 12 IOMA-port 3 -> Shelf 12 IOMA-port 1 -> Shelf 11 IOMA-port 3 -> Shelf 11 IOMA-port 1 -> Shelf 12 IOMA-port 3`

#

It would be easier to shut down and cable correctly. I have been known in the field to correct this live without disruption. In this case it is more complex. I end up using ports 2 & 4) to relocate cables to the correct module before final/correct placement. That would take significant more time to roll out a solution. Best to shut down and cable correctly.

halcyon delta Mar 28, 2025, 12:57 PM

#

also using configadvisor or activeiq to visualize the connections is helpful

I got a headache trying to figure ou the cabling from that output 😅
well done TMAC 🙂

jovial wraith Mar 28, 2025, 1:16 PM

#

Config advisor is not always correct! Many many times missing links or even worse wrong links appear for the connections. I never trust that diagram

marble stump Apr 1, 2025, 7:11 AM

#

jovial wraith Can you take downtime or not?

Maybe we ca not

#FAS8300 Two-Stack Cabling