Hi, in a specific case I wonder if is possible have multiple clusters AFF in switches at 100G, if so... there's a possibility that I can have LIFs intercluster in the same ports/switches interclusters only for snapmirror from cluster A to cluster B?, I already search for specific documentation or KB but everything is "old" docs (2013, 2016, 2019).
#Can I have multiple clusters in same switch intercluster?
tired bane
hasty cobalt
not supported, sorry. For migration purposes only, approach your SE for an FPVR for potential support. It may or may not be granted.
the cluster switches are a very important component of the cluster, so we are very adverse to approving any changes or alternative configurations for them
tired bane
Thanks for you awnser, is there some official document that says is not supported? as mentionned before I didn't find anything specific about it, I understand the point for the old switches at 10G. but with the new switches 100G I think there's enough bandwidth to support more than only 1 cluster in it.
haughty nymph
If I understand you correctly you want to connect different clusters to the same pair of cluster interconnect switches, right?
If yes, that's now possible but only with the Cisco Nexus 9336C-FX2 (procured via NetApp).
You need to use the "MultiCluster-HA" RCF-file: https://mysupport.netapp.com/site/info/cisco-ethernet-switch
Also you need to be on a current patch-release of ONTAP (9.8-9.13.1 is supported, but don't use older patch-versions). It's not yet in the HWU, so ask your NetApp guys for the versions.
tawny basalt
Except when trying to download:
No software download details available
And the download page only indicates 9.10.1-9.14.1 as supported
hasty cobalt
Please refer to my original comment - ask your SE 🙂
Sharing a switch pair between clusters would be something I’d be very very careful about, and want NetApp’s approval for
haughty nymph
@tawny basalt There's a document on Fieldportal which mentions the supported versions. It goes down to 9.8.
tawny basalt
I’ll check there. Thanks
@hasty cobalt if it’s got an actual link on the support page, it should not direct to that message about “no software download details available”. There should be better info than that result. Maybe a page saying to consult someone.
hasty cobalt
Fair. @errant knoll can you pls look into options for it? Or fixing the link 😉
errant knoll
@deep zealot can you take a look at this please?
tired bane
If I understand you correctly you want to connect different clusters to the same...
Yes that's part of the question, the other part, once I got that configuration 2 clusters A and B in the same switches for cluster interconnect then I can use the same ports of the switch and create LIF intercluster for replciation (snaomirror) between the clusters A and B?
tired bane
Sharing a switch pair between clusters would be something I’d be very very caref...
I asked and he told me was not aware of it 😦
haughty nymph
Yes that's part of the question, the other part, once I got that configuration 2...
Nope, you can't. The cluster LIFs need to be on dedicated ports. Also the cluster interconnect switches can only be used for cluster interconnect traffic (and to connect your 100GbE shelves).
SnapMirror is intercluster traffic which needs to be on front-end data ports. You need to connect them to your datacenter switches.
tawny basalt
FYI: the rcf files are now posted on the support site
haughty nymph
FYI: the rcf files are now posted on the support site
they were not before? On friday I could download everything without issues (I think I still have those RCF-files lying around on my desktop)
But shouldn't that be "at most"?
tawny basalt
No. Every time I tried to grab the multi cluster rcf files I got the message no software download details available.
I suspect Julie fixed it yesterday
tawny basalt
But shouldn't that be "at most"?
Yeah. Hey @errant knoll , would you mind getting the language fixed?
tawny basalt
But shouldn't that be "at most"?
I have complained about the grammar on the OTP process to.
At some point it says “You can proceed” and it would read so much better as “You may proceed” (or something like that. Now you will see it next time you use the OTP
can implied I was able to no matter what. May implies I now have permission
errant knoll
Let me send that back over to the the SSO group
errant knoll
But shouldn't that be "at most"?
tired bane
Nope, you can't. The cluster LIFs need to be on dedicated ports. Also the cluste...
Ok, got it... but just for curiosity, let's say there's a specific use for those clusters to get snapshots from one to another (snapmirror) but for other reasons keep it separated clusters A and B, if we check the use of the cluster network says (from a doc from 2013):
It is used for operations such as:
o Volume moves
o To synchronize cluster/node configuration and metadata among the nodes in the cluster (this is a very important communication aspect since it keeps nodes in the cluster in quorum)
o Access data that is remote to an interface
and obviously the technology is other from those years, at that time there was only 10G switches so I understand the point of keep those switches dedicated, now with the 100G switch at the end of the day there's going to be only the delta (yes, once everything is transferred) so technically is not big charge and let's say there's no disk shelfs involve only the two clusters.
I try to find documentation where's says is forbidden and I can't find anything about it... so if is not forbidden it is allowed? 😉
haughty nymph
It has always been the case that cluster interconnect switches are not allowed to be used for anything else than cluster interconnect traffic. It's basically the last walled garden NetApp still insists upon. Other kind of traffic (like HA or metrocluster) may go through "compliant" switches but cluster traffic needs to stay in these switches provided by NetApp. (I'm not even sure if the traffic between the 169.xxx cluster IPs is actually encrypted...)
You should also find it in the docs-page but here's one example from the KB: https://kb.netapp.com/onprem/ontap/os/What_is_the_cluster_interconnect
Switched clusters can only use NetApp-validated switches
- Only the approved switches that have been qualified for use with ONTAP can be used.
- The cluster interconnect infrastructure must not be shared for any other purpose
echo fractal
It has always been the case that cluster interconnect switches are not allowed t...
No, cluster traffic is not encrypted.
And technically you don't even need "switches provided by NetApp", you can use existing switches (or switches bought on the open market), as long as they are the right model and you configure them with the NetApp RCF files
tired bane
Ok, thanks everybody for the input I got my answers
tawny basalt
A vol move or snapmirror within the same cluster will use the cluster (INTRA-Cluster) ports/LIFs and any replication to another cluster must use INTER-Cluster LIFs which always go to the front end data switching
tired bane
A vol move or snapmirror within the same cluster will use the cluster (INTRA-Clu...
That's claire to me thanks, I was thinking "out of the box" because lets say if I need to replicate but my front en is 10G and I already have 100G switches (INTRA-cluster) why not use it? at the end of the day the only trafic will pass into those switches is sync of logs, vol move (when executed) and the data from an indirect path (which is only NAS trafic) so from a point of view is not 100% exploited their capacity... so I was thinking if is possible to add only the delta of the sync of the volumes (yes I undernstad the initialize will use a lot more, but in order and slowly will not afect in temrs of bandwith).
sour sonnet
It has always been the case that cluster interconnect switches are not allowed t...
not entirely true since MCC-IP also uses the switches for "intra cluster" traffic via the T6 cards