#Mediator setup for SMAS

1 messages · Page 1 of 1 (latest)

foggy vapor
#

I have followed all the details about setup of the mediator on Rocky Linux 9... As I try to add the mediator inside the cluster I need to add a certificate which I guess should be from the mediator, so I tried with pretty much all that I could find on the mediator in the path "/opt/netapp/lib/ontap_mediator/ontap_mediator/server_config" (great path btw) 🙂 but no matter what I enter it is not accepted... I have tried to change the name and password of the mediatoradmin.. also generated new self signed certificated on the mediator... all this is very poorly documented (IMHO)... so I was hopeing that someone inhere have tried this before? 😉

pearl crest
#

there should be a server.crt file in that path. This file contains the certificate. You install it on your cluster with security certificate install -type server-ca. Just copy/paste the contents of the certificate file into the CLI (or System Manager)

foggy vapor
#

I tried to add the certificate here... but I guess I need to add it under Client/Server certificated on the cluster? But why can you then add it as you configure the mediator?

Screenshot_2024-12-12_at_18.00.02.jpg
pearl crest
#

I usually did it on the CLI, but according to the docs, that is exactly the place where you copy/paste the certificate in System Manager, yes. Maybe you forgot some lines to copy/paste (you need to include the -----BEGIN CERTIFICATE--- and ---END CERTIFICATE--- lines too, for example)

foggy vapor
#

Hmm that GUI clearly needs more work... (9.15.1) it failed, but then after a few secs, it completes OK... and is now added on both clusters...

foggy vapor
#

Just another quick related question 🙂 To enable SMAS you need to create a consistency group... in this setup we have a few different datastores (volumes) which will be added to snapcenter with different policies... like /hourly snapshots/ and /daily snapshots/.. are you able to have different snapshot policies for the volumes inside a consistency group? Or do you have to create two separate consistency groups if you require this? (I haven't worked that much with consistency groups)... 🙂

plain wharf
#

This worked for a recent SMAS deployment we just wrapped up. We had issues with the cert. Thought there was a KB, looking. 1) ssh to the linux install and: sudo cat /opt/netapp/lib/ontap_mediator/ontap_mediator/server_config/ca.crt
2) Copy/paste the output to a new notepad file. The notepad file should start with: -----BEGIN CERTIFICATE----- and end with -----END CERTIFICATE-----
3) Next, ssh to the cluster
4) at the cluster admin prompt: security certificate install -type server-ca -vserver {cluster name} being sure to use YOUR cluster name
5) it'll prompt you for the certificate so just copy all of the notepad and paste into CLI.
6) repeat steps 4 and 5 for the other cluster.
7) After that, you should be able to step through the GUI for adding the Mediator via either of the clusters ONTAP System Manager. When you get to the Certificate field, just copy and paste the same info from the Notepad file.

foggy vapor
#

Yeah it's kinda impressive how they managed to *uck this up so bad... how hard can it be to streamline this to just one simple wizard?

pearl crest
#

well in the GUI it is one simple wizard 😉

plain wharf
#

The first three steps are in a different doc..good stuff...

pearl crest
#

yeah, the documentation actually assumes that you know what a certificate is and how to install it. Which kinda makes sense, but of course not to everyone

#

I guess the doc writers went like "well, we can't possibly have to explain this to people right?" 😄

#

but I agree it's stange that someone said "hey, the documentation is incomplete, we probably need to explain this step in more details for some users..." and then they went like "let's add a KB for that" instead of "let's just fix the documentation

lavish junco
foggy vapor
#

Ahh FFS... Looks like I have missed the fact that intercluster communication has to be setup on the Default ipspace in order to work with SMAS... I normally seperate intercluster in its own ipspace... It's my own fault 😉 I actually think I heard this at some point, but apparently it didn't stick 😉 but what a strange requirement.. and I bit it will be removed in the future...

#

....and ofcause the "broadcast-domain move" cannot move the ports over to the Default ipspace because the ports have a home port... (which I guess interluster links requires)... so it's a total teardown.. 😦

wary raven
#

try broadcast-domain merge which in my experience works even if the broadcast-domain has ports which are home-ports for some active LIFs

wary raven
foggy vapor
#

Nah merge cannot merge two ipspaces.. for some unknown reason 😉

#

`hadstor::*> broadcast-domain merge -ipspace ic -broadcast-domain intercluster -into-broadcast-domain Default
(network port broadcast-domain merge)

Error: command failed: Broadcast domain "Default" in IPspace "ic" not found.`

#

It's like they don't want you to use ipspaces.. if you "tab" you way with "broadcast-domain move" it gives you "-broadcast-domain Default"... you then have to specify "-ipspace" first... it really starts to annoy me 🙂

#

But not as much as the auto-generated "Default-1...." broadcast domains created... 🙂

wary raven
#

I think if you create the VLAN with vlan create -skip-broadcast-domain-placement true this will be skipped

#

But yeah it's annoying even though it also helps because you see if there are any layer2 reachability issues.
If all the necessary VLANs are correctly configured on the switch once you create all the VLANs in ONTAP they will automatically be in the correct broadcast-domains. You only need to rename them.

foggy vapor
#

So... as we are going to move this to the Default broadcast domain... we will pull this down from 9000 to 1500 because you normally have management interfaces in the Default... hmmm

#

...and I guess you will have to fiddle so that nothing tried to failover to these ports as they are in another VLAN... I'm sorry, but this is just a mess... please tell me this is fix in 9.16.1 ?

wary raven
#

Don't see any changes to that in 9.16.1RC1

foggy vapor
#

nah I also had a look at the release notes...

wary raven
#

But you can configure your ports for 9000 and then use different MTUs for the broadcast-domains, so like 1500 for mgmt and 9000 for intercluster

foggy vapor
pearl crest
wary raven
#

or create separate failover-groups 😄

foggy vapor
#

Are you sure? We now have e0M mixed with intercluster interfaces on they own VLANs... and cluster management is on e0M...

#

hadstor::*> network interface modify -vserver hadstor -lif cluster_mgmt -failover-policy ? disabled Failover disabled system-defined Next failover target selected from targets defined by the failover group limited to the home-node and next-to-next node local-only Next failover target selected from targets defined by the failover group limited to the local node only sfo-partner-only Next failover target selected from targets defined by the failover group limited to the home-node and the SFO partner only broadcast-domain-wide Next failover target selected from targets defined by the failover group containing all of the ports in a broadcast domain

pearl crest
#

you don't have management in a separate broadcast domain?

foggy vapor
#

nope

#

`hadstor::*> broadcast-domain show
(network port broadcast-domain show)
IPspace Broadcast Update
Name Domain Name MTU Port List Status Details

Cluster Cluster 9000
hadstor-02:e0a complete
hadstor-02:e1a complete
hadstor-01:e0a complete
hadstor-01:e1a complete
Default Default 9000
hadstor-02:e0M complete
hadstor-02:e3a-903 complete
hadstor-02:e5a-903 complete
hadstor-01:e0M complete
hadstor-01:e3a-903 complete
hadstor-01:e5a-903 complete
ic intercluster 9000
- -
vmware vmware 9000
hadstor-02:e3b-904 complete
hadstor-02:e5b-905 complete
hadstor-01:e3b-904 complete
hadstor-01:e5b-905 complete
4 entries were displayed.`

#

but I guess I should create a management domain then...

pearl crest
#

so the 903 vlan is not a separate one? it's the same as management ?

foggy vapor
#

no management is in an untagged vlan and not 903...

wary raven
#

Create a mgmt failover-group with the e0M ports and use that for your mgmt-LIFs, then create a peering failover-group for the other ports 903-ports and use that for the IC-LIFs

foggy vapor
#

yes that could save me some work I guess.. but it's the failover-groups depreciated? 😉

wary raven
#

nah

#

When you create a new broadcast-domain it automatically create a new failover-group with the same name.

#

Every LIF always needs a failover-group afaik. You usually don't touch them because most times you want the ports in broadcast-domain be the same as in the failover-group

foggy vapor
#

do I need a failover group for the intercluster lifs or are they configured not to failover by default?

wary raven
#

intercluster-LIFs will only failover locally

#

I usually use the "local-only" failover-policy, or "disabled" when I have port-channels

wary raven
# wary raven When you create a new broadcast-domain it automatically create a new failover-gr... 
     [-failover-group <failover-group>] - Failover Group Name
         Use this parameter to specify the name of the failover group to associate with the LIF.  Manage failover groups by using the network interface failover-groups command.  Each broadcast domain has a default failover group which is created by the system
         automatically and has the same name as the broadcast domain. The failover group associated with the broadcast domain includes all ports in the broadcast domain.  A logical interface's failover group is set to the failover group of the home port's
         broadcast domain by default, but this value can be modified.
foggy vapor
#

So the policy is still Beoadcast Domain Wide even with the failover group set... should I disable the failover policy?

wary raven
#

If you disable it that LIF will not failover, is that what you want? 🙂

#

If you have 1x IC-LIF per node, but 2x ports per node which are in the correct network, I would configure it to "local-only". If you only have 1x port per node --> "disabled"

#

same for the node-mgmt LIFs

#

basically for all LIFs which can't failover to other nodes

foggy vapor
#

ok I think I got it now...

#

the IC's have a failover policy of local only... but I need to create a failover group because you cannot remove e0M from the Default group... just great 😉

pearl crest
foggy vapor
#

Also even if this were to change in ONTAP 9.17... it will most likely be something you do not want to fiddle with, once the SMAS is up and running... 🙂

pearl crest
foggy vapor
#

Hmm as I try to setup the protection of the SVM with initiator replication I get this nice message "Setting an initiator group's replication peer requires that the peer cluster's effective cluster version be 9.15.1 or later.".. just great... of cause both clusters are on 9.15.1P6...

#

hmm of cause it then works after the volumes are in sync... but it didn't do this the first time I configured it... 🙂

foggy vapor
#

Quick question... if you were as stupid as me 🙂 and moved your temp-mediator VM onto one of your SMAS datastores, and started a failover... 🙂 The consistency group sync mirror would now be "out of sync" and none of the datastores are online... is there a way to force one of the locations online again?

pearl crest
#

They say to place the mediator at a third site for a reason, I guess...
I think I heard about an option to force-online one side without a mediator, but I don't know the details...

foggy vapor
foggy vapor
#

for some reason the OTV thinks there are still a host cluster relationship, with 5 datastores and snapmirror status is in sync... even after reboot and waiting 30 mins... I guess this information isn't updated like... ever? 😉 because we have no datastores and no snapmirror... guess the reinstall cannot be helped...