BES-53248 issues

[humble geode]

I'm raising this with my account team but bringing it up here too.
We were sold BES-53248 swtiches for our little cluster. These have RCF files, similar to what Cisco switches have. However, the Broadcom RCF does not enable ports 49-53 so if you buy the 6-port license to enable the 40/100g ports, you can't use them as intended.
The RCF file has ports 49-53 listed, but they're commented out. Either NetApp needs to uncomment them or make a second RCF available that has the proper config for people that buy the licenses.

The RCF file says:
!===== Additional Node Ports activated with Licenses ===
!Instructions:
! (1) Apply licenses purchased and reboot the switch
! (2) Run 'show port all | exclude Detach', and note the interfaces between 0/16 and 0/55.
! These are the interfaces activated with licenses
! (3) Uncomment the applicable node port license block below, including the 2 exit commands

but the FEs are not told to read and do that... Can NetApp please put a second RCF file on the download site with the 6 ports enabled to make it a little more clear?

[neon harness]

On it, Ed. Bringing it to PM. Hang tight.

[neon harness]

_We can definitely easily help with this. _

@humble geode Ed, please work with your account team to file an FPVR so it gets to prod mgmt, and they'll provide you with what you need. We're also looking into your suggestion as a permanent fix.

[normal zinc]

Um, the process is trivial

1. Use the document with the transaction key to process and download license keys to activate those ports
2. Install the licenses keys and reboot
3. Follow the instructions on the NETAPP DOCS page for bes instructions

basically it outright tells you what to do.

The base bes only includes ports 1-16 and 55/56. This matches the base rcf file. The docs outright tell you what to do. Run a command. Verify ports uncomment lines in the rcf. Then upload.

I’ve done this a bunch. Just follow the documented procedure

If the FEs are not reading that, someone needs to be retrained

I mean it’s basic.

[humble geode]

I got the very, very slim details to activate my BES licenses electronically. Whatever came via paper got shipped to my remote site instead of to me (different issue). I activated the licenses using the KB article instructions. You say to follow the instructions on the NETAPP DOCS page. What I've read does not say to update the RCF. Which document procedure are you following? Perhaps that additional doc needs to be linked to the KB article.

"This article outlines the process and steps for activating additional Ethernet ports on the Broadcom-supported BES-53248 Ethernet switch (Part numbers: X190005, X190005R). "
https://kb.netapp.com/onprem/Switches/Broadcom/How_to_add_Additional_Port_Licensing_for_the_Broadcom_BES_53248_Switch

This article is only part of the solution - that's my point.

[normal zinc]

OK Here were go
Install Licenses:

• show license
• install license file(s) - copy url nvram:license-key x
• verify license - show license
• reboot - reload
• verify ports - show port all | exclude Detach
• In the section: Activate newly licensed ports:
  -> To activate newly licensed ports, you need to edit the latest version of the RCF and uncomment the applicable port details.
  -> you "fix" the RCF file before uploading to the switch.
  I usually rename it a bit after it is modified so I know:
  BES-53248-RCF-v1.9-Cluster-HA.txt ->
  BES-53248-RCF-v1.9-Cluster-HA-mod.txt

Then upload (copy URL nvram:script BES-53248-RCF-v1.9-Cluster-HA-mod.scr)

Plus step 6 in the "Install RCF config file" also says:
-> If additional port licenses have been installed on the switch, you must modify the RCF to configure the additional licensed ports. See Activate newly licensed ports for details.

Install licenses: https://docs.netapp.com/us-en/ontap-systems-switches/switch-bes-53248/configure-licenses.html
Install RCF config file: https://docs.netapp.com/us-en/ontap-systems-switches/switch-bes-53248/configure-install-rcf.html

[humble geode]

The documentation challenge comes in if the additional ports are licensed AFTER the initial install. NetApp guy installs the switch and RCF file. Weeks later, we get a new license, add the licenses to the switch following the KB (which doesn't mention updating the RCF file), and then 2nd NetApp guy goes to migrate the cluster back-end cables to the new ports.

Buried in "install licenses" page, it says:
"When installing additional licenses, you must configure the new interfaces manually. Do not re-apply an RCF to an existing working production switch."

This section in https://docs.netapp.com/us-en/ontap-systems-switches/switch-bes-53248/configure-licenses.html is WRONG:
"For high-speed ports between 0/49 to 0/54 inclusive, uncomment each port but only uncomment one speed line in the RCF for each of these ports, either: speed 100G full-duplex or speed 40G full-duplex as shown in the example. For low-speed ports between 0/17 to 0/48 inclusive, uncomment the entire 8-port section when an appropriate license has been activated."

This leaves you with untrunked ports (which does work) but no jumbo frames and cluster ping will complain about these ports. This gives you a mix of jumbo and non-jumbo on the back-end network if you also haves 10g nodes on the back-end (which we did with an old 2720).

[normal zinc]

Ah. Yes....I would agree that you could absolutley argue the RCF and the DOCS are out of sync. Looking at the RCF 1.8 file, the speed lines are there and you must uncomment one of them if you are to use it/them

But, if you pull down the latest RCF 1.9, there is a NOTE in the beginning:
!===== Additional Node Ports activated with Licenses === !Instructions: ! (1) Apply licenses purchased and reboot the switch ! (2) Run 'show port all | exclude Detach', and note the interfaces between 0/16 and 0/55. ! These are the interfaces activated with licenses ! (3) Uncomment the applicable node port license block below, including the 2 exit commands ! ! IMPORTANT NOTES ! In certain conditions, BES-53248 may not be able to auto-negotiate port ! speed correctly, and port speed has to be manually set in config mode, e.g. ! interface 0/49 ! speed 40G full-duplex ! interface 0/49 ! speed 100G full-duplex

I guess I have done it too much recently and do not see it as an issue however, I can absolutley see where if someone is unfamiliar with the process, the documention may be a little quirkey to follow

[humble geode]

The bit about uncommented only the high-speed line makes more sense if you look at the example. The example shows 2 speed lines - one for 40g and one for 100g so they want us to uncomment only 1. HOWEVER, RCF 1.9 has no speed lines for the high-speed ports. It's mentioned in the comments but not in the code blocks. I've raised this with my account team too.