raven pasture
Greetings, I am trying to understand something PS told me today.
All shelf ID's must be unique within an HA pair.
It's been my experience that the shelf ID's only need to be unique on their Loop/Chain.
Are there any technical articles that explain this? All the KB's I was pointed to are clearly labeled as "best practice".
half ore
Ha pair is the answer. It must be documented someplace. It’s always been this way even back to fiber channels days
Believe you me that ONTAP gets very very confused with multiple shelves with the same id. I had to work with a customer earlier this year with two shelf id 0. Not fun.
It took a lot of work to get it fixed and fortunately there was enough space to evacuate and then reset the shelf plus a takeover/giveback to fully fix
odd swan
Ha pair is the answer. It must be documented someplace. It’s always been this wa...
small nitpick: FC shelves only had a single-digit shelf ID that went from 0 to 6 (or maybe 1 to 6, I'm not sure), so after the 7th (or 6th) shelf you had to re-use a shelf ID. The shelf ID only had to be unique on the FC port
And I think ever since ONTAP uses stack IDs (and not SAS port-based disk names) you could technically re-use shelf IDs across stacks in cDOT as well. Of course it's a bad idea and you shouldn't do it, but as long as you don't re-use a shelf ID within a stack (which then leads to a soft-ID consisting of the shelf's S/N, giving you very weird disk names until you power-cycle the shelf) it will technically work. But I would never ever do it because it will lead to people pulling the wrong disks if one breaks
half ore
Agreed. Definitely makes it difficult to troubleshoot. With 100 is IDs to choose from, no excuse now
odd swan
you might run into trouble if you only have DS212C shelves on a very large system (1440 disks), but if you do something like that you have other issues I would say 😄
half ore
I remember a long time ago under before 7 mode…engineering was always impressed with themselves for being able to run hundreds of spinning disks in a single filesystem
jolly breach
Omg, I remember the good old DS14 shelves (and their funny problems), I am officially old
jolly breach
Maybe it's a stupid question, but... in an IP Metrocluster you have no direct connection to the disks on the remote site. Could you theoretically have the same Shelf- IDs on both sides? (Our first IP MCC was delivered last week, will be installed in a month, so I have no experience here)
ancient spindle
It's still needed to be unique per HA-pair but recommendation is to have unique shelf IDs within each MetroCluster DR group.
Exception are some models with integrated disks where you can't change the shelf ID - they always have ID 0.
raven pasture
I understand best practice, but this is a technical question. Everything I can find states it's best practice, but from my c-dot 8 days, we definitely ran with duplicate ID's on different stacks/channels, duplicates on the same channel caused an outage. With 9 duplicates get renamed to serial number. I am trying to find out if it's technically allowed vs best practice. Good thing is I have a lab cluster being racked and built and I will test my theory. If it's a technical requirement to be unique per HA-pair, I am trying to find out which version of Ontap this changed in. 😉
odd swan
why do you want to know? just set different IDs and everything is fine
I'd like to know the use-case you're trying to solve
raven pasture
I am being told that I can not use the same ID on a second chain. In my 20+ years of experience I have never been told this. Only uniqueness on the chain. I want technical validation why I am being told this. I can not find any technical document explaining why this is or when it changed. I never accept statements like this without validation (especially when it's in conflict with my experience). LIke I said, I plan to test it in my lab. If I add two shelfs with ID 0 on different chains and one gets changed to it's serial number, then i have my answer. But I still want to know when that was changed in Ontap.
odd swan
The shelf will not get a soft ID in that case, no. However the system health monitoring will still complain, and Active IQ will show a warning for your config. So I would still advise against doing this in a production environment
I'm still unsure what you get out of giving shelves duplicate IDs. As I see it, it only has disadvantages
also, you say that every documentation shows this only as "best practice", which is definitely wrong
"must be unique"
which is from here, the official documentation on how to install/cable a shelf
half ore
With the NVMe shelf on an a400 using switches : two shelf id 0 cause very bad things to happen. I was unable on the command line to explicitly call out drives on the second shelf as ONTAP would confuse it with the first shelf. It was very difficult. Even after evacuating the shelf , setting a new id and power cycling the shelf ultimately had to do takeover/giveback to get the shelf stable