coarse depot
We currently have a total of 8-nodes cluster.
The tasks will be all done by Professional Services. So we are thinking to add 2xC800 during business hours and to swap in off-hours, because adding should be safe and no impact on the services. What would you say and why?
vestal dew
Unless the fas8200 are all flash to start with pretty sure you can’t swap them with a800’s
Add the a800’s to the cluster, vol move everything then remove the fas
gilded zephyr
by "swapping" do you mean headswaps only? Or do the A800 bring their own disks?
coarse depot
My mistake. I should say swapping A400 with A800. Adding C800 first then decommission FAS8200 later. Yes, swapping controllers only
gilded zephyr
then I have to agree with TMAC, I don't think you can headswap a FAS8200 to an A800
so yeah, add the C800, move everything from the FAS8200 over via vol move (I hope you have enough space), then remove the FAS8200 nodes from the cluster, re-cable the shelves to the A800 and add that to the cluster, then move stuff back
coarse depot
I am asking when should we do these two tasks?
gilded zephyr
since it's not a headswap, I would be perfectly comfortable to do it on any regular workday
the vol moves are nondisruptive and removing nodes too. As long as all your SAN is properly multipathed of course 🙂
coarse depot
Yes, it is headsswap from A400 to A800
gilded zephyr
huh. okay. now I'm confused. A400, A800, C800 and FAS 8200?
anyway. Headswaps in off-hours, everything else doesn't matter. That's what I always went with
or rather what our customers always went with. I mean we have found quite a few new bugs during headswaps (but we've done dozens if not hundreds of those so it's still pretty rare to hit a bug)
vestal dew
So you mean:
Add C800's to cluster
A400 -> A800 (head swap)
I have done HeadSwaps mid-day. I prefer to get a window in case something goes sideways. With that said I have had things go sideways, but everything stayed online. As Darkstar mentioned, it is nearly commonoplace to hit a bug here or there during the headswap and if you know how to get around then great, but otherwise you need to figure out what the bug is, if there is a workaround or not.
The one bit of advise i can give: reboot the nodes that will be swapped (takeover-giveback both ways) before the headswap. I suspect those bugs have already been fixed, but there were a number that were resolved with that procedure before starting.
gilded zephyr
good point about the reboots, also do a takeover/giveback after the headswap as that has also been known to cause issues
coarse depot
I have one more question about what NIC cards that I can utilize in headswapping.
A400:
We are using X1151A for cluster switch connections. X1161A for 25G client connections.
A800:
Since I can only see X1146A NIC there in A800 quote, so, I assume that it should be for cluster switch to replace X1151A. Since there are no any other NIC cards, I assume that we can continue to use X1161A for client connections, and move it to A800 controller in swapping process. But, I couldn't confirm that X1161A can be used on A800 upon HWU.
Am I right on these two assumptions?
vestal dew
The a800 has a card pre populated in slot 1
Cluster connections are e0a and e1a. Ha connections are e0b/e1b
Do you mean x1116?
That is supported in the a800
coarse depot
I meant this one currently for client connections on A400: Dual 10G/25G Ethernet Controller CX5-Mezz(X1161A)
Are you saying X1161A can be on A800 for client connections?
vestal dew
Check a sysconfig -ac
I’m not seeing x1161 on the a400. There is a x1122
The x1122 can be used in slots 2-5 on the a800
coarse depot
*> node run -node node-11 sysconfig -ac
sysconfig: slot 3 OK: X1151A: 2p 100G Naples-100 Smart IO
sysconfig: There are no configuration errors.
X1151A is currently used for cluster ports on A400
I don't have x1122
vestal dew
Check hardware universe. Hwu.netapp.com
coarse depot
I already did. Please review my questions again.
vestal dew
The a400 has a mezzanine. Onboard ports
Can’t reuse mezzanine
The x1161 is the mezzanine card
If you have no cards in the a800 you won’t have any data connections
coarse depot
OK. Again, in A800 quote, there is only A1146A listed there.
Can I continue to use X1151A on A800 ?
coarse depot
Are you saying A1146A should be my data card?
And, that pre populated card in slot 1 is not listed in the A800 quote?
vestal dew
Pretty sure about that. The card in slot 1 is 100% mandatory and can’t be used for anything else.
On that platform, there is the x1148 that can only be used with twinax (it gets too hot with optics or aoc). The x1146 is used for those that need to use optics.
The x1148 does rdma/nfs (RoCE) where the x1146 can’t
coarse depot
I have A1146A and A1146A only in the A800 quote. there s no x1148
vestal dew
If you can get a bill of materials there may be two listed. I’ve always been frustrated by Netapp doing this. It was common in the fas9000 with the cluster card in slot 4 but the sas card was called out even though you needed one
Do you have the equipment? Look at it. There will be a card in slot 1 and any other card is for data access. Note the x1148 is mandatory for nvme disk expansion
coarse depot
The equipment has not arrived yet.
One more question:
What is drive pack 2x1.92tb, NVMe (x4010A-2-I-N-C)used for, in addtion to those SSD aggrs which is migrated from A400? Should I just create am aggr on 2x1.92TB?
gilded zephyr
I think the x1146a card listed in the quote is the cluster card. We had this discussion some weeks ago here and I think that was the consensus back then...
coarse depot
Correct. Which NIC card is for client connection since there is only this card listed in the quote?
vestal dew
I just did a "fake quote". When I left it "naked" (no extra NICs), the X1146 was NOT listed in the Bill Of Materials (BOM)
The card in slot one is likely associated with "AFF-A800A-203-C" or "OC-AFF-A800A-CARDS" on the BOM
When I added the X1146, it shows up as expected. I think you are OK
@coarse depot You should be OK. there will be a card in Slot 1 and the second card may be in any other slot, but likley Slot2
gilded zephyr
It might be different if the quote is made with the Headupgrade kit though (if these are still a thing?) instead of simply an A800
vestal dew
Doubt it @gilded zephyr . The card in slot 1 is mandatory which is why it does not show up. It is bundled in another base-item component
gilded zephyr
But yeah you might be okay. I would still doublecheck and ask to be sure. Nothing more annoying than noticing that when you're right in the middle of the headswap 😂
vestal dew
The FAS9000 did exactly the same thing. The card in slot 4 was never on the BOM but showed up any way
coarse depot
One more question:
What is drive pack 2x1.92tb, NVMe (x4010A-2-I-N-C)used for, in addtion to those SSD aggrs which is migrated from A400? Should I just create am aggr on 2x1.92TB?
gilded zephyr
drive packs consist of 8(?) drives each so 2 drive packs are 16 drives I think
vestal dew
There is a quantity of drive packs. SSD/nvme are sold in 2-drive packs. Platforms have a certain minimum that you need. If you have 8 drive packs you will have 16 drives. If you are on 9.14 or higher you can enable auto partition on all drives and then use adpv2 on all drives. There is a kb to enable.
Otherwise you could put them all in one aggregate. In a fanboy of symmetry so I would still create two aggregates unless there are extenuating circumstances
vestal dew
storage raidlm policy modify -policy-name auto_partition_spare_ssds_on_aff -policy-type Shared-Disk -node * -is-enabled true
Pretty sure in Only works with ONTAP 9.14.1RC1 anf higher
*** Not for MetroCluster configs
coarse depot
There will be only 4 Packs of 2x1.92TB per node. My understanding is that auto partition will use root-data-data partition. 4x2x1.92 = 15.36TB. Thus, it result in only about 12TB usable aggregate per node.
What should I do with such small aggregate on this A800?
There are about 340TB usable currently on A400 and will be migrated over to A800
vestal dew
My gut is telling me this wasn’t sold as an “upgrade”. Rather a new system which needed a minimum amount of storage. That is the minimum number of disks with that platform. If you look at the quote you will probably just see 4 x 2-packs of the 1.92t drives.
When the head swap happens, make sure that whomever does it boots the 800 and runs option 9a or at least go into maintenance mode and remove owners from all internal disks
The upgrade will utilize the disks from the external shelves.
coarse depot
So, which means that we are not going to use these drives?
vestal dew
Up to you!.
Being so small, you would need to decide. That option/kb above I mentioned...if you have all SSDs/NVMe in the system and they are all Partitioned, then all new drives will auto partition. However, if you decide to use them as whole drives, that option/kb will not work (all drives must be participating in ADPv2)
If you leave them as ADPv2, you could removeowner on all the P2 and assign to same node as P1. You will end up with a total of 16 partitions (8 x P1 and 8 x P2). Then make an aggregate with a raidgroup of 7 and a total of 14 "disks" leaving one spare P1/P2 partition
I have done this plenty of times with customers.
coarse depot
If I understand you correct, that (4 x 2-packs of the 1.92t drives) would be assigned to one node only in this HA, right?
vestal dew
For the new drives
@coarse depot if I’ve taken in the info correctly, you are upgrading to a new platform with a head swap so there will be storage in the new heads. The new drives are just that: new/extra. is that correct?
coarse depot
Correct
vestal dew
So your can do one of four things with the new drives
coarse depot
Thank you!
vestal dew
In any case when doing the head swap make sure your ps person boots the ha pair and gets all ownership removed from the partitions and drives on the “new” disks before starting
gilded zephyr
the question remains as to why you bought 4 drive packs if you don't really have any use for them 🙂
vestal dew
@gilded zephyr -> it is a selling Hack! If you sell the "upgrade" then you dont need disks and essentially extend the warranty of the old system (i.e. pay more). If you buy a new system you must get new disks...I made the reference earlier as this is the most minimal config that can be done with a new A800...and a new warranty starts. In many cases, this is less expensive to the customer
vestal dew
By the way, this hack is for some scenarios and certain cases. YMMV