I'm sure this is possible but I am struggling to find any documentation or KBs that state it's ok to add larger capacity disks to an existing aggregate as a new raid group and retain their larger size. I've spent the last 30 mins searching KBs, NetApp docs, and Google with no luck. Most talk about downsizing disks, adding to existing raid groups, or the process of adding disks. I need official documentation to prove it's ok to customers.
Thanks in advance.
#Adding larger capacity disks to existing aggregate (new RG) KBs/Documentation?
stuck pebble
storm gazelle
Not sure where to actually find something but here is what happens
When adding a larger drive to a current raid group, they will actually become the new parity drives. They need to be replaced first. So in raid tec the first three drives will be parity, the surviving drives will be data. Any new drives will be data as parity may be accounted for. (Note that for performance reasons you should be adding like 10-12 drives otherwise the new dunes will become a hot spot) That was before ado. ONTAP may just apply adp to the drive now. If adp is not used then I believe the logic above applies
Adding a new raid group…we do this to. Especially with adp.
The data partitions are rg0 and when you expand with whole drives they end up in rg1+
ebon quartz
For adding larger disks to an existing RAID Group, I think you need to manually replace the parity disks with larger disks first. Otherwise ONTAP will just down-size the newly added disk to the size of the parity disks
For creating a new RAID group (this is the preferred method), just supply -raidgroup new to the storage aggregate add-disks command to force ONTAP to create a new RAID Group
storm gazelle
on 7-mode was automatic. If I added a 2T disk to an aggr with 1T, The 2T was zeroed, and added to the aggr and automatially made the parity. I would expect, ONTAP might fail to add a single large drive with RAID_TEC/RAID_DP and that you may need to add a minimum of three drives (two for RAID_DP, but if you are going to 10T, you should be on RAID_TEC)
@ebon quartz an easy task to play in the simulator. Create a two size drives and test away. I am 99.99% sure I am correct here (I have done it on 7-mode and that code did not change to ONTAP)
ebon quartz
I remember having to "fix" customer's aggregates in 7-mode because they added larger disks and ended up with only half of their capacity used. "Fixing" involved deleting the aggregate and re-creating it. That's why I'm pretty certain that in 7-mode, ONTAP didn't do that, at least not always (maybe it was a bug, but it impacted at least 3 or 4 customers back then)
I'll spin up a simulator and try it
storm gazelle
That was because the did some kind of disk replace or they had 2T spares and was using 1T disks. That will cause the bigger disk to right-size. If you ADD a disk, you get the whole capacity, only after the parity drives have been updated
ebon quartz
what do you mean by "spares being updated"?
storm gazelle
You "add a disk" to the aggregate and ONTAP will flip the logical position of the spare with the larger drive.
ebon quartz
but spares are not part of the aggregate. do you mean parity disks?
storm gazelle
In this case, the first larger drives must replace the spares to accomodate the deeper disk stripes
ebon quartz
yeah okay now I understand what you mean
you confused me a bit there with replacing spares (by other spares) 😄
storm gazelle
I love to @ebon quartz 🤓 confuse
ebon quartz
I was correct, ONTAP will downsize the larger disks if you just try to add them to the aggregate
even if you give it enough disks (3 in this case) to replace the two parity disks, it won't do that
uneven hinge
ONTAP has been able to support mixed drive sizes in an aggregate since FCAL shelves were offered. But, as with anything, there were caveats.
In that, If you mixed larger drive sizes in the same RG then the larger sizes would be down-sized to match the smaller drives.
If you create a new RG of the larger drives, they will remain full size.
This behavior is still the case today including data partitions.
ebon quartz
yeah, it seems you cannot even replace the parity disks with larger ones manually anymore
uneven hinge
You used to be able to do that. I think that it was bug 34881. Nowadays, I think that that has been "fixed."
ebon quartz
I was under the impression that this did work in 7-mode, at least for RAID4 aggregates
uneven hinge
And it did
ebon quartz
yeah, I remember having to change the raid type, then replacing the disk and then changing it back. or something. but even then it felt like a weird hack 🙂
uneven hinge
but it did more harm than good (from a performance aspect)
ebon quartz
yeah, mixed-disksize aggregates were terrible
uneven hinge
Yep. They work as long as they are maintained/managed properly along with available spare count.
no one likes the SPARES_LOW notifications......and no one likes larger spares being assimilated as lower-capacity because there weren't any available
storm gazelle
I have always and still dislike mixing anything in the same raid group
uneven hinge
Mixing in the same RG is a "use only if you really mean it" scenario
mixing in the same aggr across multiple RGs......you could argue either way successfully
storm gazelle
Thanks for doing the test @ebon quartz . I have actually never tried since RAID4, but back in those days, NetApp had the single monolithic aggregate
uneven hinge
@storm gazelle I'm surprised that you don't recall bug 34881 🙂
storm gazelle
No link. didnt happen
ebon quartz
Thanks for doing the test <@189749672921792512> . I have actually never tried si...
no worries, it's no big deal since I have that nice little PowerShell script that deploys an HA vsim cluster within 10 minutes in our ESX lab 🙂
stuck pebble
Thank you all for your help!
The plan will be to add 22TB disks as a new RG to an existing aggregate with 16TB disks. Do you know if any official documentation supports this capability? I miss the old "Storage Subsystem Technical FAQ" document. 🙂
uneven hinge
https://docs.netapp.com/us-en/ontap-cli/storage-aggregate-add-disks.html is a great place to start.
Check out the notes of the "-raidgroup" parameter.
And "-simulate true" will become your best friend when doing this
ebon quartz
https://docs.netapp.com/us-en/ontap-cli/storage-aggregate-add-disks.html is a gr...
sadly that says nothing about adding disks with different sizes and what happens in those cases
uneven hinge
That is a fair comment. The description says that it does and does not at the same time due to the verbiage of matching disk type, but does not specifically state anything about capacities or sizes.
ONTAP has been able to support mixed drive sizes in an aggregate since FCAL drives and shelves. I recall working support cases (7.2.4 timeline) where a customer has 150GB, 300GB, and 450GB FC drives in the same aggregate, but within respective RAID groups.
If only we had the '-simulate true' parameter back then, we could have saves a lot of "oops" moments when larger drives got down-sized
paper torrent
https://kb.netapp.com/on-prem/ontap/OHW/OHW-KBs/Does_Data_ONTAP_allow_mixing_of_drive_sizes_in_a_RAID_group
When adding larger drives to an existing RAID group of smaller drives, the larger drives will be right sized to match the usable capacity of the smaller drives.
For example:
Adding a 10TB NL-SAS drive to an existing RAID group of 4TB drives results in usable capacity of 3.63TB on the 10TB drive.
NOTE: Right sizing applies to DATA drives. It is possible for PARITY drives to not be right sized, but additional added DATA drives will continue to be right sized
Right sized drives can be replaced via storage disk replace if a more appropriate spare is available
