It's been over a decade since I've had to expand an aggregate (and before partitioning existed) so I'd like to get some current recommendations (on 9.12.1). I have an A800 pair with 36 drives auto-provisioned into aggr0 and a single large aggregate per head . We're adding 12 drives per head so we'll need to add 6 drives to each aggr1. What's the best approach here? Thanks!
#aggregate expansion
gusty hare
finite estuary
so no ADP at all?
you have 6 disks of the 36 for root (3 per node)
and then remaining 30 are split between each node? - so each existing aggr has14 disk aggrs (and 1 spare)?
and you're adding 12 total drives? or 24 drives?
gusty hare
Auto-provisioning uses ADP. 1 plex with 2 raid groups in aggr1. Currently have 18 drives owned per head, for 36 drives total (7.62TB drives). We're adding another 12 drives to the pair to fill it (A800 max is 48 drives). I have 108 partitions per pair, 4 of which are spares (2 @ 13.38GB each and 2 @ 3.49TB each).
finite estuary
ah ok. i misread your post.
so your existing aggrs - look something like this in the context of RaidGroups?
gusty hare
I'll be damned if I know where you got that screen shot from - I can't find it in SM. What's the CLI version? 🙂
finite estuary
That's from fusion.
CLI..best would proably be sysconfig -r
node run -node node01 sysconfig -r
gusty hare
looks like 24 drives in plex0/rg0 and 11 drives in plex0/rg1. The spare has an additional drive from bay 45. I can't post the output - my work security rules are restricting it.
In SM, if I go to edit the raid configuration, it's showing 24 drives in the raid group size
finite estuary
So you're going to want to add those new drives/partitions to the second RG.
You should end up looking like this -
gusty hare
cli shows maxraidsize 24 for all aggr1 aggregates
it looks like the auto stuff just set the maxraidsize to 24 and start filling it with the available drives and then created a shorter raidgroup. That used to be nasty in the spinning disk days.
finite estuary
Yeah. Best to avoid disk configs like that with spinning.
But for AFF to have one RG with 24 and one with 12 is fine / normal.
gusty hare
well this was kind of goofy. After the drives were physically installed, I did a storage aggregate show-spare-disks -original-owner foo, then added those 6 spares to rg1. I got half the new space. I then went into SM and did an Add Capacity to get the other half. I checked Add Capacity in SM before I did the CLI add-disks and only saw that I would get half the space. So it's a 2-step operation, no matter which route you go.
finite estuary
the 6 disks would get partitioned, so it added the Data part to the RG and created a second data part for the other node.
gusty hare
Yeah, I realize what it did. But SM probably should have figured out what a customer wants and that's to assign all of the space, not just half of it.
carmine minnow
that seems like a hole in ADP that you cannot pick to format a drive prior to adding it to a aggr. you can simulate the add to make sure it will partition, but I dont like to do anything automatically. I dont think spares partition until they are added, which makes me nervous that it will actually grab the drive without partions during a failure.
finite estuary
You always want to simulate. But , The drive needs to be zeroed before it’s added. So before it’s partitioned too. You can manually partition it IIRC but I never have.
gusty hare
The spare drive will stay unpartitioned until it's needed. This used to trigger false alerts for insufficient spares and I complained a fair bit about this...
restive thicket
Also you don't need to reallocate with SSDs, and it is not supported as well.
gusty hare
Found another issue, albeit minor. storage aggregate auto-provision on 36 x 15.3TB drives says the aggregate will be 194.7TB. It's actually 205TB. I think this is the extra 5% space we got in 9.12.