clear yoke
Moved some shelves from clusterA to Cluster C, Unfortunately half the disks on one shelf were assigned when the decomissioned node booted to option 9 to remove ownership and it's partner had already removed its ownership.
reassign doesn't work bc "not in takeover mode".
I recall there being a way, but don't remember. Just need to clear the ownership so new cluster can use them
orchid sage
Did you try "storage disk removeowner -disk ... -data1 true"? Or "-data2 true" or "-root true" or "-data true"? That removes the einer from the partitions. After that you should be able to remove ownership from the container disk
clear yoke
Ah, forgot about adp. Will try that.
orchid sage
Diag Mode?
clear yoke
fails in diag mode: "unable to determine sharing method" for disk
orchid sage
Hm. Okay. Never seen that, sorry. Maybe try on nodeshell?
clear yoke
the nodes see disknameP1 as unowned.
tried in the cluster shell (diag) to removeownership from disknameP0,P2 and got "no entries matching your query"
would the -root need to be -Root? LOL
didn't make a difference. Could see from disk show -partition-ownership that these disks have a Root, Data1, and Data2 partition, with only Data1 being unowned.
sto disk unpartition (in diag) fails with "disk is shared" error
orchid sage
tried in the cluster shell (diag) to removeownership from disknameP0,P2 and got ...
in nodeshell you can try disk partition removeowner, that takes a disknameP1 as parameter. But in my experience it works no better than disk removeowner -data2 true etc.
clear yoke
ran the -root true commands in cluster shell, should that have been in node shell?
orchid sage
no that is clustershell
in nodeshell I think there's only disk remove_ownership 0a.10.16P1 etc.
in nodeshell there's also disk assign -s unowned -f 0a.10.14P1 etc. (undocumented) that should remove the ownership no matter what
clear yoke
disk assigned worked, now we have all the partitions and the disk unowned.
cyan harness
For unpartitioning a disk, a single node needs to own the disk and the root and the data partitions (spinner) or needs to own the disk and the root and the data1 and data2 partitions (ssd)
You may have to play a game in diag mode. You will need to disable disk auto assign for now. You can then play around removing ownership of any partitions you need to and then assigning them to the node you need it on. You may need to do the same for the disk also
clear yoke
yep, the partitions have aggrs on them, so once owned we should see that foreign aggr
have the failed aggrs for the old system visible in node shell, but not cluster
no more aggr destroy command? We see the failed aggrs, and they are offline, but no destroy option in diag or advanced
orchid sage
It is "storage aggregate delete" now 😅
clear yoke
Yeah, in cluster shell, which can't find these offline aggrs. They only show up in 'aggr status' in node shell
Seems Maintenance Mode or moving disks back to OG node and destroy them (or option 9a). Since we're not bringing this uber important production system down, gonna try to get a shelf from RTP to move these 11 drives to and do the later.
orchid sage
Two possibilities:
cyan harness
This is so much easier
storage aggregate remove-stale-record -aggregate aggr1 -nodename node1
aggr1 = name of "odd" aggregate and nodename is the node it is on. And if that does not work, the debug-vreport-show/fix
clear yoke
will give that a go tomorrow. THanks, tmac!
cyan harness
I am pretty sure it is at least and advanced command
orchid sage
storage aggregate remove-stale-record -aggregate aggr1 -nodename node1
are you sure this also deletes the aggregate from the disk? I always thought it only removes aggregates from the DB that no longer exist. But I could be wrong, I usually go straight down the vreport fix route 🤷♂️
cyan harness
I’ve had times I had to use this and also use the vreport. Sometimes both are needed
clear yoke
TMAC, ur a god! remove-stale-record was all we needed.
orchid sage
TMAC, ur a god! remove-stale-record was all we needed.
I'm curious. Did that actually remove (delete) the aggregate? I mean if you do aggr status in the nodeshell?
cyan harness
It usually does 😁
grave bolt
Does an aggregate that doesn’t exist, exist? Sounds philosophical