#A50 ASA R2 disks encrypted and need to restore system to factory

70 messages · Page 1 of 1 (latest)

radiant yew
#

I am working on a system that was working with an external key manager and now the project is over and they deleted the External key manager. The disks show up but are unable to be seen by most commands. If i do a disk show, they are there, but cannot get any cluster commands to use the drives. Anybody have anything to help other than replace the drives?

#

This is what I am seeing:

#

DISK CHAN VENDOR PRODUCT ID REV SERIAL# HW (BLOCKS BPS) DQ


0n.0 NV:A NETAPP X4025S173A3T8NTE NA50 ? ? ? ff 0 4160 N
0n.2 NV:A NETAPP X4025S173A3T8NTE NA50 ? ? ? ff 0 4160 N
0n.1 NV:A NETAPP X4025S173A3T8NTE NA50 ? ? ? ff 0 4160 N
0n.6 NV:A NETAPP X4025S173A3T8NTE NA50 ? ? ? ff 0 4160 N

midnight cosmos
#

that's going to depend on a couple of things.
How were the drives encrypted and are they SED drives

#

you can try to set a couple of things
go into priv mode
storage encryption disk modify -fips-key-id 0x0 disk *
this will reset the key on any SED drive

#

and then, if not SED

storage encryption disk sanitize
storage encryption disk destroy

#

and, of course, remove anything related to the key manager
security key-manager delete -address ****

#

hopefully you have physical access to the disks if needed.
They should have the PSID on the label
storage encryption disk revert-to-original-state requires the PSID of each device

#

storage encryption disk modify -data-key-id 0x0 disk * is the other command i think

it has been a while since i've had to deal with that specific issue

radiant yew
#

it acts like the disks are not there for all of these commands... ::*> storage encryption disk modify -fips-key-id 0x0 -disk *
0 entries were acted on.

#

there was a star after disk but did not show up for some reason??

#

::*> storage encryption disk sanitize -disk *
0 entries were acted on.

::*> storage encryption disk sanitize -disk 0n.0

Error: There are no entries matching your query.

::*> storage encryption disk sanitize -disk localhost:0n.0
0 entries were acted on.

midnight cosmos
#

probably not SED drives then

radiant yew
#

and when I try to reset the drive i get: ::*> storage encryption disk revert-to-original-state -disk localhost:0n.0 -psid 6QDDVKHG3YL8Q*****************

Error: command failed on disk "localhost:0n.0": Disk Name must specify only one
device.
0 entries were acted on.

midnight cosmos
#

yea, using PSID you have to it disk by disk

radiant yew
#

yes I am doing it to localhost:0n.0, it just drops that last part, unless there is another way to specify the disk

midnight cosmos
#

did you try the sanitize/destroy with disk *

radiant yew
#

yes

#

see above

midnight cosmos
#

storage encryption disk destroy -disk * -force-all-states true

#

and have you removed the KMIP entries from NetApp

radiant yew
#

::*> storage encryption disk destroy -disk * -force-all-states true
24 entries were acted on.

Error: No disk characteristics changed. Disk Name must specify at least one
eligible device.

::*> storage encryption disk destroy -disk localhost:0n.0 -force-all-states true
1 entry was acted on.

Error: No disk characteristics changed. Disk Name must specify at least one
eligible device.

::*> storage encryption disk destroy -disk 0n.0 -force-all-states true

Error: There are no entries matching your query.

::*> storage encryption disk destroy -disk 0 -force-all-states true

Error: There are no entries matching your query.

#

::*> security key-manager show
No key managers configured.

#

yes

#

the cluster was destroyed because the mroots were encrypted so nothing left

#

I feel like it needs another name for the disk, when it says disk path is there another way to specify the disk. Here is what I see with a disk show

#

Usable Disk Container Container
Disk Size Shelf Bay Type Type Name


localhost:0n.0 - 0 0 unknown broken -
localhost:0n.1 - 0 1 unknown broken -
localhost:0n.2 - 0 2 unknown broken -
localhost:0n.3 - 0 3 unknown broken -

#

I tried to make the disks spare, but anytime I try any command to these disks it acts like there are not there

midnight cosmos
#

storage encryption disk show -disk 0n.0

radiant yew
#

I need to get them out of broken

#

::*> storage encryption disk show -disk 0n.0
There are no entries matching your query.

#

storage encryption disk show -disk localhost:0n.0

                             Disk Unique ID: 374A4130:58700427:00253845:0000000E:00000000:00000000:00000000:00000000:00000000:00000000
                                  Disk Name: localhost:0n.0
                             Container Name: -
                             Container Type: broken

Key ID of the Current Data Authentication Key: 0000000000000000020000000000010062164BE2BB3B9F03898EDE0F54E0657D
Key ID of the Current FIPS Authentication Key: 0x0
Is Power-On Lock Protection Enabled?: true
Mode of SED Data and FIPS-Compliance Protection: data
Disk Type: unknown
Control Standard: TCG Opal V2
Compliance Standard: -
Overall Security: -
TCG Data Range Start Sector: -
TCG Data Range Length in Sectors: -
TCG Read/Write Lock Enabled: -
TCG Read/Write Locked: -
TCG Active Key Type: -

midnight cosmos
#

to 'unbreak' a drive
storage disk unfail -disk *

radiant yew
#

::*> storage disk unfail -disk *

Error: command failed on disk "localhost:0n.17": Disk "localhost:0n.17" does
not exist.

Warning: Do you want to continue running this command? {y|n}: y

Error: command failed on disk "localhost:0n.13": Disk "localhost:0n.13" does
not exist.

midnight cosmos
#

oye
i'd drop to a loader and do a system reset to factory

radiant yew
#

that is how I got here, I tried 4, 9a, reload software.

midnight cosmos
#

when you are at the bootmenu
wipeconfig

radiant yew
#

maintenance mode, all the tricks that have worked in the past..

midnight cosmos
#

then option 4

radiant yew
#

wipe config does not see the drives. I will try that again.

midnight cosmos
#

if it isn't seeing the drives from the bootloader, not sure
Never had that happen, might need to pull them all
move one somewhere else and see if it only works with just 1, etc

radiant yew
#

if there is a PSID on the disk, it is SED right?

midnight cosmos
#

should be

#

did you do a 4a from the bootloader

radiant yew
#

I am booting now, will let you know what happens

#

should I do the wipeconfig or 4a

midnight cosmos
#

i mean.. i can show you how do wipe the disk from a non-netapp system
but i highly doubt that would be a good thing for support

#

4a, then wipeconfig

radiant yew
#

if you have something on that, that may be the answer, I just need to get this encryption key removed from the disk

#

it says 4a is not valid

#

Please choose one of the following:

(1) Normal Boot.
(2) Boot without /etc/rc.
(3) Change password.
(4) Initialize and configure system.
(5) Maintenance mode boot.
(6) Update flash from backup config.
(7) Install new software first.
(8) Reboot node.
(9) Clean System Configuration.
(10) Set Onboard Key Manager recovery secrets.
(11) Configure node for external key management.
Selection (1-11)? 4a

Please choose one of the following:

#

9a worked

#

that is what support said to try

#

command worked, did not fix the problem, sorry should be specific

midnight cosmos
#

hrm.
4a wipes disks and creates a vol0
9a unpartitions, removes ownership and destroyes the aggr
9b reinitializes nodes with ADP
usually you do a 9b after 9a

#

and you have to run 9a on both nodes

radiant yew
#

I get this when it tries to talk to the disk, this was during option 9a:

#

May 27 15:25:04 [localhost:diskown.errorDuringIO:error]: error 19 (disk not ready for requested operation) on disk 0n.6 (S/N ) while reading individual disk ownership area

midnight cosmos
#

yea, not sure.
If you run 9a on both nodes and it still fails.. i'm out of ideas other than what I said about using a non-netapp method, which support might not like

radiant yew
#

Zero disks, reset config and install a new file system?: y

NOTE: Make sure that all encrypting drives (if present) in the system are re-keyed to MSID 0x0 before proceeding with re-initialization.
This will erase all the data on the disks, are you sure?:

#

I am waiting on support to get back to me. Does this process make the disk unusable?

midnight cosmos
#

it should not. it resets the key to 0x0 so it can be rekeyed
It just makes all the data on it useless

#

fips-mode and data-mode typcially have to both be ran to clear it, then a destroy/sanitize to fully wipe/reset them

radiant yew
#

wipedisk did not work....

radiant yew
#

thanks for the help!!!

midnight cosmos
#

did you get something to work?

raven copper
#

I didn’t read everything here @radiant yew

Tell me this: do you need any data on the disks and do you have the NetApp license files for the nodes?

The simplest/easiest way to clean them is to go into maintenance mode.

Once there, find a way to get both nodes to unown (remove ownership) of all drives

Go to one node and then assign all drives

On that same node do

disk encrypt sanitize start -all

This is nearly instantaneous.
DO NOT run “disk encrypt show “ as it usually hangs (known bug). Just halt the system and reboot

Do the boot menu
Option 9a/9b etc

#

It’s brute force and it works. I’ve had to do it a few times

raven copper
#

If you don’t have the key manager your options are limited. You must “destroy” the disk (meaning reset the encryption which means you lose any data)

I’ve had scenarios where a customer misplaced the passphrase. Ended up running with a single node until all volumes were converted to NVE. Could remove the aggregate encrypt bit and start unencrypting the volumes. Once the aggregate was clear of encrypted volumes we could give back. Then finish unencrypting the rest. Finally delete the key manager and then start over