sharp stratus
Hello,
Regarding converting an existing volume to an NVE volume, how much degraded performance can we expect while it is in the process of converting? On my AFF-A220 we have one volume which acts as a vCenter data store for about 50 VM’s which provide mission critical services and require 100% uptime. Just trying to gauge how impactful it will be to enable volume encryption. I was going to use the guide below to convert it. Thanks in advance.
grave radish
Well from what I’ve seen it’s not too bad as in not noticeable as long as you only do a small number per node at a time. It’s much faster to do a “vol move start -volume xx -vserver yy -destination destaggr -encrypt-destination true” than the “cool encryption conversion start” command
Also, you need to have enough space to essentially mirror the volume.
Another option is to create a new encrypted volume and mount into VMware. Then storage vmotion into the encrypted volume
Lastly if you want to take advantage of cross volume efficiencies (since you have an aff) you should use aggregate encryption. To get there you need to encrypt to NVE first. Then set the aggregate encryption bit then encrypt the volume again (vol move start…) but use the -encrypt-with-aggr-key true option.
wraith fiber
in-place encrypt is very slow. If at all possible, do it with a volume move. We had a customer inplace-encrypt a 100TB volume and it took around 2-3 weeks or so
sharp stratus
volume move is basically creating a new volume which is encrypted, and transferring the data to that volume, correct?
wraith fiber
yep
sharp stratus
Well from what I’ve seen it’s not too bad as in not noticeable as long as you on...
when you say to do a small number per node at a time, what do you mean exactly? can you break up the encryption process with volume move? encrypt a small bit at a time so performance isn't effected?
the volume id like to encrypt is relatively small, about 6TB
grave radish
Make sure the volume is thin provisioned for the move (space guarantee = none). Otherwise ONTAP will move the “whole” volume. With thin it only moves data in use. So even though you volume is 6t, it will only have to move what is in use.
I’ve had customers convert entire clusters with hundreds of volumes. In these cases I try to limit to about 2 per node to minimize any possible impact to production data. Also, I factor in disk type. Sata, definitely 2 no more. Sas i might push to 3. SSD/nvme I might push to to 4 (all per node)
Once you start a move with encryption, let it finish
You cannot break up a single volume.
sharp stratus
With volume move start, does NetApp automatically "replace" the old non-encrypted volume with the new NVE one? is there some sort of activation process after the fact?
west crater
Nothing to do after the conversion is complete.
Other option is to create new NVE datastores and just do Stroage vMotion of the VMs into the NVE ones. decom the old ones.
brittle vale
when you say to do a small number per node at a time, what do you mean exactly? ...
You can only do 8 vol moves at a time on a single node:
https://kb.netapp.com/onprem/ontap/os/How_many_vol_move_operations_can_be_active_at_the_same_time
I will concur with everyone else though. Doing the vol move is much faster than the in place conversion.
sharp stratus
Roger, so looks like a vol move sounds like the "move". Last two questions.. does the "volume move start" command sequence create the new encrypted volume AND delete the old one? Also, I'm guessing I will need to do this for my root volume as well. Does that need to happen first?
grave radish
Yes. It is basically snapmirror on the backend. When the mirror is complete and ONTAP can “cut over” it does that and redirects I/o to the new encrypted volume and then deletes the original volume (and I am pretty sure it also clears it from the recovery-queue or the operation simply bypasses the recovery-queue)
sharp stratus
Fantastic, and I/O remains uneffected on the original volume during the encryption process?
grave radish
Yeppers
sharp stratus
Any clue how long it would take to do ~6TB of data? Within a single volume
grave radish
Nope. Depends on system utilization and type of disks. No way to give a reliable answer
sharp stratus
Roger. Thanks very much for the assistance @grave radish , @brittle vale and @west crater. Very much appreciate the valuable insight.
cunning flicker
Any clue how long it would take to do ~6TB of data? Within a single volume
On our A800 we had like 800 MB/s to 1 GB/s replication throughput, but wenn the CPUs are at round 80% the vol move had like 400-500 MB/s and the cutover was retried several times
brittle vale
Roger. Thanks very much for the assistance <@894285831248957440> , <@68993220915...
One question (hopefully before you get started): Is there a reason you're not using NAE (Aggregate Encryption)? It's a lot easier to manage than encrypting a bunch of individual volumes.
The only downside of aggregate encryption is that you cannot rotate keys. My customer requires us to rotate keys every three years, so we were forced to use NVE because of that. But, other than that, I would recommend going with NAE. It's easier to manage, and you can still benefit from cross-volume efficiencies.
grave radish
To get to nae from unencrypted you need to
fickle berry
Step 2 not necessary anymore with 9.14.1 😬
And if you can manage to move away all volumes from one aggr you can already activate NAE there. So once you move your volumes back they will already use the aggr-key. Saves you needing to encrypt twice.
grave radish
Yeah, I’m not putting ANY production on 9.14 for at least six months
So stick with my list
That’s still rc
fickle berry
But usually you don't have enough space or an empty aggr lying around 😅
Yup, just saying
grave radish
I also deal a lot with air gapped sites and purposefully stay behind
tropic breach
Well from what I’ve seen it’s not too bad as in not noticeable as long as you on...
Is encrypting the volume being mandatory for having aggregate encryption?
I'm reading netapp doc and it says:
An aggregate enabled for aggregate-level encryption is called an NAE aggregate (for NetApp Aggregate Encryption). All volumes in an NAE aggregate must be encrypted with NAE or NVE encryption. With aggregate-level encryption, volumes you create in the aggregate are encrypted with NAE encryption by default. You can override the default to use NVE encryption instead.
It seems to be that I can have:
silk dawn
Correct.
In either case the data is encrypted at rest
To convert an aggregate to encrypted, you encrypt all the volumes first
grave radish
Netapp aggregate encryption: every single volume on the aggregate MUST be encrypted or you cannot enable NAE. Don’t believe me, try it. Won’t work. Guaranteed. In order to enable NAE on a current aggregate with data, every single volume must be encrypted with NVE first. If you are not running 9.14 (which I believe may still be in rc) then you can convert every volume to NVE except SVM root volumes. Those must be moved off. I posted pretty good details a few days ago.