Software encryption: NAE or NVE, or both ?

[limber idol]

Hello,

I have just configured Onboard Key Manager and activated hardware encryption with SEDs as a first step.
In a second time, I would like to setup software encryption in order to encrypt data-at-rest on my AFF, we are using SVM DR to replicate data on a secondary site.

I had a look at ONTAP documentation and it seems that both NAE and NVE could be used in conjunction.
Is it a " best practice " ?
In real world, what do NetApp customers are using ?
What are your thoughts, the pros and cons of NAE and NVE ?

I did a test on a single volume with NVE and noticed that the destination volume is not encrypted, I guess this due as stated by the System Manager that a DP volume cannot be encrypted, am I right ?
Isn't it a bit weird that kind of cannot be encrypted ?
I mean it contains the same data as the source volume even if the SVM is down...

Thank in advance for your advices.
Regards.

[fleet roost]

If you encrypted the drives all data at rest is already encrypted

NAE -> all volumes share the same key. You are able to take advantage of cross volume dedupe when the same key is used on aff/c series platforms

NVE -> each volume has its own key. You cannot use any cross volume efficiencies

SED/NSE -> the physical drives are encrypted. Doing NVE or nae provides double encryption

Encryption is handled independently at each end or replication

You can absolutely encrypt any destination

Encryption is handled below the file system which is why the data is copied but is not encrypted

Encrypted file system->decrypted -> plain text -> snapmirror(using TLS) -> destination plain text -> encrypted to disk

[cold pine]

one important difference between NVE and NAE that might impact compliance (like gdpr etc.): If you're using NAE and delete a volume, the data is still on the disks somewhere, and since it's encrypted with the same key as all other volumes in the Aggregate, there's a theoretical chance to recover it. If using NVE, each volume has a separate key that gets deleted when the volume is deleted, so even though the data is still on the disks, it is encrypted with a key that is no longer available and thus not recoverable

we had a customer who had exactly that requirement and they ended up using NVE for that reason (even though it results in more disk usage due to no cross-volume dedup)

[fleet roost]

Change the key periodically or after a delete. Not difficult

[cold pine]

unless that physically re-encrypts every single block on the disks, this will not help. And physically re-encrypting volumes in-place can take months

[limber idol]

Thank you all for your feedback.

If I enable NAE, once all volumes are encrypted, can I revert safely and easily to unencrypted data ?
I am asking just in the unlikely case a problem would occur with volumes encrypted by NAE ...

Second point: Before writing this post, as I thought that destination volume cannot be encrypted in SVM-DR relationship, I tried manually to encrypt the test destination volume (data protection type)... It looks like it was a bad idea.
ONTAP cannot encrypt the destination volume, its status is (volume encryption conversion show):
" Paused on error.
Reason: Vserver
"xxxxxxxxxxxxxxxx" is
not in the admin
"running" state.

Documentation says that a volume conversion cannot be canceled... Does anybody knows how to stop that ?

[cold pine]

can I revert safely and easily to unencrypted data
you need a new aggregate that is unencrypted, and move the volumes over. Safe? yes. Easy? yes. However it requires a new aggregate, i.e. new disks
volume conversion cannot be canceled... Does anybody knows how to stop that
Don't think that is possible

[fleet roost]

How did you try to encrypt the destination? I’ve done it frequently.

Never ever use “volume encryption conversion start” it takes too long. (Unless you have no space for replication). The better/faster method is
Vol move start -vserver xx -volume bb -destination-aggregate same-as-src -encrypt-destination true

That will make a copy of the volume encrypting as it goes and when it’s done it cleans up

To decrypt a volume as long as the aggregate is not encryption enabled

Vol move start -vserver xx -volume bb -destination-aggregate same-as-src -encrypt-destination false

There is a kb about all this someplace

To decrypt you need to vol move to an aggregate that isn’t encryption enabled

If all aggregates are encrypted and the volumes are aggregate encrypted, you must convert to NVE first, then disable the aggr encryption bit then decrypt. It’s a multi step process

[limber idol]

Hello,

Unfortunately, I followed what is written in NetApp documentation and indeed used " volume encryption conversion start " ...
However, I only used it on small test volume so far... So no damage.
---> Understood, I will try with " vol move... " as suggested.

---> If I want to stop/delete the encryption currently paused in error of the destination SVM-DR volume, is the only way to do that is to delete the volume ?
If so, I accommodate with that.

[sweet roost]

Storage encryption only protects data on the drive while the drive is locked, in relation to NSE, or while the aggregate is unauthenticated, in relation to NAE, or the volume is unauthenticated, in relation to NVE. The encryption design is for data at rest, effectively a disk that is offline and removed from the system. NetApp's on-board key manager's job is to store the encryption keys for disks when using NSE, aggregates when using NAE, and volumes when using NVE. If you have SED drives, always use NSE as your encryption layer as the first choice, as the drives perform the encryption and you don't have to waste CPU cycles like you do with NAE or NVE, and you don't loose deduplication efficiency like you would with NVE. Technically, even if you don't enable NSE for SED drive, they are always encrypting data with a Data Encryption Key anyway, they just automatically unlock without authentication on power on. If you don't have SED drives, your next best option is NAE. It works like NVE, except instead of each volume having its own key, there is one key that all encrypted volumes on that aggregate use. Since both NAE and NVE work at the volume level, by using NAE, WAFL is able to perform aggregate wide deduplication on all data. Your last choice would be NVE. The downside of NVE is that each volume is encrypted with a separate key, so even if you had the same data on two volumes, once encrypted, the output would be different for each volume, so there would be nothing to deduplicate, thus aggregate deduplication is not allowed.

[cold pine]

it really depends on the use-case. NVE can be a better choice then NAE if you have to irrevocably delete datasets (e.g. for GDPR compliance)

also, using NSE only helps against people getting into your datacenter and stealing your disks without stealing the controller with it. So to get any "real" protection out of it you would need to enable CC mode, which brings with it a whole lot of other things to consider

[limber idol]

Hello,

Thanks for those information.
Hardware encryption is already done, so let's focus on software encryption now if you don't mind.

I understand that from a storage efficiency point of view, it's better to use NAE instead of NVE.
In my case, on my primary NetApp AFF cluster (9.14.1), I have more than 500 volumes to encrypt split across multiple aggregates, data is replicated to a secondary cluster via SVM-DR.
I have encrypted another test volume with NVE on source cluster, via " vol move start ... ", I did the same thing for its corresponding volume on destination cluster, it looks good.

NAE is not enabled on that aggregate, neither on any other aggregates of the cluster, can I simply execute this command to enable NAE : " storage aggregate modify -aggregate xxxxxxxxxxxxxxxxxxxx -encrypt-with-aggr-key " ?
Then, the same way I did with NVE, start to encrypt volume manually via " vol move start ... " ?

[fleet roost]

If you already have hardware encryption then why continue with software encryption? You are basically double encrypting your data. Nothing wrong with that.

In order to flip the aggregate bit for encrypt all volumes on the aggregate must already be encrypted with NVE. Or the aggregate must be completely empty. That’s it. That’s the only way to flip the aggr-encrypt bit

After all volumes are NVE on the aggregate, flip the bit. Then do the “vol move start” and specify the -encrypt-with-aggr set to true

Go to the support site. There’s a Netapp encryption FAQ that goes over all this and more in painstaking detail

[limber idol]

Hello,

Thank you very much @fleet roost for your explanations, I think that encryption is much clearer in my mind now !

To summarize:

• NAE can only be enabled via " storage aggregate modify -aggregate xxxxxxxxxxxxxxxxxxxx -encrypt-with-aggr-key " when:

  ---> All volumes of this aggregate are already encrypted via NVE.
  ---> OR, if the aggregate is empty.

• Once the aggregate is " NAE " enabled, then I can encrypt volumes using NAE via "vol move start ... -encrypt-with-aggr-key true ".

• This means that usage of " -encrypt-destination true " argument in vol move start command indicates that NVE is used.

• And argument " -encrypt-with-aggr-key true " is used in case of NAE.

By the way, both hardware and software encryption of our data-at-rest is a business requirement.

This may be my last question:
I created this morning a new volume on my source cluster, I was expecting that it would be natively encrypted via NVE as stated in NetApp documentation: " Brand new volumes that are not part of an NAE aggregate will have NetApp Volume Encryption (NVE) enabled by default. "
https://docs.netapp.com/us-en/ontap/encryption-at-rest/configure-netapp-volume-encryption-concept.html#understanding-nve

---> That was not the case... On the contrary, the volume newly replicated on the destination cluster via SVM-DR was encrypted !
Am I missing something, or is it an abnormal behavior of ONTAP ?
I had to manually encrypt the source volume (via System Manager this time, just to try...), now it looks good.

Regards.

[fleet roost]

First if the key manager is enabled (on board or external) a newly created volume will always be encrypted

If you have nae enabled on the aggregate the volume will default to be nae.

If you do not have nae enabled on the aggregate the volume will default to NVE

If you do not have any key manager enabled no volumes will be encrypted

When doing replication, the same criteria apply.
If the destination does not have a key manager then nothing will be encrypted. If the key manager is enabled but not nae then the destination volumes will be NVE. If the destination has the key manager enabled and the aggregates have encryption enabled then destination volumes will be nae.

Source and destination mirrors have nothing to do with encryption of each other. They are both independent events that must be treated separately

You can and should encrypt the cluster per but other that that each side is treated on its own for encryption

[limber idol]

OK, I forgot to mention it but of course, (Onboard) Key Manager is configured on both clusters.
I will create another volume on source cluster to check if again this one is not encrypted...

I managed to solve my issue reported previously with a volume conversion on a DP SVM that could not be canceled.
I just performed a vol move start ... on the same aggregate and once it completes, the volume was encrypted.

[limber idol]

Hello,

The volume I created manually has been encrypted both at source and destination automatically, so it looks good in that case.

On the contrary, another volume created automatically by Trident to protect Kubernetes environments on the source cluster has not been encrypted at creation... The weird thing is that the volume replicated via SVM DR to secondary cluster has been encrypted once again...

How can we explain that ?
Any idea to solve that ?

[fleet roost]

What platform is this? What version ONTAP?
This sounds feasible if the aggregate is not NAE. I am pretty sure a volume must be encrypted on nae.

If the aggregate is not nae then trident is sending the appropriate flags to not encrypt the volume. You can do the same when manually creating a volume also

Two options

1. Make sure NAE is enable on all aggregates. Trident volumes will be forced to encrypt
2. Modified the trident backend

https://docs.netapp.com/us-en/trident/trident-use/ontap-nas-examples.html#backend-configuration-options-for-provisioning-volumes

right from that page under encryption telling you exactly what I just said

Enable NetApp Volume Encryption (NVE) on the new volume; defaults to false. NVE must be licensed and enabled on the cluster to use this option.

If NAE is enabled on the backend, any volume provisioned in Trident will be NAE enabled.

For more information, refer to: How Trident works with NVE and NAE.