Discrepancy Available Space as shown by volume size and lun size | NetApp | Page 1

marsh bone Aug 20, 2023, 8:18 PM

#

Upon information provided below by CLI's, 8TB was assigned to the volume, the used space on the volume seems around 5+TB, 8TB was assigned to the lun. The volume has only this lun. However, why the available space on the window server only shows 623GB?

`Volume is thin provisioned
Vserver Volume Aggregate State Type Size Available Used%

iscsi-svm volume03 node1_ssd_aggr1 online RW 8.50TB 3.07TB 63%

The lun has been configured space-reserve=disabled(thin), and space-allocation=disabled.
*> lun show -volume volume03 -fields size-used, size
vserver path size size-used

iscsi-svm /vol/volume03/lun03 8TB 8.02TB

On window servers, it shows:
Used space: 7.39TB
Free space: 623GB
Capacity: 7.99TB`

graceful valley Aug 20, 2023, 8:56 PM

#

Check if fractional reserve is enabled (100%) and if there are snapshots in your volume.

marsh bone Aug 21, 2023, 12:01 AM

#

franction reserve = 0%;
there are no snapshots at all.

graceful valley Aug 21, 2023, 6:52 AM

#

Space reclamation then. Take the LUN offline and change "-space-allocation" to enabled. Your host OS needs support for SCSI UNMAP.

#

Read this: https://docs.netapp.com/us-en/ontap/san-admin/host-side-management-concept.html

Host-side space management overview

In a thinly provisioned environment, host side space management completes the process of managing space from the storage system that has been freed in the ho...

marsh bone Aug 21, 2023, 11:10 AM

#

That only explained why the discrepancy in size between lun on the storage and the host, and also why the available space on the storage size is less than on the host.

But, the question is about available space on the storage volume, which is much larger than host or the storage lun(3+TB available on the volume, but 623GB available on host lun, and full on the storage lun). How can we explain that?

Also, what are detailed steps on how to reconcile the space between two?

graceful valley Aug 21, 2023, 4:19 PM

#

hm... you maybe have Snapshot copy reserve enabled?
vol show -fields percent-snapshot-space

marsh bone Aug 21, 2023, 4:30 PM

#

no, just ran the command, it is 0%. If it was enabled, then I wouldn't be able to assign the entire volume space 8T to the lun. No, it was not enabled.

graceful valley Aug 21, 2023, 7:26 PM

#

My last guess: The available parameter of your vol show cmd is basically "physical-available". So the result of size minus available is physical-used.

If you check the following cmd you should see that used and physical-used is the same here:
vol show -vserver iscsi-svm -volume volume03 -fields size,used,physical-used,logical-used,available
(since fractional-reserve and snapshot-reserve is disabled, you have no snapshots and your vol is thin-provisioned)

The sized-used in your lun show cmd is "logical-used" though.

In other words: The efficiency features of your system shrank the data on your volume. From logical-used to physical-used.
The logical-used from vol show should be approximately the size-used from lun show. At least that fits in my tests.

If you want to go deeper, check vol show-space and vol show-footprint but I'm not really sure what you want to achieve.
If you need more space in your LUN for your Windows server: Increase the vol-size to some arbitrary number (100TB, 500TB, ... it does not really matter if your vol is thin-provisioned) and then increase the LUN size.

marsh bone Aug 21, 2023, 7:40 PM

#

What I cannot explain is that why the lun shows 623GB available on the Window server? No matter what happened or how we explained these figures on the storage side, 623GB is in fact the available space on Window that the user can use. I know how to increase the lun size. I just wanted to logically explain these figures.

graceful valley Aug 21, 2023, 7:43 PM

#

Why shouldn't it show 623GB available? Your LUN size is 8TB --> minus 7.4TB --> that's your ~623GB

#

Your Windows OS does not know anything about the physical used size in ONTAP.

#

And ONTAP does not know how much space is actually free in the LUN since space-allocation is disabled.

marsh bone Aug 21, 2023, 7:48 PM

#

Well, the reason of why I couldn't see the available space on Windows the same amount as shown on the storage (which is about 3+TB available on the storage) is because space-allocation is disabled? The answer is probably not. What caused the available space on the stoarge seems much larger than on host? That is the part I am confused about.

graceful valley Aug 21, 2023, 9:25 PM

#

You're mixing up logical-used and physical-used. Your Windows server will never know if ONTAP managed to shrink the 7.4TB of data which actually reside on the LUN (aka logical space) to 5TB physical space on the storage system. Or maybe 3TB, or even 1TB because the data was highly compressible or can be deduped very easily.
If you create a 8TB LUN, and it's formatted with a file-system by the host OS, you will not be able to save more than 8TB of data on this LUN. At least from the OS point of view. The file-system is 8TB in size - how could it save more than 8TB of data in it? The server never really knows how the data is being handled by the storage system.

#

Maybe to understand it better, think about these two personas of admins:

The server admin: He got his 8TB LUN (as requested from the storage admin) and his only concern is that the application on the Windows server does never fill up these 8TB. He does not care if the 8TB LUN is from an NetApp SAN or from Pure or some other vendor. He also does not care if this LUN needs 8TB of physical space on the storage system or if it has been reduced to maybe 1TB of physical blocks. He also does not know if the storage system uses snapshots or anything. All of this is the storage admin's business. The server admin only wants his 8TB LUN. It needs to be always available and he needs to be able to write 8TB of real data to it, delete it at his will, write it again, etc. And if he really needs more space he asks the storage admin to increase the size to 10TB or so.
The storage admin: He does not (really) care if the 8TB of the LUN are actually being used or if the LUN stays empty. His real concern is that the volume does not get full. Why? Because he needs to ensure that the LUN stays online. He got a request to provide a 8TB LUN, so he gave the server admin 8TB and now needs to make sure that the server admin can always use these 8TB, no matter what happens. The easy way for the storage admin is to make everything thick-provisioned, use snapshot-reserve and fractional-reserve and everything. Then there is no risk for him. If the 8TB LUN ever goes full that's because the server admin filled it up with 8TB of data. But the storage admin can also try to use the physical space of his storage system more efficient and use thin-provisioning. Or even features like compression and deduplication to save more logical space on the physical space available.

#

In your scenario:

The server admin used ~7.4TB of logical data on his 8TB LUN. ~0.6TB is still available, he can use that if he wants, everything's fine. If he wants more usable space in his LUN he needs to ask the storage admin to increase the LUN size.
The storage admin sees the ~3.1TB of physical space still available in his volume. Which is good. And because snapshots are disabled he does not really need to fear that the snapshot space ever fills up his volume (which might impact the LUN). Also there are no other LUNs or files on this volumes, so the LUN is safe. But since the volume is thin-provisioned, his only concern needs to be to always have enough space in the aggregate for his volume. That's what he needs to monitor. If there is not enough space for the volume ONTAP will take the LUN offline. Which the application and/or the Windows OS will not like (--> crash).

#

The storage admin can improve this scenario a bit by enabling "space-allocation". Aka "host-side space management". (Read here: https://docs.netapp.com/us-en/ontap/san-admin/host-side-management-concept.html)
Enabling "space-allocation" enables two things:

If Windows deletes something in its file system it will send SCSI UNMAP cmds to the storage system so ONTAP can free these blocks as well. That's nice-to-have so you can reuse this free space for other volumes. (In my opinion enabling "space-allocation" is only really relevant if you have big LUNs which go empty very often. And you really need that space.)
If the volume runs out of space ONTAP does not take the LUN offline but puts it in read-only mode and notifies Windows about that. This could also lead to crashes but might be a better approach.

In the end you need to make sure that your volume never runs out of space in the first place... so yeah.
I hope that clears everything up. Not sure how to explain it any further.

#Discrepancy Available Space as shown by volume size and lun size