#Root vol footprint

1 messages · Page 1 of 1 (latest)

floral ice
#

I recently ran into an issue with a root aggregate becoming 100% full leading to a failover while blocks were reclaimed. The underlying problem was that the root volume became >100% it's capacity due to delayed frees (increased due to heavy churn in the root volume).

We've manually gone through several other storage nodes' root vol footprint data to try to locate any other instances. This is fairly labor intensive so was hoping that Harvest might be able to help here.

NOTE: root vol is thick provisioned so this always appears to be completely consumed from aggr0's perspective. In order to know a thick provisioned flexvol's actual utilization, vol show-footprint command can show actual util -

#

It looks like the ONTAP: Aggr dashboard has a footprint metric by perf tier. However, we are not leveraging tiers and since this is root vol I believe that won't apply.

autumn loom
#

hi @floral ice another challenge is that ONTAP's REST API does not include root aggregates.

Looks like Harvest can collect vol show-footprint via the private CLI. Would that be helpful? If so, can you let us know which fields would be most useful in the template?

floral ice
#

Thanks Chris! I think that would be helpful here.

I would capture all from the above example. %Used is not required IMO.

autumn loom
#

Give this Rest template a try and see how it works for you.

name:                     VolumeFootprint
query:                    api/private/cli/vol/show-footprint
object:                   volume

counters:
  - ^^hostname                                  => node
  - ^^volume
  - ^^vserver                                   => svm
  - ^aggregate                                  => aggr
  - delayed_free_footprint
  - effective_total_footprint
  - flexvol_metadata_footprint
  - total_footprint
  - total_metadata_footprint
  - volume_blocks_footprint                     => blocks_footprint
  - volume_blocks_footprint_bin0                => blocks_footprint_bin0
  - volume_blocks_footprint_bin1                => blocks_footprint_bin1
  - volume_guarantee_footprint                  => guarantee_footprint

export_options:
  include_all_labels: true
#

And here's an example of the metrics exported from this template.

curl -s 'http://127.0.0.1:12998/metrics' | rg -v '^#|metadata' | sort | rg osc_vol01
volume_blocks_footprint_bin0{aggr="test1",cluster="umeng-aff300-01-02",datacenter="dc-1",node="umeng-aff300-02",svm="osc",volume="osc_vol01"} 931284537344
volume_blocks_footprint_bin1{aggr="test1",cluster="umeng-aff300-01-02",datacenter="dc-1",node="umeng-aff300-02",svm="osc",volume="osc_vol01"} 162677788672
volume_blocks_footprint{aggr="test1",cluster="umeng-aff300-01-02",datacenter="dc-1",node="umeng-aff300-02",svm="osc",volume="osc_vol01"} 1084140761088
volume_delayed_free_footprint{aggr="test1",cluster="umeng-aff300-01-02",datacenter="dc-1",node="umeng-aff300-02",svm="osc",volume="osc_vol01"} 72581120
volume_effective_total_footprint{aggr="test1",cluster="umeng-aff300-01-02",datacenter="dc-1",node="umeng-aff300-02",svm="osc",volume="osc_vol01"} 976419889152
volume_guarantee_footprint{aggr="test1",cluster="umeng-aff300-01-02",datacenter="dc-1",node="umeng-aff300-02",svm="osc",volume="osc_vol01"} 0
volume_total_footprint{aggr="test1",cluster="umeng-aff300-01-02",datacenter="dc-1",node="umeng-aff300-02",svm="osc",volume="osc_vol01"} 1113307623424
floral ice
#

Thanks @autumn loom This looks good!

#

Will this be included in the next build?

autumn loom
#

yes

autumn loom
#

@floral ice do you need these metrics from both the Rest and Zapi collector or only Rest?