Monitoring Buffer to Buffer credits and FA Queue depth on Netapp AFF SAN arrays.

[scarlet ocean]

Question about Buffer to Buffer credits in SAN environment. I do know that OCI has counters for events when number of buffer to buffer credits available goes down to 0, as described here https://docs.netapp.com/us-en/oncommand-insight/howto/how-to-troubleshoot-fibre-channel-bb-credit-0-errors.html

Does Netapp SAN array keep track of 0 buffer credits available events or does OCI pulls it from SAN fabric switch?

I'm also curious to know if number of available buffer to buffer credits is tracked somewhere in OCI or in Netapp so we can analyze those trends before they reach 0.

[swift stirrup]

Basically buffer credit 0 problems are only a problem if you get above 2-3% and usually ISL's are the point to monitor. Buffering "in front of" a target port just means the target system is overloaded. IIRC, the HBA's have a pretty huge buffer themselves to take care of bursty loads. I don't think the NetApp storage system is going to give you much insight, but your SAN switch monitoring should tell you. If you have a larger SAN, then you probably have such monitoring.

[scarlet ocean]

What i'm looking for is a metric that tell's me that FA is overloaded and we need to add/enable another FA pair on Netapp. I was hoping that buffer credit 0 can be used as such metric. Do you have aa suggestion for any other metrics I should look as "FA is overloaded" indicator?

[hardy crystal]

what do you mean by "FA"?

[swift stirrup]

Buffer credits are simply buffers... they are used by both sources and targets and one can over allocate buffers to give a skewed view. It's not a great metric for capacity because misconfiguration alone can deplete buffer credits (slow drain). As long at the storage system can respond sufficiently, buffer credits aren't really an issue. NetApp systems have enough internal metrics to tell you when performance requirements are being saturated

[ruby dock]

I think buffer credits are only measured on the switch. ONTAP uses FC queue depth.

https://kb.netapp.com/onprem/ontap/da/SAN/How_to_check_FCP_error_counters has info on checking that. I believe this command has a counter for that too:
pstejska_vsim::*> statistics catalog counter show -object fcp_port_sli

Object: fcp_port_sli
Counter Description
--------------------------- ----------------------------------------------
arbitration_fc_al_timout Number of timeouts that occurred during an
fibre channel-arbitrated loop (FC-AL)
arbitration operation
bbscn_off Number of times BB_SC_N credit recovery
mechanism has been disengaged
bbscn_on Number of times BB_SC_N credit recovery
mechanism has been engaged

[scarlet ocean]

Fibre Channel Adapter

[scarlet ocean]

Do you know if Netapp has a counter for FC queue depth?

[ruby dock]

I know we have the EMS message.

https://kb.netapp.com/onprem/ontap/da/SAN/How_to_check_FCP_error_counters