CIFS Auth Latency | NetApp | Page 1

strange monolith Feb 7, 2023, 12:17 PM

#

Hi Folks, After the 9.10.1 release we are having issues with slow LDAP and Slow auth, Is there any dashboards we could use to show this?

scenic hornet Feb 7, 2023, 1:03 PM

#

hi @strange monolith I'm not aware of any counters that would help troubleshoot LDAP. @terse sail do you know of anything?

Maybe these could help? https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/How_to_troubleshoot_LDAP_issues_in_Windows_Active_Directory

https://www.netapp.com/media/19423-tr-4835.pdf?v=10-6-2020-9-13-16-PM

NetApp Knowledge Base

How to troubleshoot LDAP issues in Windows Active Directory

strange monolith Feb 7, 2023, 1:35 PM

#

Hi Chris, Ralf in the AMS support center uploaded our perf archive to an internal grafana box and it had a nice auth latency display

scenic hornet Feb 7, 2023, 1:36 PM

#

sweet, that implies there are counters we could consume

strange monolith Feb 7, 2023, 2:23 PM

#

It would be fantastic if we could get that built into the CIFS latency views

scenic hornet Feb 7, 2023, 2:25 PM

#

if you know which counters are being used, please share. Otherwise, I'll connect with Rolf later and try to find them. Are you working with Rolf D, Rolf H, or Rolf J? 😃

terse sail Feb 7, 2023, 4:52 PM

#

Do you have a case number?

#

@strange monolith ?

#

It could be from SMB2 view in PAS, which would be the SMB2 object.

strange monolith Feb 7, 2023, 5:42 PM

#

Sorry mate, let me find it, 2009463576

#

Yeah it was from the SMB2 object I remember that much

coral python Feb 8, 2023, 11:13 AM

#

Thanks @terse sail @strange monolith
yes smb2 object is available in Ontap Zapi side. However this counter is not present in Rest side.

strange monolith Feb 8, 2023, 11:19 AM

#

Is that easy to add to restapi? is there any equivalent? Just something to check for AD latency session setup etc would be ideal

coral python Feb 8, 2023, 11:24 AM

#

ONTAP Rest performance counter API are available from 9.12 onwards. One need to request ONTAP to add any missing/new object as needed. For now, this object is available in Zapi so it should work till Zapi are available in ONTAP.

strange monolith Feb 8, 2023, 11:34 AM

#

Would it be Possible to include a range of CIFS auth latency counters into the next release on the dashboards?

coral python Feb 8, 2023, 11:40 AM

#

is your requirement to add similar smb2 view dashboard from PAS to harvest?

strange monolith Feb 8, 2023, 2:28 PM

#

Ideally mate, Just want to check auth latency is good, if that is SMB2 view that Ralf and the support guys use with PAS that will do

coral python Feb 8, 2023, 3:43 PM

#

Sure We'll add it to roadmap.

#

Added a feature request https://github.com/NetApp/harvest/issues/1696

strange monolith Feb 9, 2023, 12:28 PM

#

Thank you Rahul

dusty belfry Mar 17, 2023, 2:15 PM

#

@strange monolith Would you mind filling up this SharePoint form to address the SMB 2 REST gaps. The REST team follows this process to track incoming requests for gaps https://forms.office.com/Pages/ResponsePage.aspx?id=oBEJS5uSFUeUS8A3RRZbOtlEKM3rNwBHjLH8dubcgOVURVM2UzIzTkQzSzdTU0pQRVFFRENZWlAxNi4u

strange monolith Mar 23, 2023, 4:54 PM

#

Sure

#

We are running 9.10P11 and collecting via ZAPI (need to enable restapi for the service account) . Is the dashboard available to use now?

scenic hornet Mar 23, 2023, 5:00 PM

#

hi @strange monolith yes these changes were added a couple of weeks ago for ZapiPerf and are available in nightly. Screenshots https://github.com/NetApp/harvest/pull/1754

#

the smb2.yaml template is disable by default in conf/zapiperf/default.yaml

strange monolith Mar 23, 2023, 11:26 PM

#

Ah theres no data on the dashboard being collected

#

Ah now enabled! I will report back tomorrow with my findings

strange monolith Mar 24, 2023, 3:09 PM

#

Still showing as no data 😦 Our service account is configured as per https://nabox.org/documentation/configuration/ and dashboard, and the smb2yaml is enabled

scenic hornet Mar 24, 2023, 3:13 PM

#

let's check your poller logs - can you ssh into nabox and run dc logs nabox-harvest2 | grep SMB2 or if you'd rather, grab the logs and email them to us and we'll take a look. ng-harvest-files@netapp.com https://nabox.org/documentation/troubleshooting/#collecting-logs

strange monolith Mar 24, 2023, 5:53 PM

#

Nothing returned for that command

#

#

scenic hornet Mar 24, 2023, 5:59 PM

#

If nothing is returned that means the SMB2 collector was not started. can you ssh into nabox and check that SMB2 is enabled in /opt/harvest2-conf/conf/zapiperf/default.yaml thanks for the screenshots. That does show that smb2 is not there

#

did you download a nightly build of Harvest and add it to nabox?

strange monolith Mar 24, 2023, 6:01 PM

#

scenic hornet Mar 24, 2023, 6:03 PM

#

something's wrong since you screenshot does not match https://github.com/NetApp/harvest/blob/main/conf/zapiperf/default.yaml

strange monolith Mar 24, 2023, 6:04 PM

#

scenic hornet Mar 24, 2023, 6:04 PM

#

oh! scroll down further and see if SMB2 is down near the bottom 😄

#

can you grab the poller logs for one of your pollers that should be using SMB2 and we'll take a look, https://github.com/NetApp/harvest/wiki/FAQ#how-do-i-share-sensitive-log-files-with-netapp

strange monolith Mar 24, 2023, 6:05 PM

#

I was like, oh its down at the bottom

#

How I grab the poller logs mate?

scenic hornet Mar 24, 2023, 6:08 PM

#

dc logs nabox-harvest2 > nabox-harvest2.log zip if it's too big

strange monolith Mar 24, 2023, 6:11 PM

#

Next stupid question, where would that log file be?

scenic hornet Mar 24, 2023, 6:11 PM

#

should be in the current directory

#

do you see it there?

strange monolith Mar 24, 2023, 6:18 PM

#

Just sent them over to you

scenic hornet Mar 24, 2023, 6:20 PM

#

got it

strange monolith Mar 24, 2023, 6:20 PM

#

thank you

scenic hornet Mar 24, 2023, 6:29 PM

#

there are some REST permission issues we can work on later - the won't affect this issue. from your ssh shell can you run dc down. This will bounce the pollers

#

i see that pollers were restarted version="harvest version 23.03.24-nightly (commit cea86732) (build date 2023-03-24T00:25:15-0400) linux/amd64 but there is no mention of trying to start SMB2. This is the list of ZapiPerf collectors started from your logs

#

there are 632 instances of API request rejected => Insufficient privileges: user 'grafana' does not have read access to this resource This means the ONTAP user named grafana does not have ZAPI read permissions. Details https://netapp.github.io/harvest/23.02/prepare-cdot-clusters/#ontap-cli

#

and that's because of a new template that was added so not related to SMB2
Zapi:QosPolicyAdaptive and Zapi:QosPolicyFixed

strange monolith Mar 24, 2023, 7:53 PM

#

Ah thats fine mate, what part is missing for our privileges?

#

scenic hornet Mar 27, 2023, 12:58 PM

#

for the recently added qos policy templates these additional permissions are required
security login role create -role harvest2-role -access readonly -cmddirname "qos adaptive-policy-group" security login role create -role harvest2-role -access readonly -cmddirname "qos policy-group"

#CIFS Auth Latency