Harvest 24.11.1 update - invalid logins for application: ontapi.

[cursive cargo]

Hi Team,

Since updating to Harvest 24.11.1, ONTAP is reporting invalid ontapi login requests every 1 hour on our clusters. All clusters are set to 'Rest'' only in the NAbox.

I have re-entered the credentials, set it to 'Auto' and back to 'rest' again, restarted the NAbox appliance etc. Data is still being logged sucessfully, but every 1 hour it is trying to login with ONTAPI for some reason.

Message: security.invalid.login: Failed to authenticate login attempt to Vserver: VSERVER, username: HARVEST USERNAME, application: ontapi.

Description: This message occurs when an attempt is made to access the appliance by using invalid authentication credentials.

Corrective Action: Verify that the user made a genuine attempt to access the system. If not, use the "security login delete" command to either remove the account or use the "security login lock" command to lock the account to disallow access to the system using that user account. Also, strengthen the authentication credentials of the user account by updating it. More detail regarding the authentication failure is available in audit.log file.

Source: mgwd
Sequence#: 757700

All data and systems were migrated from v3.5.4 about a month ago.

NAbox v4.0.8b1 (3bbcb80)
Harvest 24.11.1
Grafana 11.3.0
Victoria Metrics 2.24.0

ONTAP: 9.15.1P1, 9.15.1P2 & 9.15.1P3

[gray oxide]

@cursive cargo Could you share NABox support bundle here
https://upload.nabox.org/reku-powo-vevu

[cursive cargo]

Thanks @gray oxide The file 'nabox-logs-2024-11-27_182213.tgz' has been uploaded.

[swift edge]

Thanks cashnmoney, I see three pollers, prefixed with: N, S, g. Each of those pollers has an "auth failed => 401 Unauthorized" error for the following collectors: ZapiPerf:FlexCache and ZapiPerf:Rwctx. That happens because your harvest.yml specifies the set of collectors:

        collectors:
            - Rest
            - RestPerf
            - Zapi
            - ZapiPerf
            - Ems

That set of collectors means the Rest and RestPerf collectors will be preferred over Zapi and ZapiPerf where both collectors include matching object templates.

Here's an example of what I mean, take the object Volume. All four collectors Rest, RestPerf, Zapi, and ZapiPerf define a volume template for the Volume object in their respective default.yaml. When Harvest starts, it reads each collectors default.yaml and were there are duplicate objects, Harvest unique-ifies the collector-object pairs by the order of the collectors.

Using the list above, the Rest:Volume collector will run instead of the Zapi:Volume collector because Rest is listed before Zapi. In your 401 case, what's happening is that all of the collector-object pairings results in Rest and RestPerf being selected for all objects except for the FlexCache and Rwctx objects.

That happens because those tow objects only exist in ZapiPerf and not RestPerf so the ZapiPerf collector runs. Why don't templates exist for RestPerf? That's because ONTAP has not implemented the RestPerf metrics for the cluster yet or has only recently implemented it. See https://github.com/NetApp/harvest/issues/2646 and https://github.com/NetApp/harvest/issues/2504. Once ONTAP adds those metrics, we will create templates and update the RestPerf collector to use them. If you want to remove these error messages you can remove the ZapiPerf collector from your pollers.