There doesn't seem to be a lot of information publicly available except one KB article that states "Starting from ONTAP 9.10.1, the tiering scanner can be run in the express mode which is enable by default in ONTAP 9.11.1 and later." I have 8 more or less identical nodes with litt varying loads and I'm wondering why some tiering-scanners are running in "express mode" while most aren't. Anyone care to enlighten me?
#volume object-store tiering scan-sched-class
ember surge
ancient thistle
There are reasons why it might go to express mode. If you can open a case and reference KB https://kb.netapp.com/on-prem/ontap/Perf/Perf-KBs/Slow_tiering_during_high_workload, you can get the workaround, if you need it.
My first question is, why is it so urgent to have express mode?
ember surge
i have a case open. i have one node of 4 nearly identical nodes, the partner node is identical and the load on the partner is actually higher, but the "sick" node is tiering 90% of the time at 100MB/s and the aggregate would already be full if i didn't have the option of QoS'ing the crap out of certain workloads.
express mode happens by itself as well, even the volumes in question were in "express mode" for several hours before they went back to dogging it
anyway, it looks like i'm going to have to dig up the dusty old perfstat bones again in this decade, so, fun...
SG has a better approach by keeping metrics in prometheus that one can just upload to a case if necessary, b t w
north lintel
SG has a better approach by keeping metrics in prometheus that one can just uplo...
ONTAP can also upload a perf archive that you can view in Grafana through ActiveIQ
ancient thistle
What is the case number?
ember surge
2010141032
ember surge
ONTAP can also upload a perf archive that you can view in Grafana through Active...
unfortunately, ONTap can't do that here. I've done the manual thing once before, so that's an option, but a lot more work than just being able to have a simple download with date+time ranges
ancient thistle
Ah restricted ASUPs. No wonder.
I guess you could get a perfstat too, but either way you're going to have uncensored node/volume names.
I'm not a big fan of restricted ASUPs personally because it really cuts off a lot of useful info. But if you need it you need it. Yet if we need logs we'll need that info anyway...
I've gotta go but link that KB as well I found.
Yeah it would help to have some perf data in case the tiering scanner is getting stuck due to latency.
ember surge
and i'm not a big fan of netapp tools requiring names to diagnose internal functions, so we'll probably need to agree to disagree.
the US has US FED to deal with such issues, in the EU, that approach doesn't seem feasible, somehow, and the paperwork I need to have done to blast internal info to half of the 3rd world for 24/7 help isn't worth the effort.
north lintel
if you had ever worked in Support you would probably understand the reason restricted ASUPs are terrible 😉 . And I have never heard of massive paperwork being required by the EU to allow unrestricted ASUPs. If people really want quicker/better case resolution, they can usually somehow make it happen easily. Or if not, they have to accept the fact that we cannot help them without remote access to their system (or even send someone over to diagnose, which can take days).
May I ask in which "vertical" you are working that makes this so difficult? Law Enforcement? Government? something like that?
ember surge
LE
and we spend a lot of time on Schrems II implications
what i don't get is that there is probably no single object in ONTap that doesn't have an UUID, why isn't that enough? the user can easily map UUID's to individual aggregates, volumes, interfaces (even though those are necessarily as sensitive)
why do i have to include usernames and SID's and domain names, etc to get help with internal functions? this makes no sense to me. it's a tooling issue.
north lintel
because there's no "tool" to do the troubleshooting. It's done by humans. And humans are better with names like "data", "userhoms", "domian\admin" than things like 1234-443fddfb432-59492-34923-223afd, 1234-443fddfb432-56492-34923-223afd and 1234-443fddfb832-59492-34923-223afd. How quickly do you find the typos in the first, vs. in the latter?
ember surge
there's nothing preventing mock up of names to help with that
as in, just model it with usefull names, and when one finds the problem, apply the model to reality, remap UUID's to actual objects
north lintel
yeah, as I said, in theory everything is simple. And as you said, agree to disagree. I don't think it is worth investing in such a complicated tooling with uuids and fake name translation for the 0.1% of clusters that don't send proper debug logs 😉
ember surge
i really hope that the performance data sent weekly otherwise doesn't keep such private data stored at netapp, or one may have bigger issues than one wants. One could also choose to log things differently in ONTap so that most of this just isn't captured
and i would really hope diagnostics isn't done as much by humans... there should be enough tooling to find many abberations by this point
north lintel
who has access and how ASUP data is retained is outlined in TR 4688, and I'm sure you can get the ISO audit reports from them if you really need to see them.
If you have doubts about the data that is actually sent, or the retention, then I would suggest turning off ASUP completely. It is not a requirement for having a supported system, and if it's possible that sensitive data is included it's better to be safe than sorry
north lintel
and i would really hope diagnostics isn't done as much by humans... there should...
heh, I for one like the Future you live in, how cool would it be if such things as troubleshooting and diagnostics would be automated, it would make our life sooo much easier 😀 But I guess that you, coming from a governmental background, also know how it is with buerocracy and having humans everywhere in the chain of events 🤔 whether that's a good thing or bad thing is left to everyone's interpretation
ember surge
TR-4688 doesn't actually say much except it confirms that US FED has a special setup. The point you are missing is confusing results with causes. It may well be that many simply prefer to have "easier/faster" support and don't enable the "compliance" option, or don't know it exists at all.
and yes, i know, from a business standpiont, "support" is a cost, not really a revenue center, and i imagine that performance data is (or should be used) to alert to major regressions, but... i can only guess. The information is potentially a huge sales resource, but it's hard to say if anyone has realized (in both senses) yet.