- Alerting implementation
- Bundle migrate CLI tool in NAbox 4 web server
- Minor bug fixes and improvements
#NAbox 4.0.0b3 available
polar plover
broken lynx
Hi @polar plover thanks for the new Beta. I will test it in the next few days. One question do you plan to install a node exporter per default? This would help to monitor the whole appliance in a easy way
polar plover
node exporter has been there for a while yes, and is part of NAbox 4, it's just that there's not a lot of metrics leveraged currently (disk capacity only)
CPU and memory too actually, is in NAbox dashboard
broken lynx
Nice so we can easily add it to our Prometheus and monitor it properly
polar plover
Yes indeed. NAbox 4 has properly integrated alerts for capacity, that's it for now, open to ideas ! I'll probably add memory utilization, but I need to test the right metric to use
pallid urchin
Ok...it wasn't there if you remove the 4. from that URL...
That's the link I used...
polar plover
Yes I always point to 4.nabox.org but it's confusing that NAbox4 is also on the main web site 😄
pallid urchin
Kinda hard to read the release notes if the URL is different than the one linked. :p
polar plover
What do you mean ? Anything I should fix ?
broken lynx
Yes indeed. NAbox 4 has properly integrated alerts for capacity, that's it for n...
What would be the URL to collect the node exporter metrics? Currently we use the regular <fqdn>:9100 in our config to collect node exporter metrics
polar plover
Ahhhh. Yes I’m with you now. Yeah, no it’s not exposed 😆 I didn’t realize as you’re using your own prom obviously you need a public endpoint. There is an option in NAbox to enable guest access to metrics, I’ll expose node exporter as well
broken lynx
Yes we use Harvest, NABox and Prometheus in various ways. We have a dedicated Prometheus installation for monitoring all our management VMs like Ansible controllers, NABoxes etc.
The standalone Harvest is for the monitoring of the ONTAP systems directly with reduced metrics and like 10 NABox installations for all the possible metrics. Sounds like an overhead but for it works pretty well 🙂
about the configuration of the NABox. In the section LDAP & Active Directory I can not add a second DC. If I use the comma separated value like in NABox 3 it says invalide host name
broken lynx
Just wanted to start the migration and the error occurs - docker: Error response from daemon: network nabox_default not found.
polar plover
Thanks ! Good point on the multiple LDAP server, I'll fix that
Now, not having nabox_default doesn't seem normal, can you check this for me ? ```
nabox-api:~# docker network ls
NETWORK ID NAME DRIVER SCOPE
a44e164c52eb docker-compose_default bridge local
af973e6de56c host host local
f15648fb5767 nabox_default bridge local
bd5103316d6e none null local
dcoi2nabox-2:~# docker network ls
NETWORK ID NAME DRIVER SCOPE
a583c0a7b999 docker-compose_default bridge local
85389069f924 host host local
a2ba217dceee none null local
Seems that this is missing - I will reboot the appliance
polar plover
thanks. What version of NAbox is it ?
Anyways if things are working, it's probably using docker-compose_default which is the network you can use in the migration command then
It's not you, forget it, I'll fix the doc
polar plover
broken lynx
Version 3.5.2
regarding the documentation the export SNAPSHOT_PATH=/prometheus/data/snapshots/<xxx> is the name of the snapshot created right? because when using the docker run --rm command I receive the error no such file or directory
polar plover
indeed, you should see this snapshot in /data/prometheus/data/snapshots/
broken lynx
ahhh the error has a additional / in it - /prometheus/data/snapshots//prometheus/data/snapshots/20240502T081654Z-24551bd06ad09e60
polar plover
oh the doc is wrong (again)
$ docker run --rm -v /data/prometheus:/prometheus -it --network docker-compose_default victoriametrics/vmctl:heads-master-0-g93a29fce4e prometheus --prom-snapshot=$SNAPSHOT_PATH --vm-addr=https://$NABOX_IP/vm/ --vm-insecure-skip-verify -s
SNAPSHOT_PATH is already the ful path, not just the ID
broken lynx
now it works 🙂
broken lynx
just used the manual way, but I can delete the VM and try it with the new version
polar plover
If it's no trouble really, as there is the manual way that works it could be fixed later
well you don't have to redeploy actually
you can just roll with it and cancel the current migration
broken lynx
with the migrate tool is the same error - docker: Error response from daemon: network nabox_default not found.
time=2024-05-02T09:04:55.818Z level=ERROR msg="Failed to migrate" error="exit status 125"
broken lynx
Thanks ! Good point on the multiple LDAP server, I'll fix that
Interesting with 2 LDAP server the login to Grafana with an AD account does not work anymore. As soon I delete one it works again
polar plover
Did you upgrade nabox 4 to the version I posted this morning ?
polar plover
Interesting with 2 LDAP server the login to Grafana with an AD account does not ...
Dang it
Oh wait. Grafana expects space separated list. I need to transform it
broken lynx
works now 🙂
polar plover
\o/ Thank you !
broken lynx
another question regarding customize harvest 2. Is this still possible the same way in NABox 4 like in 3 with the custom.yaml? In the doc there is a point about the REST collectors, that this wont work any more
polar plover
With the harvest team we refined the wording about that https://4.nabox.org/faq/#customize-harvest
It's not that it doesn't work, it's that Rest collector, with or without NAbox, doesn't support extending template, you have to copy the whole existing template because it will be replaced instead of merged
yes, customization works pretty much the same, except for the fact you can stick to harvest documentation, you don't have to put template file names in an array
broken lynx
Ah ok so I can create the custom default yaml in the user folder with the additional templates like nfs4pool
polar plover
That would work yes
broken lynx
Hi @polar plover is it possible to revert the migration? I just restarted it but there was no prompt for the password or a message that the systems are already configured in the 4. Even if there are none configured
polar plover
What do you mean ? The migration pushes all the systems to nabox4, skipping those already there
polar plover
/node-exporter is exposed in beta 4 (to be pulished soon).
You will have to turn this on though :
broken lynx
Thank you for that, now we have the NABox 4 prober monitored 🙂
polar plover
well... I might actually steal those dashboards 🙂 That's standard node-exporter dashboards from grafana ?
broken lynx
should, need to ask internal but as I remember right we need to translate some panels from mandarin to englisch
polar plover
lol. nice
broken lynx
sorry my bad it is the regular node exporter dashboard. He needed to translate the stuff for the windows node exporter dashboard
broken lynx
Interesting is since I enabled the export function the CPU usage / load and memory usage has no metics in the Nabox dashboard
in the container panel it looks fine
polar plover
mmm, can you access /node-exporter/ on nabox ?
polar plover
How much RAM did you give NAbox, it looks like it's struggling.
Btw that's not NAbox 4 is it ?
broken lynx
How much RAM did you give NAbox, it looks like it's struggling.
Btw that's not N...
Currently 4 CPUs and 8 GB RAM with two two clusters (4x A250) collecting data. The high usage is probably because of the migration. Let me check in the office how it looks now
polar plover
Yes check that node-exporter still runs
broken lynx
The node-exporter target is up and running
polar plover
What about /vm/targets ? You see node-exported endpoint here ?
broken lynx
Yep is around
polar plover
All good, fix will be in next beta