Managed to completely bork netdata when trying to switch from stable to nightly

[solar halo]

HI @narrow mortar , thanks for sharing this here, and sorry you're experiencing this

@fervent lily pinging you here since you're more knowledgeable on this area

[fervent lily]

Most likely the issue is a result of a file getting left behind unintentionally by the unisntall process. The installer looks for the netdata executable to detect existing installs. Assuming this is what’s going on, running command -v netdata in a shell should tell you where the offending file is, and removing that should make things work correctly.

Can you confirm if you system package manager sees a package called netdata installed on the system? That’s the other thing the installer checks to find existing installs.

That’s expected in some cases and just means the installer fell back to using a static build instead of native packages.

Auto-updates should work correctly.

Given that it fell back to a static build, it should be under /opt/netdata/etc/netdata.

I’m honestly not sure why it may have fallen back to a static build in this case.

Converting would require following the clean reinstall instructions in the reinstall documentation you linked above.

Hmm...

Do you happen to have a package installed called netdata-repo or netdata-repo-edge?

Yes, though the kickstart script should do that itself.

There are also packages in the Ubuntu and Debian repositories.

Those are probably what you’re seeing.

The installer itself has a check to make sure that it would pull in our packages instead of one of those from the distro repos.

Just a moment...

Can you run apt-get clean first and then try it?

I think this is an issue of repository metadata caching being done by APT.

Yes.

Hmm, that may actually just be an issue with APT...

It depends a bit on what type of install.

The problem is that that may choose any of three different install methods.

Most likely it’s a native install, in which case switching to a nightly build should just involve running apt install netdata-repo-edge, updating repository metadata, and then using apt to reinstall the agent.

And yes, we don’t really have this properly documented anywhere and ideally need to get around to doing so...

Ah, I keep forgetting that APT is weird and requires you to explicitly uninstall config files even if they are unmodified.

apt purge netdata-repo should fix this.

You’ll have to tell APT to explicitly downgrade because of how our nightly versions are handled (the nightly builds never ‘see’ patch version bumps from the stable releases).

Ironically, the fact that APT unconditionally preserves config files means that the easiest way to do this is apt remove -y netdata && apt install -y netdata.