I have been having this issue for a while. For some reason, my mouse pointer will randomly slow down at times which makes moving it annoying. I first suspected it might be CPU load, but when I check it, it seems to be fine. I'm not even using a mouse, but a simple trackpad. If anyone has any ideas as to what might be causing the issue, I am all ears.
I use KDE Plasma in case it's relevant.