Probably the one thing in the game that I think is worthy of r/badUIbattles. It's super easy to accidentally delete items you don't want to.
My recommendation - have a dedicated rubbish bin on the UI where you can drag items to delete.
Combine this with improved stack UI/UX and you should in a few clicks pickup a single item, whole stack of items, half of the stack, etc and then just drop them on the rubbish bin.
Having a hotkey like Ctrl + RMB might be a good way to delete quicker, and maybe something like Ctrl + Shift + RMB to delete whole stack - but best sync this with stack pickup UI/UX systems.