While trying to assign the movement to my right mouse button I've noticed that it doesn't work while assigning it to left (the default) or middle works. Apparantly the default "area selection" "Delete" binding clashes with that I've set it to middle for now.
Note: I don't expect these to clash as the area selection should only be active when I hold the modifier