I get it. It gets REALLY annoying when people constantly leave your matches. But the problem is not them anymore, its the penalty.
Getting a leave penalty is the SCARIEST thing in the entire game. I had closed the game shortly after winning / losing, and constantly checked if I somehow got a leave penalty.
The problem with the penalty is how hard it is to get out of it. In order for you to leave a penalty, you have to win two (more or less) matches with players who have also left a game. Now this doesn't sound that bad on paper.
But either the players leave your matches or there are no players in your lobbies or very few.
An easy solution is to scrap that whole thing and just make you wait an hour or a minute.