Adding another 5-10 seconds would give people a little more time to chat or to rest a little between matches. Matches are 10 minutes, so an extra 10 seconds should not be that big of a deal between matches.
If 5 of those seconds were added after the vote is decided, it would give a chance for the leavers to go, which would help the sorting algorithm more consistently do a better job.