If you did wait till the other player was finished, before you passed the token to the next player, that still seems that it would cut down on the game-time a lot.
You'd have 2 players going at once, instead of one.
You'd have 2 players going at once, instead of one.