This is a common problem I encounter when connecting up during a skirmish game.
3 of the players are all connected to one another.
The fourth player is not connected to anyone.
None of the other three players are attempting to connect to the fourth player.
It just hangs there in this state until you cancel.