I tested with every combination of being in spec, authed, with kicking players allowed and with moving to spectator enabled. It looked coherent in every case. Feel free to improve the logic, it feels like it could be more simple.

That would make things so much easier. You can have a look at the DDNet client, they've had that for quite some time already.