I was finally able to reproduce #2669 @fokkonaut , even with the vanilla server and sometimes even without rcon login.
Actually it only happens with gcc 6.3 (on Debian 9) in release mode. Even in release with debug info there is no issue =\
I think I also found the reason for the errornous client id thing. The client id is correct but the server is still sending packets although the client is offline.
Happens for example when the server is still sending rcon stuff or votes
Added a ton of log messages... Result was that GetToken doesn't work when doing the heartbeat after rcon login. After fixing net_addr_comp it's working for me