[quakenet] <Dune> "In Teeworlds before 0.6.5, connection packets could be forged. There was no challenge-response involved in the connection build up. A remote attacker could send connection packets from a spoofed IP address and occupy all server slots, or even use them for a reflection attack using map download packets."
19:25
[quakenet] <Marcel> and what about the ddos problem
19:25
[quakenet] <Dune> that is the ddos problem
19:25
[quakenet] <Dune> another attack angle is the server info packets
19:26
[quakenet] <Dune> but 0.6.5 fixes the vulnerabilities at least partially, iirc
19:26
[quakenet] <Dune> upgrade your servers (and clients).
19:27
[quakenet] <Marcel> they still could send tons of those packets and by that overload the master server
19:27
[quakenet] <Marcel> the master server right now is not working at all
19:28
[quakenet] <Dune> upgrade to 0.7
19:28
[quakenet] <Dune> masterserver works pretty well there
19:29
[quakenet] <Marcel> well it probably won't take long until they're getting ddosed
19:30
[quakenet] <minus> huh, was 0.7 released?
19:30
[quakenet] <Dune> yes minus
19:30
[quakenet] <Dune> a week ago
19:30
[quakenet] <minus> no one updated the website tho
19:30
[quakenet] <Dune> yeah
19:30
[quakenet] <minus> still 0.6.5 there
19:30
[quakenet] <Dune> take it as a public beta, 0.7.1 should come soon
19:30
[quakenet] <Marcel> the download is still not available tho
19:30
[quakenet] <Dune> you can get it on the github releases
[quakenet] <Dune> 8[22:32]heinrich5991: Marcel: yes, moving to https for master servers is the plan
20:55
[quakenet] <Marcel> so what's your current state
20:58
[quakenet] <heinrich5991> Marcel: had some working parts already
20:59
[quakenet] <heinrich5991> do you have some experience with this? I'd love to have someone to discuss this with :)
21:00
[quakenet] <Marcel> yea, got some. what language are you using server side
21:02
[quakenet] <heinrich5991> so far I just used a simple python http end point for the registering and a static file (regularly updated and served by nginx behind cloudflare) for the server list
21:05
[quakenet] <Learath2> There is a huge flood of cou2 packets to master4.teeworlds.com which makes it pretty much toast. Now is it a fault of the protocol I'm not sure as siz2 packets are small enough, any challange response scheme would require a much bigger challange packet
21:06
[quakenet] <heinrich5991> 0.6.5 challenge packet is huge
21:06
[quakenet] <heinrich5991> (512 bytes)
21:06
[quakenet] <heinrich5991> 0.7.0 not so much IIRC
21:06
[quakenet] <heinrich5991> would require breaking the protocol again ^
21:06
[quakenet] <heinrich5991> ^
21:06
[quakenet] <Marcel> and does the http request on the client side already work?
21:06
[quakenet] <Learath2> 0.6.5 one is the client challange though right?
21:06
[quakenet] <heinrich5991> Marcel: yes
21:06
[quakenet] <Learath2> I meant for the masterserver
21:06
[quakenet] <heinrich5991> Learath2: not needed for http masters
21:06
[quakenet] <Learath2> Yeah but that's a lot of work to get working
21:07
[quakenet] <Marcel> how do the master servers communicate with each other?
21:07
[quakenet] <Marcel> right now
21:07
[quakenet] <heinrich5991> missing parts are proper server registration on the game server side (and also a strategy for this) and a legacy server scraper to make the old servers compatible with the new system
21:07
[quakenet] <Learath2> They currently don't iirc
21:07
[quakenet] <heinrich5991> Marcel: currently not at all, clients fetch all server lists, game servers register at one
21:07
[quakenet] <heinrich5991> I want to change it (as matricks has advised) to game servers register at all masters, clients fetch one
21:08
[quakenet] <Marcel> hmm
21:09
[quakenet] <heinrich5991> what do you think about that?
21:10
[quakenet] <Learath2> I don't think it's the best idea tbh, it's very possible to knock out a master with the weak servers we have
21:10
[quakenet] <Marcel> what about having a sql server and the master servers syncing with it maybe every 30 seconds
21:11
[quakenet] <Learath2> It centralizes the system but maybe light caching servers that report back to a main server. Server goes through all masters until it finds one. Client does the same
21:12
[quakenet] <heinrich5991> Learath2: that's why we put the masters behind cloudflare
21:12
[quakenet] <heinrich5991> if a master is knocked out, the client tries the next one
21:12
[quakenet] <Marcel> i don't think that the client should know all the master servers. i think that it'd be better if there's one address and that cloudflare at the end decides which server to choose (i think they have a service for that)
21:13
[quakenet] <heinrich5991> I don't want to make cloudflare mandatory for teeworlds
21:13
[quakenet] <Learath2> Yes but we still have the original issue, you can knockout a master and the servers on it are practically unavailable
21:13
[quakenet] <heinrich5991> they could go out of business one day
21:13
[quakenet] <Marcel> they probably won't, they are so big and still growing
21:13
[quakenet] <Marcel> cloudflare has a really good anti ddos protection and caching system
21:13
[quakenet] <heinrich5991> yes. but this is an open-source game, it shouldn't depend on such services by design IMO
21:13
[quakenet] <heinrich5991> yes
21:14
[quakenet] <heinrich5991> I want to use them
21:14
[quakenet] <heinrich5991> but I don't want to make another implementation impossible
21:14
[quakenet] <Learath2> We could run a load balancing public facing server. Don't need cloudflare for that anyone provides that
21:14
[quakenet] <heinrich5991> so putting the masters behind cloudflare is a thing I want to do
21:15
[quakenet] <Learath2> heinrich5991: why do you think it's better for the servers to register at all masters?
21:15
[quakenet] <heinrich5991> so masters are independent
21:15
[quakenet] <heinrich5991> and if one isn't reachable by a client, it's still okay
21:16
[quakenet] <Marcel> they still'd be independent with a sql server
21:16
[quakenet] <Learath2> no they wouldn't be, they'd be dependant on a central sql server
21:16
[quakenet] <Dune> i'll go to sleep now, my people need me
21:16
[quakenet] <heinrich5991> good night Dune :)
21:16
[quakenet] <Learath2> night Dune :)
21:16
[quakenet] <Dune> i hope you'll work out a layer of protection for the masterservers
21:16
[quakenet] <heinrich5991> nice to see you here agiain
21:16
[quakenet] <Dune> sorry I can't help much
21:17
[quakenet] <Dune> good night : )
21:17
[quakenet] <Marcel> yep, but if the sql server is hidden then it also wouldn't get ddosed
21:17
[quakenet] <Dune> nice to see all that activity, yeah
21:17
[quakenet] <Dune> ;-)
21:17
[quakenet] <Marcel> gn
21:17
[quakenet] <heinrich5991> okay, but if all game servers register with all masters, we don't need a single point of failure, do we?
21:17
[quakenet] <Learath2> Marcel: no need for it though, it creates unnecessary complexity. I think I'm okay with servers registering at every master
21:18
[quakenet] <Marcel> the problem with that is that the master servers are currently hardcoded and there's no option to add additional ones
21:18
[quakenet] <Learath2> We should let the client fetch from as many masters as it needs though. So that people can run their own e.g.
21:19
[quakenet] <Learath2> Well we could run lightweight proxies on master{1,4}.teeworlds.com to translate to the new system until the legacy usage drops below a certain %
21:19
[quakenet] <rand> what about DHT ? flying away
21:20
[quakenet] <rand> http will be nice
21:20
[quakenet] <Marcel> that's a good idea
21:20
[quakenet] <Learath2> Heh think matricks suggested DHT once upon a time
21:21
[quakenet] <heinrich5991> that's not a solution that will work in one week
21:21
[quakenet] <heinrich5991> we'd better use one that might work in one week
21:21
[quakenet] <Learath2> heinrich5991: so how far are you anyways? anything we can help with?
21:21
[quakenet] <heinrich5991> well, that was based on ddnet back then
21:22
[quakenet] <heinrich5991> so currently I'd need to add the curl library to teeworlds 0.6.5, with bam and cmake
21:22
[quakenet] <Learath2> oy isn't even here, does he even accept prs now?
21:23
[quakenet] <rand> small one at least
21:23
[quakenet] <rand> for 0.7
21:23
[quakenet] <heinrich5991> it seems that I'm kinda in charge of 0.6.5 right now
21:23
[quakenet] <heinrich5991> I would port it to 0.7 afterwards, obviously
21:24
[quakenet] <heinrich5991> (after maybe adding cmake to 0.7)
21:24
[quakenet] <rand> :D that would be awesome
21:25
[quakenet] <minus> why cmake when you can have meson
21:25
[quakenet] <rand> what is meson ?
21:26
[quakenet] <heinrich5991> minus: because I know cmake
21:26
[quakenet] <heinrich5991> it seems industry-standard, and it works pretty well for ddnet
21:26
[quakenet] <minus> you can learn meson :D
21:26
[quakenet] <heinrich5991> we do cross-compilation to everything from linux
21:26
[quakenet] <minus> cmake works, yes
21:26
[quakenet] <heinrich5991> it's pretty awesome
21:26
[quakenet] <minus> it does not work well
21:27
[quakenet] <heinrich5991> what do you mean? the result or that you have to code in a language that you don't like?
21:27
[quakenet] <heinrich5991> the cmakelists I wrote for ddnet seem to work pretty well
21:27
[quakenet] <heinrich5991> I don't like the language, but that's okay for the result
21:27
[quakenet] <Marcel> what about if the client does a request to masters.teeworlds.com and then fetches all the servers from the master servers listed there
21:27
[quakenet] <Learath2> We even have tests nowadays \o/
21:28
[quakenet] <heinrich5991> yes. I don't think anyone but me has added tests(?)
21:28
[quakenet] <minus> i find it hard to impossible to produce static builds with it, and example programs of libraries produced in the same project
21:28
[quakenet] <heinrich5991> Marcel: so clients should connect to multiple masters you say?
21:28
[quakenet] <Learath2> heinrich5991: I did in a couple branches of mine that never got any review so I gave up on them :P
21:28
[quakenet] <rand> Marcel: what if masters.teeworlds.com got ddos ?
21:29
[quakenet] <Marcel> that's the problem
21:29
[quakenet] <Learath2> Marcel: another single point of failure that's not necessary
21:30
[quakenet] <Learath2> We should instead use masters.cfg. (not the way we have now where it just resets it for some reason)
21:30
[quakenet] <rand> a solution such that it's easier to get a ddos protection is fine, i guess that the threat is weaker as long as it's possible
21:30
[quakenet] <rand> that's why http alone is fine, with or without cloudfare
21:30
[quakenet] <Learath2> masters.cfg initialised with master{1,4}.teeworlds.com; client instead gets the list from one place through http
21:31
[quakenet] <Learath2> server also goes through masters.cfg to register at all masters and heartbeats through http
21:31
[quakenet] <Learath2> (maybe also add a PoW requirement for registering or atleast a challange/response)
21:34
[quakenet] <Marcel> what if the client would do a request to masters.teeworlds.com if the masters.cfg is missing and write them to the file. if the website was not accessable then it could use the hardcoded ones
21:34
[quakenet] <Learath2> What does that gain us?
21:35
[quakenet] <Marcel> the ability to expand
21:35
[quakenet] <heinrich5991> the ability to expand to what? to more than 4 master servers?
21:36
[quakenet] <heinrich5991> 4 master servers should be enough for anyone – bill gates
21:36
[quakenet] <Marcel> it's always better to have the chance than forcing everyone to update their clients and servers
21:36
[quakenet] <heinrich5991> you also add untested code paths
21:37
[quakenet] <heinrich5991> so that's not a "pure" upside
21:37
[quakenet] <heinrich5991> we haven't seen a reason to have more than 4 masters in 10 years
21:37
[quakenet] <heinrich5991> do you think that will change rather than something else we did not foresee?
21:38
[quakenet] <Learath2> heinrich5991: should we maybe add the version to the url btw? So further updates are relatively easier?
21:40
[quakenet] <Marcel> that's just something i've learned in the past few years. products should be prepared for their growth, you also thought that udp'd be good enough.
21:40
[quakenet] <heinrich5991> I doubt that anyone who thought about that thought that
21:41
[quakenet] <heinrich5991> we've had a relevant issue open on github for years
21:41
[quakenet] <heinrich5991> the problem is that you need to be aware of reflection attacks et cetera
21:41
[quakenet] <heinrich5991> if you aren't adding random things like "having the possibility to add more udp masters" adds nothing
21:46
[quakenet] <Marcel> matricks also probably thought that it'd good enough to write the system as it is now. we can see now what we've got by that. but whatever if you are sure about that
21:47
[quakenet] <heinrich5991> do you think that this was designed by someone who was aware of reflection attacks?
21:48
[quakenet] <Marcel> i don't know, it could also be that he didn't thought anyone would abuse it
21:53
[quakenet] <Marcel> do you want to write the masters in python? i could write them in php if you want
[quakenet] <heinrich5991> I like the language better, and I think not only because I know it better
21:55
[quakenet] <Learath2> What will you use as the http library? flask?
21:56
[quakenet] <Marcel> the is probably true, but php is still being widely used and is running on almost every website
21:56
[quakenet] <kebron> What about golang? Would probably be faster
21:58
[quakenet] <heinrich5991> kebron: the good thing is: you can replace it, it's running on the master
21:58
[quakenet] <heinrich5991> a quick, working PoC is fine for the start
21:58
[quakenet] <kebron> Yeah true
21:58
[quakenet] <heinrich5991> otherwise you could indeed rewrite it in go, or even rust :P
21:59
[quakenet] <heinrich5991> but we don't have to release a new version for that, because it's on our side, not the client's
21:59
[quakenet] <rand> as long as the protocol is well defined :}
21:59
[quakenet] <heinrich5991> yes
21:59
[quakenet] <rand> whatever the language
22:01
[quakenet] <Marcel> do you want to go for http or https
22:01
[quakenet] <Marcel> i'd stick to https as it already has a handshake
22:04
[quakenet] <heinrich5991> yes, https
22:04
[quakenet] <heinrich5991> http has a handshake, too
22:04
[quakenet] <heinrich5991> tcp, namely
22:06
[quakenet] <Marcel> oh yea right
22:09
[quakenet] <Marcel> soo who should write the master
22:10
[quakenet] <heinrich5991> the master was 50 lines of code or so and is already written IIRC
22:10
[quakenet] <heinrich5991> the things that are missing are: adding libcurl to teeworlds 0.6.5 (both with bam and cmake), and write the registering code
22:10
[quakenet] <heinrich5991> and the backward compatibility for servers not supporting the http masters
22:13
[quakenet] <Marcel> did you add a version number
22:13
[quakenet] <heinrich5991> where?
22:14
[quakenet] <Marcel> to the protocol
22:15
[quakenet] <heinrich5991> I wanted to add one to the url, but it's not really necessary
22:15
[quakenet] <heinrich5991> you could always add one later. the only benefit of adding it right now is that the urls will look more symmetrical to the old ones
