DDraceNetwork - developer

DDraceNetwork

Development / developer

Development discussion. Logged to https://ddnet.tw/irclogs/ Connected with DDNet's IRC channel, Matrix room and GitHub repositories — IRC: #ddnet on Quakenet | Matrix: #ddnet-developer:matrix.org GitHub: https://github.com/ddnet

Between 2017-07-16 00:00:00Z and 2017-07-17 00:00:00Z

<eeeee> what if instead of extending the protocol to allow servers to provide http map urls we started crawling and hosting all maps from public servers?

<Edible> what if you used the browser to launch teeworlds!?

01:13

<Edible> what if im losing my hair because there is a mini runway construction going on in my head?

02:43

where is irc (edited)

[ddnet-maps:master] 1 new commit

c572c02 daily update - def-

eeeee: That would be fine for me. Providing the entire map downloads from TW would not be a large amount of bandwidth. We might need a bit more storage. heinrich5991 already has all the maps: https://heinrich5991.de/teeworlds/maps/maps/

[ddnet:master] 1 new commit

a736a27 No need for writing to tmp config file anymore - def-

<deen> eeeee: But it might be a bit dangerous to provide maps like that. Someone can fake the CRC relatively easily I assume and end we end up with duplicates

<eeeee> i'd mitigate that with the (unpopular) measure of manually whitelisting ips (or domain names) which would get crawled

07:50

<eeeee> rationale being what you said and other attack vectors (spamming the crawler with lots of huge maps) which we don't have time to deal with

07:51

<eeeee> apart from not being decentralized and hipster enough, i think whitelisting would work nicely because honestly not many servers release new maps these day

07:51

<deen> yeah, we'd need something like that I guess. For a start it's probably good enough to check if a map with same name and same crc is uploaded and instead of replacing the old one send me a mail

07:51

<deen> well, but someone has to maintain the whitelist, that's a bit annoying

07:52

<eeeee> works for banning on masters in #teeworlds

07:53

<eeeee> how much traffic do http map downloads generate currently?

07:53

<deen> 3 GB / day

07:54

<eeeee> do you expect it to be more if we deployed tw-wide?

07:54

<deen> So with support for all maps I'd guess 6 GB / day

07:54

<deen> and I even added a few common maps to our maps server manually

07:54

<deen> the ones that appeared in logs of often being requested

07:55

<deen> Usually block and gores

07:55

<deen> I guess we could just use heinrich5991's script

07:55

<deen> Or we just make ddnet client download from his server directly :D

07:58

<eeeee> he'll probably start injecting ads into the maps to recover the hosting costs :>

07:58

<deen> hahaha

07:59

<deen> oh, and right now our maps are transferred over http instead of https

08:00

<deen> updates over https seemed more important, not sure if we want to rethink that. I'll try adding regular ssl cert support for now

08:02

<deen> because right now only allow my custom ca since i didn't want to trust the hundreds of default CAs

08:05

<deen> heinrich5991: fix your permissions please, i can't download new files on your maps server

08:05

<deen> Example: https://heinrich5991.de/teeworlds/maps/maps/Epix_f7c5b6db.map

08:06

<deen> from April 2017, but the one from January works

<deen> or should we just use letsencrypt for updates as well?

08:16

<eeeee> i think we should use whatever is more convenient

08:17

<eeeee> if someone can fake certs it's unlikely ddnet autoupdate would be their first choice to attack

08:17

<deen> all of our high profile players though!

08:17

<deen> prime nsa targets

08:17

<eeeee> some of them really should be, with all the hundreds of gbits of ddos they're generating

08:19

<deen> switching over to regular certs is possible, but some work

08:20

<deen> and then we have the problem that we depend on what CAs our users' systems trust

08:20

<eeeee> i forgot, why isn't map download server using cloudflare?

08:20

<deen> We could do that

08:21

<eeeee> and i thought letencrypt is trusted pretty much everywhere now

08:21

<deen> I don't like cloudflare personally

08:21

<eeeee> or is it trusted because browser vendors use their own ca lists instead of the system ones?

08:21

<deen> on winxp I'm not sure

08:22

<deen> and on old debian versions letsencrypt will probably also not work

<eeeee> can you bundle a cert for letsencrypt root ca then?

08:32

<eeeee> that doesnt make sense though, does it

08:32

<deen> for the updater or map download?

08:32

<eeeee> you can have your cert signed by letsencrypt and still bundle that with the client

08:35

<eeeee> for both? it seems like they don't expect much changes in their chain of trust: https://letsencrypt.org/certificates/

08:38

<eeeee> and you could still keep your own ca, so in case letsencrypt implodes would be able to autoupdate clients to another ca.

08:38

<deen> that sounds complicated

08:39

<deen> I'll just stay with my own CA for updates

08:39

<deen> for map download I will enable using https:// with system CAs for now, but by default keep using the http://maps.ddnet.tw

08:40

<eeeee> sure, should be fine

[ddnet:master] 1 new commit

c7c9c3c Possible to use https:// for map downloads, but... - def-

<Learath2> well bundling the LE root would mean that we trust the initial download was secure no?

08:54

<Learath2> think that's a no go

08:59

<deen> Learath2: can we use the system CAs and additionally add another CA?

09:00

<Learath2> i'll have a check at the API

09:04

<deen> what happens when a root ca becomes compromised? all OSes and browsers manually remove it from their CA list and add the new one?

09:04

<Learath2> deen: nope, you can either change the path that holds all the certs or give a path to the ca bundle

09:04

<deen> well, that's bad

09:05

<deen> So we're going to vendor-lock ourselves into LetsEncrypt?

09:05

<Learath2> deen: if a root ca becomes compromised i think its a horror show

09:06

<Learath2> that's why they only issue intermediary certs with the root iirc

09:07

<deen> and about cloudflare, any opinions?

09:09

<Learath2> well i don't have anything against it but i'd love to hear why you don't like it

09:09

<deen> I don't like the internet becing only usable if you're under the supervision of google, amazon, facebook, cloudflare. but we could try enabling it and check how the map download times change

09:09

<deen> since I have quite some stats now for map download times

09:10

<deen> So I will lock us into letsencrypt + my custom ca

09:12

<deen> can curl tell us what server we're connecting to?

09:13

<deen> or I have to str_comp_nocase_num against (https?://)?maps.ddnet.tw?

09:14

<deen> oh, and a / at the end or someone will use maps.ddnet.tw.myserver.com

<deen> We have to switch to our 3rd update subdomain for compatibility now

09:31

<deen> I'm curious if you can update super-old client versions all the way to the most recent one

<Learath2> deen: i'll try it after i grab breakfast :)

09:42

<Learath2> we could do a gettaddrinfo to get the server we are connecting to, if that's what you mean

09:47

<deen> We have > 5 GB of ddnet versions lying around on the server and that's after I removed most of the old subversions already

09:49

<deen> I downloaded DDNet 3.6, the first with auto-updater. It just downloads the same files over and over and crashes finally, haha

09:50

<deen> ah, that works correctly, but the old DDNet version links against libssl.so.0.9.8, that's why it can't start

09:53

<deen> we can't update libwinpthread-1.dll properly since we had no dll update support on old versions

09:53

<deen> and we need a different one for 32/64bit of course

[ddnet/ddnet] New tag created: 10.6.6

10:00

[ddnet:] 5 new commits

c9b8711 Also trust Let's Encrypt Root CA - def- 191eefb Only trust our own custom-selected CAs for our ... - def- e436498 Use https for map downloads by default - def- 4a8f3b0 Use new update server that runs on Let's Encrypt - def- b05693e Version 10.6.6 - def-

[ddnet:master] 5 new commits

c9b8711 Also trust Let's Encrypt Root CA - def- 191eefb Only trust our own custom-selected CAs for our ... - def- e436498 Use https for map downloads by default - def- 4a8f3b0 Use new update server that runs on Let's Encrypt - def- b05693e Version 10.6.6 - def-

[ddnet-web:master] 1 new commit

87a8eee Version 10.6.6 - def-

<deen> Learath2: oh, and with cloudflare you lose e2e encryption

10:36

<deen> Ah shit, and Cloudflare would use their own cert, so now that we're locked into letsencrypt we can't do that, haha

10:36

<deen> "Cloudflare's free Universal SSL, which does not work with Windows XP (sp3) systems"

Anyone have heinrich's script to scan for new maps?

Map download statistics in ms: count: 349627 min: 0.0 max: 255.632 mean: 1.221531861097683 variance: 24.18211954962241 stddev: 4.91753185547612 skewness: 13.87720594085525 median: 0.277 20th perc:0.09 40th perc:0.191 60th perc:0.422 80th perc:0.97 90th perc:1.951 95th perc:4.305

11:15

Someone needed 4 minutes to download a map once, average is 1 second and 80% get their map in < 1 sec, 90% in < 2 sec, 95% in 4 sec

[ddnet-web:master] 1 new commit

aaea807 Add donation - def-

@deen ddnet forum is rip

i know, updating server

[ddnet-web:master] 1 new commit

0ac7035 Update phpBB to version 3.2.0 - def-

16:57

[ddnet-web:master] 1 new commit

0c43b06 Improve search bar location a bit - def-

[ddnet-web:master] 1 new commit

8350cdf More consistent margins in forum - def-

[ddnet-web:master] 1 new commit

22d0e45 Further phpBB fixes - def-

[ddnet:master] 1 new commit

a30323d oops - def-

Configuring curl with x86_64-w64-mingw32 takes forever, for example this hangs for a few minutes for me: checking types of args and return type for send... SOCKET,const char *,int,int,int

[ddnet:master] 5 new commits

22c3c13 Turns out we also need Intermediate Certificate... - def- f94f567 New curl and openssl versions - def- b2d86d1 Add missing mysql include files - def- 4a13d47 Use update4.ddnet.tw... - def- 5785267 Version 10.6.7 - def-

[ddnet:master] 1 new commit

bf6c91a Oops, update4 - def-

[ddnet:master] 1 new commit

b1b4aa7 Windows build fixes - def-

[ddnet:master] 1 new commit

ed3b3f9 Further build fixes - def-

22:34

[ddnet/ddnet] New tag created: 10.6.7

22:39

[ddnet:master] 1 new commit

97fdc83 curl-4 on windows - def-

[ddnet-scripts:master] 1 new commit

b02df5c DDNet lib update - def-

[ddnet-web:master] 1 new commit

58c2098 Version 10.6.7 - def-

Exported 112 message(s)