DDraceNetwork - developer

DDraceNetwork

Development / developer

Development discussion. Logged to https://ddnet.tw/irclogs/ Connected with DDNet's IRC channel, Matrix room and GitHub repositories — IRC: #ddnet on Quakenet | Matrix: #ddnet-developer:matrix.org GitHub: https://github.com/ddnet

Between 2019-06-24 00:00:00Z and 2019-06-25 00:00:00Z

[ddnet-maps:master] 1 new commit

cb72878 M Airnade Ultra - ddnet-maps

[ddnet/ddnet] Issue closed: #1682 since newer versions projectiles dont appear on big maps

@snail

@archimede67 ty

np ^^

@snail if i randomly tag you do i also get a thank you?

[ddnet-web:master] 1 new commit

a1b5d9e More accurate donation info for RUS - def-

2

@n000b i guess that's a no

thank you

[ddnet/ddnet] Pull request closed: #1757 update desktop db after install

ty all

15:15

except ryozuki

mean

naem

@heinrich5991 I remember you talking about a spec for the http masters somewhere, is that correct?

@Learath2 not really, don't remember

Well does using a PATCH as a heartbeat sound reasonable to you?

I'd use a POST

16:02

patch sounds more obscure

16:03

@Learath2 make sure to include proper IPv6 support for these masters

I was thinking POST to list a server, then something like a PATCH with the current timestamp as a heartbeat

Why not trying instead of thinking?

Because I'd rather do it right once then trying 50 different variations

@Learath2 I'd just send a POST request every time, no difference between the first and later ones

16:29

(like currently)

But you will not be sure until you do it

@archimede67 APIs like these I prefer designing first, then implementing according to a spec

Which spec?

the one we think of

@heinrich5991 is that really efficient though, if we keep the packets smaller the server would be more equipped to handle attacks

16:30

@archimede67 the spec I prefer to design, as stated in the first half of the sentence

16:34

Also do you think we should keep it just ip:port? We could have the masters store more

I think we should store more

16:34

I liked this sample register request:

16:35

{
    "info": {
        "clients": [
            {
                "name": "heinrich5991",
                "clan": "",
                "country": null,
                "score": 123,
                "team": -1
            }
        ],
        "max_players": 64,
        "max_clients": 64,
        "passworded": false,
        "game_type": "DDRaceNetwork",
        "name": "DDNet RUS - Moderate [DDraceNetwork]",
        "map": {
            "name": "Sunreal",
            "crc": "0abcdef1",
            "sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
            "size": 12345
        },
        "version": "0.6.4, 11.2.1"
    },
    "port": 1234,
    "secret": "abcdef"
}

16:35

i.e. complete info that the masterserver can distribute

Yeah I had something similar in mind

16:36

The only difference I had in mind was to have the future messages be a PATCH

don't overthink that, we can add that later

with the secret and a timestamp

better get a working version first IMO

16:36

oh yea, a version is still missing

16:36

from that json

16:36

(so that lingering old requests don't override newer ones)

a timestamp or a counter should be fine for that

yea, counter I'd think

a unix timestamp would be better as we can use that to expire entries aswell

16:39

Also I'm thinking of implementing the masterserver in python, any objections to that?

no, can reimplement it in rust later

16:40

(if the need arises)

yeah, was also thinking rust later if we need the performance

we can't trust the client either way, so timestamp doesn't help there

16:40

(use flask!)

yeah already using flask

and write the server list to a static file to be served by a real http server

16:41

ok, so one complete https request with little payload data has 2.5kb of data, with data say 4kb

16:42

or let's say 3kb with data

16:42

1000 servers doing this, 3kb each, every 15 seconds (as we currently do), I get 200kb/s

that's quite managable

which seems fine (also, it should be less because we re-use https connections, so the 2.5kb overhead per connection should go away, yielding <50kb/s) (edited)

I had one more concern in mind, do you think we should account for horizontal scaling?

16:44

(i doubt we'd ever need it given we'd put this behind some run of the mill ddos protection service anyways)

16:45

e.g. should we support multiple masters?

I'd say no to horizontal scaling

16:45

I'd like multiple masters

"The HTTP PATCH request method applies partial modifications to a resource." i think it has nothing to do with a heartbeat

each server registers at each master, clients request one

why not just a GET request?

@Ryozuki it'd be modifying the "timestamp" in the server entry

because it modifies state, and that would be against the HTTP spec

POST for registering

GETs are supposed to never modify state

what means heartbeat?

the server says it's still alive

updting the info?

a message to show that the server is still alive

only alive?

16:46

hmm

well would also modify the data in this case

still i dont think patch fits

16:47

you want to make the master server be like a rest api?

yes

thats a cool idea

anyway, what do you think about my multiple masters proposal, @Learath2 ?

16:47

I think matricks originally proposed it ^^

I don't really see a problem with it, the increase of traffic for the server is negligible to register at all masters

16:48

I think we shouldn't hardcode the masters this time

16:48

maybe a request to /masters to get a list of all the masters

"The difference between PUT and POST is that PUT is idempotent: calling it once or several times successively has the same effect (that is no side effect), where successive identical POST may have additional effects, like passing an order several times." does sending a heartbeat has diferent effects over time?

meh, not sure I like that @Learath2

16:49

but I'd definitely like a masters.cfg, one url per line where the master servers are

its better heinrich way

16:49

that way u dont rely on one point

16:49

which can be ddosed

what I was proposing is a way to get that masters.cfg

e.g. https://master1.ddnet.tw/v1

if you can't reach the master, you just use your old masters.cfg

yea, I think self-update ought to be enough

ah

I don't know

16:50

what use-case do you have in mind, @Learath2 ?

e.g. we add a new master

but where whould u implement master servers? at ddnet level or at teeworlds 0.6?

16:50

will u make it backwards compatible?

it's unreasonable to expect users that can't figure out how to change their names to add a new master

yeah

yes. that'd be done with the update

16:51

like the updated client

oh, that's true, maybe letting the update handle it is a better idea

16:51

hmm, it's not quite as real-time

we specify https://master1.ddnet.tw/ to https://master4.ddnet.tw/ and that gives us some room to spare

also yeah, how general should I keep this?

16:52

also maybe some versioning

16:52

https://master1.ddnet.tw/v1

it would be cool if the master server can also provide custom endpoints for data such as total players, total amount of servers and amount of servers per gamemode etc

16:52

but thats secondary i guess

@Ryozuki yeah, I was thinking of that aswell

that way u make it someone who wants to interact weith masters not make so many calls

16:53

or whathever

@Learath2 yea, add a version to the top of the path

@heinrich5991 do we really need to output to a static file?

@Learath2 what lang will u use for the rest sv

I think we should do that. serving from database is a bad idea IMO

@Ryozuki flask python for now

hmm serving from database with a cache is better IMO

update it once per second or so

yeah we could configure nginx to cache a little

@Learath2 i think PUT fits heartbeat

16:56

@Learath2 no u would use memcached or smth like that, i would rely more in the server directly managing the cache than nginx

or I could output to a static file, and we could do some nginx configuration black magic to only serve one endpoint as a file and the rest goes to flask

but thats just me

@Learath2 that's not particularly hard to configure

16:57

so that'd be fine

16:57

I think

memcached also sounds like a decent idea, I'll just get a draft implementation for now

16:58

we can think about caching after I get the thing working

managing the cache urself directly allows u to renew it when u get a updated resource btw

16:58

nginx cant know much

yeah, it does sound more reasonable that we should be managing the cache if we don't write out to a static file

16:59

but with a static file we can just update that

idk about flask, i would personally use nodejs but i know u hate js xd

i'm not sure whether nginx is optimized enough to cache a non-changed file

im making quite a big rest api lately :p

17:00

idk how nginx cache works

what are you using? express.js?

the framework i use uses it yes

17:00

im using https://nestjs.com/

NestJS - A progressive Node.js web framework

NestJS is a framework for building efficient, scalable Node.js web applications. It uses modern JavaScript, is built with TypeScript and combines elements of OOP (Object Oriented Progamming), FP (Functional Programming), and FRP (Functional Reactive Programming).

I was thinking node at the beginning, but I didn't want to introduce a new language to ddnet and didn't want the bloat of npm and all that stuff

(with typescript)

17:01

@Learath2 its easy to implement cache and ratelimiting with it

17:01

btw u will also want rate limiting

I'll let nginx handle the rate-limiting

well yeah, but u wont be able to implement rate limiting per endpoint

nginx has zones for rate limiting

:o

17:03

does this rest api need some auth

i think they are called location blocks instead of zones tho

17:03

no need for auth

well if u need help ill be glad to help

17:04

:p

17:04

i used flask time ago too

I have a feeling you'd use JWTs

im using jwt

17:04

xd

yeah, knew it, you are the kind to use the fancy new stuff

i love it

17:04

monkaS

I usually just go for good old tokens with a key value store

¯\_(ツ)_/¯

17:06

im also using a ORM but i end up making lot of raw sql calls cuz it just cant cover my complexity sadly

ORMs are good if you are doing bog standard stuff

yeah

JWTs are good if you have multiple backends that use the same auth, great for microservice architectures

17:08

as a session token though, not really useful, bloated, and creates a potential security issue

ninja

Image attachment

sql

pepeH

@archimede67 arent u a web dev

making your own social network?

@Learath2 kinda

17:09

something weeb

ye sql worst thing ever

sql is nice

no

that's a pretty weak ORM if it cant do those, what is it?

nosql better

@Learath2 well i probs cant do it

17:10

but i cant bother waste 1 hour

17:10

to find out how

17:10

xd

17:11

i mainly relly on the orm to have easy objects and it creates me all the tables and relation tables

17:11

i probs can do it*

also migrations, if you have those

17:11

makes it a breeze to extend your schema

not yet since i dont have prod yet

17:12

yeah it has migration support

wait

17:14

@Learath2 hates js? xd

he hates new stuff

17:14

and systemd

xd

xd

17:15

oof

17:15

Image attachment

it doesnt exist

hm

well it does but its a domain that probs points to ddnet.tw

17:15

and its not added to the cert

ye

I like typescript better then js

i only code in typescript @Learath2

oh bois unicode

18:43

how to get rid of ^M ? -.-

18:43

Embedded image

18:43

i tried dos2unix.exe and unix2dos.exe nothing worked

18:43

delteting the line rewriting it

18:44

if i edit the file it messes it up

18:45

even vim from git bash messes it up

this isn't unicode btw, likely plain ascii

18:51

which file @ChillerDragon

stats.h

18:52

vanilla master

18:52

Embedded image

18:52

any idea? I dont want to switch to linux and redo my changes -.-

@ChillerDragon it's probably fine

18:57

the file has windows file endings for some weird reason

but then unix2dos.exe shoudl do the thin..

18:57

ah

18:57

then i get changes in all linex

hum?

maybe i have to edit it with notepad editor from windows? troll

troll

send me the file in the small-diff version

19:02

[sent via PM]

19:02

@ChillerDragon file looks good, just commit it like that I guess

:c

19:02

looks messed up

19:03

if my git gets triggered github git will show this as well

nah

19:03

not necessarily

19:03

I actually think I never saw these on github

grr

19:05

i want to atleast fix the #endif but git reset --hard path doesnt work

19:05

how to only reset one file?

19:09

are yo kidding me i have the bug on linux as well

19:10

what is wrong with this file

flag.cpp also has wrong endings

it's probably not a bug

19:10

it's just that the file has windows line endings

19:10

git probably only shows these ^M on changed lines

19:11

^M is \r

it triggers me

it's not shown on github

19:12

make a separate PR updating the windows line endings to unix ones

justatest

19:12

ya was thinking about that

git converts them to unix endings usually

19:18

iirc

i think thats something you can choose on windows git installation

19:30

(YEs i UsE wiNdoWs justatest

justatest

)

windows is good

u

ikr

19:30

no kek emoji

windows is bad

20:03

monkaS

do you have your autocrlf properly set?

what ever that is pr is published and github didnt show it

21:00

also tested on linux so i blame the file not my setup

21:00

tested 4 editors and 2 operating systems

blaming the file lmao

23:10

yes, the file is broken

23:10

better yet, it has a grudge against you

pepeH

Ryozuki: did you watch the new episode of aot?

Exported 274 message(s)