DDraceNetwork - developer

DDraceNetwork

Development / developer

Development discussion. Logged to https://ddnet.tw/irclogs/ Connected with DDNet's IRC channel, Matrix room and GitHub repositories — IRC: #ddnet on Quakenet | Matrix: #ddnet-developer:matrix.org GitHub: https://github.com/ddnet

Between 2021-11-04 00:00:00Z and 2021-11-05 00:00:00Z

i have a suggestion to remake the rankpoint system. right now there are big differences between maps and the rankpoints they give. for example its probably very hard to get top10 on multeasymap but when you finish a harder map quickly you can get rankpoints evenw hen you didnt really speedrun the map. thats why i think its a good idea to use the % of ranks that were recently added. lets say you get rankpoints if youre within the top 3% of all ranks on that map. this would motivate more players to get a better average rank without having to push to top10. after all... you can be rank 11 on every map and have 0 rankpoints

2

2

[ddnet-maps:master] 1 new commit

ecbb7ae M Luna - ddnet-maps

yo @deen this is me begging you to not force push anymore to ddnet master :) i have like 2 active forks merging in irregular intervals and it would cause me big pain in the ass

yikes what do i do when client launch segfaults but running with gdb just hangs it? :C

fixed it with printf driven development :D

09:55

did anyone of you ever try to configure vscode with cmake? I am a bit annoyed by the greyed out code because vscode does not know about the cmake options i use

09:56

image.png

Image attachment

chillerdragon

did anyone of you ever try to configure vscode with cmake? I am a bit annoyed by the greyed out code because vscode does not know about the cmake options i use

you can configure it probs

10:17

the cmake command

10:17

what vscode probs does is generate compile_commands.json

10:17

but it depends on the flags u pass to cmake

10:18

for dev u should enable -DDEV and any other feature u want

10:19

Image attachment

10:19

maybe

10:19

https://vector-of-bool.github.io/docs/vscode-cmake-tools/settings.html#conf-cmake-configuresettings

10:20

Image attachment

chillerdragon

yo @deen this is me begging you to not force push anymore to ddnet master :) i have like 2 active forks merging in irregular intervals and it would cause me big pain in the ass

did I?

@deen: no worries mate you did not :) just asking you to keep it that way

10:38

@Ryozuki: yea tried that it affected the build but not the syntax hightlight

u have to reconfigure

10:38

nobo

10:38

ctrl shift p

10:38

cmake: configure

oh ok lemme try

10:39

did it work for u?

lol im not a normie

10:39

i use neovim

ok sir

greenthing

omagwd it works how did it not work the first time

11:04

thanks ryo babe for motivating me to try again

11:05

image.png

Image attachment

11:05

but oof can we get some gitignore? :D

echo ".cmake/" >> .git/info/exclude (edited)

11:06

ur a nobo

ik

11:10

but that diff

[ddnet/ddnet] Pull request opened: #4290 Ignore vscode cmake extension files

[ddnet:master] 2 new commits

eafc433 Ignore vscode cmake extension files - ChillerDragon 6f86647 Merge #4290 - bors[bot]

11:43

[ddnet/ddnet] Pull request closed: #4290 Ignore vscode cmake extension files

@Ryozuki: here u go mozilla dudes accepted it https://addons.mozilla.org/en-US/firefox/addon/github_forks/

GitHub Forks – Get this Extension for

Firefox (en-US)

Download GitHub Forks for Firefox. https://github.com/ChillerDragon/GitHubForks Find the best fork of a github repository quickly by browsing a rich representation of the network tab.

Jupstar ✪ do you compile with llvm CFI?

13:17

LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e., -Clto).

13:17

i discovered this cuz rust added it https://github.com/rust-lang/rust/pull/89652 xd

13:17

Clang includes an implementation of a number of control flow integrity (CFI) schemes, which are designed to abort the program upon detecting certain forms of undefined behavior that can potentially allow attackers to subvert the program’s control flow. These schemes have been optimized for performance, allowing developers to enable them in release builds.

13:18

-fsanitize=cfi for clang

13:18

and requires lto

[ddnet:15.6-rc3] 4 new commits

01d097c Version 15.6 - def- 6fc198a Update language files for 15.6 release - def- 6e4221b Translate German - def- 74a499b Fix pickup network clipping - fokkonaut

13:19

[ddnet:15.6] 17 new commits

01d097c Version 15.6 - def- 6fc198a Update language files for 15.6 release - def- 6e4221b Translate German - def- 74a499b Fix pickup network clipping - fokkonaut b4ea62a Use correct (larger) m_NumSwitchers bound - trml be65574 update catalan - edg-l 8252f46 Update spanish.txt (#4256) - n0Ketchp 4e698d3 Update spanish.txt - n0Ketchp 3dc733c Update spanish.txt - n0Ketchp d62ca7f Update spanish.txt - n0Ketchp 74d314d Update russian.txt - NikGreens 8ed1a93 Update russian.txt - NikGreens 8ea1655 Update russian.txt - NikGreens 642eaa4 Update brazilian_portuguese.txt - rffontenelle 47e2c4c Add exception id to TeamKill - Jupeyy 0000d75 Reset selection, if gui element was changed outside of DoEditBox - Jupeyy 160e02b Fix characters disappearing (fixes #4285) - fokkonaut

13:19

[ddnet-web:pr-15.6] 1 new commit

5608338 Version 15.6 - def-

new version poggers

13:19

poggers

That is a massive release :D

[ddnet/ddnet] Pull request closed: #4219 Version 15.6

15.6: the memory safety update

13:26

monkalaugh

No

13:27

those changes are not in 15.6

oh

since they were not broken by 15.6 changes I think

[ddnet/ddnet] Issue closed: #4285 Player disappears on Divinum V2

13:28

[ddnet:master] 2 new commits

ae6bb97 Fix characters disappearing (fixes #4285) - fokkonaut 29c376c Merge #4288 - bors[bot]

13:28

[ddnet/ddnet] Pull request closed: #4288 Fix characters disappearing (fixes #4285)

If I keep adding new changes into 15.6 we need more and more rcs

yeah

[ddnet:master] 1 new commit

1bb21e1 Version 15.6 - def-

13:28

[ddnet/ddnet-web] Pull request closed: #145 Version 15.6

13:28

[ddnet-web:master] 2 new commits

5608338 Version 15.6 - def- 4aa696f Merge pull request #145 from ddnet/pr-15.6 - def-

its good our problem are too many changes

13:28

and not the other way around

13:29

ddnet

13:30

@deen the sha sums are not updated yet?

13:30

maybe im too fast

13:30

xd

true, running it now

aur is updated

great, thanks

I'm around for an hour or so incase something blows up

monkaS

[ddnet:master] 2 new commits

5413041 sv_plasma_per_sec 0 => disable firing - def- 9cb30a8 Merge #4287 - bors[bot]

so nice version rls without conflicts in version.h :D without any conflicts at all

[ddnet/ddnet] Pull request closed: #4287 sv_plasma_per_sec 0 => disable firing

[ddnet/ddnet] Issue opened: #4291 After the update i got this weird grey bar on the right

Max on Discord reports: "After the update i got this weird grey bar on the right" !20211104145652_1 @Jupeyy any ideas?

[ddnet/ddnet] Issue opened: #4292 Mouse moves faster than cursor when moving

in editor. Reported by cheeser0613 on Discord https://user-images.githubusercontent.com/2335377/140338651-a7467931-305e-40ff-a003-5164e52cf381.mp4 @Jupeyy you changed something about mouse movement in editor, right?

15:10

[ddnet/ddnet] Issue opened: #4293 Problems with client for multiple people

Since 15.6, reported on Discord. !unknown out of memory in first line looks weird.

[ddnet/ddnet] Issue opened: #4294 Lots of opengl client leaks

``` Direct leak of 80 byte(s) in 1 object(s) allocated from: #0 0x559a82d44d19 in malloc (/media/ddnet/DDNet+0x9d6d19) #1 0x559a830326c1 in CCommandProcessorFragment_OpenGL2::Cmd_CreateBufferObject(CCommandBuffer::SCommand_CreateBufferObject const) /media/ddnet/src/engine/client/backend/opengl/backend_opengl.cpp:1849:25 #2 0x559a830132bd in CCommandProcessorFragment_OpenGL::RunCommand(CCommandBuffer::SCommand const) /media/ddnet/src/engine/client/backend/opengl/backend_openg...

15:47

[ddnet/ddnet] Pull request opened: #4295 Remove upper resolution bound

fixes #4291

Checklist

[ ] Tested the change ingame
[ ] Provided screenshots if it is a visual change
[ ] Tested in combination with possibly related configuration options
[ ] Written a unit test if it works standalone, system.c especially
[ ] Considered possible null pointers and out of bounds array indexing
[ ] Changed no physics that affect existing maps
[ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssan...

[ddnet/ddnet] Issue closed: #4294 Lots of opengl client leaks

[ddnet/ddnet] Pull request opened: #4296 Test for #4293

Checklist

[ ] Tested the change ingame
[ ] Provided screenshots if it is a visual change
[ ] Tested in combination with possibly related configuration options
[ ] Written a unit test if it works standalone, system.c especially
[ ] Considered possible null pointers and out of bounds array indexing
[ ] Changed no physics that affect existing maps
[ ] Tested the change with [ASan+UBSan or valgrind's memcheck](https://github.com/ddnet/ddnet/#using-addresssanitizer--u...

16:04

[ddnet/ddnet] Issue opened: #4297 My game is crashing no matter which map I enter

My game is crashing no matter which map I enter It happened after I just updated the game to new version Reported by anonymous on Discord

rip rc

I don't get how not a single one of us gets these weird issues

well i dont play on windows

yeah really weird

16:05

especially louis and others tested it

16:06

i also asked a few ppl from my clan to test it, who are all on windows

frozen

[ddnet/ddnet-web] Pull request opened: #155 Mighty stepped down for now

we have way too few RC testers I'd say

[ddnet/ddnet-web] Pull request closed: #155 Mighty stepped down for now

16:20

[ddnet-web:master] 2 new commits

a6f75b0 Mighty stepped down for now - murpii 82d6be7 Merge pull request #155 from murpii/patch-3 - def-

Yes RC are nice to stabilize releases. I try to merge client to master often its not rly RC but at least something :D

or minimize ifdefs for windows and stuff, use the std or if the std isnt good enough some libs SDL also has some read write stuff, sadly not really powerful like e.g. c++17 filesystem (https://wiki.libsdl.org/SDL_RWFromFile "a UTF-8 string representing the filename to open")

The Simple Directmedia Layer Wiki

16:30

i could atleast imagine that most stuff used in system.h also exists in boost or std

I dont want to imagine how much we'd break trying to replace system.h

in the end boost and std have insanly stricter rules, so its probs easier to use than our randomly written system replacement functions xd

[ddnet/ddnet-web] Pull request opened: #156 유빈 stepped down

16:41

[ddnet/ddnet-web] Pull request closed: #156 유빈 stepped down

16:41

[ddnet-web:master] 2 new commits

215747e 유빈 stepped down - murpii 0435f0b Merge pull request #156 from murpii/patch-4 - def-

I pulled out all the laptops I have, none seem to be able to reproduce this :/

Maybe if you switch your Windows language to Russian

17:21

or your username to Russian

17:21

at least Nakimar reported that reverting the unicode filesystem changes fixed part of the issue

Oh, that's a lead

17:24

If only one developer could reproduce it we could find what's going wrong

We should start logging the version and githash when starting the client

definitely worse performance in 15.6

Yes, the performance is obviously my greatest concern right now

idk why you released it anyways xd

XDE

how is the performance worse?

Oh idk why, maybe since it worked for everyone in rc and it has been in testing for a long time now?

i hope its not SDL again xD

Learath2

Oh idk why, maybe since it worked for everyone in rc and it has been in testing for a long time now?

But the performance loss was known

You know we tend to write software to release not to let ferment

@fokkonaut: can you be more explicit?

17:48

bcs of the doors?

snapfinditem is slow on entityex

17:48

to match to the real entity

17:49

as learath suggested, we should have a new netmsg including them

but that only matters for zooming out far doesnt it

idk

Yep, clipped to the view and zoomed at default you shouldn't be losing much performance at all

i also benchmarked find item and most overhead comes from the insanly slow uuid manager lookup

the whole client feels a bit worse than 15.5

At zoom level 2 I lose about 5 fps from 15.5

is there anything unique for every single netmsg?

17:52

that does not require uuid lookup for example

oh, hm actually I do lose a lot more fps when checking on desktop

not every entityex in a message each, but all in one

I get 2k fps in 15.5 1k in 15.6

see

on what map tho?

but there are no entityex on that map, I'm on grandma

then its weird, are u on windows?

I spec and look at the first part on both clients

17:54

on windows yep

pls SDL

17:54

dont do this to me

17:54

can you replace SDL.dll @Learath2 ?

yep, let me try

to check if it comes from there

ah won't work

17:55

SDL_FlashWindow is missing in old sdl

ah true

17:57

https://github.com/Jupeyy/ddnet/commit/84e580b4d98d8c57b933f7bd1a7085bfaa19bdd9 completly randomly make UUID grouped in their 8Bit CRC already almost doubles the performance

test · Jupeyy/ddnet@84e580b

17:57

so the best would be if we somehow could evade calls to it

I just noticed heavy fps drops when you do /showothers and there are other players, is it due to the alpha?

mhh, showothers generally drops alot of fps

why? because /pause doesnt drop fps, while at the same position

17:58

i even get more fps in /pause

17:58

xd

ah weird

I can make a build with reverted SDL_FlashWindow. Would that help? Then we can test if SDL is causing lower fps

i guess learath could just build himself?

I can't on this computer

18:02

I rage uninstalled VS

image.png

Image attachment

18:07

lookupuuid is so slow, with my trash crc thing it almost halves thats why i want to know if there is any unique ID that can identify a uuid packet early

18:08

or can it be sorted somehow to do binary search over the packets

CAN REPRODUCE \o/

18:09

Just create a local user account with cyrillic name

the Key() is that a 100% bet?

@trml EntityEx for doors is sadly not helpful at all, since it still requires the tiles to be placed there ._. My plot doors that work with other tiles are not showing as closed with this :/

18:13

Even tho they are sent with entityex m_Number correct + SwitchState state closed

[ddnet-web:master] 1 new commit

4b34961 Revert "Version 15.6" - def-

@trml: you are not here by any chance?

@trml I dont think it is smart to not send doors to clients when they are closed anyways, bcs thats such a small change in snapshot and somehow belongs to the server

Do any of you have the capability to compile on windows?

[ddnet/ddnet] Pull request opened: #4298 Fix #4293 by reverting relevant changes

@Robyt3 @Learath2 Would like to release DDNet 15.6.1 with this

Checklist

[ ] Tested the change ingame
[ ] Provided screenshots if it is a visual change
[ ] Tested in combination with possibly related configuration options
[ ] Written a unit test if it works standalone, system.c especially
[ ] Considered possible null pointers and out of bounds array indexing
[ ] Changed no physics that affect existing maps
[ ] Tested the change with [ASan+UBSan or valgrind's memch...

Learath2

Do any of you have the capability to compile on windows?

yes

GitHub

Click to see attachment 🖼️

Um @deen this is nice and all but that build you gave out on #bugs didn't fix it for me, I'm not sure that's enough

fokkonaut

yes

If you have some time can you create a user account with a russian name and try to bisect the bug?

ah, too bad

wym user acc?

windows username

what would that change

18:22

ah nvm

18:22

i see

It completely breaks the game for some reason

18:22

Reverting the most obvious commits didn't help

18:22

not sure what we broke there

hm, i also had a unicode fs change

18:23

but it was only fs_rename so probably unrelated

@Learath2 i can ask someone about the russian name, sec

@fokkonaut: or @trml what would this break? https://github.com/Jupeyy/ddnet/commit/46b9a7abf8967315c30503c5191686e7dae559da

test · Jupeyy/ddnet@46b9a7a

18:25

this fixes the performance basically

very cool, thanks for looking into the performance so quickly jupstar

you can always ask deen or compile with the github CIs :D (@Learath2)

Do any of you have the capability to compile on windows?

Jupstar ✪

@fokkonaut: or @trml what would this break? https://github.com/Jupeyy/ddnet/commit/46b9a7abf8967315c30503c5191686e7dae559da

it wouldnt respect https://github.com/Jupeyy/ddnet/blob/46b9a7abf8967315c30503c5191686e7dae559da/src/engine/shared/snapshot.cpp#L24-L49 anymore

18:27

Means, it would break extended messages

why not ask robyte, he'll glady do it i bet

deen

very cool, thanks for looking into the performance so quickly jupstar

Dont apply that, lol

he also tested ime stuff for teeworlds

Jupstar ✪

you can always ask deen or compile with the github CIs :D (@Learath2)

I doubt we can bisect remotely like this :D, with 500 commits it'd take 9 compilations

so, 9 builds? 20 min turnaround time, we'd be done in 3 hours

I'll just get mingw back on this windows installation

[ddnet/ddnet] Pull request closed: #4296 Test for #4293

18:30

[ddnet/ddnet] Pull request closed: #4298 Fix #4293 by reverting relevant changes

ok

18:30

so its not possible anyway?

18:30

todo binary search

https://github.com/Jupeyy/ddnet/blob/46b9a7abf8967315c30503c5191686e7dae559da/src/engine/shared/uuid_manager.cpp#L128-L136

@fokkonaut: can you please just say how your client does it?, do you use a different system for extensions?

You mean F-Client?

18:31

It does the same as DDNet

I think 0.7 is better at doing it's searches in snaps

https://github.com/fokkonaut/F-Client/blob/F-Client/src/engine/shared/uuid_manager.cpp#L100-L110

F-Client/uuid_manager.cpp at F-Client · fokkonaut/F-Client

F-Client is a client mod for Teeworlds 0.7 developed by fokkonaut. - F-Client/uuid_manager.cpp at F-Client · fokkonaut/F-Client

18:32

my client uses heinrichs older approach, i htink

18:32

no idea if this is slower or not

this sucks i know that

We can go even further and unpack snaps into hashmaps so all further lookups are O(1)

ok so you have the same performance regressions

18:33

maybe the uuid manager is just overkill

18:33

i dont see an easy way to binary search if you HAVE to use the UUID

Jupstar ✪

maybe the uuid manager is just overkill

no other vanilla compatible way i think, because it sends this snap as if it has the id 0

18:33

iirc

yeah i already optimized it a bit, bcs nameplates were so slow in the past

18:34

hashmaps will be very slow i think

18:34

we need a 32bit lookup

18:34

everything else gets slow very fast

Is that really the part that is getting overwhelmed?

yes

18:36

Embedded image

Jupstar ✪

image.png

seems so

oh sry

18:36

this

image.png

18:37

if there is a way to sort them somehow or not call uuid its fixable

18:37

else its a design flaw

Sort what?

sort keys to call the uuid loopup as less as possible

18:37

the uuid lookup is already binary, and still so slow, bcs its called so often

18:38

is heinrich online

Nope

he could just tell xd

Okay, iirc currently the snap includes an object that has the mappings from snap-type-id -> uuid

wait netobject extensions have an unique identifier type dont they?

18:40

did we have some documentation somewhere?

18:41

https://github.com/heinrich5991/libtw2/blob/master/doc/snapshot.md

libtw2/snapshot.md at master · heinrich5991/libtw2

Some Teeworlds stuff in Rust. Contribute to heinrich5991/libtw2 development by creating an account on GitHub.

fokkonaut

@trml EntityEx for doors is sadly not helpful at all, since it still requires the tiles to be placed there ._. My plot doors that work with other tiles are not showing as closed with this :/

wrong information as it seems, let me debug

Basically NETOBJTYPE_EX objects provide a mapping from type-id inside the snap to a uuid

ok

18:42

i mean we just need to "original key" and check against that i guess, snapfind item uses the lookedup item type

18:42

we need to original item type, is that always NETOBJTYPE_EX

Extended objects get typeid > OFFSET_UUID_TYPE, NETOBJTYPE_EX objects provide the mapping from this typeid to the uuid

ok

https://github.com/teeworlds/teeworlds/pull/2129 here is what 0.7 did to optimize snap lookups

Optimise CClient::SnapFindItem by LordSk · Pull Request #2129 · tee...

So I was doing some test with a lot of dummies and ran into performances issues. Naturally I did some profiling and SnapFindItem was among the top culprits (it is called twice per client every fram...

yeah, but that doesnt like, like its doable

Why not?

ok another question you might know lets say you have 2 extended objects of the same type, do they share the same offset to the item uuid?

18:50

bcs type + id is not the same refering @fokkonaut

they do

18:50

Well to be more correct they will have the exact same type within that one snap

it is the same, but extended objects have type 0

18:51

to be vanilla compatible

18:51

"compatible"

18:51

i other words: so they can be sent to vanilla clients, so they can drop them

fokkonaut

it is the same, but extended objects have type 0

Um, nope. I don't think so

yes, they have (edited)

Eeeh, no they don't or snapshot.cpp is lying to me

They are found by the uuid

Type 0 is used to provide the mapping,

18:52

Types > OFFSET_UUID_TYPE are decoded using objects of type 0

@Learath2: are the extended types dynamic, e.g. can it have a different internal index from snapshot to snapshot?

Jupstar ✪

@Learath2: are the extended types dynamic, e.g. can it have a different internal index from snapshot to snapshot?

Yes

ok

18:53

so we only need to store these mappings?

yea, but you dont send them as lets say 65557, it can differ from the type number thats on the server

from snapshot to snapshot?

18:53

then we can doi smth like this https://github.com/Jupeyy/ddnet/commit/46b9a7abf8967315c30503c5191686e7dae559da just with the mapped type

test · Jupeyy/ddnet@46b9a7a

18:54

e.g. a map that has ENTITY_EX as key and the internal index as data entry of a std::map so we can just see what ENTITY_EX requires as internal type

18:54

and than do the thing in my commit with that mapped data

fokkonaut

yea, but you dont send them as lets say 65557, it can differ from the type number thats on the server

I'm not sure what you are really referring to, the first extended type in a snapshot will be type 32766

yes

18:57

i'll just try it out and you have to judge on if it will work xd

Jupstar ✪

e.g. a map that has ENTITY_EX as key and the internal index as data entry of a std::map so we can just see what ENTITY_EX requires as internal type

I'm not sure what part of the process is the bottleneck. Is it the fact that we are looking for objects of type ENTITY_EX and that has to go through uuid which is too slow?

i mean linear search is slow, but most bottleneck simply comes from the fact we do uuid lookup and uuid is 16bytes

but what call stack leads to the lookup?

19:00

Is the new void CGameClient::SnapCollectEntities() taking too long?

i assume all calls, bcs it always hits extended objects

19:00

and they are "unwrapped"

19:01

that function too yes

19:01

its 50% of the calls, the rest is onnewsnapshot

19:01

ok no

19:01

its -5% of all perf

19:01

and onnewsnapshot does 50% of all

I don't follow, I'll just get a profile of my own

image.png

Image attachment

19:03

oh thats the optimized version

19:03

image.png

Image attachment

Hm, I don't know what SnapFindItem call is causing this

probably from entityex

There is no SnapFindItem for entityex tho, they are collected differently

ah

it also doesnt really matter which one it is, i think all netobjects use it

19:08

the problem is that entity ex are part of the snapshot

19:08

so they are still looked up

19:08

(?)

There are rather few calls to snapfinditem in the code :/

Jupstar ✪

so they are still looked up

yes, what i thought too

19:09

right now xd

Maybe the new entityex just make the snap too big for linear search to be viable?

so @Learath2 you are saying that we can just ignore objectex?

19:09

in that call

19:09

that would be even easier

i think so, yes

Jupstar ✪

so @Learath2 you are saying that we can just ignore objectex?

Wym? The id 0? that already should be early returning from GetItemType

so just check if the item type is over UUID_OFFSET, and only check else

19:11

it doesnt rearly return if there is a object_ex somewhere in the snapshot

19:11

early*

19:11

bcs that object_ex is looked up, even if it doesnt matter for this find call

19:12

it makes no difference between the type of the snapshot thing

What do you mean by object_ex? NETOBJTYPE_EX?

yes

NETOBJTYPE_EX == 0 which is < OFFSET_UUID_TYPE so GetItemType should never result in a uuid lookup

19:13

Idk what else you'd be meaning by lookup

ok

19:15

then not objtype_ex

19:15

but whatever uses the uuid thing

Can we somehow learn where this snapfinditem is being called from?

NETMSG_MAP_DETAILS

You can try caching the mapping OnNewSnapshot actually

if snapfinditem doesnt require extended protocol, it will be fast enough

Get all the NETOBJTYPE_EX objects, build a map from internal id directly to externalid, then use that for this snapshot

19:17

NETMSG_MAP_DETAILS isn't even a snap object, how would it be going thru snapfinditem?

well i dunno what message

19:19

but one of them must be in

19:19

or the server sends buggy snapshots

19:19

https://github.com/Jupeyy/ddnet/commit/7b91d861c65ef152f4ed0f16ff8a52be03627951

testtest · Jupeyy/ddnet@7b91d86

19:19

safe or not?

I don't see how that's helpful

19:22

You removed the ability to find extended objects?

NETOBJTYPE_GAMEINFOEX

19:23

thats the question (@Learath2)

You removed the ability to find extended objects?

19:23

is it ever useful? the client only 2 times does not try to find a non vanilla object type

I'm not following your logic here. You are still doing GetItemType, which is already cheap for non extended object

19:25

s

ok NETOBJTYPE_DDNETPROJECTILE seems like a problem atleast?

19:25

thats what i suggested yes (@Learath2)

Get all the NETOBJTYPE_EX objects, build a map from internal id directly to externalid, then use that for this snapshot

Jupstar ✪

ok NETOBJTYPE_DDNETPROJECTILE seems like a problem atleast?

I actually don't have any idea, you managed to completely confuse me

ok lets see it like this:

19:26

NETOBJTYPE_GAMEINFOEX is in the snapshot every time isnt it?

19:26

every tick

yes

and NETOBJTYPE_GAMEINFOEX is uuid so even if you search for something completly unrelated, like a character, it might still lookup this ID type

Ah, I see what you mean, as said I think building a map at the start will work

ok

Your patch makes more sense too now, it's not correct but it does make sense

19:29

Check if the type we are looking for is an extended type. If not don't even do the lookup is what you are trying to do, right?

from what i see only NETOBJTYPE_DDNETPROJECTILE is used in snapfinditem

19:32

so we basically only would require mapping for this single uuid

19:33

(the only non vanilla netobj to be precise)

from what i see only NETOBJTYPE_DDNETPROJECTILE is used in snapfinditem

Branch on Type < OFFSET_UUID_TYPE if true check the raw type, else check thru getItemType. it's not pretty but it will show if that's the issue

but i guess i can also check against OFFSET_GAME_UUID-1 from protocol.h so i know if the type expects a uuid or not the uuid manager always adds + OFFSET_UUID

Oh please don't confuse me more, these are all different uuids

19:39

offsets*

19:40

OFFSET_UUID_TYPE is the offset you are looking for, types < that are internal vanilla types, types >= are extended types period

https://github.com/Jupeyy/ddnet/commit/a990669050f70dbac7a14f89e0a4e19438668f56

test2 · Jupeyy/ddnet@a990669

19:45

that fix atleast doubles my fps

[ddnet/ddnet] Pull request opened: #4299 Fix fs_makedir and fs_storage_path with unicode on windows

Use the wide char variants of getenv and _mkdir. Tested the change on upstream with japanese characters and german umlauts in username. This should fix the issues with cyrillic, I don't know if that "out of memory" was also caused by that (#4293).

Checklist

[ ] Tested the change ingame
[ ] Provided screenshots if it is a visual change
[ ] Tested in combination with possibly related configuration options
[ ] Written a unit test if it works standalone, system.c esp...

19:52

[ddnet/ddnet] Pull request opened: #4300 Don't call uuid lookup, if the expected type isn't an uuid extended t…

…ype anyway Most performance loss currently comes from the simple fact that lookupuuid is slow. This addresses the problem by making sure its only called for net messages, that are extended anyway @trml or @heinrich5991 can you check if that makes sense without breaking something?

Checklist

[ ] Tested the change ingame
[ ] Provided screenshots if it is a visual change
[ ] Tested in combination with possibly related configuration options
[ ] Written a unit test if it...

this better be all the unicode fixes justatest

justatest

fokkonaut

@trml EntityEx for doors is sadly not helpful at all, since it still requires the tiles to be placed there ._. My plot doors that work with other tiles are not showing as closed with this :/

oh. do normal doors show correctly for you? doors are only rendered from the snapshot, so the client doesn't know if it's actually a "real" door or anything else. perhaps you can compare the values (and m_Number/m_Layer) with normal doors

All fine, I was only sending m_NumSwitchers amount for SwitchState, forgot to add + m_NumPlots from my side

21:09

All working

ah ok, nice

If door prediction will base on this, and not tiles, it will work so good :)

yes, the idea is to make it possible to get more general prediction (and in theory you could even add other types of doors, and get prediction)

oo

if someone ever fixes the "pass through doors" thing for example:P (edited)

21:13

(or maybe it already is)

its not

21:14

I dont think its needed to fix that for prediction tho

Jupstar ✪

from what i see only NETOBJTYPE_DDNETPROJECTILE is used in snapfinditem

I have a branch where I got rid of even more calls to SnapFindItem, including this one (I think), by caching the SNAP_CUR->SNAP_PREV mapping in addition to the entity ex mapping, so I could try to finish up that one

21:28

I dont know much about how UUIDs/SnapFindItem works otherwise, but noticed previously that the uuid lookup took most of the time there

Robyt3

this better be all the unicode fixes justatest

justatest

oh, you're here too. hi, thanks for fix

[ddnet:master] 4 new commits

3df3feb fix fs_makedir and fs_storage_path with unicode on windows - Robyt3 83d8e2b fix fs_getcwd with unicode on windows - Robyt3 3854814 fix fs_chdir with unicode on windows - Robyt3 1c809fa Merge #4299 - bors[bot]

21:49

[ddnet/ddnet] Pull request closed: #4299 Fix fs_makedir, fs_storage_path, fs_getcwd and fs_chdir with unicode on windows

@trml || anyone looking into the uuid thing: The mapping of type to uuid is not static, it's per snap. Just be careful with that

oh, so its a mapping to the types, not the objects? then I probably misread a few things of the discussion above

Okay, let's use concrete terminology here. When I say type I refer to the type part of the "typeandid" construct inside any given netobject

22:11

That type if below OFFSET_UUID_TYPE refers to vanilla object types. Except for type 0 (NETMSGTYPE_EX). Objects of type 0 instead denote a mapping from their id (which refers to the id inside the netobject) to a uuid.

22:12

Types above OFFSET_UUID_TYPE are mapped to uuids using the mappings sent in the NETMSGTYPE_EX objects. Then these uuids are mapped to types using the uuid manager

22:13

That last sentence should have been s/types/concrete types/ as in NETOBJTYPE_ENTITYEX e.g.

[ddnet:15.6.1-rc1] 22 new commits

01d097c Version 15.6 - def- 6fc198a Update language files for 15.6 release - def- 6e4221b Translate German - def- 74a499b Fix pickup network clipping - fokkonaut b4ea62a Use correct (larger) m_NumSwitchers bound - trml be65574 update catalan - edg-l 8252f46 Update spanish.txt (#4256) - n0Ketchp 4e698d3 Update spanish.txt - n0Ketchp 3dc733c Update spanish.txt - n0Ketchp d62ca7f Update spanish.txt - n0Ketchp 74d314d Update russian.txt - NikGreens 8ed1a93 Update russian.txt - NikGreens 8ea1655 Update russian.txt - NikGreens 642eaa4 Update brazilian_portuguese.txt - rffontenelle 47e2c4c Add exception id to TeamKill - Jupeyy 0000d75 Reset selection, if gui element was changed outside of DoEditBox - Jupeyy 160e02b Fix characters disappearing (fixes #4285) - fokkonaut dc95187 fix fs_makedir and fs_storage_path with unicode on windows - Robyt3 4bdcbef fix fs_getcwd with unicode on windows - Robyt3 10ec409 fix fs_chdir with unicode on windows - Robyt3 d65a8e8 Don't call uuid lookup, if the expected type isn't an uuid extended type anyway - Jupeyy c396711 Version 15.6.1 - def-

Optimizations that come to mind are first keeping this a mapping from extended concrete types to snap internal types

22:15

Second optimizing the linear lookup in SnapFindItem

22:16

Third re-engineering the extended net objects to negotiate the mapping before the first snap through netmsgs

yes, thanks, that clarified it quite a bit (wasn't aware of how uuids worked internally)

22:19

something I thought about previously (and tried a bit) was to do something like SnapCollectEntities to also gather the snap_prev and snap_cur versions of netobjects in the same list, to reduce the calls to SnapFindItem more, but not sure if that would be needed with those other optimizations?

I actually have no idea what is the problem here, I haven't profiled it myself so I'll refrain from saying anything concrete

22:20

I think the optimization jupeyy came up with + taking the binary search for snaps from 0.7 should get us back to the same performance, but not exactly sure

22:21

I'd also like to get @heinrich5991 to take a look at the optimization we thought of, I'm sure he'd have a more elegant way of doing it since my idea breaks abstraction

22:23

Oh and an extreme optimization I don't remember who came up with is to unpack the snaps as they arrive into a hashmap

22:23

This would make all lookups O(1), though I think most of the operations on a snap seems to be iterating which is not that good in a hashmap

yes, lookups on ID are perhaps mostly useful for things like matching cur+prev

SIMD snap parsing when?

Exported 408 message(s)