Guild icon
DDraceNetwork
Development / developer
Development discussion. Logged to https://ddnet.tw/irclogs/ Connected with DDNet's IRC channel, Matrix room and GitHub repositories — IRC: #ddnet on Quakenet | Matrix: #ddnet-developer:matrix.org GitHub: https://github.com/ddnet
Between 2020-04-08 00:00:00Z and 2020-04-09 00:00:00Z
Avatar
@Learath2 from the official libsdl.org site
05:28
I'm trying to update SDL to 2.0.12 in travis. Everything is working except for mac OS, which throws the following error during make install: CMake Error at /usr/local/Cellar/cmake/3.15.5/sh...
06:29
ae330bb Try to fix SDL2 hidapi rpath on MacOSX - def- 724f292 Merge pull request #2121 from def-/pr-mac-hidapi - def-
Avatar
@noby @Learath2 can you try again on mac osx?
06:50
(i fixed the 13.0.1 release on osx)
Avatar
ok ill try
Avatar
Did you upload the dmg?
Avatar
Identifier: org.DDNetClient.app Version: 13.0.1 Code Type: X86-64 (Native) ... Crashed Thread: 0 Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY Termination Reason: DYLD, [0x1] Library missing Application Specific Information: dyld: launch, loading dependent libraries Dyld Error Message: Library not loaded: @rpath/hidapi.framework/Versions/A/hidapi Referenced from: /Users/nobody1/Desktop/DDNet.app/Contents/Frameworks/SDL2.framework/SDL2 Reason: image not found
06:54
Binary Images: 0x10ff4a000 - 0x110287fff +org.DDNetClient.app (13.0.1) <F0B0E60D-540B-396A-81D2-CA5F8986284F> /Users/USER/Desktop/DDNet.app/Contents/MacOS/DDNet 0x110541000 - 0x11062efff +libfreetype.6.dylib (24.1) <FF599A1F-62D1-3478-83B6-54282A253C8B> /Users/USER/Desktop/DDNet.app/Contents/Frameworks/libfreetype.6.dylib 0x11064d000 - 0x11078efff +org.libsdl.SDL2 (2.0.12 - 2.0.12) <3E6D5145-6DE8-3734-95A7-AE31ED09A1A9> /Users/USER/Desktop/DDNet.app/Contents/Frameworks/SDL2.framework/SDL2 0x11747e000 - 0x1174c8acf dyld (551.4) <8A72DE9C-A136-3506-AA02-4BA2B82DCAF3> /usr/lib/dyld
Avatar
@deen didn't work for me either
06:58
The hidapi rpath is wrong in SDL2 itself, not the client
06:58
last argument of install_name_tool should be sdl2
Avatar
[quakenet] deen BOT 2020-04-08 06:58:32Z
ah, too bad...
06:59
so we have to overwrite the rpath in SDL2 directly
Avatar
Yeah, they should be the ones doing it tbh, but we can for the time being
Avatar
Should I merge and try another build or can someone test locally? ^
Avatar
Wait for a second for CI to build and I can test that
07:05
I think in this case executable path would be SDL2 not DDNet but ill try
Avatar
I'm not completely sure, let me try first
07:07
Mach-O binaries are weird sometimes
07:11
dyld: Library not loaded: @executable_path/../Frameworks/SDL2.framework/SDL2/Versions/A/Frameworks/hidapi.framework/Versions/A/hidapi Referenced from: /Volumes/DDNet-13.0.1-osx/DDNet.app/Contents/Frameworks/SDL2.framework/SDL2 Reason: no suitable image found. Did find: /Volumes/DDNet-13.0.1-osx/DDNet.app/Contents/MacOS/../Frameworks/SDL2.framework/SDL2/Versions/A/Frameworks/hidapi.framework/Versions/A/hidapi: stat() failed with errno=20 /Volumes/DDNet-13.0.1-osx/DDNet.app/Contents/MacOS/../Frameworks/SDL2.framework/SDL2/Versions/A/Frameworks/hidapi.framework/Versions/A/hidapi: stat() failed with errno=20 Abort trap: 6
Avatar
oops, I also had a wrong path in there, pushed again
07:17
so we don't know where executable_path points to yet?
Avatar
Not sure :/
Avatar
[quakenet] Test123 BOT 2020-04-08 07:19:43Z
Hi
Avatar
It should be tho, how else could you link a library to a library?
Avatar
[quakenet] deen BOT 2020-04-08 07:21:39Z
I think executable_path will be the path to DDNet.app
Avatar
Then this should work
07:25
I wonder if we can mount and run the dmg on the CI
Avatar
[quakenet] deen BOT 2020-04-08 07:25:40Z
would be great
07:26
can you try the version I pushed again?
Avatar
yep, it works
Avatar
[quakenet] deen BOT 2020-04-08 07:26:31Z
ok, thanks
07:26
18aec44 New try for hidapi SDL2 rpath - def- 392412b Merge pull request #2122 from def-/pr-hidapi-sdl2-rpath - def-
Avatar
@deen there is a patch I got into sdl, but it wasnt released on a version yet, you think we should apply that to the macOS ddnet sdl?
Avatar
[quakenet] deen BOT 2020-04-08 07:27:17Z
I'm not building my own sdl2 for mac
07:27
do we need it?
Avatar
Oooooh, maybe you should be compiling with the 10.14 or 13 sdk then
07:27
I forgot about that
Avatar
[quakenet] deen BOT 2020-04-08 07:27:49Z
too late!
Avatar
on binaries linked on 10.15 there is a new check enabled in AppKit which crashes on launch
07:28
if you compile on older AppKit compatibility disables the asserts
Avatar
[quakenet] deen BOT 2020-04-08 07:28:40Z
so we're crashing?
Avatar
ae330bb Try to fix SDL2 hidapi rpath on MacOSX - def- 724f292 Merge pull request #2121 from def-/pr-mac-hidapi - def- 18aec44 New try for hidapi SDL2 rpath - def- 392412b Merge pull request #2122 from def-/pr-hidapi-sdl2-rpath - def-
Avatar
Yeah, at launch it hits an assert
Avatar
[quakenet] deen BOT 2020-04-08 07:31:40Z
so I should downgrade to 10.13 sdk and we're fine?
Avatar
Yeah I think so
07:32
Or you apply my sdl patch
07:32
But compiling sdl on our own for mac too is just a pain
Avatar
exactly, so I'll downgrade sdl
Avatar
We can upgrade sdk when the patch is in a stable sdl
Avatar
good to know
Avatar
Its broken on old sdl too, you need old sdk
Avatar
do i need to recompile libs?
Avatar
I never tried, mach-o binaries should be forward compatible though
Avatar
a98818a MacOSX SDK 10.15 -> 10.13 - def-
Avatar
@deen lmk when you have a build, I can test
Avatar
@Learath2 available now
Avatar
I'm sad to report that that didn't work
08:25
I mean it did do what's intended but now we get some lazy loading issue
08:25
what was your MACOSX_DEPLOYMENT_TARGET?
08:27
dyld: lazy symbol binding failed: Symbol not found: ___isOSVersionAtLeast Referenced from: /Volumes/DDNet-13.0.1-osx/DDNet.app/Contents/MacOS/DDNet Expected in: flat namespace dyld: Symbol not found: ___isOSVersionAtLeast Referenced from: /Volumes/DDNet-13.0.1-osx/DDNet.app/Contents/MacOS/DDNet Expected in: flat namespace Abort trap: 6 logout Saving session...completed.
Avatar
I don't set one for the DDNet build, for the libs -mmacosx-version-min=10.9
08:30
should I try setting -mmacosx-version-min=10.9 on ddnet as well?
08:31
ah, CMakeLists.txt sets 10.7
08:31
Also, we have something in CMakeLists.txt just for that failure you see
08:32
target_link_libraries(${target} -Wl,-undefined -Wl,dynamic_lookup)
Avatar
yeah noticed that, but that enables lazy binding, idk why it won't bind though
Avatar
let's try setting all to 10.9?
Avatar
yeah sure, let's try
08:35
I don't think you can cross compile if you disable the lazy binding @deen
08:35
you probably don't have libclang_rt.osx.a as it's part of the toolchain not the sdk
08:37
this release is really annoying
Avatar
apparently osxcross has a way to build compiler_rt
08:41
I'll get a linux vm
Avatar
I guess I'll have to recompile the libs with 10.7
08:42
10.9 also fails in central thing
08:43
@Learath2 you can try 13.0.1 again on macosx? I rebuilt with 10.9
Avatar
I'm not sure what's the issue, lazy binding should resolve that symbol just fine
08:43
@deen is it on ddnet.tw?
Avatar
nope still failing
Avatar
then I'll rebuild the libs
Avatar
I don't get this, how was this not a problem when you compiled with the 10.15 sdk?
Avatar
maybe it was but we never got this far in booting up?
08:52
*starting up
08:52
since we always crashed before
Avatar
ah hidapi wasn't lazily linked, yeah it probably was a problem
Avatar
curl doesn't build with 10.7 vtls/sectransp.c:1313:6: error: 'SSLSetProtocolVersionMax' is only available on macOS 10.8 or newer [-Werror,-Wunguarded-availability]
Avatar
weren't you going to recompile everything with 10.9 anyway?
Avatar
no, i had everything with 10.9 and you said it doesn't work
08:57
so now I'm going everything 10.7
Avatar
ah, that was with all 10.9
Avatar
maybe the sdk change caused the problem? should i try rebuilding all?
08:59
with 10.9 again
09:00
i'm kind of searching in the dark here :/
09:00
and before I didn't notice this curl error since ssl was accidentally disabled 😄
Avatar
I'm setting up osxcross on a vm
09:02
if you are bored of this I can finish it up
Avatar
I'll give you another dmg file to test soon
Avatar
@deen are you building with --without-ssl btw?
Avatar
try --without-ssl --with-secure-transport
Avatar
uhm when I'm cross building cmake can't find sdl framework
09:24
@deen can you find the bundled SDL2 properly?
09:24
cmake on linux even when cross compiling doesn't seem to want to find frameworks
Avatar
what do you mean by find?
Avatar
I don't have SDL2 on my VM, but it should use the bundled lib anyway
09:33
it says SDL2 not found
Avatar
-- * SDL2 not found (using bundled version)
09:34
for me
09:34
did you clone ddnet-libs?
09:34
it finds curl just fine
09:34
what version of cmake do you use?
Avatar
I also set -DPREFER_BUNDLED_LIBS=ON -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/darwin.toolchain
09:34
cmake version 3.17.0
Avatar
let me upgrade, maybe that'll help, but I set the exact same thing you set
Avatar
why is libqt a dependency of cmake?
09:43
it pulled in emacs even
Avatar
probably they have a graphical interface
09:50
@Learath2 please try dmg from ddnet.tw again, now rebuilt all with all 10.7
Avatar
party time
09:53
wierd graphical thing goin on here
09:53
but worth it for new ddnet with good fps
Avatar
@deen wait, onby is on an old osx
09:53
let me check just to be sure
Avatar
the lines inbetween? they look like opengl3.3
Avatar
[ddnet/ddnet-libs] New branch created: pr-macosx-10.7
09:54
3e960b3 Rebuild Mac OSX libs with -mmacosx-version-min=10.7 (previously 10.9) - def-
Avatar
it works, but for a different reason
Avatar
clang optimizes out the version check
Avatar
i guess it's fine for now
Avatar
if I ever get cmake working with osxcross I'll fix it
09:56
it's much easier when you have access to a mac
Avatar
i now have good fps and no fan spinning on ddnet 13 :D
Avatar
I accidentally killed my terminal, which killed the cmake build I was running....
09:58
Anyone here running archlinux?
Avatar
da5aead -mmacosx-version-min=10.7 & --without-ssl --with-secure-transport - def-
Avatar
Do you know why pacman might be stuck not upgrading things?
09:58
cmake is 3.17 in extra and it seems to be 3.17 in all mirrors
Avatar
IgnorePkg/IgnoreGroup in /etc/pacman.conf?
Avatar
ah let me check
10:00
nope
Avatar
pacman -S cmake?
Avatar
Packages (1) cmake-3.16.4-1
Avatar
[quakenet] deen BOT 2020-04-08 10:03:05Z
pacman -Syu cmake?
Avatar
:: Synchronizing package databases... core is up to date extra is up to date community is up to date :: Starting full system upgrade... resolving dependencies... looking for conflicting packages... Packages (1) cmake-3.16.4-1 Total Download Size: 6.77 MiB Total Installed Size: 35.53 MiB
10:04
oh rip irc
Avatar
weird, maybe you use an out-of-date mirror
Avatar
oh the top mirror is indeed out of date
10:05
who would do such a thing
10:09
and mirror status says that mirror is up to date
10:10
oh nvm, they do list it as out of sync
10:10
imagine hosting a widely used mirror and not actually mirroring
Avatar
do u only have 1 mirror enabled?
10:11
shouldnt it check several mirrors or smth
Avatar
no, the first that works is always used
Avatar
it's the top mirror and it connected fine so it never checks another
10:12
they should really add a way to check sync, like a last sync timestamp at the repo root
10:13
pacman could then display a WARNING: selected mirror could be out of date (last sync %d)
Avatar
tbh I never had an out-of-date mirror
Avatar
btw kernel 5.6 made a thing which theorically makes "make" way faster when parallel
Avatar
well if it happens it's quite hard to diagnose
Avatar
we are up to 5.6 now?
Avatar
@Learath2 i see updates every day, if there are none something is wrong with the mirror
Avatar
"In a simplified test case written by Linus Torvalds, this patch caused the number of context switches on the test program to drop from 11 million to just 1.2 million. "
10:14
poggers
Avatar
oh wow, that's an entire order of magnitude better
10:16
here its explained in detail xd
Avatar
context switches are pretty cheap tbf, but when you have millions happening it does stack up
Avatar
uname -a Linux arch 5.6.2-arch1-2 #1 SMP PREEMPT Sun, 05 Apr 2020 05:13:14 +0000 x86_64 GNU/Linux
10:17
5.6.2
Avatar
@Learath2 It really depends on your CPU and the mitigations on Intel made it much slower, see bottom here: https://www.phoronix.com/scan.php?page=article&item=3900x-9900k-400&num=7
Avatar
PROPELLER: Profile Guided Optimizing Large Scale LLVM-based Relinker - google/llvm-propeller
Avatar
oh the meltdown/spectre mitigations? I can see them raising the cost of a context switch
Avatar
can u optimize ddnet with this?
10:19
poggers
Avatar
anyway even cmake 3.17 doesn't find SDL2.framework
Avatar
yes, you can use PGO, can also try LTO
10:20
but in my experience you don't win much with PGO
Avatar
"2% to 6%", sounds realistic
10:22
using a profiler and improving the slow spots selectively should help more
Avatar
what profiler you recommend
Avatar
VTune is what we usually use at work, but I think it costs money
Avatar
is gprof good
Avatar
gprof is a sampling profiler, keeping in mind the limitations of sampling profilers it's decent
Avatar
and valgrind --tool=callgrind is what I use privately, but that's just to get a rough estimate since it changes runtimes hugely
Avatar
yeah callgrind actually logs all the calls but at the cost of runtime performance
10:25
you are kinda hoping that everything is affected pretty much equally
Avatar
yes, but it's not true
Avatar
I wonder if there is an architecture that allows a hardware profiler of sorts
Avatar
if you have cache misses you definitely won't find those wil callgrind. I've seen perf used to find cache misses
Avatar
what about "perf·
Avatar
@Learath2 yes, that's what vtune does. intel cpus have specific registers for profiling
Avatar
and perf uses them as well
Avatar
but aren't they all just counters? you wouldn't get any callstack information from those
Avatar
yeah, right
Avatar
it's a little beyond my ability to go from counter information to optimization
Avatar
perf report monkaS
Avatar
i didnt test on windows
Avatar
@Ryozuki did you test it? (open the client, go somewhere?)
Avatar
i tested on linux works fine
Avatar
okay
Avatar
@heinrich5991 macos failed xd
Avatar
seen it
Avatar
Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_INCLUDE_DIR) -- Could NOT find Crypto (missing: CRYPTO_LIBRARY CRYPTO_INCLUDEDIR)
14:00
i think its not related?
Avatar
that's not the error I think
Avatar
it has so many warnings
Avatar
DDraceNetwork, a cooperative racing mod of Teeworlds - ddnet/ddnet
14:01
that's the reason the build failed
14:01
that warning
14:01
not sure why it appeared now though
Avatar
@heinrich5991 note: before #include "SDL_opengl.h" was only on non macos
14:02
but theorically
14:02
sdl_opengl handles cross platform stuff
14:02
it should include opengl fine on macos
14:03
in my change its included on every os
14:03
like in tw
14:03
note: insert '_Nullable' if the pointer may be null
14:03
never seen this
14:03
maybe some compiler flag on macos?
14:04
@heinrich5991 this nullable stuff is objective-c
14:04
In Xcode 7 GM I started to get this warning: Pointer is missing a nullability type specifier (_Nonnull, _Nullable, or _Null_unspecified) In the following function declaration (NSUserDefaults
Avatar
it's also C I think
Avatar
@heinrich5991 i think -Wno-nullability-completeness
14:05
disables this
14:05
or -Wno-nonnull
14:05
idk
Avatar
why does it start happening with this commit though?
Avatar
no idea
14:09
maybe its cuz SDL_opengl.h
Avatar
maybe also something on the CI was updated
14:10
I can't re-run master CI unfortunately
Avatar
fix a typo somewhere
14:10
xd
Avatar
@Learath2 ^?
Avatar
i ran valgrind on ddnet client, maybe someone finds something
Avatar
Don't merge yet. I want to use this pr to fix several leaks and stuff, if you find something you can report it here. valgrind-out.txt
Avatar
@heinrich5991 CUIRect MenuBar, CModeBar, ToolBar, StatusBar, ExtraEditor, UndoList, ToolBox; declaring a value like this initializes them explicitly right?
15:55
or maybe not
15:56
ah nevermind this isnt the issue i think
Avatar
it only initializes them if the constructor does something or the constructors of the struct members do something
Avatar
sometimes it feels like valgrind tells u nothing lol
16:11
Uninitialised value was created by a stack allocation ==84015== at 0x1F998E: CDemoRecorder::Start(IStorage*, IConsole*, char const*, char const*, char const*, SHA256_DIGEST*, unsigned int, char const*, unsigned int, unsigned char*, IOINTERNAL*, bool (*)(void const*, int, void*), void*) (demo.cpp:45) sure but why u dont tell me exactly what stack allocated variable it is 😤
Avatar
rip IRC, but it says here:
16:12
ah, message too long, I'm not allowed to rip IRC
16:12
make a pastebin or smth
Avatar
it says in the lines above that it comes from this function: ==84015== by 0x1FA13A: CDemoRecorder::Start(IStorage*, IConsole*, char const*, char const*, char const*, SHA256_DIGEST*, unsigned int, char const*, unsigned int, unsigned char*, IOINTERNAL*, bool (*)(void const*, int, void*), void*) (demo.cpp:167)
16:14
check line 167 and you see that it's aChunk
Avatar
oh true
16:14
it isnt initialized
Avatar
I don't really see how it can not be initialized though (edited)
16:15
maybe i should memzero it?
Avatar
no
Avatar
@heinrich5991 i think -Wno-nullability-completeness
@Ryozuki I tried that and there were even more other failures: https://github.com/ddnet/ddnet/pull/2124/files
Avatar
@heinrich5991 then i suppose this should be ignored right?
16:16
@deen :o
Avatar
@Ryozuki I don't think valgrind has false positives, so I'm kinda suprised here
Avatar
well valgrind says it isnt perfect in their web xD
Avatar
ok
16:18
link?
Avatar
Official Home Page for valgrind, a suite of tools for debugging and profiling. Automatically detect memory management and threading bugs, and perform detailed profiling. The current stable version is valgrind-3.15.0.
16:19
in the caveeats section
Avatar
broken encoding, nice
16:20
maybe its not aChunk?
16:20
but only aChunk makes sense
16:21
well its definitly aChunk
16:21
cuz the top error is Syscall param write(buf) points to uninitialised byte(s
16:21
and io_write calls fwrite
16:22
@heinrich5991 why shouldnt we memzero it? does fwrite handle it?
Avatar
fread should initialize all the bytes up to the number returned
16:23
fwrite should only read as far as the parameter goes
16:23
so there shouldn't™ be uninitialized bytes there
Avatar
mem_zero would probably silence that warning, but it would do unnecessary work
Avatar
You could create a suppression file if we have actual false positives
16:24
then we don't have to keep looking into the same stuff again and again
16:24
and add it to the repository I mean
16:24
i guess i have to rerun with -gen-suppressions=all
Avatar
you could write the suppression file manually, but probably easier that way
Avatar
it complains aBuf is uninitialized but i dont see how, m_pTooltip is initialized to 0 in the ctor
16:44
is valgrind not that good?
Avatar
@Ryozuki can you point me to the line in the valgrind-out.txt file?
16:47
notification from github: @Ryozuki pushed 0 commits.
Avatar
i removed a commit i made
Avatar
ah
Avatar
cuz it was already initialized
16:47
how to print current line in less?
16:47
xD
Avatar
idk
16:48
What is the command to make less display line numbers in the left column?
Avatar
You can also toggle line numbers without quitting less by typing -N.
16:49
You can also press = while less is open to just display (at the bottom of the screen) information about the current screen, including line numbers, with format:
16:50
use the new file i uploaded with gen suppressions btw
16:51
valgrind maybe doesnt know what str_format does
Avatar
it shouldn't care what str_format does, afaik it tracks which bytes have been initialized and which have not
Avatar
well its either str_format or str_copy
Avatar
or str_utf8_decode
16:55
or even TextEx?
Avatar
all those functions just use ptext
16:56
which is aBuf
16:56
iirc
Avatar
well, but if they read past the 0 byte, then they read uninit data
16:57
(I'm looking into it right now)
Avatar
if this leaks it would be big right? its called every render call xD
Avatar
this isn't mem leaks, but reading of uninit data
Avatar
ah yeah
16:57
which could lead to ub?
Avatar
which is UB
16:59
unlikely to cause problems that deep in the call stack, should still be fixed
Avatar
well ill take a break now, ill come later at night maybe
Avatar
I'll highlight you if I find something
Avatar
@Ryozuki ah, found it
17:00
line 920 in TextEx reads two characters ahead
17:01
i.e. one past the null byte
Avatar
Ok send remind me it on git, i wont be at pc for a whilw
Avatar
@heinrich5991 i guess this fixes it?
19:24
hm wait
19:25
i guess this
19:28
i think a check like this is also good
Avatar
@heinrich5991 can you help me debug a cmake problem while compiling with osxcross?
Avatar
@Ryozuki you shouldn't check for !pText and return
19:53
it is not in the function's responsibility to check that the user passed in a non-null pointer
19:53
if you want to check, do it with an assert
19:58
@Learath2 what's the issue?
19:58
(playing cs right now though)
Avatar
ee0d5f8 Update settingscommands - def-
Avatar
Trafalgar Law 2020-04-08 21:05:10Z
@deen did you failed here? https://i.imgur.com/GoUljV7.png
21:05
You delete rifle but you added again
Avatar
I nuked my VM, will set it up again tomorrow
21:06
@heinrich5991 it was having trouble finding the bundled SDL2
Exported 374 message(s)